ICL-MoE
MoDirectRSICLRegressor
ICL-MoE post-processor for DirectRS regressors.
Takes a fitted MoDirectRSRegressor and builds a soft-gated mixture of experts using kNN attention in the DirectRS stretch embedding space.
Parameters
directrs_model : MoDirectRSRegressor
A fitted DirectRS regressor (must have ._core set).
name : str, optional
Model identifier. Default: “ICL-MoE”.
variant : str, default=“hierarchical”
One of “point_expert”, “local_linear”, “leaf_expert”, “hierarchical”.
k : int, default=50
Number of nearest neighbors for kNN gating.
tau : float, default=1.0
Temperature for softmax gating weights.
ridge_lambda : float, default=1.0
Regularisation for local linear / residual ridge.
top_m : int, default=5
Number of top leaf experts per tree (variant C / hierarchical).
__init__(directrs_model, name=None, variant='hierarchical', k=50, tau=1.0, ridge_lambda=1.0, top_m=5)
fit(X, y, sample_weight=None, verbose=False)
Fit ICL-MoE on training data.
Parameters
X : np.ndarray of shape (n_samples, n_features) y : np.ndarray of shape (n_samples,) sample_weight : ignored (kept for API compatibility) verbose : bool, default=False
Returns
self
explain_local(X, feature_names=None, sample_index=0)
Per-sample additive decomposition in original feature space.
Parameters
X : array-like of shape (n_samples, n_features) feature_names : list of str, optional sample_index : int, default=0 Sample to show in the default plot.
Returns
ValidationResult
result.value contains all samples (intercept, contributions arrays). result.plot() shows the selected sample.
importance_global(X, feature_names=None, mode='contrib_abs')
Global feature importance via explain_local aggregation.
Parameters
X : array-like of shape (n_samples, n_features) feature_names : list of str, optional mode : str, default=“contrib_abs” “contrib_abs” or “contrib_rms”.
Returns
ValidationResult
Horizontal bar plot of feature importance.
get_neighbor_analysis(X, sample_index=0, feature_names=None)
Return neighbor analysis for a query point.
Parameters
X : array-like of shape (n_samples, n_features) sample_index : int, default=0 feature_names : list of str, optional
Returns
ValidationResult
Accessible plots via result.plot('weight_bar') and result.plot('neighbor_scatter'), or result.plot() for all.
get_leaf_gating_analysis(X, sample_index=0, tree_index=0, feature_names=None)
Leaf gating weights for a query point.
Parameters
X : array-like of shape (n_samples, n_features) sample_index : int, default=0 tree_index : int, default=0 feature_names : list of str, optional
Returns
ValidationResult
Bar plot of per-leaf gating weights.
reset_calibrate_interval()
reset_calibrate_proba()
calibrate_interval(X, y, alpha=0.1, max_depth: int=5)
Fit a conformal prediction model to the given data.
This method computes the model’s prediction interval calibrated to the given data.
If the model is a regressor, splits the data with 50% for fitting lower (alpha / 5) and upper (1 - alpha / 2) gradient boosting trees-based quantile regression to the model’s residual; and 50% for calibration.
If the model is a binary classifiers, it computes the calibration quantile based on predicted probabilities for the positive class.
Parameters
X : X : np.ndarray of shape (n_samples, n_features)
Feature matrix for prediction.
y : array-like of shape (n_samples, )
Target values.
alpha : float, default=0.1
Expected miscoverage for the conformal prediction.
max_depth : int, default=5
Maximum depth of the gradient boosting trees for regression tasks. Only used when task_type is REGRESSION.
Raises
ValueError: If the model is neither a regressor nor a classifier.
predict_interval(X)
Predict the prediction interval for the given data based on the conformal prediction model.
It splits the data with 50% for fitting lower (alpha / 5) and upper (1 - alpha / 2) gradient boosting trees-based quantile regression to the model’s residual; and 50% for calibration.
Parameters
X : np.ndarray of shape (n_samples, n_features)
Feature matrix for prediction.
Returns
np.ndarray: The lower and upper bounds of the prediction intervals for each sample
in the format [n_samples, 2] for regressors or a flattened array for classifiers.
Raises
ValueError: If fit_conformal has not been called to fit the conformal prediction model
before calling this method.
predict(X)
Model predictions, calling the child class’s ‘_predict’ method.
Parameters
X : np.ndarray of shape (n_samples, n_features)
Feature matrix for prediction.
Returns
np.ndarray: The (calibrated) final prediction
name()
version()
save(file_name: str)
Save the model into file system.
Parameters
file_name: str
The path and name of the file.
load(file_name: str)
Load the model into memory from file system.
Parameters
file_name: str
The path and name of the file.
Returns
estimator object
MoDirectRSICLClassifier
ICL-MoE post-processor for DirectRS classifiers.
Takes a fitted MoDirectRSClassifier and builds a soft-gated mixture of experts using kNN attention in the DirectRS stretch embedding space. Operates in logit space internally; predictions use sigmoid.
Parameters
directrs_model : MoDirectRSClassifier
A fitted DirectRS classifier (must have ._core set).
name : str, optional
Model identifier. Default: “ICL-MoE-Cls”.
variant : str, default=“hierarchical”
One of “point_expert”, “local_linear”, “leaf_expert”, “hierarchical”.
k : int, default=50
Number of nearest neighbors for kNN gating.
tau : float, default=1.0
Temperature for softmax gating weights.
ridge_lambda : float, default=1.0
Regularisation for local linear / residual ridge.
top_m : int, default=5
Number of top leaf experts per tree (variant C / hierarchical).
__init__(directrs_model, name=None, variant='hierarchical', k=50, tau=1.0, ridge_lambda=1.0, top_m=5)
fit(X, y, sample_weight=None, verbose=False)
Fit ICL-MoE on training data for classification.
Parameters
X : np.ndarray of shape (n_samples, n_features) y : np.ndarray of shape (n_samples,) sample_weight : ignored verbose : bool, default=False
Returns
self
explain_local(X, feature_names=None, sample_index=0)
Per-sample additive decomposition in original feature space.
Parameters
X : array-like of shape (n_samples, n_features) feature_names : list of str, optional sample_index : int, default=0 Sample to show in the default plot.
Returns
ValidationResult
result.value contains all samples (intercept, contributions arrays). result.plot() shows the selected sample.
importance_global(X, feature_names=None, mode='contrib_abs')
Global feature importance via explain_local aggregation.
Parameters
X : array-like of shape (n_samples, n_features) feature_names : list of str, optional mode : str, default=“contrib_abs” “contrib_abs” or “contrib_rms”.
Returns
ValidationResult
Horizontal bar plot of feature importance.
get_neighbor_analysis(X, sample_index=0, feature_names=None)
Return neighbor analysis for a query point.
Parameters
X : array-like of shape (n_samples, n_features) sample_index : int, default=0 feature_names : list of str, optional
Returns
ValidationResult
Accessible plots via result.plot('weight_bar') and result.plot('neighbor_scatter'), or result.plot() for all.
get_leaf_gating_analysis(X, sample_index=0, tree_index=0, feature_names=None)
Leaf gating weights for a query point.
Parameters
X : array-like of shape (n_samples, n_features) sample_index : int, default=0 tree_index : int, default=0 feature_names : list of str, optional
Returns
ValidationResult
Bar plot of per-leaf gating weights.
reset_calibrate_interval()
reset_calibrate_proba()
calibrate_interval(X, y, alpha=0.1)
Fit a conformal prediction model to the given data.
This method computes the model’s prediction interval calibrated to the given data.
It computes the calibration quantile based on predicted probabilities for the positive class.
Parameters
X : X : np.ndarray of shape (n_samples, n_features)
Feature matrix for prediction.
y : array-like of shape (n_samples, )
Target values.
alpha : float, default=0.1
Expected miscoverage for the conformal prediction.
Raises
ValueError: If the model is neither a regressor nor a classifier.
predict_interval(X)
Predict the prediction set for the given data based on the conformal prediction model.
This method computes the model prediction interval (regression) or prediction sets (classification) using conformal prediction.
Parameters
X : np.ndarray of shape (n_samples, n_features)
Feature matrix for prediction.
Returns
np.ndarray: The lower and upper bounds of the prediction intervals for each sample
in the format [n_samples, 2] for regressors or a flattened array for classifiers.
Raises
ValueError: If fit_conformal has not been called to fit the conformal prediction model
before calling this method.
calibrate_proba(X, y, sample_weight=None, method='sigmoid')
Fit the calibration method on the model’s predictions.
Parameters
X : np.ndarray of shape (n_samples, n_features)
Feature matrix for prediction.
y : np.ndarray of shape (n_samples, )
Ground truth labels.
sample_weight : array-like, shape (n_samples,), default=None
Sample weights.
method : {‘sigmoid’, ‘isotonic’}, default=‘sigmoid’
The calibration method.
- ‘sigmoid’: Platt’s method, i.e., fit a logistic regression on predicted probabilities and y
- ‘isotonic’: Fit an isotonic regression on predicted probabilities and y.
Returns
self: Calibrated estimator
predict_proba(X, calibration: bool=True)
Predict (calibrated) probabilities for X.
Parameters
X : np.ndarray of shape (n_samples, n_features)
Feature matrix for prediction.
calibration : bool, default=True
If True, will return calibrated probability if calibration is done. Otherwise, will return raw probability.
Returns
np.ndarray: The (calibrated) predicted probabilities
predict(X, calibration: bool=True)
Model predictions, calling the child class’s ‘_predict’ method.
Parameters
X : np.ndarray of shape (n_samples, n_features)
Feature matrix for prediction.
calibration : bool, default=True
If True, will use calibrated probability if calibration is done. Otherwise, will use raw probability.
Returns
np.ndarray: The (calibrated) final prediction
decision_function(X, calibration: bool=True)
Computes the decision function for the given input data.
Parameters
X : np.ndarray of shape (n_samples, n_features)
Feature matrix for prediction.
calibration : bool, default=True
If True, will use calibrated probability if calibration is done. Otherwise, will use raw probability.
Returns
logit_prediction : array, shape (n_samples,) or (n_samples, n_classes)
Array of (calibrated) logit predictions.
name()
version()
save(file_name: str)
Save the model into file system.
Parameters
file_name: str
The path and name of the file.
load(file_name: str)
Load the model into memory from file system.
Parameters
file_name: str
The path and name of the file.
Returns
estimator object