DirectRS

MoDirectRSRegressor

DirectRS post-processor for tree ensemble regressors.

Takes a pre-trained tree ensemble (XGBoost, LightGBM, or CatBoost) and builds a piecewise-linear model that can match or improve the original with per-tree Ridge regression on geometric embeddings φ_t(x) = [1, S’x].

Parameters

base_model : fitted tree ensemble
Pre-trained XGBoost, LightGBM, or CatBoost regressor (or MoDeVa wrapper).

name : str, optional
Model identifier. Default: “DirectRS”.

construction : str, default=“C”
Operator construction for geometry extraction: “A” (gain-weighted), “B” (co-occurrence), “C” (value-weighted), “M” (count).

ridge_alpha : float, default=100.0
Ridge regularization for per-tree linear heads.

n_passes : int, default=1
Number of backfitting coordinate descent passes.

n_trees_used : int, optional
Number of trees to use. Default: all trees.

__init__(base_model, name=None, construction='C', ridge_alpha=100.0, n_passes=1, n_trees_used=None)

fit(X, y, sample_weight=None, X_val=None, y_val=None, verbose=False)

Fit DirectRS on training data.

Parameters

X : np.ndarray of shape (n_samples, n_features) y : np.ndarray of shape (n_samples,) sample_weight : ignored (kept for API compatibility) X_val : np.ndarray, optional y_val : np.ndarray, optional verbose : bool, default=False

Returns

self

get_global_stretch_analysis(feature_names=None)

Return eigendecomposition of S’ matrix.

Returns

ValidationResult
Accessible plots via result.plot('eigenvalue_spectrum'), result.plot('feature_activity'), result.plot('feature_loadings'), result.plot('g_matrix'), or result.plot() for all.

explain_local(X, feature_names=None, sample_index=0)

Exact per-sample additive decomposition in original feature space.

Parameters

X : array-like of shape (n_samples, n_features) feature_names : list of str, optional sample_index : int, default=0 Sample to show in the default plot.

Returns

ValidationResult
result.value contains all samples (intercept, contributions arrays). result.plot() shows the selected sample.

importance_global(X=None, feature_names=None, mode='slope')

Exact global feature importance in original feature space.

Parameters

X : array-like, optional
Required for data-weighted modes (“contrib_abs”, “contrib_rms”).

feature_names : list of str, optional mode : str, default=“slope” “slope”, “slope2”, “contrib_abs”, or “contrib_rms”.

Returns

ValidationResult
Horizontal bar plot of feature importance.

importance_main_interaction(X, feature_names=None, n_bins=20)

Exact empirical main-effect and interaction decomposition (Eq 10-14).

Parameters

X : array-like of shape (n_samples, n_features)
Data used to compute binned main effects and interaction residuals.

feature_names : list of str, optional n_bins : int, default=20 Number of quantile bins for continuous features.

Returns

ValidationResult
Grouped bar plot of main vs excess importance per feature. result.value contains all raw arrays including variance identity diagnostics (var_f, E_g2, rho, eta2_main, etc.).

geometric_interaction_traces(feature_names=None, K=3, gamma=0.5)

Geometric higher-order interaction tracing via stretch matrix (Section 6).

Parameters

feature_names : list of str, optional K : int, default=3 Maximum coupling order. gamma : float, default=0.5 Decay factor for cumulative map (0 < gamma < 1).

Returns

ValidationResult
Accessible plots via result.plot('spectrum'), result.plot('adjacency'), result.plot('cumulative_coupling'), or result.plot() for all.

get_off_diagonal_analysis(feature_names=None, top_k=10)

Return top off-diagonal G entries (feature interactions).

Parameters

feature_names : list of str, optional top_k : int, default=10 Number of top feature pairs to return, ranked by |G_ij|.

Returns

ValidationResult
Horizontal bar plot of top feature-pair coupling strengths.

get_feature_importance(feature_names=None)

Return Ridge-coefficient weighted feature importance.

reset_calibrate_interval()

reset_calibrate_proba()

calibrate_interval(X, y, alpha=0.1, max_depth: int=5)

Fit a conformal prediction model to the given data.

This method computes the model’s prediction interval calibrated to the given data.

If the model is a regressor, splits the data with 50% for fitting lower (alpha / 5) and upper (1 - alpha / 2) gradient boosting trees-based quantile regression to the model’s residual; and 50% for calibration.

If the model is a binary classifiers, it computes the calibration quantile based on predicted probabilities for the positive class.

Parameters

X : X : np.ndarray of shape (n_samples, n_features)
Feature matrix for prediction.

y : array-like of shape (n_samples, )
Target values.

alpha : float, default=0.1
Expected miscoverage for the conformal prediction.

max_depth : int, default=5
Maximum depth of the gradient boosting trees for regression tasks. Only used when task_type is REGRESSION.

Raises

ValueError: If the model is neither a regressor nor a classifier.

predict_interval(X)

Predict the prediction interval for the given data based on the conformal prediction model.

It splits the data with 50% for fitting lower (alpha / 5) and upper (1 - alpha / 2) gradient boosting trees-based quantile regression to the model’s residual; and 50% for calibration.

Parameters

X : np.ndarray of shape (n_samples, n_features)
Feature matrix for prediction.

Returns

np.ndarray: The lower and upper bounds of the prediction intervals for each sample
in the format [n_samples, 2] for regressors or a flattened array for classifiers.

Raises

ValueError: If fit_conformal has not been called to fit the conformal prediction model
before calling this method.

predict(X)

Model predictions, calling the child class’s ‘_predict’ method.

Parameters

X : np.ndarray of shape (n_samples, n_features)
Feature matrix for prediction.

Returns

np.ndarray: The (calibrated) final prediction

name()

version()

save(file_name: str)

Save the model into file system.

Parameters

file_name: str
The path and name of the file.

load(file_name: str)

Load the model into memory from file system.

Parameters

file_name: str
The path and name of the file.

Returns

estimator object

MoDirectRSClassifier

DirectRS post-processor for tree ensemble classifiers.

Takes a pre-trained tree ensemble classifier and builds a piecewise-linear model using logistic ridge via IRLS (weighted Ridge on working responses in logit space). Predictions use sigmoid for probabilities.

Parameters

base_model : fitted tree ensemble
Pre-trained XGBoost, LightGBM, or CatBoost classifier (or MoDeVa wrapper).

name : str, optional
Model identifier. Default: “DirectRS-Cls”.

construction : str, default=“C”
Operator construction for geometry extraction.

ridge_alpha : float, default=100.0
Ridge regularization for per-tree linear heads.

n_passes : int, default=1
Number of backfitting coordinate descent passes.

n_trees_used : int, optional
Number of trees to use. Default: all trees.

__init__(base_model, name=None, construction='C', ridge_alpha=100.0, n_passes=1, n_trees_used=None)

fit(X, y, sample_weight=None, X_val=None, y_val=None, verbose=False)

Fit DirectRS on training data for classification.

Parameters

X : np.ndarray of shape (n_samples, n_features) y : np.ndarray of shape (n_samples,) sample_weight : ignored X_val : np.ndarray, optional y_val : np.ndarray, optional verbose : bool, default=False

Returns

self

get_global_stretch_analysis(feature_names=None)

Return eigendecomposition of S’ matrix.

Returns

ValidationResult
Accessible plots via result.plot('eigenvalue_spectrum'), result.plot('feature_activity'), result.plot('feature_loadings'), result.plot('g_matrix'), or result.plot() for all.

explain_local(X, feature_names=None, sample_index=0)

Exact per-sample additive decomposition in original feature space.

Parameters

X : array-like of shape (n_samples, n_features) feature_names : list of str, optional sample_index : int, default=0 Sample to show in the default plot.

Returns

ValidationResult
result.value contains all samples (intercept, contributions arrays). result.plot() shows the selected sample.

importance_global(X=None, feature_names=None, mode='slope')

Exact global feature importance in original feature space.

Parameters

X : array-like, optional
Required for data-weighted modes (“contrib_abs”, “contrib_rms”).

feature_names : list of str, optional mode : str, default=“slope” “slope”, “slope2”, “contrib_abs”, or “contrib_rms”.

Returns

ValidationResult
Horizontal bar plot of feature importance.

importance_main_interaction(X, feature_names=None, n_bins=20)

Exact empirical main-effect and interaction decomposition (Eq 10-14).

Parameters

X : array-like of shape (n_samples, n_features)
Data used to compute binned main effects and interaction residuals.

feature_names : list of str, optional n_bins : int, default=20 Number of quantile bins for continuous features.

Returns

ValidationResult
Grouped bar plot of main vs excess importance per feature. result.value contains all raw arrays including variance identity diagnostics (var_f, E_g2, rho, eta2_main, etc.).

geometric_interaction_traces(feature_names=None, K=3, gamma=0.5)

Geometric higher-order interaction tracing via stretch matrix (Section 6).

Parameters

feature_names : list of str, optional K : int, default=3 Maximum coupling order. gamma : float, default=0.5 Decay factor for cumulative map (0 < gamma < 1).

Returns

ValidationResult
Accessible plots via result.plot('spectrum'), result.plot('adjacency'), result.plot('cumulative_coupling'), or result.plot() for all.

get_off_diagonal_analysis(feature_names=None, top_k=10)

Return top off-diagonal G entries (feature interactions).

Parameters

feature_names : list of str, optional top_k : int, default=10 Number of top feature pairs to return, ranked by |G_ij|.

Returns

ValidationResult
Horizontal bar plot of top feature-pair coupling strengths.

get_feature_importance(feature_names=None)

Return Ridge-coefficient weighted feature importance.

reset_calibrate_interval()

reset_calibrate_proba()

calibrate_interval(X, y, alpha=0.1)

Fit a conformal prediction model to the given data.

This method computes the model’s prediction interval calibrated to the given data.

It computes the calibration quantile based on predicted probabilities for the positive class.

Parameters

X : X : np.ndarray of shape (n_samples, n_features)
Feature matrix for prediction.

y : array-like of shape (n_samples, )
Target values.

alpha : float, default=0.1
Expected miscoverage for the conformal prediction.

Raises

ValueError: If the model is neither a regressor nor a classifier.

predict_interval(X)

Predict the prediction set for the given data based on the conformal prediction model.

This method computes the model prediction interval (regression) or prediction sets (classification) using conformal prediction.

Parameters

X : np.ndarray of shape (n_samples, n_features)
Feature matrix for prediction.

Returns

np.ndarray: The lower and upper bounds of the prediction intervals for each sample
in the format [n_samples, 2] for regressors or a flattened array for classifiers.

Raises

ValueError: If fit_conformal has not been called to fit the conformal prediction model
before calling this method.

calibrate_proba(X, y, sample_weight=None, method='sigmoid')

Fit the calibration method on the model’s predictions.

Parameters

X : np.ndarray of shape (n_samples, n_features)
Feature matrix for prediction.

y : np.ndarray of shape (n_samples, )
Ground truth labels.

sample_weight : array-like, shape (n_samples,), default=None
Sample weights.

method : {‘sigmoid’, ‘isotonic’}, default=‘sigmoid’
The calibration method.

• ‘sigmoid’: Platt’s method, i.e., fit a logistic regression on predicted probabilities and y
• ‘isotonic’: Fit an isotonic regression on predicted probabilities and y.

Returns

self: Calibrated estimator

predict_proba(X, calibration: bool=True)

Predict (calibrated) probabilities for X.

Parameters

X : np.ndarray of shape (n_samples, n_features)
Feature matrix for prediction.

calibration : bool, default=True
If True, will return calibrated probability if calibration is done. Otherwise, will return raw probability.

Returns

np.ndarray: The (calibrated) predicted probabilities

predict(X, calibration: bool=True)

Model predictions, calling the child class’s ‘_predict’ method.

Parameters

X : np.ndarray of shape (n_samples, n_features)
Feature matrix for prediction.

calibration : bool, default=True
If True, will use calibrated probability if calibration is done. Otherwise, will use raw probability.

Returns

np.ndarray: The (calibrated) final prediction

decision_function(X, calibration: bool=True)

Computes the decision function for the given input data.

Parameters

X : np.ndarray of shape (n_samples, n_features)
Feature matrix for prediction.

calibration : bool, default=True
If True, will use calibrated probability if calibration is done. Otherwise, will use raw probability.

Returns

logit_prediction : array, shape (n_samples,) or (n_samples, n_classes)
Array of (calibrated) logit predictions.

name()

version()

save(file_name: str)

Save the model into file system.

Parameters

file_name: str
The path and name of the file.

load(file_name: str)

Load the model into memory from file system.

Parameters

file_name: str
The path and name of the file.

Returns

estimator object