GBDT Leaf Kernel

Gradient-boosted-tree leaf co-membership kernel.

class modeva.models.MoGBDTKernelRegressor(base_model, name=None, kernel_topk=200, kernel_rho=1.0, ridge_lambda=1.0, n_clusters=8, gate_mode='defensive', nystrom_landmarks=300, residual_gamma_max=1.5, gate_tmin=1.96, shrink_tau=20.0, random_state=0)

GBDT-as-learned-kernel regressor with five interchangeable heads.

Wraps a pre-trained tree ensemble and exposes prediction heads on the induced leaf kernel. Default head exact_gbdt recovers the base GBDT prediction exactly via the generalized Nadaraya-Watson representation (Theorem 1 of the GBDT-GNW paper).

Parameters:

base_modelfitted XGBoost regressor (or MoXGBRegressor wrapper)

Pre-trained ensemble whose leaf geometry will be re-used.

namestr, optional

Identifier. Default: "GBDTKernel".

kernel_topkint, default=200

Number of training neighbors used by the NW heads.

kernel_rhofloat, default=1.0

Sharpening exponent: w_i \propto K(x, x_i)^rho.

ridge_lambdafloat, default=1.0

KRR regularization on the leaf one-hot basis.

n_clustersint, default=8

Number of behavioral cohorts for cluster-gated residual repair.

gate_mode{“defensive”, “adaptive”}, default=”defensive”

nystrom_landmarksint, default=300

residual_gamma_maxfloat, default=1.5

gate_tminfloat, default=1.96

shrink_taufloat, default=20.0

random_stateint, default=0

calibrate_interval(X, y, alpha=0.1, max_depth: int = 5)

Fit a conformal prediction model to the given data.

This method computes the model’s prediction interval calibrated to the given data.

If the model is a regressor, splits the data with 50% for fitting lower (alpha / 5) and upper (1 - alpha / 2) gradient boosting trees-based quantile regression to the model’s residual; and 50% for calibration.

If the model is a binary classifiers, it computes the calibration quantile based on predicted probabilities for the positive class.

Parameters:

XXnp.ndarray of shape (n_samples, n_features)

Feature matrix for prediction.

yarray-like of shape (n_samples, )

Target values.

alphafloat, default=0.1

Expected miscoverage for the conformal prediction.

max_depthint, default=5

Maximum depth of the gradient boosting trees for regression tasks. Only used when task_type is REGRESSION.

Raises:

ValueError: If the model is neither a regressor nor a classifier.

diagnose_weakness(top_features=8, n_bins=10)

Per-cluster weakness scores + JS feature profile (paper_cluster_gated §4).

explain_local(X, sample_index=0, feature_names=None)

Local TKLE evidence ledger for a single query.

Returns:

ValidationResult

result.value contains the full ledger. result.table is a top-20-neighbor table. result.plot('weight_bar') and result.plot('neighbor_scatter') show the prediction’s evidence.

fit(X, y, sample_weight=None, X_val=None, y_val=None, feature_names=None, verbose=False)

get_evidence_diagnostics(X)

Six TKLE diagnostics (neff, Delta_y, Delta_q, G_q, Delta_K, C_cal) across a query batch.

Returns:

ValidationResult

result.table summarizes each diagnostic’s distribution; result.plot('neff') etc. show individual histograms.

get_metadata_routing()

Get metadata routing of this object.

Please check User Guide on how the routing mechanism works.

Returns:

routingMetadataRequest

A MetadataRequest encapsulating routing information.

get_neighbor_analysis(X, sample_index=0)

Top-k neighbor table for a single query.

get_params(deep=True)

Get parameters for this estimator.

Parameters:

deepbool, default=True

If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

paramsdict

Parameter names mapped to their values.

importance_global(X, feature_names=None)

Kernel-weighted feature importance: features whose neighbors are tightly concentrated (small within-neighborhood dispersion) are deemed important under the learned geometry.

load(file_name: str)

Load the model into memory from file system.

Parameters:

file_name: str

The path and name of the file.

Returns:

estimator object

nominate_head(X_val, y_val)

Run the GNW-paper diagnostics and recommend a head.

predict(X, head='exact_gbdt')

Predict in target space using the chosen head.

predict_interval(X)

Predict the prediction interval for the given data based on the conformal prediction model.

It splits the data with 50% for fitting lower (alpha / 5) and upper (1 - alpha / 2) gradient boosting trees-based quantile regression to the model’s residual; and 50% for calibration.

Parameters:

Xnp.ndarray of shape (n_samples, n_features)

Feature matrix for prediction.

Returns:

np.ndarray: The lower and upper bounds of the prediction intervals for each sample

in the format [n_samples, 2] for regressors or a flattened array for classifiers.

Raises:

ValueError: If fit_conformal has not been called to fit the conformal prediction model

before calling this method.

predict_with_context(X_query, context_X, context_y, head='gnw_label')

In-context prediction: use the supplied (context_X, context_y) pool as the NW memory instead of the original training set.

The base kernel geometry is unchanged (it is the fitted GBDT’s leaf kernel); only the keys/values are replaced by the user-provided context. Useful for “what if I had observed this set of cases?” explanations.

Parameters:

X_queryndarray

Queries to predict.

context_Xndarray

Context features (acts as the NW memory).

context_yndarray

Context labels.

head{“gnw_label”, “gnw_leaf”}, default=”gnw_label”

Only neighbor-based heads support an external context.

Returns:

predictionsndarray

weightsndarray of shape (n_query, k)

Normalized kernel weights against the context pool.

idxndarray of shape (n_query, k)

Indices into context_X of the top-k retrieved cases.

reset_calibrate_interval()

reset_calibrate_proba()

save(file_name: str)

Save the model into file system.

Parameters:

file_name: str

The path and name of the file.

score(X, y, sample_weight=None)

Return the coefficient of determination of the prediction.

The coefficient of determination \(R^2\) is defined as \((1 - \frac{u}{v})\), where \(u\) is the residual sum of squares ((y_true - y_pred)** 2).sum() and \(v\) is the total sum of squares ((y_true - y_true.mean()) ** 2).sum(). The best possible score is 1.0 and it can be negative (because the model can be arbitrarily worse). A constant model that always predicts the expected value of y, disregarding the input features, would get a \(R^2\) score of 0.0.

Parameters:

Xarray-like of shape (n_samples, n_features)

Test samples. For some estimators this may be a precomputed kernel matrix or a list of generic objects instead with shape (n_samples, n_samples_fitted), where n_samples_fitted is the number of samples used in the fitting for the estimator.

yarray-like of shape (n_samples,) or (n_samples, n_outputs)

True values for X.

sample_weightarray-like of shape (n_samples,), default=None

Sample weights.

Returns:

scorefloat

\(R^2\) of self.predict(X) w.r.t. y.

Notes

The \(R^2\) score used when calling score on a regressor uses multioutput='uniform_average' from version 0.23 to keep consistent with default value of r2_score(). This influences the score method of all the multioutput regressors (except for MultiOutputRegressor).

set_fit_request(*, X_val: bool | None | str = '$UNCHANGED$', feature_names: bool | None | str = '$UNCHANGED$', sample_weight: bool | None | str = '$UNCHANGED$', verbose: bool | None | str = '$UNCHANGED$', y_val: bool | None | str = '$UNCHANGED$') → MoGBDTKernelRegressor

Request metadata passed to the fit method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to fit.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:

X_valstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for X_val parameter in fit.

feature_namesstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for feature_names parameter in fit.

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in fit.

verbosestr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for verbose parameter in fit.

y_valstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for y_val parameter in fit.

Returns:

selfobject

The updated object.

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:

**paramsdict

Estimator parameters.

Returns:

selfestimator instance

Estimator instance.

set_predict_request(*, head: bool | None | str = '$UNCHANGED$') → MoGBDTKernelRegressor

Request metadata passed to the predict method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to predict if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to predict.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:

headstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for head parameter in predict.

Returns:

selfobject

The updated object.

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → MoGBDTKernelRegressor

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to score.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in score.

Returns:

selfobject

The updated object.

property name

property version

class modeva.models.MoGBDTKernelClassifier(base_model, name=None, kernel_topk=200, kernel_rho=1.0, ridge_lambda=1.0, n_clusters=8, gate_mode='defensive', nystrom_landmarks=300, residual_gamma_max=1.5, gate_tmin=1.96, shrink_tau=20.0, random_state=0)

GBDT-as-learned-kernel binary classifier with five heads.

Same parameters and head menu as MoGBDTKernelRegressor. All predictions are produced in logit space internally; predict_proba applies the sigmoid.

calibrate_interval(X, y, alpha=0.1)

Fit a conformal prediction model to the given data.

This method computes the model’s prediction interval calibrated to the given data.

It computes the calibration quantile based on predicted probabilities for the positive class.

Parameters:

XXnp.ndarray of shape (n_samples, n_features)

Feature matrix for prediction.

yarray-like of shape (n_samples, )

Target values.

alphafloat, default=0.1

Expected miscoverage for the conformal prediction.

Raises:

ValueError: If the model is neither a regressor nor a classifier.

calibrate_proba(X, y, sample_weight=None, method='sigmoid')

Fit the calibration method on the model’s predictions.

Parameters:

Xnp.ndarray of shape (n_samples, n_features)

Feature matrix for prediction.

ynp.ndarray of shape (n_samples, )

Ground truth labels.

sample_weightarray-like, shape (n_samples,), default=None

Sample weights.

method{‘sigmoid’, ‘isotonic’}, default=’sigmoid’

The calibration method.

• ‘sigmoid’: Platt’s method, i.e., fit a logistic regression on predicted probabilities and y

• ‘isotonic’: Fit an isotonic regression on predicted probabilities and y.

Returns:

self: Calibrated estimator

decision_function(X, head='exact_gbdt')

Margin / logit prediction under the chosen head.

diagnose_weakness(top_features=8, n_bins=10)

Per-cluster weakness scores + JS feature profile (paper_cluster_gated §4).

explain_local(X, sample_index=0, feature_names=None)

Local TKLE evidence ledger for a single query.

Returns:

ValidationResult

result.value contains the full ledger. result.table is a top-20-neighbor table. result.plot('weight_bar') and result.plot('neighbor_scatter') show the prediction’s evidence.

fit(X, y, sample_weight=None, X_val=None, y_val=None, feature_names=None, verbose=False)

get_evidence_diagnostics(X)

Six TKLE diagnostics (neff, Delta_y, Delta_q, G_q, Delta_K, C_cal) across a query batch.

Returns:

ValidationResult

result.table summarizes each diagnostic’s distribution; result.plot('neff') etc. show individual histograms.

get_metadata_routing()

Get metadata routing of this object.

Please check User Guide on how the routing mechanism works.

Returns:

routingMetadataRequest

A MetadataRequest encapsulating routing information.

get_neighbor_analysis(X, sample_index=0)

Top-k neighbor table for a single query.

get_params(deep=True)

Get parameters for this estimator.

Parameters:

deepbool, default=True

If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

paramsdict

Parameter names mapped to their values.

importance_global(X, feature_names=None)

Kernel-weighted feature importance: features whose neighbors are tightly concentrated (small within-neighborhood dispersion) are deemed important under the learned geometry.

load(file_name: str)

Load the model into memory from file system.

Parameters:

file_name: str

The path and name of the file.

Returns:

estimator object

nominate_head(X_val, y_val)

Run the GNW-paper diagnostics and recommend a head.

predict(X, head='exact_gbdt')

Model predictions, calling the child class’s ‘_predict’ method.

Parameters:

Xnp.ndarray of shape (n_samples, n_features)

Feature matrix for prediction.

calibrationbool, default=True

If True, will use calibrated probability if calibration is done. Otherwise, will use raw probability.

Returns:

np.ndarray: The (calibrated) final prediction

predict_interval(X)

Predict the prediction set for the given data based on the conformal prediction model.

This method computes the model prediction interval (regression) or prediction sets (classification) using conformal prediction.

Parameters:

Xnp.ndarray of shape (n_samples, n_features)

Feature matrix for prediction.

Returns:

np.ndarray: The lower and upper bounds of the prediction intervals for each sample

in the format [n_samples, 2] for regressors or a flattened array for classifiers.

Raises:

ValueError: If fit_conformal has not been called to fit the conformal prediction model

before calling this method.

predict_proba(X, head='exact_gbdt')

Class probabilities under the chosen head.

predict_with_context(X_query, context_X, context_y, head='gnw_label')

In-context prediction: use the supplied (context_X, context_y) pool as the NW memory instead of the original training set.

The base kernel geometry is unchanged (it is the fitted GBDT’s leaf kernel); only the keys/values are replaced by the user-provided context. Useful for “what if I had observed this set of cases?” explanations.

Parameters:

X_queryndarray

Queries to predict.

context_Xndarray

Context features (acts as the NW memory).

context_yndarray

Context labels.

head{“gnw_label”, “gnw_leaf”}, default=”gnw_label”

Only neighbor-based heads support an external context.

Returns:

predictionsndarray

weightsndarray of shape (n_query, k)

Normalized kernel weights against the context pool.

idxndarray of shape (n_query, k)

Indices into context_X of the top-k retrieved cases.

reset_calibrate_interval()

reset_calibrate_proba()

save(file_name: str)

Save the model into file system.

Parameters:

file_name: str

The path and name of the file.

score(X, y, sample_weight=None)

Return the mean accuracy on the given test data and labels.

In multi-label classification, this is the subset accuracy which is a harsh metric since you require for each sample that each label set be correctly predicted.

Parameters:

Xarray-like of shape (n_samples, n_features)

Test samples.

yarray-like of shape (n_samples,) or (n_samples, n_outputs)

True labels for X.

sample_weightarray-like of shape (n_samples,), default=None

Sample weights.

Returns:

scorefloat

Mean accuracy of self.predict(X) w.r.t. y.

set_decision_function_request(*, head: bool | None | str = '$UNCHANGED$') → MoGBDTKernelClassifier

Request metadata passed to the decision_function method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to decision_function if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to decision_function.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:

headstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for head parameter in decision_function.

Returns:

selfobject

The updated object.

set_fit_request(*, X_val: bool | None | str = '$UNCHANGED$', feature_names: bool | None | str = '$UNCHANGED$', sample_weight: bool | None | str = '$UNCHANGED$', verbose: bool | None | str = '$UNCHANGED$', y_val: bool | None | str = '$UNCHANGED$') → MoGBDTKernelClassifier

Request metadata passed to the fit method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to fit.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:

X_valstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for X_val parameter in fit.

feature_namesstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for feature_names parameter in fit.

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in fit.

verbosestr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for verbose parameter in fit.

y_valstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for y_val parameter in fit.

Returns:

selfobject

The updated object.

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:

**paramsdict

Estimator parameters.

Returns:

selfestimator instance

Estimator instance.

set_predict_proba_request(*, head: bool | None | str = '$UNCHANGED$') → MoGBDTKernelClassifier

Request metadata passed to the predict_proba method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to predict_proba if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to predict_proba.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:

headstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for head parameter in predict_proba.

Returns:

selfobject

The updated object.

set_predict_request(*, head: bool | None | str = '$UNCHANGED$') → MoGBDTKernelClassifier

Request metadata passed to the predict method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to predict if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to predict.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:

headstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for head parameter in predict.

Returns:

selfobject

The updated object.

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → MoGBDTKernelClassifier

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to score.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in score.

Returns:

selfobject

The updated object.

property name

property version