GBDT Leaf Kernel
MoGBDTKernelRegressor
GBDT-as-learned-kernel regressor with five interchangeable heads.
Wraps a pre-trained tree ensemble and exposes prediction heads on the induced leaf kernel. Default head exact_gbdt recovers the base GBDT prediction exactly via the generalized Nadaraya-Watson representation (Theorem 1 of the GBDT-GNW paper).
Parameters
base_model : fitted XGBoost regressor (or MoXGBRegressor wrapper)
Pre-trained ensemble whose leaf geometry will be re-used.
name : str, optional
Identifier. Default: "GBDTKernel".
kernel_topk : int, default=200
Number of training neighbors used by the NW heads.
kernel_rho : float, default=1.0
Sharpening exponent: w_i \propto K(x, x_i)^rho.
ridge_lambda : float, default=1.0
KRR regularization on the leaf one-hot basis.
n_clusters : int, default=8
Number of behavioral cohorts for cluster-gated residual repair.
gate_mode : {“defensive”, “adaptive”}, default=“defensive” nystrom_landmarks : int, default=300 residual_gamma_max : float, default=1.5 gate_tmin : float, default=1.96 shrink_tau : float, default=20.0 random_state : int, default=0
__init__(base_model, name=None, kernel_topk=200, kernel_rho=1.0, ridge_lambda=1.0, n_clusters=8, gate_mode='defensive', nystrom_landmarks=300, residual_gamma_max=1.5, gate_tmin=1.96, shrink_tau=20.0, random_state=0)
fit(X, y, sample_weight=None, X_val=None, y_val=None, feature_names=None, verbose=False)
predict(X, head='exact_gbdt')
Predict in target space using the chosen head.
explain_local(X, sample_index=0, feature_names=None)
Local TKLE evidence ledger for a single query.
Returns
ValidationResult
result.value contains the full ledger. result.table is a top-20-neighbor table. result.plot('weight_bar') and result.plot('neighbor_scatter') show the prediction’s evidence.
get_evidence_diagnostics(X)
Six TKLE diagnostics (neff, Delta_y, Delta_q, G_q, Delta_K, C_cal) across a query batch.
Returns
ValidationResult
result.table summarizes each diagnostic’s distribution; result.plot('neff') etc. show individual histograms.
importance_global(X, feature_names=None)
Kernel-weighted feature importance: features whose neighbors are tightly concentrated (small within-neighborhood dispersion) are deemed important under the learned geometry.
get_neighbor_analysis(X, sample_index=0)
Top-k neighbor table for a single query.
nominate_head(X_val, y_val)
Run the GNW-paper diagnostics and recommend a head.
diagnose_weakness(top_features=8, n_bins=10)
Per-cluster weakness scores + JS feature profile (paper_cluster_gated §4).
predict_with_context(X_query, context_X, context_y, head='gnw_label')
In-context prediction: use the supplied (context_X, context_y) pool as the NW memory instead of the original training set.
The base kernel geometry is unchanged (it is the fitted GBDT’s leaf kernel); only the keys/values are replaced by the user-provided context. Useful for “what if I had observed this set of cases?” explanations.
Parameters
X_query : ndarray
Queries to predict.
context_X : ndarray
Context features (acts as the NW memory).
context_y : ndarray
Context labels.
head : {“gnw_label”, “gnw_leaf”}, default=“gnw_label”
Only neighbor-based heads support an external context.
Returns
predictions : ndarray weights : ndarray of shape (n_query, k) Normalized kernel weights against the context pool. idx : ndarray of shape (n_query, k) Indices into context_X of the top-k retrieved cases.
reset_calibrate_interval()
reset_calibrate_proba()
calibrate_interval(X, y, alpha=0.1, max_depth: int=5)
Fit a conformal prediction model to the given data.
This method computes the model’s prediction interval calibrated to the given data.
If the model is a regressor, splits the data with 50% for fitting lower (alpha / 5) and upper (1 - alpha / 2) gradient boosting trees-based quantile regression to the model’s residual; and 50% for calibration.
If the model is a binary classifiers, it computes the calibration quantile based on predicted probabilities for the positive class.
Parameters
X : X : np.ndarray of shape (n_samples, n_features)
Feature matrix for prediction.
y : array-like of shape (n_samples, )
Target values.
alpha : float, default=0.1
Expected miscoverage for the conformal prediction.
max_depth : int, default=5
Maximum depth of the gradient boosting trees for regression tasks. Only used when task_type is REGRESSION.
Raises
ValueError: If the model is neither a regressor nor a classifier.
predict_interval(X)
Predict the prediction interval for the given data based on the conformal prediction model.
It splits the data with 50% for fitting lower (alpha / 5) and upper (1 - alpha / 2) gradient boosting trees-based quantile regression to the model’s residual; and 50% for calibration.
Parameters
X : np.ndarray of shape (n_samples, n_features)
Feature matrix for prediction.
Returns
np.ndarray: The lower and upper bounds of the prediction intervals for each sample
in the format [n_samples, 2] for regressors or a flattened array for classifiers.
Raises
ValueError: If fit_conformal has not been called to fit the conformal prediction model
before calling this method.
name()
version()
save(file_name: str)
Save the model into file system.
Parameters
file_name: str
The path and name of the file.
load(file_name: str)
Load the model into memory from file system.
Parameters
file_name: str
The path and name of the file.
Returns
estimator object
MoGBDTKernelClassifier
GBDT-as-learned-kernel binary classifier with five heads.
Same parameters and head menu as MoGBDTKernelRegressor. All predictions are produced in logit space internally; predict_proba applies the sigmoid.
__init__(base_model, name=None, kernel_topk=200, kernel_rho=1.0, ridge_lambda=1.0, n_clusters=8, gate_mode='defensive', nystrom_landmarks=300, residual_gamma_max=1.5, gate_tmin=1.96, shrink_tau=20.0, random_state=0)
fit(X, y, sample_weight=None, X_val=None, y_val=None, feature_names=None, verbose=False)
decision_function(X, head='exact_gbdt')
Margin / logit prediction under the chosen head.
predict_proba(X, head='exact_gbdt')
Class probabilities under the chosen head.
predict(X, head='exact_gbdt')
explain_local(X, sample_index=0, feature_names=None)
Local TKLE evidence ledger for a single query.
Returns
ValidationResult
result.value contains the full ledger. result.table is a top-20-neighbor table. result.plot('weight_bar') and result.plot('neighbor_scatter') show the prediction’s evidence.
get_evidence_diagnostics(X)
Six TKLE diagnostics (neff, Delta_y, Delta_q, G_q, Delta_K, C_cal) across a query batch.
Returns
ValidationResult
result.table summarizes each diagnostic’s distribution; result.plot('neff') etc. show individual histograms.
importance_global(X, feature_names=None)
Kernel-weighted feature importance: features whose neighbors are tightly concentrated (small within-neighborhood dispersion) are deemed important under the learned geometry.
get_neighbor_analysis(X, sample_index=0)
Top-k neighbor table for a single query.
nominate_head(X_val, y_val)
Run the GNW-paper diagnostics and recommend a head.
diagnose_weakness(top_features=8, n_bins=10)
Per-cluster weakness scores + JS feature profile (paper_cluster_gated §4).
predict_with_context(X_query, context_X, context_y, head='gnw_label')
In-context prediction: use the supplied (context_X, context_y) pool as the NW memory instead of the original training set.
The base kernel geometry is unchanged (it is the fitted GBDT’s leaf kernel); only the keys/values are replaced by the user-provided context. Useful for “what if I had observed this set of cases?” explanations.
Parameters
X_query : ndarray
Queries to predict.
context_X : ndarray
Context features (acts as the NW memory).
context_y : ndarray
Context labels.
head : {“gnw_label”, “gnw_leaf”}, default=“gnw_label”
Only neighbor-based heads support an external context.
Returns
predictions : ndarray weights : ndarray of shape (n_query, k) Normalized kernel weights against the context pool. idx : ndarray of shape (n_query, k) Indices into context_X of the top-k retrieved cases.
reset_calibrate_interval()
reset_calibrate_proba()
calibrate_interval(X, y, alpha=0.1)
Fit a conformal prediction model to the given data.
This method computes the model’s prediction interval calibrated to the given data.
It computes the calibration quantile based on predicted probabilities for the positive class.
Parameters
X : X : np.ndarray of shape (n_samples, n_features)
Feature matrix for prediction.
y : array-like of shape (n_samples, )
Target values.
alpha : float, default=0.1
Expected miscoverage for the conformal prediction.
Raises
ValueError: If the model is neither a regressor nor a classifier.
predict_interval(X)
Predict the prediction set for the given data based on the conformal prediction model.
This method computes the model prediction interval (regression) or prediction sets (classification) using conformal prediction.
Parameters
X : np.ndarray of shape (n_samples, n_features)
Feature matrix for prediction.
Returns
np.ndarray: The lower and upper bounds of the prediction intervals for each sample
in the format [n_samples, 2] for regressors or a flattened array for classifiers.
Raises
ValueError: If fit_conformal has not been called to fit the conformal prediction model
before calling this method.
calibrate_proba(X, y, sample_weight=None, method='sigmoid')
Fit the calibration method on the model’s predictions.
Parameters
X : np.ndarray of shape (n_samples, n_features)
Feature matrix for prediction.
y : np.ndarray of shape (n_samples, )
Ground truth labels.
sample_weight : array-like, shape (n_samples,), default=None
Sample weights.
method : {‘sigmoid’, ‘isotonic’}, default=‘sigmoid’
The calibration method.
- ‘sigmoid’: Platt’s method, i.e., fit a logistic regression on predicted probabilities and y
- ‘isotonic’: Fit an isotonic regression on predicted probabilities and y.
Returns
self: Calibrated estimator
name()
version()
save(file_name: str)
Save the model into file system.
Parameters
file_name: str
The path and name of the file.
load(file_name: str)
Load the model into memory from file system.
Parameters
file_name: str
The path and name of the file.
Returns
estimator object