Note
Go to the end to download the full example code.
FuseKernel Classification
For classification FuseKernel fits the fused kernel ridge regression on one-hot targets and reads probabilities off a temperature-calibrated softmax. This example fuses the XGBoost tree co-membership kernel with a learned multi-scale spectral kernel (MS-SKM) on the TaiwanCredit dataset.
Installation
# To install the required package, use the following command:
# !pip install modeva
Authentication
# To get authentication, use the following command: (To get full access please replace the token to your own token)
# from modeva.utils.authenticate import authenticate
# authenticate(auth_code='eaaa4301-b140-484c-8e93-f9f633c8bacb')
Import required modules
import warnings
warnings.filterwarnings("ignore")
import numpy as np
import pandas as pd
from modeva import DataSet, ModelZoo, TestSuite
Load and prepare dataset
ds = DataSet()
ds.load(name="TaiwanCredit")
ds.set_random_split()
Train FuseKernel
For classification FuseKernel fits the fused KRR on one-hot targets and reads probabilities off a temperature-calibrated softmax. Binary and multiclass are both supported, and the simplex weights are selected on the held-out query by accuracy.
from modeva.models import MoFuseKernelClassifier
model = MoFuseKernelClassifier(
name="FuseKernel-Cls",
use_xgb=True, use_spectral=True,
spectral_params={"H": 4, "K": 8, "kernel": "laplace", "epochs": 50},
interpret_ref_size=400, interpret_n_grid=20,
random_state=0,
)
mz = ModelZoo(dataset=ds)
mz.add_model(model)
mz.train_all()
mz.leaderboard()
0%| | 0/1 [00:00<?, ?it/s]
100%|██████████| 1/1 [14:05<00:00, 845.40s/it]
100%|██████████| 1/1 [14:05<00:00, 845.40s/it]
Accuracy
AUC, accuracy, F1, log-loss and Brier, train vs test.
ts = TestSuite(ds, model)
ts.diagnose_accuracy_table().table
Probabilities
predict_proba returns calibrated class probabilities (rows sum to 1).
proba = model.predict_proba(ds.test_x)
pd.DataFrame(proba[:5], columns=[f"P(class={c})" for c in model.classes_])
Inherent Interpretation (FANOVA)
The same functional-ANOVA decomposition applies to the classification scores.
results = ts.interpret_fi()
results.table
results = ts.interpret_ei()
results.table
results = ts.interpret_effects(features="PAY_1")
results.plot(results.get_figure_names()[0])
results = ts.interpret_local_fi(sample_index=1)
results.table
Reliability
Conformal prediction sets and coverage for the classifier.
ts.diagnose_reliability().table
Total running time of the script: (44 minutes 40.452 seconds)