FuseKernel Regression

FuseKernel fuses complementary kernel channels and decodes them with kernel ridge regression. This example fuses the XGBoost tree co-membership kernel with a learned multi-scale spectral kernel (MS-SKM) on the CaliforniaHousing dataset.

Installation

# To install the required package, use the following command:
# !pip install modeva

Authentication

# To get authentication, use the following command: (To get full access please replace the token to your own token)
# from modeva.utils.authenticate import authenticate
# authenticate(auth_code='eaaa4301-b140-484c-8e93-f9f633c8bacb')

Import required modules

import warnings
warnings.filterwarnings("ignore")

import numpy as np
import pandas as pd
from modeva import DataSet, ModelZoo, TestSuite

Load and prepare dataset

ds = DataSet()
ds.load(name="CaliforniaHousing")
ds.set_random_split()

Train FuseKernel

FuseKernel fuses complementary kernel channels – here the XGBoost tree co-membership kernel and the learned multi-scale spectral kernel (MS-SKM) – and decodes them with kernel ridge regression. Every channel is fit on a support fold and the mixture weights and ridge are selected on a held-out query fold (fit_method="grid"), so the selection is leakage-free.

from modeva.models import MoFuseKernelRegressor

model = MoFuseKernelRegressor(
    name="FuseKernel",
    use_xgb=True, use_spectral=True,            # channels to fuse
    fit_method="grid",                          # leakage-free, query-scored
    spectral_params={"H": 4, "K": 8, "kernel": "laplace", "epochs": 50},
    interpret_ref_size=400, interpret_n_grid=20,
    random_state=0,
)

mz = ModelZoo(dataset=ds)
mz.add_model(model)
mz.train_all()
mz.leaderboard()

  0%|          | 0/1 [00:00<?, ?it/s]
100%|██████████| 1/1 [08:09<00:00, 489.64s/it]
100%|██████████| 1/1 [08:09<00:00, 489.64s/it]

	start_time	end_time	Duration	train MSE	test MSE	train MAE	test MAE	train R2	test R2
FuseKernel	2026-07-09 14:24:53	2026-07-09 14:31:20	386.4657	0.1346	0.1842	0.2248	0.2815	0.8994	0.8587

Accuracy

FuseKernel is a standard MoDeVa model, so the whole TestSuite applies.

ts = TestSuite(ds, model)
ts.diagnose_accuracy_table().table

	MSE	MAE	R2
train	0.1346	0.2248	0.8994
test	0.1842	0.2815	0.8587
GAP	0.0496	0.0567	-0.0407

Inherent Interpretation (FANOVA)

With a spectral channel present, FuseKernel exposes a functional-ANOVA decomposition of the fused predictor through the standard interpret_* methods.

results = ts.interpret_fi()
results.table

	Name	Score
0	MedInc	0.1146
1	HouseAge	0.0034
2	AveRooms	0.1080
3	AveBedrms	0.0007
4	Population	0.0009
5	AveOccup	0.0595
6	Latitude	0.4171
7	Longitude	0.2957

results = ts.interpret_ei()
results.table

	Name	Importance
0	MedInc	0.1519
1	HouseAge	0.0011
2	AveRooms	0.1017
3	AveBedrms	0.0002
4	Population	0.0009
5	AveOccup	0.0660
6	Latitude	0.3315
7	Longitude	0.1926
8	Longitude & Latitude	0.1152
9	Longitude & AveOccup	0.0042
10	Longitude & HouseAge	0.0008
11	Longitude & MedInc	0.0094
12	Longitude & AveBedrms	0.0005
13	Latitude & AveOccup	0.0046
14	Latitude & HouseAge	0.0034
15	Latitude & MedInc	0.0080
16	Latitude & AveBedrms	0.0007
17	AveOccup & HouseAge	0.0074

results = ts.interpret_effects(features="MedInc")
results.plot(results.get_figure_names()[0])

results = ts.interpret_effects(features="Latitude")
results.plot(results.get_figure_names()[0])

results = ts.interpret_local_fi(sample_index=1)
results.table

	Name	Importance	Values
0	MedInc	-0.2900	2.0804
1	HouseAge	0.0348	42.0000
2	AveRooms	-0.2295	4.2941
3	AveBedrms	-0.0016	1.1176
4	Population	-0.0015	1206.0000
5	AveOccup	0.3865	2.0269
6	Latitude	-0.3738	37.8400
7	Longitude	0.6149	-122.2600

Channel Decomposition

The fused prediction is an exact additive sum over channels. channel_contributions returns each channel’s contribution in target units.

result = model.channel_contributions(ds.test_x)
result.table

	Channel	MeanAbsContribution
0	spectral	0.6280
1	tree	0.3481

Inherent GP Predictive Uncertainty

For regression the prediction intervals are the closed-form Gaussian-process posterior of the fused kernel, so diagnose_reliability reflects the model’s own uncertainty.

ts.diagnose_reliability().table

	Avg.Width	Avg.Coverage
0	1.2511	0.907

mean, var = model.predict_dist(ds.test_x)
intervals = model.predict_interval(ds.test_x)      # (n, 2) at the calibrated level
pd.DataFrame({"mean": mean[:5], "std": np.sqrt(var)[:5],
              "low": intervals[:5, 0], "high": intervals[:5, 1]})

	mean	std	low	high
0	2.6644	0.4574	1.9120	3.4168
1	1.7150	0.4660	0.9484	2.4815
2	2.1091	0.4491	1.3703	2.8479
3	1.9841	0.4648	1.2195	2.7487
4	1.3017	0.4524	0.5577	2.0458

Weakness Localization

Slicing surfaces low-accuracy regions of input space.

ts.diagnose_slicing_accuracy(features="MedInc").table

	Feature	Segment	Size	MSE	Threshold	Weak
6	MedInc	[9.20, 10.65)	38	0.2969	0.1842	True
5	MedInc	[7.75, 9.20)	91	0.2368	0.1842	True
0	MedInc	[0.50, 1.95)	450	0.2264	0.1842	True
4	MedInc	[6.30, 7.75)	196	0.2013	0.1842	True
2	MedInc	[3.40, 4.85)	1178	0.1806	0.1842	False
9	MedInc	[13.55, 15.00]	14	0.1755	0.1842	False
1	MedInc	[1.95, 3.40)	1537	0.1734	0.1842	False
3	MedInc	[4.85, 6.30)	604	0.1699	0.1842	False
8	MedInc	[12.10, 13.55)	6	0.0978	0.1842	False
7	MedInc	[10.65, 12.10)	14	0.0872	0.1842	False