FuseKernel Regression

FuseKernel fuses complementary kernel channels and decodes them with kernel ridge regression. This example fuses the XGBoost tree co-membership kernel with a learned multi-scale spectral kernel (MS-SKM) on the CaliforniaHousing dataset.

Installation

# To install the required package, use the following command:
# !pip install modeva

Authentication

# To get authentication, use the following command: (To get full access please replace the token to your own token)
# from modeva.utils.authenticate import authenticate
# authenticate(auth_code='eaaa4301-b140-484c-8e93-f9f633c8bacb')

Import required modules

import warnings
warnings.filterwarnings("ignore")

import numpy as np
import pandas as pd
from modeva import DataSet, ModelZoo, TestSuite

Load and prepare dataset

ds = DataSet()
ds.load(name="CaliforniaHousing")
ds.set_random_split()

Train FuseKernel

FuseKernel fuses complementary kernel channels – here the XGBoost tree co-membership kernel and the learned multi-scale spectral kernel (MS-SKM) – and decodes them with kernel ridge regression. Every channel is fit on a support fold and the mixture weights and ridge are selected on a held-out query fold (fit_method="grid"), so the selection is leakage-free.

from modeva.models import MoFuseKernelRegressor

model = MoFuseKernelRegressor(
    name="FuseKernel",
    use_xgb=True, use_spectral=True,            # channels to fuse
    fit_method="grid",                          # leakage-free, query-scored
    spectral_params={"H": 4, "K": 8, "kernel": "laplace", "epochs": 50},
    interpret_ref_size=400, interpret_n_grid=20,
    random_state=0,
)

mz = ModelZoo(dataset=ds)
mz.add_model(model)
mz.train_all()
mz.leaderboard()

  0%|          | 0/1 [00:00<?, ?it/s]
100%|██████████| 1/1 [09:04<00:00, 544.57s/it]
100%|██████████| 1/1 [09:04<00:00, 544.57s/it]

	start_time	end_time	Duration	train MSE	test MSE	train MAE	test MAE	train R2	test R2
FuseKernel	2026-07-08 00:41:11	2026-07-08 00:48:36	445.6114	0.1342	0.1834	0.2245	0.2813	0.8997	0.8594

Accuracy

FuseKernel is a standard MoDeVa model, so the whole TestSuite applies.

ts = TestSuite(ds, model)
ts.diagnose_accuracy_table().table

	MSE	MAE	R2
train	0.1342	0.2245	0.8997
test	0.1834	0.2813	0.8594
GAP	0.0492	0.0569	-0.0404

Inherent Interpretation (FANOVA)

With a spectral channel present, FuseKernel exposes a functional-ANOVA decomposition of the fused predictor through the standard interpret_* methods.

results = ts.interpret_fi()
results.table

	Name	Score
0	MedInc	0.1123
1	HouseAge	0.0028
2	AveRooms	0.0991
3	AveBedrms	0.0007
4	Population	0.0010
5	AveOccup	0.0550
6	Latitude	0.4359
7	Longitude	0.2932

results = ts.interpret_ei()
results.table

	Name	Importance
0	MedInc	0.1490
1	HouseAge	0.0011
2	AveRooms	0.0933
3	AveBedrms	0.0003
4	Population	0.0009
5	AveOccup	0.0612
6	Latitude	0.3520
7	Longitude	0.2024
8	Longitude & Latitude	0.1066
9	Longitude & AveOccup	0.0035
10	Longitude & HouseAge	0.0005
11	Longitude & MedInc	0.0084
12	Longitude & AveBedrms	0.0004
13	Latitude & AveOccup	0.0044
14	Latitude & HouseAge	0.0028
15	Latitude & MedInc	0.0060
16	Latitude & AveBedrms	0.0007
17	AveOccup & HouseAge	0.0066

results = ts.interpret_effects(features="MedInc")
results.plot(results.get_figure_names()[0])

results = ts.interpret_effects(features="Latitude")
results.plot(results.get_figure_names()[0])

results = ts.interpret_local_fi(sample_index=1)
results.table

	Name	Importance	Values
0	MedInc	-0.2842	2.0804
1	HouseAge	0.0244	42.0000
2	AveRooms	-0.2264	4.2941
3	AveBedrms	-0.0103	1.1176
4	Population	0.0056	1206.0000
5	AveOccup	0.3829	2.0269
6	Latitude	-0.4301	37.8400
7	Longitude	0.6310	-122.2600

Channel Decomposition

The fused prediction is an exact additive sum over channels. channel_contributions returns each channel’s contribution in target units.

result = model.channel_contributions(ds.test_x)
result.table

	Channel	MeanAbsContribution
0	spectral	0.6341
1	tree	0.3350

Inherent GP Predictive Uncertainty

For regression the prediction intervals are the closed-form Gaussian-process posterior of the fused kernel, so diagnose_reliability reflects the model’s own uncertainty.

ts.diagnose_reliability().table

	Avg.Width	Avg.Coverage
0	1.2165	0.9075

mean, var = model.predict_dist(ds.test_x)
intervals = model.predict_interval(ds.test_x)      # (n, 2) at the calibrated level
pd.DataFrame({"mean": mean[:5], "std": np.sqrt(var)[:5],
              "low": intervals[:5, 0], "high": intervals[:5, 1]})

	mean	std	low	high
0	2.7104	0.4574	1.9580	3.4627
1	1.8376	0.4659	1.0713	2.6039
2	2.0367	0.4477	1.3004	2.7730
3	1.9972	0.4630	1.2356	2.7588
4	1.3327	0.4516	0.5899	2.0755

Weakness Localization

Slicing surfaces low-accuracy regions of input space.

ts.diagnose_slicing_accuracy(features="MedInc").table

	Feature	Segment	Size	MSE	Threshold	Weak
6	MedInc	[9.20, 10.65)	38	0.2995	0.1834	True
5	MedInc	[7.75, 9.20)	91	0.2384	0.1834	True
0	MedInc	[0.50, 1.95)	450	0.2287	0.1834	True
4	MedInc	[6.30, 7.75)	196	0.1970	0.1834	True
2	MedInc	[3.40, 4.85)	1178	0.1810	0.1834	False
1	MedInc	[1.95, 3.40)	1537	0.1726	0.1834	False
3	MedInc	[4.85, 6.30)	604	0.1654	0.1834	False
9	MedInc	[13.55, 15.00]	14	0.1535	0.1834	False
8	MedInc	[12.10, 13.55)	6	0.0991	0.1834	False
7	MedInc	[10.65, 12.10)	14	0.0919	0.1834	False