Weak-Cluster Detection and Repair

Overview

Even a well-performing model has regions it serves poorly. This workflow detects those regions with FuseKernel and repairs them with a Mixture of Experts:

1. Detect (FuseKernel) — FuseKernel learns a kernel over the data and clusters samples in that kernel geometry. Breaking performance down per cluster automatically surfaces the model’s weak regions, without having to guess where to look.

2. Repair (Mixture of Experts) — a residual-driven Mixture of Experts (MoMoERegressor / MoMoEClassifier) clusters samples by their learning trajectories and fits a specialised expert per cluster, with a gate routing between them. The experts can carry the same monotonicity constraints as the base model, so the repaired model stays interpretable.

The improvement is largest exactly where the base model was weakest.

Detection with FuseKernel

diagnose_weak_clusters embeds the fitted fused kernel with a Nyström spectral map, partitions it into n_clusters clusters, and reports the model metric per cluster on train and test. The result table includes an ALL aggregate row; value["worst_clusters"] ranks the weakest clusters and value["labels_test"] maps each test row to its cluster.

For fast detection the FuseKernel needs only the tree co-membership kernel at the base model’s depth — no dense RBF or learned spectral channel is required just to locate the weak clusters, so use_rbf=False and use_spectral=False with the Nyström solver keeps the fit cheap.

from modeva import DataSet
from modeva.models import MoXGBRegressor, MoFuseKernelRegressor

ds = DataSet()
ds.load(name="CaliforniaHousing")
ds.set_random_split()

# Base model whose weak regions we detect and repair
base = MoXGBRegressor(name="base", max_depth=3, n_estimators=100)
base.fit(ds.train_x, ds.train_y.ravel())

# Fast FuseKernel: tree co-membership kernel only, at the base model's depth
fk = MoFuseKernelRegressor(name="FuseKernel", use_xgb=True, use_rbf=False,
                           use_spectral=False, fit_method="grid", solver="nystrom",
                           gbdt_params={"n_estimators": 100, "max_depth": 3})
fk.fit(ds.train_x, ds.train_y.ravel())

# Detect weak clusters
diag = fk.diagnose_weak_clusters(ds, n_clusters=6)
diag.table                        # per-cluster train/test breakdown

import numpy as np
worst_id = int(diag.value["worst_clusters"].iloc[0]["cluster"])
weak_mask = np.asarray(diag.value["labels_test"]) == worst_id   # weak test rows

Repair with a Mixture of Experts

The weak region is repaired with MoMoERegressor (or MoMoEClassifier). With cluster_method="ltc" it fits a baseline model, computes residuals, clusters samples by their learning trajectories — where the base model struggles — and fits a specialised expert per cluster. For a classification model, pass monotone_constraints to keep every expert monotone (as in the monotone-credit demo).

from modeva.models import MoMoERegressor

moe = MoMoERegressor(name="MoE-repair", n_clusters=5, cluster_method="ltc",
                     expert="xgboost", max_depth=3, n_estimators=100)
moe.fit(ds.train_x, ds.train_y.ravel())

Before vs After

Compare the base model and the repaired MoE, both overall and on the FuseKernel-detected weak region. The repair should help everywhere a little, and most where the base model was weakest.

from sklearn.metrics import r2_score

Xte, yte = np.asarray(ds.test_x), np.asarray(ds.test_y).ravel()

def r2(model, mask=None):
    p = model.predict(Xte)
    if mask is None:
        mask = np.ones(len(yte), dtype=bool)
    return round(r2_score(yte[mask], p[mask]), 4)

import pandas as pd
pd.DataFrame({
    ("overall", "base"):                          {"R2": r2(base)},
    ("overall", "MoE repair"):                     {"R2": r2(moe)},
    (f"weak cluster {worst_id}", "base"):          {"R2": r2(base, weak_mask)},
    (f"weak cluster {worst_id}", "MoE repair"):    {"R2": r2(moe, weak_mask)},
}).T

Parameters

diagnose_weak_clusters (FuseKernel detection):

Parameter	Default	Description
dataset	(required)	A DataSet with train/test splits
n_clusters	5	Number of spectral clusters in the breakdown

MoMoERegressor / MoMoEClassifier (repair):

Parameter	Default	Description
n_clusters	10	Number of experts (clusters)
cluster_method	"kmeans"	"ltc" (learning-trajectory, residual-driven) or "kmeans"
expert	"xgboost"	Expert model (xgboost/lightgbm/catboost)
monotone_constraints	None	Per-feature monotonicity, aligned to features

Examples

See the runnable Weak-Cluster Detection and Repair (FuseKernel + MoE) example in the Weakness Region Detection gallery for the full workflow end to end.

Key Methods

• MoFuseKernelRegressor.diagnose_weak_clusters()

• MoMoEClassifier / MoMoERegressor

See the FuseKernel API Reference and the Models API Reference for full method documentation.