{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "\n# DirectRS Classification\n\nThis example demonstrates DirectRS post-processing on a pre-trained XGBoost\nclassifier. DirectRS works on the raw score (logit) space, providing exact\nadditive decomposition of the decision function while improving or maintaining\nclassification accuracy.\n\nWe use the TaiwanCredit dataset with a depth-2 XGBoost base model.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Setup\nImport libraries and suppress warnings.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "import warnings\nwarnings.filterwarnings(\"ignore\")\n\nimport numpy as np\nfrom modeva import DataSet, TestSuite\nfrom modeva.models import MoXGBClassifier" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Load Dataset\nLoad the TaiwanCredit dataset and create a random train/test split.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "ds = DataSet()\nds.load(name=\"TaiwanCredit\")\nds.set_random_split()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Train Base Model\nTrain an XGBoost classifier with depth 2.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "model = MoXGBClassifier(\n name=\"XGB-cls-depth2\",\n n_estimators=200, max_depth=2, learning_rate=0.1,\n random_state=42, verbosity=0\n)\nmodel.fit(ds.train_x, ds.train_y.ravel())\n\nts = TestSuite(ds, model)\nts.diagnose_accuracy_table().table" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Fit DirectRS\nPost-process the trained XGBoost classifier with DirectRS.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "from modeva.models import MoDirectRSClassifier\n\ndrs = MoDirectRSClassifier(\n base_model=model, ridge_alpha=100.0, n_passes=1\n)\ndrs.fit(ds.train_x, ds.train_y.ravel(), verbose=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Accuracy Comparison\nCompare AUC and accuracy between the base XGBoost model and DirectRS.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "from sklearn.metrics import roc_auc_score, accuracy_score\n\ny_test = ds.test_y.ravel()\nbase_proba = model.predict_proba(ds.test_x)[:, 1]\ndrs_proba = drs.predict_proba(ds.test_x)[:, 1]\nbase_acc = accuracy_score(y_test, model.predict(ds.test_x))\ndrs_acc = accuracy_score(y_test, drs.predict(ds.test_x))\n\nprint(f\"{'Metric':<10s} {'Base XGB':>10s} {'DirectRS':>10s}\")\nprint(\"-\" * 32)\nprint(f\"{'AUC':<10s} {roc_auc_score(y_test, base_proba):>10.4f} {roc_auc_score(y_test, drs_proba):>10.4f}\")\nprint(f\"{'Accuracy':<10s} {base_acc:>10.4f} {drs_acc:>10.4f}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## S' Stretch Analysis\nAnalyze the global stretch matrix S' extracted from tree geometry.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "result = drs.get_global_stretch_analysis(ds.feature_names)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Eigenvalue spectrum of S'.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "result.plot(\"eigenvalue_spectrum\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Feature activity scores.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "result.plot(\"feature_activity\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Local Explanation\nFor classification, the decomposition operates on raw scores (logits).\nWe verify using ``drs._core.predict`` which returns raw scores.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "result = drs.explain_local(ds.test_x, feature_names=ds.feature_names)\nlocal = result.value\n\nraw_pred = drs._core.predict(ds.test_x)\nrecon = local['intercept'] + local['contributions'].sum(axis=1)\nmax_err = np.max(np.abs(raw_pred - recon))\n\nprint(f\"Max |raw_score - (intercept + sum contributions)|: {max_err:.2e}\")\nprint(f\"Decomposition exact to machine precision: {max_err < 1e-10}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Local explanation waterfall plot.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "result.plot()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Global Feature Importance\nCompute global feature importance using the default ``slope`` mode.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "result = drs.importance_global(feature_names=ds.feature_names)\nresult.plot()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Main/Interaction Decomposition\nDecompose model variance into main effects and interactions.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "result = drs.importance_main_interaction(ds.test_x, feature_names=ds.feature_names)\nmi = result.value\n\nprint(f\"Orthogonalized variance split:\")\nprint(f\" eta2_main = {mi['eta2_main']:.4f} ({mi['eta2_main']*100:.1f}%)\")\nprint(f\" eta2_int = {mi['eta2_int']:.4f} ({mi['eta2_int']*100:.1f}%)\")\nprint(f\" rho(g, r) = {mi['rho']:.4f}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Main vs interaction importance bar chart.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "result.plot()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Geometric Interaction Traces\nTrace feature interactions through the adjacency matrix A.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "result = drs.geometric_interaction_traces(\n feature_names=ds.feature_names, K=4, gamma=0.5\n)\ntraces = result.value\n\nprint(\"Interaction spectrum:\")\nfor k in range(len(traces['T'])):\n print(f\" k={k+1}: T_k = {traces['T'][k]:.6f}, E_k = {traces['E'][k]:.6f}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Adjacency matrix heatmap.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "result.plot(\"adjacency\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Interaction energy spectrum.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "result.plot(\"spectrum\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Top Feature Interactions\nShow the strongest off-diagonal entries in the stretch matrix.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "result = drs.get_off_diagonal_analysis(ds.feature_names, top_k=15)\nresult.plot()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## FANOVA Comparison\nCompare DirectRS feature importance with FANOVA decomposition.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "results = ts.interpret_fi()\nresults.plot()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "results = ts.interpret_ei()\nresults.plot()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Side-by-side comparison of importance scores.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "result_drs = drs.importance_global(feature_names=ds.feature_names, mode=\"slope\")\nresult_fanova = ts.interpret_fi()\n\ndrs_imp = result_drs.value['importance']\nfanova_table = result_fanova.table\nfanova_imp = dict(zip(fanova_table[\"Name\"], fanova_table[\"Score\"]))\n\nprint(f\"{'Feature':<16s} {'DirectRS':>10s} {'FANOVA':>10s}\")\nprint(\"-\" * 38)\nfor i, feat in enumerate(ds.feature_names):\n print(f\"{feat:<16s} {drs_imp[i]:>10.4f} {fanova_imp.get(feat, 0.0):>10.4f}\")" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.11" } }, "nbformat": 4, "nbformat_minor": 0 }