{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "\n# FuseKernel Classification\n\nFor classification FuseKernel fits the fused kernel ridge regression on one-hot\ntargets and reads probabilities off a temperature-calibrated softmax. This\nexample fuses the XGBoost tree co-membership kernel with a learned multi-scale\nspectral kernel (MS-SKM) on the TaiwanCredit dataset.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Installation\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "# To install the required package, use the following command:\n# !pip install modeva"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Authentication\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "# To get authentication, use the following command: (To get full access please replace the token to your own token)\n# from modeva.utils.authenticate import authenticate\n# authenticate(auth_code='eaaa4301-b140-484c-8e93-f9f633c8bacb')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Import required modules\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "import warnings\nwarnings.filterwarnings(\"ignore\")\n\nimport numpy as np\nimport pandas as pd\nfrom modeva import DataSet, ModelZoo, TestSuite"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Load and prepare dataset\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "ds = DataSet()\nds.load(name=\"TaiwanCredit\")\nds.set_random_split()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Train FuseKernel\nFor classification FuseKernel fits the fused KRR on one-hot targets and reads\nprobabilities off a temperature-calibrated softmax. Binary and multiclass are\nboth supported, and the simplex weights are selected on the held-out query by\naccuracy.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "from modeva.models import MoFuseKernelClassifier\n\nmodel = MoFuseKernelClassifier(\n    name=\"FuseKernel-Cls\",\n    use_xgb=True, use_spectral=True,\n    spectral_params={\"H\": 4, \"K\": 8, \"kernel\": \"laplace\", \"epochs\": 50},\n    interpret_ref_size=400, interpret_n_grid=20,\n    random_state=0,\n)\n\nmz = ModelZoo(dataset=ds)\nmz.add_model(model)\nmz.train_all()\nmz.leaderboard()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Accuracy\nAUC, accuracy, F1, log-loss and Brier, train vs test.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "ts = TestSuite(ds, model)\nts.diagnose_accuracy_table().table"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Probabilities\n``predict_proba`` returns calibrated class probabilities (rows sum to 1).\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "proba = model.predict_proba(ds.test_x)\npd.DataFrame(proba[:5], columns=[f\"P(class={c})\" for c in model.classes_])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Inherent Interpretation (FANOVA)\nThe same functional-ANOVA decomposition applies to the classification scores.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "results = ts.interpret_fi()\nresults.table"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "results = ts.interpret_ei()\nresults.table"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "results = ts.interpret_effects(features=\"PAY_1\")\nresults.plot(results.get_figure_names()[0])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "results = ts.interpret_local_fi(sample_index=1)\nresults.table"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Reliability\nConformal prediction sets and coverage for the classifier.\n\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": false
      },
      "outputs": [],
      "source": [
        "ts.diagnose_reliability().table"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.11"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}