Linear Tree Classification

Installation

# To install the required package, use the following command:
# !pip install modeva

Authentication

# To get authentication, use the following command: (To get full access please replace the token to your own token)
# from modeva.utils.authenticate import authenticate
# authenticate(auth_code='eaaa4301-b140-484c-8e93-f9f633c8bacb')

Import required modules

from modeva import DataSet
from modeva import TestSuite
from modeva.models import MoLGBMClassifier, MoGLMTreeBoostClassifier, MoNeuralTreeClassifier

Load and prepare dataset

ds = DataSet()
ds.load(name="TaiwanCredit")
ds.set_random_split()
ds.set_target("FlagDefault")

LGBM Linear Tree model

model = MoLGBMClassifier(linear_trees=True, max_depth=2, verbose=-1, random_state=0)
model.fit(ds.train_x, ds.train_y.ravel())

MoLGBMClassifier(boosting_type='gbdt', class_weight=None, colsample_bytree=1.0,
                 importance_type='split', learning_rate=0.1, linear_trees=True,
                 max_depth=2, min_child_samples=20, min_child_weight=0.001,
                 min_split_gain=0.0, n_estimators=100, n_jobs=None,
                 num_leaves=31, objective=None, random_state=0, reg_alpha=0.0,
                 reg_lambda=0.0, subsample=1.0, subsample_for_bin=200000,
                 subsample_freq=0, verbose=-1)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Basic accuracy analysis

ts = TestSuite(ds, model)
results = ts.diagnose_accuracy_table()
results.table

	AUC	ACC	F1	LogLoss	Precision	Recall	Brier
train	0.7925	0.8217	0.4775	0.4232	0.6856	0.3664	0.1326
test	0.7840	0.8298	0.4846	0.4198	0.7018	0.3701	0.1307
GAP	-0.0086	0.0082	0.0071	-0.0034	0.0162	0.0037	-0.0019

Feature importance analysis

results = ts.interpret_fi()
results.plot()

Local feature importance analysis

results = ts.interpret_local_fi(sample_index=1, centered=True)
results.plot()

Boosted GLMTree model

model = MoGLMTreeBoostClassifier(max_depth=1, n_estimators=100,
                                 reg_lambda=0.001, verbose=True, random_state=0)
model.fit(ds.train_x, ds.train_y.ravel())

Iteration 1 with validation loss 0.46286
Iteration 2 with validation loss 0.45391
Iteration 3 with validation loss 0.45545
Iteration 4 with validation loss 0.45813
Iteration 5 with validation loss 0.45458
Iteration 6 with validation loss 0.45238
Iteration 7 with validation loss 0.45290
Iteration 8 with validation loss 0.45338
Iteration 9 with validation loss 0.45465
Iteration 10 with validation loss 0.45425
Iteration 11 with validation loss 0.45525
Iteration 12 with validation loss 0.45535
Early stop as validation loss does not decrease for certain iterations.

MoGLMTreeBoostClassifier(name='MoGLMTreeBoostClassifier', reg_lambda=0.001,
                         verbose=True)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Basic accuracy analysis

ts = TestSuite(ds, model)
results = ts.diagnose_accuracy_table()
results.table

	AUC	ACC	F1	LogLoss	Precision	Recall	Brier
train	0.7779	0.8199	0.4858	0.4384	0.6658	0.3825	1.3578e-01
test	0.7695	0.8207	0.4694	0.4394	0.6512	0.3670	1.3582e-01
GAP	-0.0085	0.0008	-0.0164	0.0010	-0.0146	-0.0155	4.0008e-05

Main effect plot for numerical feature

results = ts.interpret_effects(features="PAY_1")
results.plot()

Main effect plot for categorical feature

results = ts.interpret_effects(features="EDUCATION")
results.plot()

Neural Tree model with Monotonicity Constraints

modelnn = MoNeuralTreeClassifier(estimator=model,
                                 nn_temperature=0.0001,
                                 nn_max_epochs=20,
                                 feature_names=ds.feature_names,
                                 mono_increasing_list=("PAY_1",),
                                 mono_sample_size=1000,
                                 reg_mono=10,
                                 verbose=True,
                                 random_state=0)
modelnn.fit(ds.train_x, ds.train_y.ravel())

#### #### MoNeuralTree Training Stage 1: Use Fitted MoGLMTreeBoost ####
#### MoNeuralTree Training Stage 2: Fine-tuning via Gradient Descent ####
Initial training and validation loss: 0.4342 and 0.4552
Epoch 0: Train loss 14.5726, Validation loss 9.1768, Monotonicity loss 0.0592
Epoch 1: Train loss 5.2107, Validation loss 1.3408, Monotonicity loss 0.0150
Epoch 2: Train loss 4.4272, Validation loss 3.1641, Monotonicity loss 0.0101
Epoch 3: Train loss 6.2526, Validation loss 8.4962, Monotonicity loss 0.0033
Epoch 4: Train loss 4.3140, Validation loss 3.2423, Monotonicity loss 0.0030
Epoch 5: Train loss 3.8045, Validation loss 10.1239, Monotonicity loss 0.0026
Epoch 6: Train loss 4.7665, Validation loss 5.4289, Monotonicity loss 0.0022
Epoch 7: Train loss 5.1385, Validation loss 5.1316, Monotonicity loss 0.0044
Epoch 8: Train loss 4.1558, Validation loss 10.4380, Monotonicity loss 0.0013
Epoch 9: Train loss 4.8965, Validation loss 2.1641, Monotonicity loss 0.0008
Epoch 10: Train loss 5.3007, Validation loss 8.5860, Monotonicity loss 0.0002
Epoch 11: Train loss 4.3431, Validation loss 3.5167, Monotonicity loss 0.0005
Epoch 12: Train loss 3.9529, Validation loss 5.5719, Monotonicity loss 0.0000
Training is terminated as validation loss stops decreasing.

MoNeuralTreeClassifier(clip_predict=False, device='cpu',
                       estimator=MoGLMTreeBoostClassifier(name='MoGLMTreeBoostClassifier',
                                                          reg_lambda=0.001,
                                                          verbose=True),
                       learning_rate=1.0, max_depth=1, min_impurity_decrease=0,
                       min_samples_leaf=50, n_epoch_no_change=5,
                       n_estimators=100, n_feature_search=5, n_screen_grid=1,
                       n_split_grid=20, name='MoGLMTreeBoostClassifier',
                       nn_max_epochs=20, reg_lambda=0.001, simplified=True,
                       split_custom=None, verbose=True)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Basic accuracy analysis

ts = TestSuite(ds, modelnn)
results = ts.diagnose_accuracy_table()
results.table

	AUC	ACC	F1	LogLoss	Precision	Recall	Brier
train	0.7067	0.7965	0.4624	4.2875	0.5607	0.3933	0.1979
test	0.7119	0.8007	0.4549	4.0884	0.5563	0.3847	0.1935
GAP	0.0052	0.0042	-0.0075	-0.1990	-0.0044	-0.0086	-0.0044

Feature importance analysis

results = ts.interpret_fi()
results.plot()

Main effect plot

results = ts.interpret_effects(features="PAY_1")
results.plot()