MoReLUDNN Classification

Installation

# To install the required package, use the following command:
# !pip install modeva

Authentication

# To get authentication, use the following command: (To get full access please replace the token to your own token)
# from modeva.utils.authenticate import authenticate
# authenticate(auth_code='eaaa4301-b140-484c-8e93-f9f633c8bacb')

Import required modules

from modeva import DataSet
from modeva import TestSuite
from modeva.models import MoReLUDNNClassifier

Load and prepare dataset

ds = DataSet()
ds.load(name="TaiwanCredit")
ds.set_random_split()
ds.set_target("FlagDefault")

ds.scale_numerical(method="minmax")
ds.preprocess()

Train model

model = MoReLUDNNClassifier(max_epochs=100, verbose=True)
model.fit(ds.train_x, ds.train_y)

#### MoReLUDNN Training ####
Epoch 0: Train loss 0.5701, Validation loss 0.5288
Epoch 1: Train loss 0.5169, Validation loss 0.5138
Epoch 2: Train loss 0.4999, Validation loss 0.4939
Epoch 3: Train loss 0.4784, Validation loss 0.4739
Epoch 4: Train loss 0.4648, Validation loss 0.4654
Epoch 5: Train loss 0.4582, Validation loss 0.4637
Epoch 6: Train loss 0.4554, Validation loss 0.4597
Epoch 7: Train loss 0.4540, Validation loss 0.4594
Epoch 8: Train loss 0.4531, Validation loss 0.4567
Epoch 9: Train loss 0.4512, Validation loss 0.4553
Epoch 10: Train loss 0.4497, Validation loss 0.4542
Epoch 11: Train loss 0.4486, Validation loss 0.4530
Epoch 12: Train loss 0.4475, Validation loss 0.4526
Epoch 13: Train loss 0.4470, Validation loss 0.4523
Epoch 14: Train loss 0.4458, Validation loss 0.4521
Epoch 15: Train loss 0.4458, Validation loss 0.4502
Epoch 16: Train loss 0.4464, Validation loss 0.4506
Epoch 17: Train loss 0.4440, Validation loss 0.4492
Epoch 18: Train loss 0.4435, Validation loss 0.4485
Epoch 19: Train loss 0.4430, Validation loss 0.4489
Epoch 20: Train loss 0.4424, Validation loss 0.4479
Epoch 21: Train loss 0.4418, Validation loss 0.4488
Epoch 22: Train loss 0.4427, Validation loss 0.4493
Epoch 23: Train loss 0.4416, Validation loss 0.4466
Epoch 24: Train loss 0.4406, Validation loss 0.4462
Epoch 25: Train loss 0.4402, Validation loss 0.4475
Epoch 26: Train loss 0.4400, Validation loss 0.4462
Epoch 27: Train loss 0.4394, Validation loss 0.4451
Epoch 28: Train loss 0.4390, Validation loss 0.4455
Epoch 29: Train loss 0.4388, Validation loss 0.4441
Epoch 30: Train loss 0.4396, Validation loss 0.4451
Epoch 31: Train loss 0.4375, Validation loss 0.4439
Epoch 32: Train loss 0.4378, Validation loss 0.4451
Epoch 33: Train loss 0.4381, Validation loss 0.4475
Epoch 34: Train loss 0.4382, Validation loss 0.4447
Epoch 35: Train loss 0.4363, Validation loss 0.4431
Epoch 36: Train loss 0.4363, Validation loss 0.4432
Epoch 37: Train loss 0.4368, Validation loss 0.4469
Epoch 38: Train loss 0.4371, Validation loss 0.4431
Epoch 39: Train loss 0.4354, Validation loss 0.4426
Epoch 40: Train loss 0.4351, Validation loss 0.4416
Epoch 41: Train loss 0.4350, Validation loss 0.4415
Epoch 42: Train loss 0.4349, Validation loss 0.4414
Epoch 43: Train loss 0.4343, Validation loss 0.4415
Epoch 44: Train loss 0.4343, Validation loss 0.4425
Epoch 45: Train loss 0.4341, Validation loss 0.4414
Epoch 46: Train loss 0.4336, Validation loss 0.4411
Epoch 47: Train loss 0.4336, Validation loss 0.4410
Epoch 48: Train loss 0.4329, Validation loss 0.4407
Epoch 49: Train loss 0.4330, Validation loss 0.4405
Epoch 50: Train loss 0.4334, Validation loss 0.4416
Epoch 51: Train loss 0.4335, Validation loss 0.4415
Epoch 52: Train loss 0.4338, Validation loss 0.4404
Epoch 53: Train loss 0.4329, Validation loss 0.4410
Epoch 54: Train loss 0.4318, Validation loss 0.4410
Epoch 55: Train loss 0.4320, Validation loss 0.4411
Epoch 56: Train loss 0.4323, Validation loss 0.4471
Epoch 57: Train loss 0.4329, Validation loss 0.4400
Epoch 58: Train loss 0.4316, Validation loss 0.4407
Epoch 59: Train loss 0.4316, Validation loss 0.4402
Epoch 60: Train loss 0.4310, Validation loss 0.4397
Epoch 61: Train loss 0.4306, Validation loss 0.4410
Epoch 62: Train loss 0.4317, Validation loss 0.4401
Epoch 63: Train loss 0.4306, Validation loss 0.4396
Epoch 64: Train loss 0.4307, Validation loss 0.4407
Epoch 65: Train loss 0.4306, Validation loss 0.4412
Epoch 66: Train loss 0.4305, Validation loss 0.4400
Epoch 67: Train loss 0.4311, Validation loss 0.4400
Epoch 68: Train loss 0.4296, Validation loss 0.4397
Epoch 69: Train loss 0.4306, Validation loss 0.4410
Epoch 70: Train loss 0.4301, Validation loss 0.4404
Epoch 71: Train loss 0.4297, Validation loss 0.4432
Epoch 72: Train loss 0.4301, Validation loss 0.4398
Epoch 73: Train loss 0.4297, Validation loss 0.4399
Epoch 74: Train loss 0.4296, Validation loss 0.4419
Epoch 75: Train loss 0.4292, Validation loss 0.4401
Epoch 76: Train loss 0.4297, Validation loss 0.4413
Epoch 77: Train loss 0.4294, Validation loss 0.4399
Epoch 78: Train loss 0.4288, Validation loss 0.4412
Epoch 79: Train loss 0.4295, Validation loss 0.4393
Epoch 80: Train loss 0.4290, Validation loss 0.4405
Epoch 81: Train loss 0.4283, Validation loss 0.4428
Epoch 82: Train loss 0.4281, Validation loss 0.4394
Epoch 83: Train loss 0.4288, Validation loss 0.4394
Epoch 84: Train loss 0.4286, Validation loss 0.4416
Epoch 85: Train loss 0.4286, Validation loss 0.4402
Epoch 86: Train loss 0.4282, Validation loss 0.4393
Epoch 87: Train loss 0.4281, Validation loss 0.4401
Epoch 88: Train loss 0.4281, Validation loss 0.4394
Epoch 89: Train loss 0.4279, Validation loss 0.4400
Epoch 90: Train loss 0.4281, Validation loss 0.4403
Epoch 91: Train loss 0.4277, Validation loss 0.4390
Epoch 92: Train loss 0.4278, Validation loss 0.4460
Epoch 93: Train loss 0.4289, Validation loss 0.4394
Epoch 94: Train loss 0.4286, Validation loss 0.4393
Epoch 95: Train loss 0.4285, Validation loss 0.4406
Epoch 96: Train loss 0.4283, Validation loss 0.4402
Epoch 97: Train loss 0.4276, Validation loss 0.4435
Epoch 98: Train loss 0.4278, Validation loss 0.4395
Epoch 99: Train loss 0.4279, Validation loss 0.4393
Training is terminated as max_epoch is reached.

MoReLUDNNClassifier(device='cpu', max_epochs=100, name='MoReLUDNNClassifier',
                    verbose=True)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Basic accuracy analysis

ts = TestSuite(ds, model)
results = ts.diagnose_accuracy_table()
results.table

# Global feature importance
# ----------------------------------------------------------
results = ts.interpret_fi()
results.plot()

LLM summary table

results = ts.interpret_llm_summary(dataset="train")
results.table

	Count	Response Mean	Response Std	Local AUC	Global AUC
0	696.0	0.0977	0.2971	0.6379	0.6056
1	599.0	0.0484	0.2148	0.5224	0.6221
2	372.0	0.0780	0.2685	0.6400	0.5864
3	324.0	0.1235	0.3295	0.6694	0.5900
4	312.0	0.1154	0.3200	0.6101	0.5716
...	...	...	...	...	...
6489	1.0	0.0000	NaN	NaN	0.7180
6490	1.0	0.0000	NaN	NaN	0.7374
6491	1.0	1.0000	NaN	NaN	0.6641
6492	1.0	1.0000	NaN	NaN	0.7240
6493	1.0	1.0000	NaN	NaN	0.7319

6494 rows × 5 columns

LLM parallel coordinate plot

results = ts.interpret_llm_pc(dataset="train")
results.plot()

LLM profile plot against a feature

results = ts.interpret_llm_profile(feature="PAY_1", dataset="train")
results.plot()

Local feature importance analysis

results = ts.interpret_local_linear_fi(dataset="train", sample_index=15, centered=True)
results.plot()

Extract the last hidden layer outputs

model.predict_last_hidden_layer(ds.train_x)

array([[0.        , 0.        , 0.75400305, ..., 0.00544053, 0.5844969 ,
        0.38274243],
       [0.        , 0.        , 0.53666425, ..., 0.        , 0.        ,
        0.1593619 ],
       [0.        , 0.        , 0.71073663, ..., 0.        , 0.0747483 ,
        0.36387306],
       ...,
       [0.        , 0.        , 0.8394985 , ..., 0.2557177 , 0.75262344,
        0.46638003],
       [0.        , 0.        , 0.59678817, ..., 0.        , 0.        ,
        0.22723463],
       [0.        , 0.        , 0.64763045, ..., 0.        , 0.08511826,
        0.30503824]], dtype=float32)