Diagnostic Suite

  • Performance and Residual Analysis

    • Introduction

    • Classification Metrics

    • Regression Metrics

    • Performance Evaluation in MoDeVa

    • Residual Analysis

    • Residual Analysis in MoDeVa

    • Performance Comparison

    • Supervised Learning for Residual Analysis

    • Examples

  • Weakness Detection

    • Introduction

    • Slicing

      • Weakness Detection in MoDeVa

      • Error Slicing for Weakness Detection

      • Weakness Comparison

      • Advanced Slicing

    • Error Decomposition (AMIF)

    • Error-Aware Clustering (Random Forest and LTC)

    • Weak-Cluster Detection and Repair (FuseKernel detection + Mixture of Experts)

    • Examples

  • Underfitting and Overfitting

    • Empirical Risk and Generalization Gap

    • Slicing Generalization Gap

    • Overfitting Slicing in MoDeVa

    • Overfit Comparison

    • Overfitting and Model Robustness

    • Remediation Strategies for Model Weaknesses Identified by Gap Analysis

    • Examples

  • Reliability

    • Conformal Prediction

    • Reliability Analysis in MoDeVa

    • Identification of Reliability Issue and Impactful Variables

    • Reliability Diagnostics in MoDeVa

    • Reliability Comparison

    • Strategies for Addressing Model Weaknesses

    • Supervised Machine Learning: Random Forest Clustering

    • Examples

  • Robustness

    • Input Perturbation for Robustness Test

    • Robustness Analysis in MoDeVa

    • Identification of Robustness Issue and Impactful Variables

    • Robustness Comparison

    • Supervised Machine Learning: Random Forest Clustering

    • Strategies for Addressing Model Weaknesses

    • Examples

  • Resilience

    • Resilience Analysis in MoDeVa

    • Measuring Distribution Drift

    • Strategies for Addressing Model Weaknesses

    • Examples

  • Fairness

    • Definitions of Group Fairness

    • Metrics for Group Fairness

    • Fairness Metrics in MoDeVa

    • Fairness Evaluation in MoDeVa

    • Fairness Comparison

    • Fairness Mitigation

    • Examples