Diagnostic Suite

• Performance and Residual Analysis
  • Introduction
  • Classification Metrics
  • Regression Metrics
  • Performance Evaluation in MoDeVa
  • Residual Analysis
  • Residual Analysis in MoDeVa
  • Performance Comparison
  • Supervised Learning for Residual Analysis
  • Examples
• Weakness Detection
  • Introduction
  • Weakness Detection in MoDeVa
  • Error Slicing for Weakness Detection
  • Weakness Comparison
  • Advanced Slicing
  • Examples
• AMIF Weakness Region Diagnostics
  • Overview
  • Method
  • Implementation in MoDeVa
  • Output Interpretation
  • Parameters
  • Key Method
• Error-Aware Clustering
  • Overview
  • Method
  • Implementation in MoDeVa
  • Output Interpretation
  • Parameters
  • Key Method
• Weak-Cluster Detection and Repair
  • Overview
  • Detection with FuseKernel
  • Repair with a Mixture of Experts
  • Before vs After
  • Parameters
  • Examples
  • Key Methods
• Underfitting and Overfitting
  • Empirical Risk and Generalization Gap
  • Slicing Generalization Gap
  • Overfitting Slicing in MoDeVa
  • Overfit Comparison
  • Overfitting and Model Robustness
  • Remediation Strategies for Model Weaknesses Identified by Gap Analysis
  • Examples
• Reliability
  • Conformal Prediction
  • Reliability Analysis in MoDeVa
  • Identification of Reliability Issue and Impactful Variables
  • Reliability Diagnostics in MoDeVa
  • Reliability Comparison
  • Strategies for Addressing Model Weaknesses
  • Supervised Machine Learning: Random Forest Clustering
  • Examples
• Robustness
  • Input Perturbation for Robustness Test
  • Robustness Analysis in MoDeVa
  • Identification of Robustness Issue and Impactful Variables
  • Robustness Comparison
  • Supervised Machine Learning: Random Forest Clustering
  • Strategies for Addressing Model Weaknesses
  • Examples
• Resilience
  • Resilience Analysis in MoDeVa
  • Measuring Distribution Drift
  • Strategies for Addressing Model Weaknesses
  • Examples
• Fairness
  • Definitions of Group Fairness
  • Metrics for Group Fairness
  • Fairness Metrics in MoDeVa
  • Fairness Evaluation in MoDeVa
  • Fairness Comparison
  • Fairness Mitigation
  • Examples