Test Suite

The TestSuite class (exposed as modeva.TestSuite) is the single entry point for post-hoc explanation, inherent interpretation, diagnostics and model comparison. Its methods are documented by area on the following pages:

• Post-hoc Explainability

• Inherent Interpretation

• Performance, Weakness Detection, Residual Analysis, Overfit, Reliability, Robustness, Resilience and Fairness diagnostics

• Model Comparison

class modeva.TestSuite(dataset=None, model=None, models=None, name='testsuite')

A comprehensive model evaluation and analysis toolkit that provides methods for explaining, diagnosing, comparing and interpreting machine-learning models. Construct it with a dataset and one or more fitted models, then call the area methods documented on the pages above.

Parameters:

• dataset – The DataSet to evaluate on.

• model – A single fitted model to test.

• models – A list of fitted models, used by the comparison tests.

• name – Name of the test-suite run, default "testsuite".

TestSuite.register(name: str, test_result: ValidationResult, register_name: str = None, description: str = None, tags: Dict[str, Any] | None = None, run_id: str = None)

Register a test into MLFlow.

Parameters:

namestr

The current name of the test to be registered.

test_resultValidationResult

The validation result object of test.

register_namestr, default=None

The register name of the test in MLFlow. If None, will be the same as name.

descriptionstr, default=None

The description of this test.

tagsdict, default=None

The tags.

run_idstr, default=None

The run id in MLFLow.