Utilities
Utility functions.
- modeva.testsuite.utils.slicing_utils.get_data_info(res_value)[source]
Extract data information from the result values.
It is designed for extracting the “good” / “bad” samples of slicing-based tests, and the results can be further used for testing data distribution drift.
- Parameters:
- res_valuelist of dict
List containing result values with feature and sample information.
- Returns:
- dict
A dictionary containing data information for each feature. The structure is as follows:
{ 'feature_name': { 'dataset1': str, # Name of the dataset 'dataset2': str, # Name of the dataset 'sample_idx1': list, # List of sample IDs for "good" samples 'sample_idx2': list, # List of sample IDs for "bad" samples 'name1': str, # Label for "good" samples 'name2': str # Label for "bad" samples } }
Examples
results = ts.diagnose_slicing_robustness(features="PAY_1", perturb_features=("PAY_1", "EDUCATION",), noise_levels=0.1, metric="AUC", method="auto-xgb1", threshold=0.7) data_info = get_data_info(res_value=results.value)["PAY_1"]
- modeva.utils.mlflow.set_mlflow_home(mlflow_home: str = None)[source]
Set the environment variable MLFLOW_HOME, which is the path of the mlflow home directory.
- Parameters:
- mlflow_homestr, default=None
The path to mlflow directory. If None, the default path is ~/modeva_mlflow.
- modeva.utils.mlflow.get_mlflow_home()[source]
Return the path of the mlflow home directory.
The directory is set to a folder named ‘modeva_mlflow’ in the user home folder by default.
Alternatively, it can be set by the ‘MLFLOW_HOME’ environment variable or programmatically by giving an explicit folder path. The ‘~’ symbol is expanded to the user home folder.
If the folder does not already exist, it is automatically created.
- Returns:
- mlflow_home: str
The path to mlflow directory.