Model Zoo

The ModelZoo class (exposed as modeva.ModelZoo) registers, trains and manages models and produces leaderboards.

LocalModelZoo

A class for managing multiple models, their training, and evaluation.

LocalModelZoo provides functionality to: - Add and manage multiple models - Train models on a given dataset - Track model performance metrics - Register models with MLflow for experiment tracking - Compare model performance through a leaderboard

Parameters

dataset : Dataset
The dataset object to be used for model training and evaluation.

models : Dict, default=None
Dictionary of models to initialize with. If None, starts with empty dict.

name : str, default=None
Name for the model zoo. If None, a UUID-based name will be generated.

random_state : int, default=0
Random seed for reproducibility.

experiment_name : str, default=None
Name for the MLflow experiment. If None, will use name parameter or generate UUID-based name.

Examples

__init__(dataset, models: Dict=None, name: str=None, random_state: int=0, experiment_name: str=None)

dataset()

Return the dataset object.

models()

Return the model dictionary.

add_model(model, name: str=None, replace: bool=False)

Add a new model together with its name to models dictionary.

Parameters

name : str
The name of model.

model : object
The model object.

replace : bool
Whether to replace old model when new model with same name

list_model_names()

Return the list of model names.

get_model(name: str)

Return a model object.

Parameters

name : str
The name of model to be extracted.

train(name: str)

Train a model object.

Parameters

name : str
The name of model to be trained.

train_all(silent: bool=False)

Train all models.

Parameters

silent : bool, default=False
If False, will show the progress bar during model training.

leaderboard(order_by: str=None, ascending=False)

Show the leaderboard of all models.

Parameters

order_by : str, default=None
The leaderboard will be ordered by this metric. If None, will show the results by the order of model training.

ascending : bool, default=False
The ordering parameter used when order_by is not None.

register(name: str, register_name: str=None, description: str=None, tags: Optional[Dict[str, Any]]={}, run_id: str=None)

Register a model into MLFlow.

Parameters

name : str
The current name of the model to be registered.

register_name : str, default=None
The register name of the model in MLFlow. If None, will be the same as name.

description : str, default=None
The description of this model.

tags : dict, default=None
The tags.

run_id : str, default=None
The run id in MLFLow.

load_registered_model(name: str, version: int=None)

Return the list all registered models.

Parameters

name : str
The name of model used for filtering.

version : int, default=None
Model version.

list_registered_models(name: str=None, format: str='frame', flat: bool=False)

Return the list all registered models.

Parameters

name : str
The name of model used for filtering.

format : str, default=“frame”
The format of displayed model list.

flat : bool, default=False
Whether to flatting the results.

delete_registered_model(name: str, dataset: str)

Delete registered model.

Parameters

name : str
The model name to delete

dataset : str
The dataset name to trained model