Model Zoo

The ModelZoo class (exposed as modeva.ModelZoo) registers, trains and manages models and produces leaderboards.

class modeva.models.local_model_zoo.LocalModelZoo(dataset, models: Dict = None, name: str = None, random_state: int = 0, experiment_name: str = None)

A class for managing multiple models, their training, and evaluation.

LocalModelZoo provides functionality to: - Add and manage multiple models - Train models on a given dataset - Track model performance metrics - Register models with MLflow for experiment tracking - Compare model performance through a leaderboard

Parameters:

datasetDataset

The dataset object to be used for model training and evaluation.

modelsDict, default=None

Dictionary of models to initialize with. If None, starts with empty dict.

namestr, default=None

Name for the model zoo. If None, a UUID-based name will be generated.

random_stateint, default=0

Random seed for reproducibility.

experiment_namestr, default=None

Name for the MLflow experiment. If None, will use name parameter or generate UUID-based name.

Examples

add_model(model, name: str = None, replace: bool = False)

Add a new model together with its name to models dictionary.

Parameters:

namestr

The name of model.

modelobject

The model object.

replacebool

Whether to replace old model when new model with same name

delete_registered_model(name: str, dataset: str)

Delete registered model.

Parameters:

namestr

The model name to delete

datasetstr

The dataset name to trained model

get_model(name: str)

Return a model object.

Parameters:

namestr

The name of model to be extracted.

leaderboard(order_by: str = None, ascending=False)

Show the leaderboard of all models.

Parameters:

order_bystr, default=None

The leaderboard will be ordered by this metric. If None, will show the results by the order of model training.

ascendingbool, default=False

The ordering parameter used when order_by is not None.

list_model_names()

Return the list of model names.

list_registered_models(name: str = None, format: str = 'frame', flat: bool = False)

Return the list all registered models.

Parameters:

namestr

The name of model used for filtering.

formatstr, default=”frame”

The format of displayed model list.

flatbool, default=False

Whether to flatting the results.

load_registered_model(name: str, version: int = None)

Return the list all registered models.

Parameters:

namestr

The name of model used for filtering.

versionint, default=None

Model version.

register(name: str, register_name: str = None, description: str = None, tags: Dict[str, Any] | None = {'dataset': 'SimuCredit'}, run_id: str = None)

Register a model into MLFlow.

Parameters:

namestr

The current name of the model to be registered.

register_namestr, default=None

The register name of the model in MLFlow. If None, will be the same as name.

descriptionstr, default=None

The description of this model.

tagsdict, default=None

The tags.

run_idstr, default=None

The run id in MLFLow.

train(name: str)

Train a model object.

Parameters:

namestr

The name of model to be trained.

train_all(silent: bool = False)

Train all models.

Parameters:

silentbool, default=False

If False, will show the progress bar during model training.

property dataset

Return the dataset object.

property models

Return the model dictionary.