Feature Engineering Basics

This example demonstrates the basic feature engineering workflow in MoDeVa: queue transformers, execute them, and inspect the results. We use the CaliforniaHousing dataset with three unsupervised transformers.

Setup

Import libraries and suppress warnings.

import warnings
warnings.filterwarnings("ignore")

import numpy as np
from modeva import DataSet

Load Dataset

Load the CaliforniaHousing dataset and create a random train/test split.

ds = DataSet()
ds.load(name="CaliforniaHousing")
ds.set_random_split()

# Keep a copy of the raw (un-engineered) data to demonstrate transforming
# new data later; ds.to_df() reflects the engineered frame after execution.
raw_df = ds.to_df()

print(f"Features: {ds.feature_names}")
print(f"Number of features: {len(ds.feature_names)}")
print(f"Training samples: {ds.train_x.shape[0]}")

Features: ['MedInc', 'HouseAge', 'AveRooms', 'AveBedrms', 'Population', 'AveOccup', 'Latitude', 'Longitude']
Number of features: 8
Training samples: 16512

Queue Feature Engineering Steps

Feature engineering uses a deferred execution pattern: first queue the transformers, then execute them all at once.

# Bivector features: pairwise products x_i * x_j
ds.fe_bivector(max_features=20, selection="variance")

# Whitening: SVD-based whitening (GA Mahalanobis transform)
ds.fe_whitening(regularization=1e-8)

# Density: RBF kernel density estimate per sample
ds.fe_density(bandwidth="median", n_reference=200)

Execute Feature Engineering

Run all queued steps. Each step fits on the data and appends new columns.

ds.engineer_features()

print(f"Features after engineering: {len(ds.all_feature_names)}")
print(f"New feature names (first 10): {ds.all_feature_names[:10]}")

Features after engineering: 59
New feature names (first 10): ['MedInc', 'HouseAge', 'AveRooms', 'AveBedrms', 'Population', 'AveOccup', 'Latitude', 'Longitude', 'MedHouseVal', 'bv_AveBedrms*Longitude']

Inspect Results

Check the feature engineering status.

fe = ds.get_feature_engineer()
status = fe.get_status()
print(f"Executed steps: {[s['name'] for s in status['executed_steps']]}")
print(f"Pending steps: {[s['name'] for s in status['pending_steps']]}")

Executed steps: ['fe_bivector', 'fe_whitening', 'fe_density']
Pending steps: []

Transform New Data

Apply the fitted pipeline to the test set.

test_df = raw_df.iloc[:5]
transformed = ds.transform_features(test_df)
print(f"Transformed shape: {transformed.shape}")
print(f"Columns: {list(transformed.columns[:15])}...")

Transformed shape: (5, 59)
Columns: ['MedInc', 'HouseAge', 'AveRooms', 'AveBedrms', 'Population', 'AveOccup', 'Latitude', 'Longitude', 'MedHouseVal', 'bv_AveBedrms*Longitude', 'bv_MedInc*Latitude', 'bv_MedInc*HouseAge', 'bv_AveOccup*Longitude', 'bv_HouseAge*AveRooms', 'bv_AveRooms*Latitude']...

Reset

Reset feature engineering to restore original features.

ds.reset_feature_engineering()
print(f"Features after reset: {len(ds.all_feature_names)}")

Features after reset: 9