Supervised Feature Engineering

This example demonstrates supervised feature engineering transformers that use the target variable to generate features. These include RF proximity, spectral proximity, and DirectRS stretch features.

Setup

Import libraries and suppress warnings.

import warnings
warnings.filterwarnings("ignore")

import numpy as np
from modeva import DataSet

Load Dataset

Load the CaliforniaHousing dataset.

ds = DataSet()
ds.load(name="CaliforniaHousing")
ds.set_random_split()

print(f"Features: {len(ds.feature_names)}")
print(f"Target: {ds.target_feature_name}")

Features: 8
Target: MedHouseVal

RF Proximity Features

Train a random forest and use leaf co-occurrence as a similarity kernel. The Nystrom method computes a fixed-width feature map from landmark points.

ds.fe_rf_proximity(n_estimators=50, max_depth=4, n_landmarks=20)

DirectRS Stretch Features

Build a weighted outer-product matrix from per-leaf centroids and variances, decompose via eigendecomposition, and apply the resulting stretch matrix.

ds.fe_direct_rs(n_estimators=30, max_depth=3, regularization=1e-8)

Execute All Steps

ds.engineer_features()

print(f"Total features after engineering: {len(ds.all_feature_names)}")

Total features after engineering: 58

Feature Engineering Status

fe = ds.get_feature_engineer()
status = fe.get_status()
for step in status['executed_steps']:
    print(f"  - {step['name']}")

- fe_rf_proximity
- fe_direct_rs