Feature Engineering

Overview

MoDeVa’s Feature Engineering module provides 13 geometric algebra-based transformers that generate new features from existing numerical data. Unlike traditional preprocessing (which replaces columns), feature engineering appends new columns to the dataset, enriching the feature space for downstream modeling.

The module is built on GAML (Geometric Algebra Machine Learning), which is bundled inside MoDeVa for self-contained distribution. All transformers follow the scikit-learn fit/transform API and are wrapped with MoDeVa’s deferred execution queue pattern.

Categories

The 13 transformers are organized into six categories:

Bivector Features (pairwise interactions)

• fe_bivector() — All-pairs products \(x_i \cdot x_j\) with variance-based selection

• fe_wedge_bivector() — Antisymmetric wedge products \(x_i x_j - x_j x_i\) (GA exterior product)

Rotation & Shear (coordinate transforms)

• fe_whitening() — SVD-based whitening (GA Mahalanobis transform)

• fe_shear() — Pairwise shear features \(x_i \cdot x_j / \|x_j\|\)

Shape Features (local geometry)

• fe_grade_ratio() — Ratio of bivector norm to vector norm for feature triplets

• fe_multi_grade_shape() — Multi-grade shape descriptors across configurable grades

Row Geometry (per-sample structure)

• fe_density() — RBF kernel density estimate per sample

• fe_blade_spectrum() — Local blade energy spectrum from k-NN neighborhoods

Tree-Based Features (supervised, requires target)

• fe_rf_proximity() — Random forest leaf co-occurrence proximity via Nystrom approximation

• fe_spectral_proximity() — Spectral embedding from RF proximity kernel

• fe_direct_rs() — DirectRS stretch features from random forest leaf geometry

Encoding & Temporal

• fe_phase_encoder() — Smooth threshold-proximity encoding via phase differences

• fe_kinematics() — Velocity, acceleration, and kinematic summaries from temporal sequences

Implementation in MoDeVa

Feature engineering follows the same deferred execution pattern as preprocessing: queue steps, then execute all at once.

from modeva import DataSet

# Load data
ds = DataSet()
ds.load(name="CaliforniaHousing")
ds.set_random_split()

# Queue feature engineering steps
ds.fe_bivector(max_features=50, selection="variance")
ds.fe_whitening(regularization=1e-8)
ds.fe_density(bandwidth="median", n_reference=200)

# Execute all queued steps
ds.engineer_features()

# Check results
print(f"Original features: 8")
print(f"After engineering: {len(ds.all_feature_names)}")

Supervised Transformers

Some transformers require a target variable. These automatically access the dataset’s target column:

ds = DataSet()
ds.load(name="CaliforniaHousing")
ds.set_random_split()

# Tree-based features (supervised — uses target automatically)
ds.fe_rf_proximity(n_estimators=100, max_depth=6, n_landmarks=50)
ds.fe_direct_rs(n_estimators=50, max_depth=4)

ds.engineer_features()

Transform on New Data

After fitting, apply the same feature engineering pipeline to new data:

# Apply fitted pipeline to test data
new_data = ds.transform_features(test_df)

Pipeline Management

# Get the feature engineering aggregator
fe = ds.get_feature_engineer()

# Check status
status = fe.get_status()
print(f"Executed steps: {status['executed']}")
print(f"Pending steps:  {status['pending']}")

# Reset (restore original features)
ds.reset_feature_engineering()

Parameters

Common Parameters

All transformers share these parameters:

Parameter	Default	Description
features	None	Features to use. If None, all numerical features are used.
dataset	"main"	Which dataset split to fit on ("main", "train", "test").

Bivector Parameters

Parameter	Default	Description
max_features	300	Maximum number of bivector features to retain.
selection	"variance"	Selection criterion: "variance" or "abs_mean".

Rotation & Shear Parameters

Parameter	Default	Description
regularization	1e-8	Regularization for whitening (prevents division by zero).
threshold	0.01	Minimum shear magnitude to keep a feature.

Tree-Based Parameters

Parameter	Default	Description
n_estimators	50-100	Number of trees in the forest.
max_depth	4-6	Maximum tree depth.
n_landmarks	50	Number of Nystrom anchor points (RF proximity).
n_components	10	Number of spectral embedding components.

Key Classes

Wrapper classes (user-facing):

• FEBivector

• FEWedgeBivector

• FEWhitening

• FEShear

• FEGradeRatio

• FEMultiGradeShape

• FEDensity

• FEBladeSpectrum

• FERFProximity

• FESpectralProximity

• FEDirectRS

• FEPhaseEncoder

• FEKinematics

Aggregator:

• FeatureEngineering

See the API Reference for full method documentation.