Data is often in the wrong shape, scale, or format for the intended analysis or model algorithm.
Converting data from one format, structure, or scale to another to make it suitable for analysis and modeling.
- Reshaping: pivot, melt, or cast data between wide and long formats
- Scaling: normalize (0-1) or standardize (mean=0, std=1) numeric features
- Encoding: convert categorical variables to numeric (one-hot, label encoding)
- Log/power transforms: handle skewed distributions
- Aggregation: group by dimensions and compute summary statistics
- Algorithm-dependent: some models require specific data shapes or scales
- Information-preserving: good transforms maintain relationships in data
- Reversible: many transforms can be undone (important for interpretability)
- Standardization: consistent transforms across train and test sets
- Built from: Data Cleaning — clean data is prerequisite
- Builds into: Feature Engineering — transformed features become inputs
- Related: Data Wrangling — transformation is a wrangling step
- Related: EDA — transforms often inspired by EDA findings
- Applying different transforms to train vs test sets causes data leakage
- One-hot encoding high-cardinality categoricals creates dimensionality explosion
- Standardization before train/test split causes information leakage
- Log transforms fail on zero or negative values without adjustment