We need to predict a continuous numeric value (price, temperature, probability) but the relationship between inputs and output is complex.
Supervised learning task where the goal is to predict a continuous numeric output variable from input features.
- Collect labeled data: features X and continuous target y
- Choose regression model: linear regression, decision trees, neural networks, etc.
- Define loss function: typically mean squared error (MSE) or mean absolute error (MAE)
- Train: find model parameters that minimize average prediction error
- Predict: for new inputs, compute continuous output value
- Continuous output: predicts values on a real-valued scale (not categories)
- Probabilistic: can model uncertainty via prediction intervals
- Interpretable: linear regression coefficients show feature importance directly
- Metric-dependent: MSE penalizes large errors more than MAE
- Built from: Supervised Learning — regression is a supervised task
- Contrasts with: Classification — predicts categories instead of numbers
- Built from: Decision Tree Structure — trees solve regression via leaf mean values
- Related: Decision Tree Flexibility — trees handle both classification and regression
- Related: Train-Test Split — regression models need held-out evaluation
- Related: Overfitting — regression models can overfit with too many features
- Heteroscedasticity: error variance changes with input values (violates OLS assumptions)
- Multicollinearity: correlated features distort coefficient estimates
- Outliers: heavily influence linear regression (consider robust regression)
- Non-linearity: linear regression fails when true relationship is curved