The Bias-Variance Tradeoff

One of the most fundamental concepts in ML is understanding why models fail to generalize.

Decomposing Error

Total prediction error = Bias² + Variance + Irreducible Noise

flowchart LR
    Model --> Underfitting
    Model --> Overfitting
    Underfitting --> HighBias["High Bias / Low Variance"]
    Overfitting --> HighVar["Low Bias / High Variance"]
    HighBias --> Fix1[More complex model]
    HighVar --> Fix2[Regularization / More data]

Practical Tips

1. Start simple — linear models first
2. Diagnose with learning curves
3. Use cross-validation to detect overfitting
4. Regularization (L1/L2) reduces variance

from sklearn.linear_model import Ridge
from sklearn.model_selection import cross_val_score

model = Ridge(alpha=1.0)
scores = cross_val_score(model, X, y, cv=5, scoring='neg_mean_squared_error')
print(f'CV Score: {scores.mean():.3f} ± {scores.std():.3f}')