1.3 Model Training Dynamics: Underfitting vs. Overfitting

🎯 Key AWS AI Exam Takeaways

• Underfitting (High Bias) occurs when a model is too simplistic to capture underlying trends (high error on both train and val sets).
• Overfitting (High Variance) occurs when a model memorizes training noise and fails to generalize to validation data.
• Mitigate overfitting using L1/L2 regularization, dropout, data augmentation, and early stopping.

During model training, balancing capacity against generalization is governed by the bias-variance tradeoff. Underfitting (high bias) happens when the model architecture lacks the mathematical complexity to capture real relationships, resulting in poor accuracy across both training and validation sets.

Conversely, overfitting (high variance) arises when an overly expressive model memorizes individual training examples, including random statistical noise. In an overfitted model, training accuracy appears near-perfect while validation accuracy collapses on unseen samples.

To prevent overfitting, engineers apply regularization techniques that penalize excessive parameter weights (L1 Lasso or L2 Ridge regularization), introduce dropout layers in neural networks to prevent co-adaptation, expand dataset diversity via data augmentation, and enforce early stopping.

⚠️ Common Pearson VUE Exam Traps

• Adding layers or training longer fixes underfitting, but severely worsens overfitting.
• Never evaluate models solely on training data; always reserve isolated validation and test splits.