Concept guide · Foundations
What is the bias-variance tradeoff?
A slower explanation for when the short interview answer is not enough. Start with the intuition, then connect it to the technical idea and the way an interviewer expects you to reason about it.
ML area
Foundations → Model complexity
Interview format
Core
Question
ML-012
Start with the idea
In simple terms
Bias is the error caused by a model being too limited. Variance is the error caused by a model reacting too strongly to the particular training sample.
Make it concrete
A straight line may miss a curved relationship because it is too simple. A very deep tree may fit every training example but change dramatically when the sample changes.
Build the answer
How to reason about it
- 1Compare training and validation behaviour to identify underfitting or overfitting.
- 2Ask whether more flexibility, more data, or stronger regularisation addresses the problem.
- 3Choose the model that performs reliably on the deployment-like evaluation, not the most complex one.
Go one level deeper
The technical explanation
The tradeoff is a way to diagnose model behaviour, not a command to make the two terms equal. More flexibility often lowers approximation error but increases sensitivity to noise. Regularisation, more data, better features, and a model family with the right inductive bias can change the balance. The useful choice depends on the deployment loss and the amount and quality of data available.
Mental model
As flexibility grows, bias often falls while variance rises; total error has a useful middle region.
Say this in the interview
Bias is error from a model that is too restrictive to represent the real relationship. Variance is error from a model that changes too much with the particular training sample and chases noise. Increasing flexibility often lowers bias and raises variance, so model selection is about finding a useful balance for the available data and the deployment loss; it is not a rule that every problem must have equal parts of both.
Keep preparing