← All topics

Bias–Variance Tradeoff with KNN Regression

The true function is f(x) = sin(2πx) with noise σ = 0.3, and the n = 100 training inputs x are fixed. Each thin curve is a K-nearest-neighbour fit to a different simulated training set. Drag the slider: small K gives jagged fits that scatter around f (variance); large K gives smooth fits that all miss the peaks in the same way (bias). The bars and the error curves show the exact decomposition, averaged over 2001 test points on [0, 1]. (Lecture notes §4.3; ISL 2.2.2, ESL 7.3.)

Keyboard and screen reader users: Tab to the K slider and use the arrow keys to change K by 1, Shift+arrow or Page Up/Page Down to change it by 10, and Home/End for K = 1 and K = 100. The current numbers are announced in the statistics line below the slider, and each chart's description also contains its numbers. The table under "Show the numbers" lists every value.

K = 10
K = 10 df = n/K = 10 bias² = – variance = – σ² = 0.090 test error = – training error = –

Twenty KNN fits from twenty training sets

Error decomposition at this K

Test error = bias² + variance + σ². Each bar is labelled with its value.

Test and training error against df = n/K

Flexibility increases to the right (log scale). The vertical line marks the current K.
Show the numbers

Exact expected values, with the training inputs held fixed. Rows from the lecture table are included; the current K is highlighted.

Bias squared, variance, test error and training error for selected K
Kdf = n/Kbias²variance σ²/Ktest errortraining error