A Very Detailed Bias-Variance Breakdown

Although the concept of bias-variance trade-off is often discussed in machine learning textbooks, e.g., Bishop (2006), Hastie, Tibshirani, and Friedman (2009), James et al. (2013), I also find it important in almost any occasions in which we need to fit a statistical model on a data set with a limited number of observations.