MACHINE LEARNING RESEARCH
ANNOTATED BIBLIOGRAPHY
Learning theory, classifiers, ensembles, boosting, and deep learning
2
RANDOM FORESTS
Breiman, L. (2001). Random forests. Machine Learning, 45, 5-32.
https://doi.org/10.1023/A:1010933404324
ANNOTATION
Breiman develops random forests as an ensemble of tree predictors built with randomized subsets
and feature selection. The article explains how the strength and correlation of individual trees affect
generalization error and shows how the method can be used for both classification and regression. It
is useful for machine-learning research because it connects ensemble construction with measurable
predictive behavior and includes internal estimates of error and variable importance. As a primary
research article, it gives a precise technical foundation for later work that uses tree ensembles in
applied prediction problems.
GRADIENT BOOSTING
Friedman, J. H. (2001). Greedy function approximation: A gradient boosting machine. The Annals of
Statistics, 29(5), 1189-1232. https://doi.org/10.1214/aos/1013203451
ANNOTATION
Friedman develops a general gradient-descent framework for boosting additive models by viewing
function estimation as numerical optimization in function space. He presents versions for several
loss functions and develops specific procedures using regression trees for classification and
regression. The article is valuable because it explains the mathematical idea behind gradient
boosting rather than presenting it only as a software implementation. It is particularly useful for
research that compares ensemble methods, studies loss functions, or needs to explain why
sequentially adding weak learners can produce a strong predictive model.
DEEP LEARNING
LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444.
https://doi.org/10.1038/nature14539
ANNOTATION
LeCun, Bengio, and Hinton review deep learning methods that use multiple processing layers to
learn representations at increasing levels of abstraction. They describe the role of backpropagation,
convolutional neural networks, recurrent networks, and large data sets in the development of
modern machine learning. The article is useful for connecting classical machine-learning methods
with the rise of representation learning and neural-network systems. Because it is a broad review in
Nature, it also helps a researcher place individual deep-learning techniques within the larger
historical development of the field.
