1,108 research outputs found
Understanding Random Forests: From Theory to Practice
Data analysis and machine learning have become an integrative part of the
modern scientific methodology, offering automated procedures for the prediction
of a phenomenon based on past observations, unraveling underlying patterns in
data and providing insights about the problem. Yet, caution should avoid using
machine learning as a black-box tool, but rather consider it as a methodology,
with a rational thought process that is entirely dependent on the problem under
study. In particular, the use of algorithms should ideally require a reasonable
understanding of their mechanisms, properties and limitations, in order to
better apprehend and interpret their results.
Accordingly, the goal of this thesis is to provide an in-depth analysis of
random forests, consistently calling into question each and every part of the
algorithm, in order to shed new light on its learning capabilities, inner
workings and interpretability. The first part of this work studies the
induction of decision trees and the construction of ensembles of randomized
trees, motivating their design and purpose whenever possible. Our contributions
follow with an original complexity analysis of random forests, showing their
good computational performance and scalability, along with an in-depth
discussion of their implementation details, as contributed within Scikit-Learn.
In the second part of this work, we analyse and discuss the interpretability
of random forests in the eyes of variable importance measures. The core of our
contributions rests in the theoretical characterization of the Mean Decrease of
Impurity variable importance measure, from which we prove and derive some of
its properties in the case of multiway totally randomized trees and in
asymptotic conditions. In consequence of this work, our analysis demonstrates
that variable importances [...].Comment: PhD thesis. Source code available at
https://github.com/glouppe/phd-thesi
- …