18 research outputs found
Advances in random forests with application to classification
Thesis (MCom)--Stellenbosch University, 2016.ENGLISH SUMMARY : Since their introduction, random forests have successfully been employed in a vast array of application areas. Fairly recently, a number of algorithms that adhere to Leo Breiman’s definition of a random forest have been proposed in the literature. Breiman’s popular random forest algorithm (Forest-RI), and related ensemble classification algorithms which followed, form the focus of this study. A review of random forest algorithms that were developed since the introduction of Forest-RI is given. This includes a novel taxonomy of random forest classification algorithms, which is based on their sources of randomization, and on deterministic modifications. Also, a visual conceptualization of contributions to random forest algorithms in the literature is provided by means of multidimensional scaling.
Towards an analysis of advances in random forest algorithms, decomposition of the expected prediction error into bias and variance components is considered. In classification, such decompositions are not as straightforward as in the case of using squared-error loss for regression. Hence various definitions of bias and variance for classification can be found in the literature. Using a particular bias-variance decomposition, an empirical study of ensemble learners, including bagging, boosting and Forest-RI, is presented. From the empirical results and insights into the way in which certain mechanisms of random forests affect bias and variance, a novel random forest framework, viz. oblique random rotation forests, is proposed. Although not entirely satisfactory, the framework serves as an example of a heuristic approach towards novel proposals based on bias-variance analyses, instead of an ad hoc approach, as is often found in the literature.
The analysis of comparative studies regarding advances in random forest algorithms is also considered. It is of interest to critically evaluate the conclusions
that can be drawn from these studies, and to infer whether novel random forest algorithms are found to significantly outperform Forest-RI. For this purpose,
a meta-analysis is conducted in which an evaluation is given of the state of research on random forests based on all (34) papers that could be found in which a novel random forest algorithm was proposed and compared to already existing random forest algorithms. Using the reported performances in each paper, a novel two-step procedure is proposed, which allows for multiple algorithms to be compared over multiple data sets, and across different papers. The meta analysis results indicate weighted voting strategies and variable weighting in high-dimensional settings to provide significantly improved performances over the performance of Breiman’s popular Forest-RI algorithm.AFRIKAANSE OPSOMMING : Sedert hulle bekendstelling is random forests met groot sukses in ’n wye verskeidenheid toepassings geimplementeer. ’n Aantal algoritmes wat aan Leo Breiman se definisie van ’n random forest voldoen, is redelik onlangs in die literatuur voorgestel. Breiman se gewilde random forest (Forest-RI) algoritme en verwante ensemble klassifikasie algoritmes wat daaruit ontwikkel is, vorm die fokus van die studie. ’n Oorsig van nuut ontwikkelde random forest algoritmes wat sedert die bekendstellig van Forest-RI voorgestel is, word gegee. Dit sluit ’n nuwe kategoriseringsraamwerk van random forest algoritmes in, wat gebaseer is op hulle bron van ewekansigheid, asook op hulle tipe deterministiese wysigings. Met behulp van meerdimensionele skalering word ’n visuele voorstelling van bydraes in die literatuur ten opsigte van random forest algoritmes ook gegee.
Met die oog op ’n analise van ontwikkelings rondom random forest algoritmes, word die opdeling van die verwagte vooruitskattingsfout in ’n sydigheiden
variansie komponent beskou. In vergelyking met regressie wanneer die gekwadreerde-fout verliesfunksie gebruik word, is hierdie opdeling in klassifikasie
minder voor-die-hand-liggend. Derhalwe kom verskeie definisies van sydigheid en variansie vir klassifikasie in die literatuur voor. Deur gebruik te maak van ’n spesifieke sydigheid-variansie opdeling word ’n empiriese studie van ensemble algoritmes, ingesluit bagging, boosting en Forest-RI, uitgevoer.
Uit die empiriese resultate en insigte rakende die manier waarop sekere meganismes van random forests sydigheid en variansie beinvloed, word ’n nuwe
random forest raamwerk voorgestel, nl. oblique random rotation forests. Hoewel nie in geheel bevredigend nie, dien die raamwerk as ’n voorbeeld van ’n
heuristiese benadering tot nuwe voorstelle gebaseer op sydigheid-variansie analises in plaas van ’n ad hoc benadering, soos wat dikwels gevind word in die literatuur. Verder word vergelykende studies met betrekking tot random forests geanaliseer. Hier is dit van belang om gevolgtrekkings wat uit vergelykende studies gemaak is, krities te evalueer, en om te verifieer of nuwe random forest algoritmes betekenisvol verbeter op Forest-RI. Met bogaande doelwitte in gedagte is ’n meta-analise uitgevoer waarin die stand van random forest navorsing geevalueer is. Die analise is gebaseer op al (34) artikels waarin ’n nuwe random forest algoritme voorgestel is en vergelyk word met reeds bestaande random forest algoritmes. Deur gebruik te maak van die gerapporteerde prestasie-maatstawwe in elke artikel, is ’n nuwe prosedure voorgestel waarvolgens ’n aantal algoritmes oor ’n aantal datastelle en oor verskillende artikels vergelyk kan word. Die resultate van die meta-analise toon aan dat geweegde stem-strategiee en die weging van veranderlikes in hoe-dimensionele data ’n betekenisvolle verbetering lewer op die akkuraatheid van Breiman se gewilde Forest-RI algoritme
Learning Dynamics of Linear Denoising Autoencoders
Denoising autoencoders (DAEs) have proven useful for unsupervised
representation learning, but a thorough theoretical understanding is still
lacking of how the input noise influences learning. Here we develop theory for
how noise influences learning in DAEs. By focusing on linear DAEs, we are able
to derive analytic expressions that exactly describe their learning dynamics.
We verify our theoretical predictions with simulations as well as experiments
on MNIST and CIFAR-10. The theory illustrates how, when tuned correctly, noise
allows DAEs to ignore low variance directions in the inputs while learning to
reconstruct them. Furthermore, in a comparison of the learning dynamics of DAEs
to standard regularised autoencoders, we show that noise has a similar
regularisation effect to weight decay, but with faster training dynamics. We
also show that our theoretical predictions approximate learning dynamics on
real-world data and qualitatively match observed dynamics in nonlinear DAEs.Comment: 14 pages, 7 figures, accepted at the 35th International Conference on
Machine Learning (ICML) 201
Off-the-Grid MARL: Datasets with Baselines for Offline Multi-Agent Reinforcement Learning
Being able to harness the power of large datasets for developing cooperative
multi-agent controllers promises to unlock enormous value for real-world
applications. Many important industrial systems are multi-agent in nature and
are difficult to model using bespoke simulators. However, in industry,
distributed processes can often be recorded during operation, and large
quantities of demonstrative data stored. Offline multi-agent reinforcement
learning (MARL) provides a promising paradigm for building effective
decentralised controllers from such datasets. However, offline MARL is still in
its infancy and therefore lacks standardised benchmark datasets and baselines
typically found in more mature subfields of reinforcement learning (RL). These
deficiencies make it difficult for the community to sensibly measure progress.
In this work, we aim to fill this gap by releasing off-the-grid MARL (OG-MARL):
a growing repository of high-quality datasets with baselines for cooperative
offline MARL research. Our datasets provide settings that are characteristic of
real-world systems, including complex environment dynamics, heterogeneous
agents, non-stationarity, many agents, partial observability, suboptimality,
sparse rewards and demonstrated coordination. For each setting, we provide a
range of different dataset types (e.g. Good, Medium, Poor, and Replay) and
profile the composition of experiences for each dataset. We hope that OG-MARL
will serve the community as a reliable source of datasets and help drive
progress, while also providing an accessible entry point for researchers new to
the field.Comment: Extended Abstract at Autonomous Agents and Multi-Agent Systems
Conference 202
Towards Learning to Speak and Hear Through Multi-Agent Communication over a Continuous Acoustic Channel
While multi-agent reinforcement learning has been used as an effective means
to study emergent communication between agents, existing work has focused
almost exclusively on communication with discrete symbols. Human communication
often takes place (and emerged) over a continuous acoustic channel; human
infants acquire language in large part through continuous signalling with their
caregivers. We therefore ask: Are we able to observe emergent language between
agents with a continuous communication channel trained through reinforcement
learning? And if so, what is the impact of channel characteristics on the
emerging language? We propose an environment and training methodology to serve
as a means to carry out an initial exploration of these questions. We use a
simple messaging environment where a "speaker" agent needs to convey a concept
to a "listener". The Speaker is equipped with a vocoder that maps symbols to a
continuous waveform, this is passed over a lossy continuous channel, and the
Listener needs to map the continuous signal to the concept. Using deep
Q-learning, we show that basic compositionality emerges in the learned language
representations. We find that noise is essential in the communication channel
when conveying unseen concept combinations. And we show that we can ground the
emergent communication by introducing a caregiver predisposed to "hearing" or
"speaking" English. Finally, we describe how our platform serves as a starting
point for future work that uses a combination of deep reinforcement learning
and multi-agent systems to study our questions of continuous signalling in
language learning and emergence.Comment: 12 pages, 6 figures, 3 tables; under review as a conference paper at
ICLR 202
Combinatorial Optimization with Policy Adaptation using Latent Space Search
Combinatorial Optimization underpins many real-world applications and yet,
designing performant algorithms to solve these complex, typically NP-hard,
problems remains a significant research challenge. Reinforcement Learning (RL)
provides a versatile framework for designing heuristics across a broad spectrum
of problem domains. However, despite notable progress, RL has not yet
supplanted industrial solvers as the go-to solution. Current approaches
emphasize pre-training heuristics that construct solutions but often rely on
search procedures with limited variance, such as stochastically sampling
numerous solutions from a single policy or employing computationally expensive
fine-tuning of the policy on individual problem instances. Building on the
intuition that performant search at inference time should be anticipated during
pre-training, we propose COMPASS, a novel RL approach that parameterizes a
distribution of diverse and specialized policies conditioned on a continuous
latent space. We evaluate COMPASS across three canonical problems - Travelling
Salesman, Capacitated Vehicle Routing, and Job-Shop Scheduling - and
demonstrate that our search strategy (i) outperforms state-of-the-art
approaches on 11 standard benchmarking tasks and (ii) generalizes better,
surpassing all other approaches on a set of 18 procedurally transformed
instance distributions.Comment: Accepted at Neurips 2023. Small updates in results reporte
Efficiently Quantifying Individual Agent Importance in Cooperative MARL
Measuring the contribution of individual agents is challenging in cooperative
multi-agent reinforcement learning (MARL). In cooperative MARL, team
performance is typically inferred from a single shared global reward. Arguably,
among the best current approaches to effectively measure individual agent
contributions is to use Shapley values. However, calculating these values is
expensive as the computational complexity grows exponentially with respect to
the number of agents. In this paper, we adapt difference rewards into an
efficient method for quantifying the contribution of individual agents,
referred to as Agent Importance, offering a linear computational complexity
relative to the number of agents. We show empirically that the computed values
are strongly correlated with the true Shapley values, as well as the true
underlying individual agent rewards, used as the ground truth in environments
where these are available. We demonstrate how Agent Importance can be used to
help study MARL systems by diagnosing algorithmic failures discovered in prior
MARL benchmarking work. Our analysis illustrates Agent Importance as a valuable
explainability component for future MARL benchmarks.Comment: 8 pages, AAAI XAI4DRL workshop 2023; references updated, figure 8
style updated, typo
How much can change in a year? Revisiting Evaluation in Multi-Agent Reinforcement Learning
Establishing sound experimental standards and rigour is important in any
growing field of research. Deep Multi-Agent Reinforcement Learning (MARL) is
one such nascent field. Although exciting progress has been made, MARL has
recently come under scrutiny for replicability issues and a lack of
standardised evaluation methodology, specifically in the cooperative setting.
Although protocols have been proposed to help alleviate the issue, it remains
important to actively monitor the health of the field. In this work, we extend
the database of evaluation methodology previously published by containing
meta-data on MARL publications from top-rated conferences and compare the
findings extracted from this updated database to the trends identified in their
work. Our analysis shows that many of the worrying trends in performance
reporting remain. This includes the omission of uncertainty quantification, not
reporting all relevant evaluation details and a narrowing of algorithmic
development classes. Promisingly, we do observe a trend towards more difficult
scenarios in SMAC-v1, which if continued into SMAC-v2 will encourage novel
algorithmic development. Our data indicate that replicability needs to be
approached more proactively by the MARL community to ensure trust in the field
as we move towards exciting new frontiers.Comment: 6 pages, AAAI XAI4DRL workshop 2023; typos corrected, images updated,
page count update