2,279 research outputs found
Counting Carbon: A Survey of Factors Influencing the Emissions of Machine Learning
Machine learning (ML) requires using energy to carry out computations during
the model training process. The generation of this energy comes with an
environmental cost in terms of greenhouse gas emissions, depending on quantity
used and the energy source. Existing research on the environmental impacts of
ML has been limited to analyses covering a small number of models and does not
adequately represent the diversity of ML models and tasks. In the current
study, we present a survey of the carbon emissions of 95 ML models across time
and different tasks in natural language processing and computer vision. We
analyze them in terms of the energy sources used, the amount of CO2 emissions
produced, how these emissions evolve across time and how they relate to model
performance. We conclude with a discussion regarding the carbon footprint of
our field and propose the creation of a centralized repository for reporting
and tracking these emissions
Multi-Fidelity Active Learning with GFlowNets
In the last decades, the capacity to generate large amounts of data in
science and engineering applications has been growing steadily. Meanwhile, the
progress in machine learning has turned it into a suitable tool to process and
utilise the available data. Nonetheless, many relevant scientific and
engineering problems present challenges where current machine learning methods
cannot yet efficiently leverage the available data and resources. For example,
in scientific discovery, we are often faced with the problem of exploring very
large, high-dimensional spaces, where querying a high fidelity, black-box
objective function is very expensive. Progress in machine learning methods that
can efficiently tackle such problems would help accelerate currently crucial
areas such as drug and materials discovery. In this paper, we propose the use
of GFlowNets for multi-fidelity active learning, where multiple approximations
of the black-box function are available at lower fidelity and cost. GFlowNets
are recently proposed methods for amortised probabilistic inference that have
proven efficient for exploring large, high-dimensional spaces and can hence be
practical in the multi-fidelity setting too. Here, we describe our algorithm
for multi-fidelity active learning with GFlowNets and evaluate its performance
in both well-studied synthetic tasks and practically relevant applications of
molecular discovery. Our results show that multi-fidelity active learning with
GFlowNets can efficiently leverage the availability of multiple oracles with
different costs and fidelities to accelerate scientific discovery and
engineering design.Comment: Code: https://github.com/nikita-0209/mf-al-gf
On the importance of catalyst-adsorbate 3D interactions for relaxed energy predictions
The use of machine learning for material property prediction and discovery
has traditionally centered on graph neural networks that incorporate the
geometric configuration of all atoms. However, in practice not all this
information may be readily available, e.g.~when evaluating the potentially
unknown binding of adsorbates to catalyst. In this paper, we investigate
whether it is possible to predict a system's relaxed energy in the OC20 dataset
while ignoring the relative position of the adsorbate with respect to the
electro-catalyst. We consider SchNet, DimeNet++ and FAENet as base
architectures and measure the impact of four modifications on model
performance: removing edges in the input graph, pooling independent
representations, not sharing the backbone weights and using an attention
mechanism to propagate non-geometric relative information. We find that while
removing binding site information impairs accuracy as expected, modified models
are able to predict relaxed energies with remarkably decent MAE. Our work
suggests future research directions in accelerated materials discovery where
information on reactant configurations can be reduced or altogether omitted
Abundance and Antimicrobial Resistance of Three Bacterial Species along a Complete Wastewater Pathway
After consumption, antibiotic residues and exposed bacteria end up via the feces in wastewater, and therefore wastewater is believed to play an important role in the spread of antimicrobial resistance (AMR). We investigated the abundance and AMR profiles of three different species over a complete wastewater pathway during a one-year sampling campaign, as well as including antimicrobial consumption and antimicrobial concentrations analysis. A total of 2886 isolates (997 Escherichia coli, 863 Klebsiella spp., and 1026 Aeromonas spp.) were cultured from the 211 samples collected. The bacterial AMR profiles mirrored the antimicrobial consumption in the respective locations, which were highest in the hospital. However, the contribution of hospital wastewater to AMR found in the wastewater treatment plant (WWTP) was below 10% for all antimicrobials tested. We found high concentrations (7-8 logs CFU/L) of the three bacterial species in all wastewaters, and they survived the wastewater treatment (effluent concentrations were around 5 log CFU/L), showing an increase of E. coli in the receiving river after the WWTP discharge. Although the WWTP had no effect on the proportion of AMR, bacterial species and antimicrobial residues were still measured in the effluent, showing the role of wastewater contamination in the environmental surface water.</p
Multi-Objective GFlowNets
We study the problem of generating diverse candidates in the context of
Multi-Objective Optimization. In many applications of machine learning such as
drug discovery and material design, the goal is to generate candidates which
simultaneously optimize a set of potentially conflicting objectives. Moreover,
these objectives are often imperfect evaluations of some underlying property of
interest, making it important to generate diverse candidates to have multiple
options for expensive downstream evaluations. We propose Multi-Objective
GFlowNets (MOGFNs), a novel method for generating diverse Pareto optimal
solutions, based on GFlowNets. We introduce two variants of MOGFNs: MOGFN-PC,
which models a family of independent sub-problems defined by a scalarization
function, with reward-conditional GFlowNets, and MOGFN-AL, which solves a
sequence of sub-problems defined by an acquisition function in an active
learning loop. Our experiments on wide variety of synthetic and benchmark tasks
demonstrate advantages of the proposed methods in terms of the Pareto
performance and importantly, improved candidate diversity, which is the main
contribution of this work.Comment: 23 pages, 8 figures. ICML 2023. Code at:
https://github.com/GFNOrg/multi-objective-gf
Physics-Constrained Deep Learning for Climate Downscaling
The availability of reliable, high-resolution climate and weather data is
important to inform long-term decisions on climate adaptation and mitigation
and to guide rapid responses to extreme events. Forecasting models are limited
by computational costs and, therefore, often generate coarse-resolution
predictions. Statistical downscaling, including super-resolution methods from
deep learning, can provide an efficient method of upsampling low-resolution
data. However, despite achieving visually compelling results in some cases,
such models frequently violate conservation laws when predicting physical
variables. In order to conserve physical quantities, we develop methods that
guarantee physical constraints are satisfied by a deep learning downscaling
model while also improving their performance according to traditional metrics.
We compare different constraining approaches and demonstrate their
applicability across different neural architectures as well as a variety of
climate and weather datasets. Besides enabling faster and more accurate climate
predictions, we also show that our novel methodologies can improve
super-resolution for satellite data and standard datasets
Fourier Neural Operators for Arbitrary Resolution Climate Data Downscaling
Climate simulations are essential in guiding our understanding of climate
change and responding to its effects. However, it is computationally expensive
to resolve complex climate processes at high spatial resolution. As one way to
speed up climate simulations, neural networks have been used to downscale
climate variables from fast-running low-resolution simulations, but
high-resolution training data are often unobtainable or scarce, greatly
limiting accuracy. In this work, we propose a downscaling method based on the
Fourier neural operator. It trains with data of a small upsampling factor and
then can zero-shot downscale its input to arbitrary unseen high resolution.
Evaluated both on ERA5 climate model data and on the Navier-Stokes equation
solution data, our downscaling model significantly outperforms state-of-the-art
convolutional and generative adversarial downscaling models, both in standard
single-resolution downscaling and in zero-shot generalization to higher
upsampling factors. Furthermore, we show that our method also outperforms
state-of-the-art data-driven partial differential equation solvers on
Navier-Stokes equations. Overall, our work bridges the gap between simulation
of a physical process and interpolation of low-resolution output, showing that
it is possible to combine both approaches and significantly improve upon each
other.Comment: Presented at the ICLR 2023 workshop on "Tackling Climate Change with
Machine Learning
Crystal-GFN: sampling crystals with desirable properties and constraints
Accelerating material discovery holds the potential to greatly help mitigate
the climate crisis. Discovering new solid-state materials such as
electrocatalysts, super-ionic conductors or photovoltaic materials can have a
crucial impact, for instance, in improving the efficiency of renewable energy
production and storage. In this paper, we introduce Crystal-GFN, a generative
model of crystal structures that sequentially samples structural properties of
crystalline materials, namely the space group, composition and lattice
parameters. This domain-inspired approach enables the flexible incorporation of
physical and structural hard constraints, as well as the use of any available
predictive model of a desired physicochemical property as an objective
function. To design stable materials, one must target the candidates with the
lowest formation energy. Here, we use as objective the formation energy per
atom of a crystal structure predicted by a new proxy machine learning model
trained on MatBench. The results demonstrate that Crystal-GFN is able to sample
highly diverse crystals with low (median -3.1 eV/atom) predicted formation
energy.Comment: Main paper (10 pages) + references + appendi
- …