19 research outputs found
Assessing Resource-Performance Trade-off of Natural Language Models using Data Envelopment Analysis
Natural language models are often summarized through a high-dimensional set
of descriptive metrics including training corpus size, training time, the
number of trainable parameters, inference times, and evaluation statistics that
assess performance across tasks. The high dimensional nature of these metrics
yields challenges with regard to objectively comparing models; in particular it
is challenging to assess the trade-off models make between performance and
resources (compute time, memory, etc.).
We apply Data Envelopment Analysis (DEA) to this problem of assessing the
resource-performance trade-off. DEA is a nonparametric method that measures
productive efficiency of abstract units that consume one or more inputs and
yield at least one output. We recast natural language models as units suitable
for DEA, and we show that DEA can be used to create an effective framework for
quantifying model performance and efficiency. A central feature of DEA is that
it identifies a subset of models that live on an efficient frontier of
performance. DEA is also scalable, having been applied to problems with
thousands of units. We report empirical results of DEA applied to 14 different
language models that have a variety of architectures, and we show that DEA can
be used to identify a subset of models that effectively balance resource
demands against performance.Comment: 9 pages, 1 figure, Eval4NLP worksho
A Code of Conduct for Marine Carbon Dioxide Removal Research
Given the clear need to inform societal decision-making on the role marine Carbon Dioxide Removal (mCDR) can play in solving the climate crisis, it is imperative that researchers begin to answer questions about its effectiveness and impacts. Yet overly hasty deployment of new ocean-based climate interventions risks harm to communities and ecosystems and could jeopardize public perception of the field as a whole. In addition, the harms, risks and benefits of mCDR efforts are unlikely to be evenly distributed. Unabated, climate change could have a devastating impact on global ecosystems and human populations, and the impacts of mCDR should be contemplated in this context. This Code of Conduct exclusively applies to mCDR research and does not attempt to put any affiliated risk in the context of the risk of delaying climate action. Its purpose is to ensure that the impacts of mCDR research activities themselves are adequately understood and accounted for as they progress. It provides a roadmap of processes, procedures, and activities that project leads should follow to ensure that decisions regarding whether, when, where, and how to conduct mCDR research are informed by relevant ethical, scientific, economic, environmental, and regulatory considerations
Something old, something new: historical perspectives provide lessons for blue growth agendas
This is the author accepted manuscript. The final version is available from Wiley via the DOI in this record.The concept of ‘blue growth’, which aims to promote the growth of ocean economies whilst holistically managing marine socio-ecological systems, is emerging within national and international marine policy. The concept is often promoted as being novel, however, we show that, historical analogies exist which can provide insights for contemporary planning and implementation of blue growth. Using a case study approach based on expert knowledge, we identified 20 historical fisheries or aquaculture examples from 13 countries, spanning the last 40–800 years, that we contend embody blue growth concepts. This is the first time, to our knowledge, that blue growth has been investigated across such broad spatial and temporal scales. The past societies managed to balance exploitation with equitable access, ecological integrity, and/or economic growth for varying periods of time. Four main trajectories existed that led to the success or failure of blue growth. Success was linked to equitable rather than open access, innovation, and management that was responsive, holistic, and based on scientific knowledge and monitoring. The inability to achieve or maintain blue growth resulted from failures to address limits to industry growth and/or anticipate the impacts of adverse extrinsic events and drivers (e.g., changes in international markets, war), the prioritisation of short-term gains over long-term sustainability, and loss of supporting systems. Fourteen cross-cutting lessons and 10 recommendations were derived that can improve understanding and implementation of blue growth. Despite the contemporary literature broadly supporting our findings, these recommendations are not adequately addressed by agendas seeking to realize blue growth.European Commissio
Recommended from our members
The Linguistic Ecology of Northwestern California: Contact, Functional Convergence and Dialectology
The linguistic ecology of Northwestern California : contact, functional convergence and dialectology
Recommended from our members