3,401 research outputs found
Meta-learning algorithms and applications
Meta-learning in the broader context concerns how an agent learns about their own learning, allowing them to improve their learning process. Learning how to learn is not only beneficial for humans, but it has also shown vast benefits for improving how machines learn. In the context of machine learning, meta-learning enables models to improve their learning process by selecting suitable meta-parameters that influence the learning. For deep learning specifically, the meta-parameters typically describe details of the training of the model but can also include description of the model itself - the architecture. Meta-learning is usually done with specific goals in mind, for example trying to improve ability to generalize or learn new concepts from only a few examples.
Meta-learning can be powerful, but it comes with a key downside: it is often computationally costly. If the costs would be alleviated, meta-learning could be more accessible to developers of new artificial intelligence models, allowing them to achieve greater goals or save resources. As a result, one key focus of our research is on significantly improving the efficiency of meta-learning. We develop two approaches: EvoGrad and PASHA, both of which significantly improve meta-learning efficiency in two common scenarios. EvoGrad allows us to efficiently optimize the value of a large number of differentiable meta-parameters, while PASHA enables us to efficiently optimize any type of meta-parameters but fewer in number.
Meta-learning is a tool that can be applied to solve various problems. Most commonly it is applied for learning new concepts from only a small number of examples (few-shot learning), but other applications exist too. To showcase the practical impact that meta-learning can make in the context of neural networks, we use meta-learning as a novel solution for two selected problems: more accurate uncertainty quantification (calibration) and general-purpose few-shot learning. Both are practically important problems and using meta-learning approaches we can obtain better solutions than the ones obtained using existing approaches. Calibration is important for safety-critical applications of neural networks, while general-purpose few-shot learning tests model's ability to generalize few-shot learning abilities across diverse tasks such as recognition, segmentation and keypoint estimation.
More efficient algorithms as well as novel applications enable the field of meta-learning to make more significant impact on the broader area of deep learning and potentially solve problems that were too challenging before. Ultimately both of them allow us to better utilize the opportunities that artificial intelligence presents
Indie encounters: exploring indie music socialising in China
Indie music, a genre deeply rooted in rock and punk music, is renowned for its independence from major commercial record labels. It has emerged as a choice for music consumers seeking alternatives to mainstream popular music, catering to a niche music preference. The minority nature of indie music not only provides its lovers with a profound space for individual expression and a sense of collective belonging but also introduces other challenges into their social lives. Recently, the field of music sociology has proposed a more diverse perspective to observe and analyse the intricate role of music for individuals and society. In this context, regarding Chinese indie music lovers with niche music preferences, how their indie music practices integrate into their social lives and how they navigate their niche music tastes have become worthwhile topics of exploration.
Drawing on interviews with 31 Chinese indie music lovers and extensive online ethnography, this thesis investigates how Chinese indie music lovers comprehend and engage with indie music, and how the power of indie music shapes them and their social behaviours.
I employ the theoretical framework of âmusic in actionâ (Hennion, 2001; DeNora, 2011, 2016) and symbolic interactionism (Mead, 1934; Goffman, 1959; Blumer, 1969) to examine the dynamic and multifaceted roles of indie music in the social lives of Chinese indie music lovers. I develop a concept of âmusic socialisingâ to delve into several key aspects of music loversâ social practices. I contend that through various forms of musical activities such as music selection, live music attendance, and digital practices, indie music lovers exhibit strategic and reflexive characteristics in their music practices. These practices actively contribute to constructing and maintaining self and identity, negotiating social ties, and forming and mediating collectivity within a broader social landscape. It is through these processes that the music practices of Chinese indie music lovers are endowed with meanings, thereby shaping their social reality. This thesis presents a rich and nuanced picture of the social experiences of Chinese indie music lovers, uncovering the transformative power of their indie music practices. It presents a compelling argument for the significance of music as a social agency, highlighting the complex interactions between music, individuals, and society. By bridging theoretical insights with rich empirical data, this thesis contributes to our understanding of the socio-cultural dimensions of music, offering fresh perspectives on the role of indie music in contemporary Chinese society
Examining systemic and dispositional factors impacting historically disenfranchised schools across North Carolina
This mixed method sequential explanatory study provided analysis of North Carolina (NC) school leadersâ dispositions in eliminating opportunity gaps, outlined in NCâs strategic plan. The studyâs quantitative phase used descriptive and correlation analysis of eight Likert subscales around four tenets of transformative leadership (Shields, 2011) and aspects of critical race theory (Bell, 1992; Ladson-Billings, 1998; Ladson-Billings & Tate, 2006) to understand systemic inequities and leadership attitudes.
The qualitative phase comprised three analyses of education leadership dispositions and systemic factors in NC schools. The first analysis of State Board of Education meeting minutes from 2018â2023 quantified and analyzed utterances of racism and critical race, outlined the sociopolitical context of such utterances, and identified systemic patterns and state leader dispositions. The second analysis of five interviews of Kâ12 graduates identified persistent and systemic factors influencing NC education 3 decades after Brown v. Board of Education (1954) and within the context of Leandro v. State of NC (1997), where the NC Supreme Court recognized the state constitutional right for every student to access a âsound basic education.â The final qualitative analysis consisted of five interviews of current NC public school system leaders, for personal narratives of the state of NC schools compared to patterns from lived experiences of NC Kâ12 graduates.
The studyâs findings suggested NC school and state education leaders experience a racialized dichotomy between willingness for change (equity intentions) and execution of transformative action (practice). Although leaders at the board and school levels recognize the need for inclusivity and equity, a struggle to transcend systemic challenges, especially rooted in racial biases and power dynamics is evident. This study may identify leadership qualities needed for change in NC to address systemic inequities for improving educational access and inform policy to uphold all studentsâ constitutional right to a sound, basic education
Multidisciplinary perspectives on Artificial Intelligence and the law
This open access book presents an interdisciplinary, multi-authored, edited collection of chapters on Artificial Intelligence (âAIâ) and the Law. AI technology has come to play a central role in the modern data economy. Through a combination of increased computing power, the growing availability of data and the advancement of algorithms, AI has now become an umbrella term for some of the most transformational technological breakthroughs of this age. The importance of AI stems from both the opportunities that it offers and the challenges that it entails. While AI applications hold the promise of economic growth and efficiency gains, they also create significant risks and uncertainty. The potential and perils of AI have thus come to dominate modern discussions of technology and ethics â and although AI was initially allowed to largely develop without guidelines or rules, few would deny that the law is set to play a fundamental role in shaping the future of AI. As the debate over AI is far from over, the need for rigorous analysis has never been greater. This book thus brings together contributors from different fields and backgrounds to explore how the law might provide answers to some of the most pressing questions raised by AI. An outcome of the CatĂłlica Research Centre for the Future of Law and its interdisciplinary working group on Law and Artificial Intelligence, it includes contributions by leading scholars in the fields of technology, ethics and the law.info:eu-repo/semantics/publishedVersio
Backpropagation Beyond the Gradient
Automatic differentiation is a key enabler of deep learning: previously, practitioners were limited to models
for which they could manually compute derivatives. Now, they can create sophisticated models with almost
no restrictions and train them using first-order, i. e. gradient, information. Popular libraries like PyTorch
and TensorFlow compute this gradient efficiently, automatically, and conveniently with a single line of
code. Under the hood, reverse-mode automatic differentiation, or gradient backpropagation, powers the
gradient computation in these libraries. Their entire design centers around gradient backpropagation.
These frameworks are specialized around one specific taskâcomputing the average gradient in a mini-batch.
This specialization often complicates the extraction of other information like higher-order statistical moments
of the gradient, or higher-order derivatives like the Hessian. It limits practitioners and researchers to methods
that rely on the gradient. Arguably, this hampers the field from exploring the potential of higher-order
information and there is evidence that focusing solely on the gradient has not lead to significant recent
advances in deep learning optimization.
To advance algorithmic research and inspire novel ideas, information beyond the batch-averaged gradient
must be made available at the same level of computational efficiency, automation, and convenience.
This thesis presents approaches to simplify experimentation with rich information beyond the gradient
by making it more readily accessible. We present an implementation of these ideas as an extension to the
backpropagation procedure in PyTorch. Using this newly accessible information, we demonstrate possible use
cases by (i) showing how it can inform our understanding of neural network training by building a diagnostic
tool, and (ii) enabling novel methods to efficiently compute and approximate curvature information.
First, we extend gradient backpropagation for sequential feedforward models to Hessian backpropagation
which enables computing approximate per-layer curvature. This perspective unifies recently proposed block-
diagonal curvature approximations. Like gradient backpropagation, the computation of these second-order
derivatives is modular, and therefore simple to automate and extend to new operations.
Based on the insight that rich information beyond the gradient can be computed efficiently and at the
same time, we extend the backpropagation in PyTorch with the BackPACK library. It provides efficient and
convenient access to statistical moments of the gradient and approximate curvature information, often at a
small overhead compared to computing just the gradient.
Next, we showcase the utility of such information to better understand neural network training. We build
the Cockpit library that visualizes what is happening inside the model during training through various
instruments that rely on BackPACKâs statistics. We show how Cockpit provides a meaningful statistical
summary report to the deep learning engineer to identify bugs in their machine learning pipeline, guide
hyperparameter tuning, and study deep learning phenomena.
Finally, we use BackPACKâs extended automatic differentiation functionality to develop ViViT, an approach
to efficiently compute curvature information, in particular curvature noise. It uses the low-rank structure
of the generalized Gauss-Newton approximation to the Hessian and addresses shortcomings in existing
curvature approximations. Through monitoring curvature noise, we demonstrate how ViViTâs information
helps in understanding challenges to make second-order optimization methods work in practice.
This work develops new tools to experiment more easily with higher-order information in complex deep
learning models. These tools have impacted works on Bayesian applications with Laplace approximations,
out-of-distribution generalization, differential privacy, and the design of automatic differentia-
tion systems. They constitute one important step towards developing and establishing more efficient deep
learning algorithms
Conversations on Empathy
In the aftermath of a global pandemic, amidst new and ongoing wars, genocide, inequality, and staggering ecological collapse, some in the public and political arena have argued that we are in desperate need of greater empathy â be this with our neighbours, refugees, war victims, the vulnerable or disappearing animal and plant species. This interdisciplinary volume asks the crucial questions: How does a better understanding of empathy contribute, if at all, to our understanding of others? How is it implicated in the ways we perceive, understand and constitute others as subjects? Conversations on Empathy examines how empathy might be enacted and experienced either as a way to highlight forms of otherness or, instead, to overcome what might otherwise appear to be irreducible differences. It explores the ways in which empathy enables us to understand, imagine and create sameness and otherness in our everyday intersubjective encounters focusing on a varied range of "radical others" â others who are perceived as being dramatically different from oneself. With a focus on the importance of empathy to understand difference, the book contends that the role of empathy is critical, now more than ever, for thinking about local and global challenges of interconnectedness, care and justice
Distributed System Fuzzing
Grey-box fuzzing is the lightweight approach of choice for finding bugs in
sequential programs. It provides a balance between efficiency and effectiveness
by conducting a biased random search over the domain of program inputs using a
feedback function from observed test executions. For distributed system
testing, however, the state-of-practice is represented today by only black-box
tools that do not attempt to infer and exploit any knowledge of the system's
past behaviours to guide the search for bugs.
In this work, we present Mallory: the first framework for grey-box
fuzz-testing of distributed systems. Unlike popular black-box distributed
system fuzzers, such as Jepsen, that search for bugs by randomly injecting
network partitions and node faults or by following human-defined schedules,
Mallory is adaptive. It exercises a novel metric to learn how to maximize the
number of observed system behaviors by choosing different sequences of faults,
thus increasing the likelihood of finding new bugs. The key enablers for our
approach are the new ideas of timeline-driven testing and timeline abstraction
that provide the feedback function guiding a biased random search for failures.
Mallory dynamically constructs Lamport timelines of the system behaviour,
abstracts these timelines into happens-before summaries, and introduces faults
guided by its real-time observation of the summaries.
We have evaluated Mallory on a diverse set of widely-used industrial
distributed systems. Compared to the start-of-the-art black-box fuzzer Jepsen,
Mallory explores more behaviours and takes less time to find bugs. Mallory
discovered 22 zero-day bugs (of which 18 were confirmed by developers),
including 10 new vulnerabilities, in rigorously-tested distributed systems such
as Braft, Dqlite, and Redis. 6 new CVEs have been assigned
- âŠ