1,205 research outputs found
Computational techniques to interpret the neural code underlying complex cognitive processes
Advances in large-scale neural recording technology have significantly improved the
capacity to further elucidate the neural code underlying complex cognitive processes.
This thesis aimed to investigate two research questions in rodent models. First, what
is the role of the hippocampus in memory and specifically what is the underlying
neural code that contributes to spatial memory and navigational decision-making.
Second, how is social cognition represented in the medial prefrontal cortex at the
level of individual neurons. To start, the thesis begins by investigating memory and
social cognition in the context of healthy and diseased states that use non-invasive
methods (i.e. fMRI and animal behavioural studies). The main body of the thesis
then shifts to developing our fundamental understanding of the neural mechanisms
underpinning these cognitive processes by applying computational techniques to ana lyse stable large-scale neural recordings. To achieve this, tailored calcium imaging
and behaviour preprocessing computational pipelines were developed and optimised
for use in social interaction and spatial navigation experimental analysis. In parallel,
a review was conducted on methods for multivariate/neural population analysis. A
comparison of multiple neural manifold learning (NML) algorithms identified that non linear algorithms such as UMAP are more adaptable across datasets of varying noise
and behavioural complexity. Furthermore, the review visualises how NML can be
applied to disease states in the brain and introduces the secondary analyses that
can be used to enhance or characterise a neural manifold. Lastly, the preprocessing
and analytical pipelines were combined to investigate the neural mechanisms in volved in social cognition and spatial memory. The social cognition study explored
how neural firing in the medial Prefrontal cortex changed as a function of the social
dominance paradigm, the "Tube Test". The univariate analysis identified an ensemble
of behavioural-tuned neurons that fire preferentially during specific behaviours such
as "pushing" or "retreating" for the animalâs own behaviour and/or the competitorâs
behaviour. Furthermore, in dominant animals, the neural population exhibited greater
average firing than that of subordinate animals. Next, to investigate spatial memory,
a spatial recency task was used, where rats learnt to navigate towards one of three
reward locations and then recall the rewarded location of the session. During the
task, over 1000 neurons were recorded from the hippocampal CA1 region for five rats
over multiple sessions. Multivariate analysis revealed that the sequence of neurons encoding an animalâs spatial position leading up to a rewarded location was also active
in the decision period before the animal navigates to the rewarded location. The result
posits that prospective replay of neural sequences in the hippocampal CA1 region
could provide a mechanism by which decision-making is supported
Deep generative models for network data synthesis and monitoring
Measurement and monitoring are fundamental tasks in all networks, enabling the down-stream management and optimization of the network.
Although networks inherently
have abundant amounts of monitoring data, its access and effective measurement is
another story. The challenges exist in many aspects. First, the inaccessibility of network monitoring data for external users, and it is hard to provide a high-fidelity dataset
without leaking commercial sensitive information. Second, it could be very expensive
to carry out effective data collection to cover a large-scale network system, considering the size of network growing, i.e., cell number of radio network and the number of
flows in the Internet Service Provider (ISP) network. Third, it is difficult to ensure fidelity and efficiency simultaneously in network monitoring, as the available resources
in the network element that can be applied to support the measurement function are
too limited to implement sophisticated mechanisms. Finally, understanding and explaining the behavior of the network becomes challenging due to its size and complex
structure. Various emerging optimization-based solutions (e.g., compressive sensing)
or data-driven solutions (e.g. deep learning) have been proposed for the aforementioned challenges. However, the fidelity and efficiency of existing methods cannot yet
meet the current network requirements.
The contributions made in this thesis significantly advance the state of the art in
the domain of network measurement and monitoring techniques. Overall, we leverage
cutting-edge machine learning technology, deep generative modeling, throughout the
entire thesis. First, we design and realize APPSHOT , an efficient city-scale network
traffic sharing with a conditional generative model, which only requires open-source
contextual data during inference (e.g., land use information and population distribution). Second, we develop an efficient drive testing system â GENDT, based on generative model, which combines graph neural networks, conditional generation, and quantified model uncertainty to enhance the efficiency of mobile drive testing. Third, we
design and implement DISTILGAN, a high-fidelity, efficient, versatile, and real-time
network telemetry system with latent GANs and spectral-temporal networks. Finally,
we propose SPOTLIGHT , an accurate, explainable, and efficient anomaly detection system of the Open RAN (Radio Access Network) system. The lessons learned through
this research are summarized, and interesting topics are discussed for future work in
this domain. All proposed solutions have been evaluated with real-world datasets and
applied to support different applications in real systems
Efficient Deep Learning for Real-time Classification of Astronomical Transients
A new golden age in astronomy is upon us, dominated by data. Large astronomical surveys are broadcasting unprecedented rates of information, demanding machine learning as a critical component in modern scientific pipelines to handle the deluge of data. The upcoming Legacy Survey of Space and Time (LSST) of the Vera C. Rubin Observatory will raise the big-data bar for time- domain astronomy, with an expected 10 million alerts per-night, and generating many petabytes of data over the lifetime of the survey. Fast and efficient classification algorithms that can operate in real-time, yet robustly and accurately, are needed for time-critical events where additional resources can be sought for follow-up analyses. In order to handle such data, state-of-the-art deep learning architectures coupled with tools that leverage modern hardware accelerators are essential.
The work contained in this thesis seeks to address the big-data challenges of LSST by proposing novel efficient deep learning architectures for multivariate time-series classification that can provide state-of-the-art classification of astronomical transients at a fraction of the computational costs of other deep learning approaches. This thesis introduces the depthwise-separable convolution and the notion of convolutional embeddings to the task of time-series classification for gains in classification performance that are achieved with far fewer model parameters than similar methods. It also introduces the attention mechanism to time-series classification that improves performance even further still, with significant improvement in computational efficiency, as well as further reduction in model size. Finally, this thesis pioneers the use of modern model compression techniques to the field of photometric classification for efficient deep learning deployment. These insights informed the final architecture which was deployed in a live production machine learning system, demonstrating the capability to operate efficiently and robustly in real-time, at LSST scale and beyond, ready for the new era of data intensive astronomy
Under construction: infrastructure and modern fiction
In this dissertation, I argue that infrastructural development, with its technological promises but widening geographic disparities and social and environmental consequences, informs both the narrative content and aesthetic forms of modernist and contemporary Anglophone fiction. Despite its prevalent material formsâroads, rails, pipes, and wiresâinfrastructure poses particular formal and narrative problems, often receding into the background as mere setting. To address how literary fiction theorizes the experience of infrastructure requires reading âinfrastructurallyâ: that is, paying attention to the seemingly mundane interactions between characters and their built environments. The writers central to this projectâJames Joyce, William Faulkner, Karen Tei Yamashita, and Mohsin Hamidâtake up the representational challenges posed by infrastructure by bringing transit networks, sanitation systems, and electrical grids and the histories of their development and use into the foreground. These writers call attention to the political dimensions of built environments, revealing the ways infrastructures produce, reinforce, and perpetuate racial and socioeconomic fault lines. They also attempt to formalize the material relations of power inscribed by and within infrastructure; the novel itself becomes an imaginary counterpart to the technologies of infrastructure, a form that shapes and constrains what types of social action and affiliation are possible
Prestressing wire breakage monitoring using sound event detection
Detecting prestressed wire breakage in concrete bridges is essential for ensuring safety and longevity and preventing catastrophic failures. This study proposes a novel approach for wire breakage detection using Mel-frequency cepstral coefficients (MFCCs) and back-propagation neural network (BPNN). Experimental data from two bridges in Italy were acquired to train and test the models. To overcome the limited availability of real-world training data, data augmentation techniques were employed to increase the data set size, enhancing the capability of the models and preventing over-fitting problems. The proposed method uses MFCCs to extract features from acoustic emission signals produced by wire breakage, which are then classified by the BPNN. The results show that the proposed method can detect and classify sound events effectively, demonstrating the promising potential of BPNN for real-time monitoring and diagnosis of bridges. The significance of this work lies in its contribution to improving bridge safety and preventing catastrophic failures. The combination of MFCCs and BPNN offers a new approach to wire breakage detection, while the use of real-world data and data augmentation techniques are significant contributions to overcoming the limited availability of training data. The proposed method has the potential to be a generalized and robust model for real-time monitoring of bridges, ultimately leading to safer and longer-lasting infrastructure
Improving the provision of services in the community for people with a sexual attraction to minors, people at risk of and/or who have perpetrated sexual abuse against children
The aim of this mixed methods research was to understand how the current provision of community-based services in the UK for people who have a sexual attraction to minors, people at risk of committing and/or who have committed sexual abuse against children can be improved. This is essential to enhance the wellbeing of this population and safeguard children. This study's quantitative phase included an online survey of 318 members of the general public, while the qualitative phase consisted in interviewing 20 people who work or have worked with people who are sexually attracted to minors and/or have committed or are at risk of committing sexual offences against children. According to the survey results, the general public supports a programme like the German Dunkelfeld (free, confidential and widely advertised service for people sexually attracted to minors), despite an undecided/slightly negative attitude towards people who commit sexual offences against children. Although, there is some variance in their opinions. In turn, the thematic analysis of the interviews revealed six themes: the level of acceptability of the programme is influenced by the level of knowledge on the topic; various large-scale strategies need to be implemented to ease the public attitudes; Dunkelfeld's confidentiality approach provides unique benefits; all-round responsivity is required; employees must possess a unique set of aptitudes; funding is essential, but this should not fall onto the service users. This research showed that a policy and practise model for programmes in the UK must consider current best practises while also addressing current shortcomings. First, programmes must be widely accepted and advertised in the community, so that people are aware of them and may use/recommend them. Second, they must secure adequate funding to ensure that supply meets demand. Third, Dunkelfeld's approach to confidentiality is a delicate subject that needs to be clarified but can be beneficial. Finally, a sex-positive approach may aid an all-round responsive programme's ability to become normalised and welcoming to diverse clients
Modular lifelong machine learning
Deep learning has drastically improved the state-of-the-art in many important fields, including computer vision and natural language processing (LeCun et al., 2015). However, it is expensive to train a deep neural network on a machine learning problem. The overall training cost further increases when one wants to solve additional problems. Lifelong machine learning (LML) develops algorithms that aim to efficiently learn to solve a sequence of problems, which become available one at a time. New problems are solved with less resources by transferring previously learned knowledge. At the same time, an LML algorithm needs to retain good performance on all encountered problems, thus avoiding catastrophic forgetting. Current approaches do not possess all the desired properties of an LML algorithm. First, they primarily focus on preventing catastrophic forgetting (Diaz-Rodriguez et al., 2018; Delange et al., 2021). As a result, they neglect some knowledge transfer properties. Furthermore, they assume that all problems in a sequence share the same input space. Finally, scaling these methods to a large sequence of problems remains a challenge.
Modular approaches to deep learning decompose a deep neural network into sub-networks, referred to as modules. Each module can then be trained to perform an atomic transformation, specialised in processing a distinct subset of inputs. This modular approach to storing knowledge makes it easy to only reuse the subset of modules which are useful for the task at hand.
This thesis introduces a line of research which demonstrates the merits of a modular approach to lifelong machine learning, and its ability to address the aforementioned shortcomings of other methods. Compared to previous work, we show that a modular approach can be used to achieve more LML properties than previously demonstrated. Furthermore, we develop tools which allow modular LML algorithms to scale in order to retain said properties on longer sequences of problems.
First, we introduce HOUDINI, a neurosymbolic framework for modular LML. HOUDINI represents modular deep neural networks as functional programs and accumulates a library of pre-trained modules over a sequence of problems. Given a new problem, we use program synthesis to select a suitable neural architecture, as well as a high-performing combination of pre-trained and new modules. We show that our approach has most of the properties desired from an LML algorithm. Notably, it can perform forward transfer, avoid negative transfer and prevent catastrophic forgetting, even across problems with disparate input domains and problems which require different neural architectures.
Second, we produce a modular LML algorithm which retains the properties of HOUDINI but can also scale to longer sequences of problems. To this end, we fix the choice of a neural architecture and introduce a probabilistic search framework, PICLE, for searching through different module combinations. To apply PICLE, we introduce two probabilistic models over neural modules which allows us to efficiently identify promising module combinations.
Third, we phrase the search over module combinations in modular LML as black-box optimisation, which allows one to make use of methods from the setting of hyperparameter optimisation (HPO). We then develop a new HPO method which marries a multi-fidelity approach with model-based optimisation. We demonstrate that this leads to improvement in anytime performance in the HPO setting and discuss how this can in turn be used to augment modular LML methods.
Overall, this thesis identifies a number of important LML properties, which have not all been attained in past methods, and presents an LML algorithm which can achieve all of them, apart from backward transfer
- âŠ