1,511 research outputs found

    Multivariate Modeling of Quasar Variability with an Attention-based Variational Autoencoder

    Get PDF
    This thesis applied HeTVAE, an attention-based VAE neural network capable of multivariate modeling of time series, to a dataset of several thousand multi-band AGN light curves from ZTF and was one of the first attempts to use a neural network to harness the stochastic light curves in their multivariate form. Whereas standard models of AGN variability make prior assumptions, HeTVAE uses no prior knowledge and is able to learn the data distribution in a regularized latent space, reading semantic information via its up-to-date self-supervised training regimen. We have successfully created a dataset class for preprocessing the irregular multivariate time series and in order to interface with the quasi-off-the-shelf network more conveniently. Also, we have trained several different model iterations using one, two or all three of the filter dimensions from ZTF on Durham’s NCC compute cluster, while configuring useful hyper parameter choices to work robustly for the astronomical dataset. In the network's training, we employed the Adam optimizer with a reduce-on-plateau learning rate schedule and a KL-annealing schedule optimize the VAE’s performance. In experimenting, we show how the VAE has learned the data distribution of the light curves by generating simulated light curves and its interpretability by visualizing attention scores and by visualizing the way the light curves are distributed along the continuous latent space using PCA. We show it orders the light curves across a smooth gradient from those those that have both low amplitude short-term variation and high amplitude long-term variation, to those with little variability, to those with both short-term and long-term high-amplitude variation in the condensed space. We also use PCA to display a potential filtering algorithm that enables parsing through large datasets in an intuitive way and present some of the pitfalls of algorithmic bias in anomaly detection. Finally, we fine-tuned the structurally correct but imprecise multivariate interpolations output by HeTVAE to three objects to show how they could improve constraints on time-delay estimates in the context of reverberation mapping for the relatively poor-cadenced ZTF data. In short, HeTVAE's use cases are ranged and it is a step in the right direction as far as being able to help organize and process the millions of AGN light curves incoming from Vera C. Rubin Observatory’s Legacy Survey of Space and Time in their full 6 optical broadband filter multivariate form

    An examination of the verbal behaviour of intergroup discrimination

    Get PDF
    This thesis examined relationships between psychological flexibility, psychological inflexibility, prejudicial attitudes, and dehumanization across three cross-sectional studies with an additional proposed experimental study. Psychological flexibility refers to mindful attention to the present moment, willing acceptance of private experiences, and engaging in behaviours congruent with one’s freely chosen values. Inflexibility, on the other hand, indicates a tendency to suppress unwanted thoughts and emotions, entanglement with one’s thoughts, and rigid behavioural patterns. Study 1 found limited correlations between inflexibility and sexism, racism, homonegativity, and dehumanization. Study 2 demonstrated more consistent positive associations between inflexibility and prejudice. And Study 3 controlled for right-wing authoritarianism and social dominance orientation, finding inflexibility predicted hostile sexism and racism beyond these factors. While showing some relationships, particularly with sexism and racism, psychological inflexibility did not consistently correlate with varied prejudices across studies. The proposed randomized controlled trial aims to evaluate an Acceptance and Commitment Therapy intervention to reduce sexism through enhanced psychological flexibility. Overall, findings provide mixed support for the utility of flexibility-based skills in addressing complex societal prejudices. Research should continue examining flexibility integrated with socio-cultural approaches to promote equity

    Robustness, Heterogeneity and Structure Capturing for Graph Representation Learning and its Application

    Get PDF
    Graph neural networks (GNNs) are potent methods for graph representation learn- ing (GRL), which extract knowledge from complicated (graph) structured data in various real-world scenarios. However, GRL still faces many challenges. Firstly GNN-based node classification may deteriorate substantially by overlooking the pos- sibility of noisy data in graph structures, as models wrongly process the relation among nodes in the input graphs as the ground truth. Secondly, nodes and edges have different types in the real-world and it is essential to capture this heterogeneity in graph representation learning. Next, relations among nodes are not restricted to pairwise relations and it is necessary to capture the complex relations accordingly. Finally, the absence of structural encodings, such as positional information, deterio- rates the performance of GNNs. This thesis proposes novel methods to address the aforementioned problems: 1. Bayesian Graph Attention Network (BGAT): Developed for situations with scarce data, this method addresses the influence of spurious edges. Incor- porating Bayesian principles into the graph attention mechanism enhances robustness, leading to competitive performance against benchmarks (Chapter 3). 2. Neighbour Contrastive Heterogeneous Graph Attention Network (NC-HGAT): By enhancing a cutting-edge self-supervised heterogeneous graph neural net- work model (HGAT) with neighbour contrastive learning, this method ad- dresses heterogeneity and uncertainty simultaneously. Extra attention to edge relations in heterogeneous graphs also aids in subsequent classification tasks (Chapter 4). 3. A novel ensemble learning framework is introduced for predicting stock price movements. It adeptly captures both group-level and pairwise relations, lead- ing to notable advancements over the existing state-of-the-art. The integration of hypergraph and graph models, coupled with the utilisation of auxiliary data via GNNs before recurrent neural network (RNN), provides a deeper under- standing of long-term dependencies between similar entities in multivariate time series analysis (Chapter 5). 4. A novel framework for graph structure learning is introduced, segmenting graphs into distinct patches. By harnessing the capabilities of transformers and integrating other position encoding techniques, this approach robustly capture intricate structural information within a graph. This results in a more comprehensive understanding of its underlying patterns (Chapter 6)

    Adaptive migratory orientation of an invasive pest on a new continent

    Get PDF
    This is the final version. Available on open access from Cell Press via the DOI in this recordData and code availability: The authors declare that the data supporting the findings of this study are available within the paper and its Supplemental Information. Individual data files used to generate the figures in the paper have been deposited in Mendeley Data (https://doi.org/10.17632/6jkvpybswd.1). Custom-written Visual Basic software Flash Flight Simulator Data Acquisition System and Mapflows have been deposited in Mendeley Data (https://doi.org/10.17632/6jkvpybswd.1). Custom single-chip used for data acquisition is available from the corresponding authors upon reasonable request ([email protected], [email protected]). Any additional information required to reanalyze the data reported in this work paper is available from the lead contact upon request.Many species of insects undertake long-range, seasonally reversed migrations, displaying sophisticated orientation behaviors to optimize their migratory trajectories. However, when invasive insects arrive in new biogeographical regions, it is unclear if migrants retain (or how quickly they regain) ancestral migratory traits, such as seasonally preferred flight headings. Here we present behavioral evidence that an invasive migratory pest, the fall armyworm moth (Spodoptera frugiperda), a native of the Americas, exhibited locally adaptive migratory orientation less than three years after arriving on a new continent. Specimens collected from China showed flight orientations directed north-northwest in spring and southwest in autumn, and this would promote seasonal forward and return migrations in East Asia. We also show that the driver of the seasonal switch in orientation direction is photoperiod. Our results thus provide a clear example of an invasive insect that has rapidly exhibited adaptive migratory behaviors, either inherited or newly evolved, in a completely alien environment.National Key Research and Development Program of Chin

    A Survey on Few-Shot Class-Incremental Learning

    Get PDF
    Large deep learning models are impressive, but they struggle when real-time data is not available. Few-shot class-incremental learning (FSCIL) poses a significant challenge for deep neural networks to learn new tasks from just a few labeled samples without forgetting the previously learned ones. This setup can easily leads to catastrophic forgetting and overfitting problems, severely affecting model performance. Studying FSCIL helps overcome deep learning model limitations on data volume and acquisition time, while improving practicality and adaptability of machine learning models. This paper provides a comprehensive survey on FSCIL. Unlike previous surveys, we aim to synthesize few-shot learning and incremental learning, focusing on introducing FSCIL from two perspectives, while reviewing over 30 theoretical research studies and more than 20 applied research studies. From the theoretical perspective, we provide a novel categorization approach that divides the field into five subcategories, including traditional machine learning methods, meta learning-based methods, feature and feature space-based methods, replay-based methods, and dynamic network structure-based methods. We also evaluate the performance of recent theoretical research on benchmark datasets of FSCIL. From the application perspective, FSCIL has achieved impressive achievements in various fields of computer vision such as image classification, object detection, and image segmentation, as well as in natural language processing and graph. We summarize the important applications. Finally, we point out potential future research directions, including applications, problem setups, and theory development. Overall, this paper offers a comprehensive analysis of the latest advances in FSCIL from a methodological, performance, and application perspective

    LIPIcs, Volume 251, ITCS 2023, Complete Volume

    Get PDF
    LIPIcs, Volume 251, ITCS 2023, Complete Volum

    Safe Collaborative Filtering

    Full text link
    Excellent tail performance is crucial for modern machine learning tasks, such as algorithmic fairness, class imbalance, and risk-sensitive decision making, as it ensures the effective handling of challenging samples within a dataset. Tail performance is also a vital determinant of success for personalised recommender systems to reduce the risk of losing users with low satisfaction. This study introduces a "safe" collaborative filtering method that prioritises recommendation quality for less-satisfied users rather than focusing on the average performance. Our approach minimises the conditional value at risk (CVaR), which represents the average risk over the tails of users' loss. To overcome computational challenges for web-scale recommender systems, we develop a robust yet practical algorithm that extends the most scalable method, implicit alternating least squares (iALS). Empirical evaluation on real-world datasets demonstrates the excellent tail performance of our approach while maintaining competitive computational efficiency

    Differences in well-being:the biological and environmental causes, related phenotypes, and real-time assessment

    Get PDF
    Well-being is a complex, and multifaceted construct that includes feeling good and functioning well. There is a growing global recognition of well-being as an important research topic and public policy goal. Well-being is related to less behavioral and emotional problems, and is associated with many positive aspects of daily life, including longevity, higher educational achievement, happier marriage, and more productivity at work. People differ in their levels of well-being, i.e., some people are in general happier or more satisfied with their lives than others. These individual differences in well-being can arise from many different factors, including biological (genetic) influences and environmental influences. To enhance the development of future mental health prevention and intervention strategies to increase well-being, more knowledge about these determinants and factors underlying well-being is needed. In this dissertation, I aimed to increase the understanding of the etiology in a series of studies using different methods, including systematic reviews, meta-analyses, twin designs, and molecular genetic designs. In part I, we brought together all published studies on the neural and physiological factors underlying well-being. This overview allowed us to critically investigate the claims made about the biology involved in well-being. The number of studies on the neural and physiological factors underlying well-being is increasing and the results point towards potential correlates of well-being. However, samples are often still small, and studies focus mostly on a single biomarker. Therefore, more well-powered, data-driven, and integrative studies across biological categories are needed to better understand the neural and physiological pathways that play a role in well-being. In part II, we investigated the overlap between well-being and a range of other phenotypes to learn more about the etiology of well-being. We report a large overlap with phenotypes including optimism, resilience, and depressive symptoms. Furthermore, when removing the genetic overlap between well-being and depressive symptoms, we showed that well-being has unique genetic associations with a range of phenotypes, independently from depressive symptoms. These results can be helpful in designing more effective interventions to increase well-being, taking into account the overlap and possible causality with other phenotypes. In part III, we used the extreme environmental change during the COVID-19 pandemic to investigate individual differences in the effects of such environmental changes on well-being. On average, we found a negative effect of the pandemic on different aspects of well-being, especially further into the pandemic. Whereas most previous studies only looked at this average negative effect of the pandemic on well-being, we focused on the individual differences as well. We reported large individual differences in the effects of the pandemic on well-being in both chapters. This indicates that one-size-fits-all preventions or interventions to maintain or increase well-being during the pandemic or lockdowns will not be successful for the whole population. Further research is needed for the identification of protective factors and resilience mechanisms to prevent further inequality during extreme environmental situations. In part IV, we looked at the real-time assessment of well-being, investigating the feasibility and results of previous studies. The real-time assessment of well-being, related variables, and the environment can lead to new insights about well-being, i.e., results that we cannot capture with traditional survey research. The real-time assessment of well-being is therefore a promising area for future research to unravel the dynamic nature of well-being fluctuations and the interaction with the environment in daily life. Integrating all results in this dissertation confirmed that well-being is a complex human trait that is influenced by many interrelated and interacting factors. Future directions to understand individual differences in well-being will be a data-driven approach to investigate the complex interplay of neural, physiological, genetic, and environmental factors in well-being

    Search to Fine-tune Pre-trained Graph Neural Networks for Graph-level Tasks

    Full text link
    Recently, graph neural networks (GNNs) have shown its unprecedented success in many graph-related tasks. However, GNNs face the label scarcity issue as other neural networks do. Thus, recent efforts try to pre-train GNNs on a large-scale unlabeled graph and adapt the knowledge from the unlabeled graph to the target downstream task. The adaptation is generally achieved by fine-tuning the pre-trained GNNs with a limited number of labeled data. Despite the importance of fine-tuning, current GNNs pre-training works often ignore designing a good fine-tuning strategy to better leverage transferred knowledge and improve the performance on downstream tasks. Only few works start to investigate a better fine-tuning strategy for pre-trained GNNs. But their designs either have strong assumptions or overlook the data-aware issue for various downstream datasets. Therefore, we aim to design a better fine-tuning strategy for pre-trained GNNs to improve the model performance in this paper. Given a pre-trained GNN, we propose to search to fine-tune pre-trained graph neural networks for graph-level tasks (S2PGNN), which adaptively design a suitable fine-tuning framework for the given labeled data on the downstream task. To ensure the improvement brought by searching fine-tuning strategy, we carefully summarize a proper search space of fine-tuning framework that is suitable for GNNs. The empirical studies show that S2PGNN can be implemented on the top of 10 famous pre-trained GNNs and consistently improve their performance. Besides, S2PGNN achieves better performance than existing fine-tuning strategies within and outside the GNN area. Our code is publicly available at \url{https://anonymous.4open.science/r/code_icde2024-A9CB/}

    Learning and Control of Dynamical Systems

    Get PDF
    Despite the remarkable success of machine learning in various domains in recent years, our understanding of its fundamental limitations remains incomplete. This knowledge gap poses a grand challenge when deploying machine learning methods in critical decision-making tasks, where incorrect decisions can have catastrophic consequences. To effectively utilize these learning-based methods in such contexts, it is crucial to explicitly characterize their performance. Over the years, significant research efforts have been dedicated to learning and control of dynamical systems where the underlying dynamics are unknown or only partially known a priori, and must be inferred from collected data. However, much of these classical results have focused on asymptotic guarantees, providing limited insights into the amount of data required to achieve desired control performance while satisfying operational constraints such as safety and stability, especially in the presence of statistical noise. In this thesis, we study the statistical complexity of learning and control of unknown dynamical systems. By utilizing recent advances in statistical learning theory, high-dimensional statistics, and control theoretic tools, we aim to establish a fundamental understanding of the number of samples required to achieve desired (i) accuracy in learning the unknown dynamics, (ii) performance in the control of the underlying system, and (iii) satisfaction of the operational constraints such as safety and stability. We provide finite-sample guarantees for these objectives and propose efficient learning and control algorithms that achieve the desired performance at these statistical limits in various dynamical systems. Our investigation covers a broad range of dynamical systems, starting from fully observable linear dynamical systems to partially observable linear dynamical systems, and ultimately, nonlinear systems. We deploy our learning and control algorithms in various adaptive control tasks in real-world control systems and demonstrate their strong empirical performance along with their learning, robustness, and stability guarantees. In particular, we implement one of our proposed methods, Fourier Adaptive Learning and Control (FALCON), on an experimental aerodynamic testbed under extreme turbulent flow dynamics in a wind tunnel. The results show that FALCON achieves state-of-the-art stabilization performance and consistently outperforms conventional and other learning-based methods by at least 37%, despite using 8 times less data. The superior performance of FALCON arises from its physically and theoretically accurate modeling of the underlying nonlinear turbulent dynamics, which yields rigorous finite-sample learning and performance guarantees. These findings underscore the importance of characterizing the statistical complexity of learning and control of unknown dynamical systems.</p
    • …
    corecore