5,233 research outputs found

    Modular lifelong machine learning

    Get PDF
    Deep learning has drastically improved the state-of-the-art in many important fields, including computer vision and natural language processing (LeCun et al., 2015). However, it is expensive to train a deep neural network on a machine learning problem. The overall training cost further increases when one wants to solve additional problems. Lifelong machine learning (LML) develops algorithms that aim to efficiently learn to solve a sequence of problems, which become available one at a time. New problems are solved with less resources by transferring previously learned knowledge. At the same time, an LML algorithm needs to retain good performance on all encountered problems, thus avoiding catastrophic forgetting. Current approaches do not possess all the desired properties of an LML algorithm. First, they primarily focus on preventing catastrophic forgetting (Diaz-Rodriguez et al., 2018; Delange et al., 2021). As a result, they neglect some knowledge transfer properties. Furthermore, they assume that all problems in a sequence share the same input space. Finally, scaling these methods to a large sequence of problems remains a challenge. Modular approaches to deep learning decompose a deep neural network into sub-networks, referred to as modules. Each module can then be trained to perform an atomic transformation, specialised in processing a distinct subset of inputs. This modular approach to storing knowledge makes it easy to only reuse the subset of modules which are useful for the task at hand. This thesis introduces a line of research which demonstrates the merits of a modular approach to lifelong machine learning, and its ability to address the aforementioned shortcomings of other methods. Compared to previous work, we show that a modular approach can be used to achieve more LML properties than previously demonstrated. Furthermore, we develop tools which allow modular LML algorithms to scale in order to retain said properties on longer sequences of problems. First, we introduce HOUDINI, a neurosymbolic framework for modular LML. HOUDINI represents modular deep neural networks as functional programs and accumulates a library of pre-trained modules over a sequence of problems. Given a new problem, we use program synthesis to select a suitable neural architecture, as well as a high-performing combination of pre-trained and new modules. We show that our approach has most of the properties desired from an LML algorithm. Notably, it can perform forward transfer, avoid negative transfer and prevent catastrophic forgetting, even across problems with disparate input domains and problems which require different neural architectures. Second, we produce a modular LML algorithm which retains the properties of HOUDINI but can also scale to longer sequences of problems. To this end, we fix the choice of a neural architecture and introduce a probabilistic search framework, PICLE, for searching through different module combinations. To apply PICLE, we introduce two probabilistic models over neural modules which allows us to efficiently identify promising module combinations. Third, we phrase the search over module combinations in modular LML as black-box optimisation, which allows one to make use of methods from the setting of hyperparameter optimisation (HPO). We then develop a new HPO method which marries a multi-fidelity approach with model-based optimisation. We demonstrate that this leads to improvement in anytime performance in the HPO setting and discuss how this can in turn be used to augment modular LML methods. Overall, this thesis identifies a number of important LML properties, which have not all been attained in past methods, and presents an LML algorithm which can achieve all of them, apart from backward transfer

    Modelling, Monitoring, Control and Optimization for Complex Industrial Processes

    Get PDF
    This reprint includes 22 research papers and an editorial, collected from the Special Issue "Modelling, Monitoring, Control and Optimization for Complex Industrial Processes", highlighting recent research advances and emerging research directions in complex industrial processes. This reprint aims to promote the research field and benefit the readers from both academic communities and industrial sectors

    Affinity-Based Reinforcement Learning : A New Paradigm for Agent Interpretability

    Get PDF
    The steady increase in complexity of reinforcement learning (RL) algorithms is accompanied by a corresponding increase in opacity that obfuscates insights into their devised strategies. Methods in explainable artificial intelligence seek to mitigate this opacity by either creating transparent algorithms or extracting explanations post hoc. A third category exists that allows the developer to affect what agents learn: constrained RL has been used in safety-critical applications and prohibits agents from visiting certain states; preference-based RL agents have been used in robotics applications and learn state-action preferences instead of traditional reward functions. We propose a new affinity-based RL paradigm in which agents learn strategies that are partially decoupled from reward functions. Unlike entropy regularisation, we regularise the objective function with a distinct action distribution that represents a desired behaviour; we encourage the agent to act according to a prior while learning to maximise rewards. The result is an inherently interpretable agent that solves problems with an intrinsic affinity for certain actions. We demonstrate the utility of our method in a financial application: we learn continuous time-variant compositions of prototypical policies, each interpretable by its action affinities, that are globally interpretable according to customers’ financial personalities. Our method combines advantages from both constrained RL and preferencebased RL: it retains the reward function but generalises the policy to match a defined behaviour, thus avoiding problems such as reward shaping and hacking. Unlike Boolean task composition, our method is a fuzzy superposition of different prototypical strategies to arrive at a more complex, yet interpretable, strategy.publishedVersio

    Dynamical and functional mechanisms of visual attention

    Get PDF
    Attention is a crucial brain function for selectively processing behaviorally relevant stimuli over irrelevant ones. Several decades of psychophysical and neurophysiological studies have established that attention is highly dynamic and flexible, yet the mechanism underlying such flexible functionality remains unknown. In this thesis, we focus on the circuit and functional mechanisms of visual attention. We first investigate the flexible corticocortical communication mechanism by using a biophysically realistic large-scale circuit model consisting of sensory and association areas. We illustrate that spiking activity patterns with complex spatiotemporal dynamics emerging in both areas exhibit dynamically coordinated interactions, based on which the flexible gamma synchrony-based and subspace-based interareal communication can naturally emerge. We further demonstrate that such dynamic communication can be modulated during attention tasks and such modulated communication provides a mechanistic account for a great variety of neural effects of attention. We then apply this large-scale circuit model to attention-guided visual search, and illustrate that the interactions of bottom-up object saliency and top-down attention modulate the dynamics of spiking activity patterns; the modulation process can explain the hallmarks of neural and behavioral features of visual search, including the linear increase of reaction time with the number of search items. Finally, we identify a dynamical working regime of the visual cortex, in which a great variety of neural dynamics at the individual neuron and circuit levels can be reconciled and explained; these include super-Poisson spiking variability, theta-gamma coupling, and normalization

    Program and Proceedings: The Nebraska Academy of Sciences 1880-2023. 142th Anniversary Year. One Hundred-Thirty-Third Annual Meeting April 21, 2023. Hybrid Meeting: Nebraska Wesleyan University & Online, Lincoln, Nebraska

    Get PDF
    AERONAUTICS & SPACE SCIENCE Chairperson(s): Dr. Scott Tarry & Michaela Lucas HUMANS PAST AND PRESENT Chairperson(s): Phil R. Geib & Allegra Ward APPLIED SCIENCE & TECHNOLOGY SECTION Chairperson(s): Mary Ettel BIOLOGY Chairpersons: Lauren Gillespie, Steve Heinisch, and Paul Davis BIOMEDICAL SCIENCES Chairperson(s): Annemarie Shibata, Kimberly Carlson, Joseph Dolence, Alexis Hobbs, James Fletcher, Paul Denton CHEM Section Chairperson(s): Nathanael Fackler EARTH SCIENCES Chairpersons: Irina Filina, Jon Schueth, Ross Dixon, Michael Leite ENVIRONMENTAL SCIENCE Chairperson: Mark Hammer PHYSICS Chairperson(s): Dr. Adam Davis SCIENCE EDUCATION Chairperson: Christine Gustafson 2023 Maiben Lecturer: Jason Bartz 2023 FRIEND OF SCIENCE AWARD TO: Ray Ward and Jim Lewi

    Machine learning enabled millimeter wave cellular system and beyond

    Get PDF
    Millimeter-wave (mmWave) communication with advantages of abundant bandwidth and immunity to interference has been deemed a promising technology for the next generation network and beyond. With the help of mmWave, the requirements envisioned of the future mobile network could be met, such as addressing the massive growth required in coverage, capacity as well as traffic, providing a better quality of service and experience to users, supporting ultra-high data rates and reliability, and ensuring ultra-low latency. However, due to the characteristics of mmWave, such as short transmission distance, high sensitivity to the blockage, and large propagation path loss, there are some challenges for mmWave cellular network design. In this context, to enjoy the benefits from the mmWave networks, the architecture of next generation cellular network will be more complex. With a more complex network, it comes more complex problems. The plethora of possibilities makes planning and managing a complex network system more difficult. Specifically, to provide better Quality of Service and Quality of Experience for users in the such network, how to provide efficient and effective handover for mobile users is important. The probability of handover trigger will significantly increase in the next generation network, due to the dense small cell deployment. Since the resources in the base station (BS) is limited, the handover management will be a great challenge. Further, to generate the maximum transmission rate for the users, Line-of-sight (LOS) channel would be the main transmission channel. However, due to the characteristics of mmWave and the complexity of the environment, LOS channel is not feasible always. Non-line-of-sight channel should be explored and used as the backup link to serve the users. With all the problems trending to be complex and nonlinear, and the data traffic dramatically increasing, the conventional method is not effective and efficiency any more. In this case, how to solve the problems in the most efficient manner becomes important. Therefore, some new concepts, as well as novel technologies, require to be explored. Among them, one promising solution is the utilization of machine learning (ML) in the mmWave cellular network. On the one hand, with the aid of ML approaches, the network could learn from the mobile data and it allows the system to use adaptable strategies while avoiding unnecessary human intervention. On the other hand, when ML is integrated in the network, the complexity and workload could be reduced, meanwhile, the huge number of devices and data could be efficiently managed. Therefore, in this thesis, different ML techniques that assist in optimizing different areas in the mmWave cellular network are explored, in terms of non-line-of-sight (NLOS) beam tracking, handover management, and beam management. To be specific, first of all, a procedure to predict the angle of arrival (AOA) and angle of departure (AOD) both in azimuth and elevation in non-line-of-sight mmWave communications based on a deep neural network is proposed. Moreover, along with the AOA and AOD prediction, a trajectory prediction is employed based on the dynamic window approach (DWA). The simulation scenario is built with ray tracing technology and generate data. Based on the generated data, there are two deep neural networks (DNNs) to predict AOA/AOD in the azimuth (AAOA/AAOD) and AOA/AOD in the elevation (EAOA/EAOD). Furthermore, under an assumption that the UE mobility and the precise location is unknown, UE trajectory is predicted and input into the trained DNNs as a parameter to predict the AAOA/AAOD and EAOA/EAOD to show the performance under a realistic assumption. The robustness of both procedures is evaluated in the presence of errors and conclude that DNN is a promising tool to predict AOA and AOD in a NLOS scenario. Second, a novel handover scheme is designed aiming to optimize the overall system throughput and the total system delay while guaranteeing the quality of service (QoS) of each user equipment (UE). Specifically, the proposed handover scheme called O-MAPPO integrates the reinforcement learning (RL) algorithm and optimization theory. An RL algorithm known as multi-agent proximal policy optimization (MAPPO) plays a role in determining handover trigger conditions. Further, an optimization problem is proposed in conjunction with MAPPO to select the target base station and determine beam selection. It aims to evaluate and optimize the system performance of total throughput and delay while guaranteeing the QoS of each UE after the handover decision is made. Third, a multi-agent RL-based beam management scheme is proposed, where multiagent deep deterministic policy gradient (MADDPG) is applied on each small-cell base station (SCBS) to maximize the system throughput while guaranteeing the quality of service. With MADDPG, smart beam management methods can serve the UEs more efficiently and accurately. Specifically, the mobility of UEs causes the dynamic changes of the network environment, the MADDPG algorithm learns the experience of these changes. Based on that, the beam management in the SCBS is optimized according the reward or penalty when severing different UEs. The approach could improve the overall system throughput and delay performance compared with traditional beam management methods. The works presented in this thesis demonstrate the potentiality of ML when addressing the problem from the mmWave cellular network. Moreover, it provides specific solutions for optimizing NLOS beam tracking, handover management and beam management. For NLOS beam tracking part, simulation results show that the prediction errors of the AOA and AOD can be maintained within an acceptable range of ±2. Further, when it comes to the handover optimization part, the numerical results show the system throughput and delay are improved by 10% and 25%, respectively, when compared with two typical RL algorithms, Deep Deterministic Policy Gradient (DDPG) and Deep Q-learning (DQL). Lastly, when it considers the intelligent beam management part, numerical results reveal the convergence performance of the MADDPG and the superiority in improving the system throughput compared with other typical RL algorithms and the traditional beam management method

    The bone microenvironment as a master regulator of disseminated tumour cells responsible for breast cancer recurrence

    Get PDF
    Advanced breast cancer is frequently associated with skeletal metastases characterised by dormancy and subsequent incurable metastatic outgrowth accompanied by skeletal related events. Disseminated tumour cells putatively assume residence in metastatic niches within the bone microenvironment, in locations that may be regularly occupied by resident cells and regulated by neighbouring cells in response to local and systemic signals. My studies explored how modification of the bone microenvironment using a dietary approach (low calcium), a surgical intervention (ovariectomy) and pharmacological inhibition of bone resorption (Zol) impacted breast cancer cell development and progression within the bone in vivo. The effect of a low calcium diet (0.1%) on the bone microenvironment of mature (12-week old) and dormant disseminated tumour cells was investigated in the absence and presence of bone-colonising tumour cells. In the absence of tumour cells, minor reductions of percent bone volume was detected after 28 days, with a reduction in PINP and increase in TRAP, without significant change to gene expression. Without existing literature on the effect of a low calcium diet on the outgrowth of dormant disseminated breast cancer cells, my research showed a low calcium diet in isolation did not sufficiently alter the bone microenvironment to trigger the outgrowth of dormant tumour cells. To investigate the effect of repeated doses of Zol on outgrowing tumour cells in bone with rapid turnover, young (6-week old) mice were treated with four once-weekly clinically-relevant doses. Zol significantly increased trabecular percent bone volume, trabecular number and reduced trabecular separation. In agreement with existing literature Zol did not prevent or significantly delay the outgrowth of tumours in the bone with rapid turnover. Profiling transcriptional changes to the mature (12-week old) bone microenvironment 14 days after ovariectomy, which has been shown to induce the outgrowth of dormant disseminated tumour cells, by RNA-seq revealed POSTN, MMP2, THBS2 and OPN as genes influencing the altered bone microenvironment. Comparison with publicly available RNA-seq data on the bone microenvironment of young (~30 years old), old (~73 years old) and old women treated with oestrogen for 3 weeks (~ 70 years old) found overlap between genes expressed by both young women and old women, suggesting caution in comparison. Comparing genes altered in the bone microenvironment of ovariectomised mice did not show overlap with genes found from in vitro dormant tumour cells. Deconvolution of bulk RNA-seq data to infer cell types did not show any significant differences. Further studies on dormancy gene signatures within the bone microenvironment are required
    • …
    corecore