Cyber-Physical Embedded Systems with Transient Supervisory Command and Control: A Framework for Validating Safety Response in Automated Collision Avoidance Systems

The ability to design and engineer complex and dynamical Cyber-Physical Systems (CPS) requires a systematic view that requires a definition of level of automation intent for the system. Since CPS covers a diverse range of systemized implementations of smart and intelligent technologies networked within a system of systems (SoS), the terms “smart” and “intelligent” is frequently used in describing systems that perform complex operations with a reduced need of a human-agent. The difference between this research and most papers in publication on CPS is that most other research focuses on the performance of the CPS rather than on the correctness of its design. However, by using both human and machine agency at different levels of automation, or autonomy, the levels of automation have profound implications and affects to the reliability and safety of the CPS. The human-agent and the machine-agent are in a tidal lock of decision-making using both feedforward and feedback information flows in similar processes, where a transient shift within the level of automation when the CPS is operating can have undesired consequences. As CPS systems become more common, and higher levels of autonomy are embedded within them, the relationship between human-agent and machine-agent also becomes more complex, and the testing methodologies for verification and validation of performance and correctness also become more complex and less clear. A framework then is developed to help the practitioner to understand the difficulties and pitfalls of CPS designs and provides guidance to test engineering design of soft computational systems using combinations of modeling, simulation, and prototyping

Reinforcement Learning with Quantum Variational Circuits

The development of quantum computational techniques has advanced greatly in recent years, parallel to the advancements in techniques for deep reinforcement learning. This work explores the potential for quantum computing to facilitate reinforcement learning problems. Quantum computing approaches offer important potential improvements in time and space complexity over traditional algorithms because of its ability to exploit the quantum phenomena of superposition and entanglement. Specifically, we investigate the use of quantum variational circuits, a form of quantum machine learning. We present our techniques for encoding classical data for a quantum variational circuit, we further explore pure and hybrid quantum algorithms for DQN and Double DQN. Our results indicate both hybrid and pure quantum variational circuit have the ability to solve reinforcement learning tasks with a smaller parameter space. These comparison are conducted with two OpenAI Gym environments: CartPole and Blackjack, The success of this work is indicative of a strong future relationship between quantum machine learning and deep reinforcement learning.Comment: Accepted to AIIDE 2020 Updated to better reflect AAAI formattin

Goal-Directed Decision Making with Spiking Neurons.

UNLABELLED: Behavioral and neuroscientific data on reward-based decision making point to a fundamental distinction between habitual and goal-directed action selection. The formation of habits, which requires simple updating of cached values, has been studied in great detail, and the reward prediction error theory of dopamine function has enjoyed prominent success in accounting for its neural bases. In contrast, the neural circuit mechanisms of goal-directed decision making, requiring extended iterative computations to estimate values online, are still unknown. Here we present a spiking neural network that provably solves the difficult online value estimation problem underlying goal-directed decision making in a near-optimal way and reproduces behavioral as well as neurophysiological experimental data on tasks ranging from simple binary choice to sequential decision making. Our model uses local plasticity rules to learn the synaptic weights of a simple neural network to achieve optimal performance and solves one-step decision-making tasks, commonly considered in neuroeconomics, as well as more challenging sequential decision-making tasks within 1 s. These decision times, and their parametric dependence on task parameters, as well as the final choice probabilities match behavioral data, whereas the evolution of neural activities in the network closely mimics neural responses recorded in frontal cortices during the execution of such tasks. Our theory provides a principled framework to understand the neural underpinning of goal-directed decision making and makes novel predictions for sequential decision-making tasks with multiple rewards. SIGNIFICANCE STATEMENT: Goal-directed actions requiring prospective planning pervade decision making, but their circuit-level mechanisms remain elusive. We show how a model circuit of biologically realistic spiking neurons can solve this computationally challenging problem in a novel way. The synaptic weights of our network can be learned using local plasticity rules such that its dynamics devise a near-optimal plan of action. By systematically comparing our model results to experimental data, we show that it reproduces behavioral decision times and choice probabilities as well as neural responses in a rich set of tasks. Our results thus offer the first biologically realistic account for complex goal-directed decision making at a computational, algorithmic, and implementational level.This research was supported by the Swiss National Science Foundation (J.F., Grant PBBEP3 146112) and the Wellcome Trust (J.F. and M.L.).This is the author accepted manuscript. It is currently under an indefinite embargo pending publication by the Society for Neuroscience

Gambling risk perception and decision making

Cognitive and biopsychological research has identified a significant relationship between perception, decision making and the negative consequences associated with sustained gambling. Drug and alcohol research suggests that how individuals navigate decisions involving motivating but risky activities involves several important, distinct, but interrelated aspects of cognition. Nevertheless, risk perception and decision making has received little attention in the gambling literature. The aim of the current thesis therefore was to investigate risk perception in gambling, and to develop a model of gambling decision making mindful of risk perception concepts. The project applied the Mental Models methodology and included: a literature review, a qualitative study evaluating expert opinions regarding gambling risk decision making, a second qualitative study evaluating lay gambler mental models of risk, and a quantitative evaluation of risk perception and decision making concepts via a self-report questionnaire. Data from all phases of the project were used to develop an assessment tool (Gambling Risk Decisions Questionnaire) and theoretical model of gambling risk decision making. It was anticipated that understanding the processes by which risk perception predisposes an individual to maintain gambling despite adverse consequences would act as an invaluable guide for preventative educational campaigns, clinical treatment, and social policy interventions. Taken together, results of the four studies confirmed the importance of relationships between decision making, behaviour, consequences, and disorder, with disorder largely predictable based on several core decision making factors, despite individual variation in clinical presentation

Gambling risk perception and decision making

Cognitive and biopsychological research has identified a significant relationship between perception, decision making and the negative consequences associated with sustained gambling. Drug and alcohol research suggests that how individuals navigate decisions involving motivating but risky activities involves several important, distinct, but interrelated aspects of cognition. Nevertheless, risk perception and decision making has received little attention in the gambling literature. The aim of the current thesis therefore was to investigate risk perception in gambling, and to develop a model of gambling decision making mindful of risk perception concepts. The project applied the Mental Models methodology and included: a literature review, a qualitative study evaluating expert opinions regarding gambling risk decision making, a second qualitative study evaluating lay gambler mental models of risk, and a quantitative evaluation of risk perception and decision making concepts via a self-report questionnaire. Data from all phases of the project were used to develop an assessment tool (Gambling Risk Decisions Questionnaire) and theoretical model of gambling risk decision making. It was anticipated that understanding the processes by which risk perception predisposes an individual to maintain gambling despite adverse consequences would act as an invaluable guide for preventative educational campaigns, clinical treatment, and social policy interventions. Taken together, results of the four studies confirmed the importance of relationships between decision making, behaviour, consequences, and disorder, with disorder largely predictable based on several core decision making factors, despite individual variation in clinical presentation

Automatic detection of problem-gambling signs from online texts using large language models

Problem gambling is a major public health concern and is associated with profound psychological distress and economic problems. There are numerous gambling communities on the internet where users exchange information about games, gambling tactics, as well as gambling-related problems. Individuals exhibiting higher levels of problem gambling engage more in such communities. Online gambling communities may provide insights into problem-gambling behaviour. Using data scraped from a major German gambling discussion board, we fine-tuned a large language model, specifically a Bidirectional Encoder Representations from Transformers (BERT) model, to predict signs of problem-gambling from forum posts. Training data were generated by manual annotation and by taking into account diagnostic criteria and gambling-related cognitive distortions. Using k-fold cross-validation, our models achieved a precision of 0.95 and F1 score of 0.71, demonstrating that satisfactory classification performance can be achieved by generating high-quality training material through manual annotation based on diagnostic criteria. The current study confirms that a BERT-based model can be reliably used on small data sets and to detect signatures of problem gambling in online communication data. Such computational approaches may have potential for the detection of changes in problem-gambling prevalence among online users

The Neural Basis of Decision-Making and Reward Processing in Adults with Euthymic Bipolar Disorder or Attention-Deficit/Hyperactivity Disorder (ADHD)

Attention-deficit/hyperactivity disorder (ADHD) and bipolar disorder (BD) share DSM-IV criteria in adults and cause problems in decision-making. Nevertheless, no previous report has assessed a decision-making task that includes the examination of the neural correlates of reward and gambling in adults with ADHD and those with BD