Optimizing the depth and the direction of prospective planning using information values

Evaluating the future consequences of actions is achievable by simulating a mental search tree into the future. Expanding deep trees, however, is computationally taxing. Therefore, machines and humans use a plan-until-habit scheme that simulates the environment up to a limited depth and then exploits habitual values as proxies for consequences that may arise in the future. Two outstanding questions in this scheme are “in which directions the search tree should be expanded?”, and “when should the expansion stop?”. Here we propose a principled solution to these questions based on a speed/accuracy tradeoff: deeper expansion in the appropriate directions leads to more accurate planning, but at the cost of slower decision-making. Our simulation results show how this algorithm expands the search tree effectively and efficiently in a grid-world environment. We further show that our algorithm can explain several behavioral patterns in animals and humans, namely the effect of time-pressure on the depth of planning, the effect of reward magnitudes on the direction of planning, and the gradual shift from goal-directed to habitual behavior over the course of training. The algorithm also provides several predictions testable in animal/human experiments

Bandit Models of Human Behavior: Reward Processing in Mental Disorders

Drawing an inspiration from behavioral studies of human decision making, we propose here a general parametric framework for multi-armed bandit problem, which extends the standard Thompson Sampling approach to incorporate reward processing biases associated with several neurological and psychiatric conditions, including Parkinson's and Alzheimer's diseases, attention-deficit/hyperactivity disorder (ADHD), addiction, and chronic pain. We demonstrate empirically that the proposed parametric approach can often outperform the baseline Thompson Sampling on a variety of datasets. Moreover, from the behavioral modeling perspective, our parametric framework can be viewed as a first step towards a unifying computational model capturing reward processing abnormalities across multiple mental conditions.Comment: Conference on Artificial General Intelligence, AGI-1

Performance robustness analysis in machine-assisted design of photonic devices

Machine-assisted design of integrated photonic devices (e.g. through optimization and inverse design methods) is opening the possibility of exploring very large design spaces, novel functionalities and non-intuitive geometries. These methods are generally used to optimize performance figures-of-merit. On the other hand, the effect of manufacturing variability remains a fundamental challenge since small fabrication errors can have a significant impact on light propagation, especially in high-index-contrast platforms. Brute-force analysis of these variabilities during the main optimization process can become prohibitive, since a large number of simulations would be required. To this purpose, efficient stochastic techniques integrated in the design cycle allow to quickly assess the performance robustness and the expected fabrication yield of each tentative device generated by the optimization. In this invited talk we present an overview of the recent advances in the implementation of stochastic techniques in photonics, focusing in particular on stochastic spectral methods that have been regarded as a promising alternative to the classical Monte Carlo method. Polynomial chaos expansion techniques generate so called surrogate models by means of an orthogonal set of polynomials to efficiently represent the dependence of a function to statistical variabilities. They achieve a considerable reduction of the simulation time compared to Monte Carlo, at least for mid-scale problems, making feasible the incorporation of tolerance analysis and yield optimization within the photonic design flow

NK cells armed with chimeric antigen receptors (CAR): roadblocks to successful development

In recent years, cell-based immunotherapies have demonstrated promising results in the treatment of cancer. Chimeric antigen receptors (CARs) arm effector cells with a weapon for targeting tumor antigens, licensing engineered cells to recognize and kill cancer cells. The quality of the CAR-antigen interaction strongly depends on the selected tumor antigen and its expression density on cancer cells. CD19 CAR-engineered T cells approved by the Food and Drug Administration have been most frequently applied in the treatment of hematological malignancies. Clinical challenges in their application primarily include cytokine release syndrome, neurological symptoms, severe inflammatory responses, and/or other off-target effects most likely mediated by cytotoxic T cells. As a consequence, there remains a significant medical need for more potent technology platforms leveraging cell-based approaches with enhanced safety profiles. A promising population that has been advanced is the natural killer (NK) cell, which can also be engineered with CARs. NK cells which belong to the innate arm of the immune system recognize and kill virally infected cells as well as (stressed) cancer cells in a major histocompatibility complex I independent manner. NK cells play an important role in the host’s immune defense against cancer due to their specialized lytic mechanisms which include death receptor (i.e., Fas)/death receptor ligand (i.e., Fas ligand) and granzyme B/perforin-mediated apoptosis, and antibody-dependent cellular cytotoxicity, as well as their immunoregulatory potential via cytokine/chemokine release. To develop and implement a highly effective CAR NK cell-based therapy with low side effects, the following three principles which are specifically addressed in this review have to be considered: unique target selection, well-designed CAR, and optimized gene delivery

Imbalanced decision hierarchy in addicts emerging from drug-hijacked dopamine spiraling circuit

Despite explicitly wanting to quit, long-term addicts find themselves powerless to resist drugs, despite knowing that drug-taking may be a harmful course of action. Such inconsistency between the explicit knowledge of negative consequences and the compulsive behavioral patterns represents a cognitive/behavioral conflict that is a central characteristic of addiction. Neurobiologically, differential cue-induced activity in distinct striatal subregions, as well as the dopamine connectivity spiraling from ventral striatal regions to the dorsal regions, play critical roles in compulsive drug seeking. However, the functional mechanism that integrates these neuropharmacological observations with the above-mentioned cognitive/behavioral conflict is unknown. Here we provide a formal computational explanation for the drug-induced cognitive inconsistency that is apparent in the addicts' “self-described mistake”. We show that addictive drugs gradually produce a motivational bias toward drug-seeking at low-level habitual decision processes, despite the low abstract cognitive valuation of this behavior. This pathology emerges within the hierarchical reinforcement learning framework when chronic exposure to the drug pharmacologically produces pathologicaly persistent phasic dopamine signals. Thereby the drug hijacks the dopaminergic spirals that cascade the reinforcement signals down the ventro-dorsal cortico-striatal hierarchy. Neurobiologically, our theory accounts for rapid development of drug cue-elicited dopamine efflux in the ventral striatum and a delayed response in the dorsal striatum. Our theory also shows how this response pattern depends critically on the dopamine spiraling circuitry. Behaviorally, our framework explains gradual insensitivity of drug-seeking to drug-associated punishments, the blocking phenomenon for drug outcomes, and the persistent preference for drugs over natural rewards by addicts. The model suggests testable predictions and beyond that, sets the stage for a view of addiction as a pathology of hierarchical decision-making processes. This view is complementary to the traditional interpretation of addiction as interaction between habitual and goal-directed decision systems

Newborn screening for presymptomatic diagnosis of complement and phagocyte deficiencies

The clinical outcomes of primary immunodeficiencies (PIDs) are greatly improved by accurate diagnosis early in life. However, it is not common to consider PIDs before the manifestation of severe clinical symptoms. Including PIDs in the nation-wide newborn screening programs will potentially improve survival and provide better disease management and preventive care in PID patients. This calls for the detection of disease biomarkers in blood and the use of dried blood spot samples, which is a part of routine newborn screening programs worldwide. Here, we developed a newborn screening method based on multiplex protein profiling for parallel diagnosis of 22 innate immunodeficiencies affecting the complement system and respiratory burst function in phagocytosis. The proposed method uses a small fraction of eluted blood from dried blood spots and is applicable for population-scale performance. The diagnosis method is validated through a retrospective screening of immunodeficient patient samples. This diagnostic approach can pave the way for an earlier, more comprehensive and accurate diagnosis of complement and phagocytic disorders, which ultimately lead to a healthy and active life for the PID patientsThis work was supported by the Swedish Research Council (VR) and grants provided by the Stockholm County Council (ALF)

QTL Analysis of Shading Sensitive Related Traits in Maize under Two Shading Treatments

During maize development and reproduction, shading stress is an important abiotic factor influencing grain yield. To elucidate the genetic basis of shading stress in maize, an F2:3 population derived from two inbred lines, Zhong72 and 502, was used to evaluate the performance of six traits under shading treatment and full-light treatment at two locations. The results showed that shading treatment significantly decreased plant height and ear height, reduced stem diameter, delayed day-to-tassel (DTT) and day-to-silk (DTS), and increased anthesis-silking interval (ASI). Forty-three different QTLs were identified for the six measured traits under shading and full light treatment at two locations, including seven QTL for plant height, nine QTL for ear height, six QTL for stem diameter, seven QTL for day-to-tassel, six QTL for day-to-silk, and eight QTL for ASI. Interestingly, three QTLs, qPH4, qEH4a, and qDTT1b were detected under full sunlight and shading treatment at two locations simultaneously, these QTL could be used for selecting elite hybrids with high tolerance to shading and high plant density. And the two QTL, qPH10 and qDTS1a, were only detected under shading treatment at two locations, should be quit for selecting insensitive inbred line in maize breeding procedure by using MAS method