Dynamic multi-objective optimisation using deep reinforcement learning::benchmark, algorithm and an application to identify vulnerable zones based on water quality

Dynamic multi-objective optimisation problem (DMOP) has brought a great challenge to the reinforcement learning (RL) research area due to its dynamic nature such as objective functions, constraints and problem parameters that may change over time. This study aims to identify the lacking in the existing benchmarks for multi-objective optimisation for the dynamic environment in the RL settings. Hence, a dynamic multi-objective testbed has been created which is a modified version of the conventional deep-sea treasure (DST) hunt testbed. This modified testbed fulfils the changing aspects of the dynamic environment in terms of the characteristics where the changes occur based on time. To the authors’ knowledge, this is the first dynamic multi-objective testbed for RL research, especially for deep reinforcement learning. In addition to that, a generic algorithm is proposed to solve the multi-objective optimisation problem in a dynamic constrained environment that maintains equilibrium by mapping different objectives simultaneously to provide the most compromised solution that closed to the true Pareto front (PF). As a proof of concept, the developed algorithm has been implemented to build an expert system for a real-world scenario using Markov decision process to identify the vulnerable zones based on water quality resilience in São Paulo, Brazil. The outcome of the implementation reveals that the proposed parity-Q deep Q network (PQDQN) algorithm is an efficient way to optimise the decision in a dynamic environment. Moreover, the result shows PQDQN algorithm performs better compared to the other state-of-the-art solutions both in the simulated and the real-world scenario

Representation of the verb's argument-structure in the human brain

<p>Abstract</p> <p>Background</p> <p>A verb's argument structure defines the number and relationships of participants needed for a complete event. One-argument (intransitive) verbs require only a subject to make a complete sentence, while two- and three-argument verbs (transitives and ditransitives) normally take direct and indirect objects. Cortical responses to verbs embedded into sentences (correct or with syntactic violations) indicate the processing of the verb's argument structure in the human brain. The two experiments of the present study examined whether and how this processing is reflected in distinct spatio-temporal cortical response patterns to isolated verbs and/or verbs presented in minimal context.</p> <p>Results</p> <p>The magnetoencephalogram was recorded while 22 native German-speaking adults saw 130 German verbs, presented one at a time for 150 ms each in experiment 1. Verb-evoked electromagnetic responses at 250 – 300 ms after stimulus onset, analyzed in source space, were higher in the left middle temporal gyrus for verbs that take only one argument, relative to two- and three-argument verbs. In experiment 2, the same verbs (presented in different order) were preceded by a proper name specifying the subject of the verb. This produced additional activation between 350 and 450 ms in or near the left inferior frontal gyrus, activity being larger and peaking earlier for one-argument verbs that required no further arguments to form a complete sentence.</p> <p>Conclusion</p> <p>Localization of sources of activity suggests that the activation in temporal and frontal regions varies with the degree by which representations of an event as a part of the verbs' semantics are completed during parsing.</p

Dissociable Influences of Auditory Object vs. Spatial Attention on Visual System Oscillatory Activity

Given that both auditory and visual systems have anatomically separate object identification (“what”) and spatial (“where”) pathways, it is of interest whether attention-driven cross-sensory modulations occur separately within these feature domains. Here, we investigated how auditory “what” vs. “where” attention tasks modulate activity in visual pathways using cortically constrained source estimates of magnetoencephalograpic (MEG) oscillatory activity. In the absence of visual stimuli or tasks, subjects were presented with a sequence of auditory-stimulus pairs and instructed to selectively attend to phonetic (“what”) vs. spatial (“where”) aspects of these sounds, or to listen passively. To investigate sustained modulatory effects, oscillatory power was estimated from time periods between sound-pair presentations. In comparison to attention to sound locations, phonetic auditory attention was associated with stronger alpha (7–13 Hz) power in several visual areas (primary visual cortex; lingual, fusiform, and inferior temporal gyri, lateral occipital cortex), as well as in higher-order visual/multisensory areas including lateral/medial parietal and retrosplenial cortices. Region-of-interest (ROI) analyses of dynamic changes, from which the sustained effects had been removed, suggested further power increases during Attend Phoneme vs. Location centered at the alpha range 400–600 ms after the onset of second sound of each stimulus pair. These results suggest distinct modulations of visual system oscillatory activity during auditory attention to sound object identity (“what”) vs. sound location (“where”). The alpha modulations could be interpreted to reflect enhanced crossmodal inhibition of feature-specific visual pathways and adjacent audiovisual association areas during “what” vs. “where” auditory attention

Right hemisphere has the last laugh: neural dynamics of joke appreciation

Understanding a joke relies on semantic, mnemonic, inferential, and emotional contributions from multiple brain areas. Anatomically constrained magnetoencephalography (aMEG) combining high-density whole-head MEG with anatomical magnetic resonance imaging allowed us to estimate where the humor-specific brain activations occur and to understand their temporal sequence. Punch lines provided either funny, not funny (semantically congruent), or nonsensical (incongruent) replies to joke questions. Healthy subjects rated them as being funny or not funny. As expected, incongruous endings evoke the largest N400m in left-dominant temporo-prefrontal areas, due to integration difficulty. In contrast, funny punch lines evoke the smallest N400m during this initial lexical–semantic stage, consistent with their primed “surface congruity” with the setup question. In line with its sensitivity to ambiguity, the anteromedial prefrontal cortex may contribute to the subsequent “second take” processing, which, for jokes, presumably reflects detection of a clever “twist” contained in the funny punch lines. Joke-selective activity simultaneously emerges in the right prefrontal cortex, which may lead an extended bilateral temporo-frontal network in establishing the distant unexpected creative coherence between the punch line and the setup. This progression from an initially promising but misleading integration from left frontotemporal associations, to medial prefrontal ambiguity evaluation and right prefrontal reprocessing, may reflect the essential tension and resolution underlying humor

The rank reversal problem in multi-criteria decision making : a literature review

Despite the importance of multicriteria decision-making (MCDM) techniques for constructing effective decision models, there are many criticisms due to the occurrence of a problem called rank reversal. Nevertheless, there is a lack of a systematic literature review on this important subject which involves different methods. This study reviews the pertinent literature on rank reversal, based on 130 related articles published from 1980 to 2015 in international journals, which were gathered and analyzed according to the following perspectives: multicriteria technique, year and journal in which the papers were published, co-authorship network, rank reversal types, and research goal. Thus our survey provides recommendations for future research, besides useful information and knowledge regarding rank reversal in the MCDM field

Patients' beliefs towards contingency management: target behaviours, incentives and the remote application of these interventions

Abstract: Introduction: Contingency management interventions are among the most efficacious psychosocial interventions in promoting abstinence from smoking, alcohol and substance use. The aim of this study was to assess the beliefs and objections towards contingency management among patients in UK‐based drug and alcohol services to help understand barriers to uptake and support the development and implementation of these interventions. Methods: The Service User Survey of Incentives was developed and implemented among patients (N = 181) at three UK‐based drug and alcohol treatment services. Descriptive analyses were conducted to ascertain positive and negative beliefs about contingency management, acceptability of different target behaviours, incentives and delivery mechanisms including delivering incentives remotely using technology devices such as mobile telephones. Results: Overall, 81% of participants were in favour of incentive programs, with more than 70% of respondents agreeing with the majority of positive belief statements. With the exception of two survey items, less than a third of participants agreed with negative belief statements. The proportion of participants indicating a neutral response was higher for negative statements (27%) indicating greater levels of ambiguity towards objections and concerns regarding contingency management. Discussion and Conclusions: Positive beliefs towards contingency management interventions were found, including high levels of acceptability towards a range of target behaviours, incentives and the use of technology devices to remotely monitor behaviour and deliver incentives. These findings have implications for the development and implementation of remote contingency management interventions within the UK drug treatment services

A Genome-Wide Association Study of Diabetic Kidney Disease in Subjects With Type 2 Diabetes

dentification of sequence variants robustly associated with predisposition to diabetic kidney disease (DKD) has the potential to provide insights into the pathophysiological mechanisms responsible. We conducted a genome-wide association study (GWAS) of DKD in type 2 diabetes (T2D) using eight complementary dichotomous and quantitative DKD phenotypes: the principal dichotomous analysis involved 5,717 T2D subjects, 3,345 with DKD. Promising association signals were evaluated in up to 26,827 subjects with T2D (12,710 with DKD). A combined T1D+T2D GWAS was performed using complementary data available for subjects with T1D, which, with replication samples, involved up to 40,340 subjects with diabetes (18,582 with DKD). Analysis of specific DKD phenotypes identified a novel signal near GABRR1 (rs9942471, P = 4.5 x 10(-8)) associated with microalbuminuria in European T2D case subjects. However, no replication of this signal was observed in Asian subjects with T2D or in the equivalent T1D analysis. There was only limited support, in this substantially enlarged analysis, for association at previously reported DKD signals, except for those at UMOD and PRKAG2, both associated with estimated glomerular filtration rate. We conclude that, despite challenges in addressing phenotypic heterogeneity, access to increased sample sizes will continue to provide more robust inference regarding risk variant discovery for DKD.Peer reviewe

Genetic analysis for a shared biological basis between migraine and coronary artery disease

Objective: To apply genetic analysis of genome-wide association data to study the extent and nature of a shared biological basis between migraine and coronary artery disease (CAD).Methods: Four separate methods for cross-phenotype genetic analysis were applied on data from 2 large-scale genome-wide association studies of migraine (19,981 cases, 56,667 controls) and CAD (21,076 cases, 63,014 controls). The first 2 methods quantified the extent of overlapping risk variants and assessed the load of CAD risk loci in migraineurs. Genomic regions of shared risk were then identified by analysis of covariance patterns between the 2 phenotypes and by querying known genome-wide significant loci.Results: We found a significant overlap of genetic risk loci for migraine and CAD. When stratified by migraine subtype, this was limited to migraine without aura, and the overlap was protective in that patients with migraine had a lower load of CAD risk alleles than controls. Genes indicated by 16 shared risk loci point to mechanisms with potential roles in migraine pathogenesis and CAD, including endothelial dysfunction (PHACTR1) and insulin homeostasis (GIP).Conclusions: The results suggest that shared biological processes contribute to risk of migraine and CAD, but surprisingly this commonality is restricted to migraine without aura and the impact is in opposite directions. Understanding the mechanisms underlying these processes and their opposite relationship to migraine and CAD may improve our understanding of both disorders.</div

Meta-analysis of 375,000 individuals identifies 38 susceptibility loci for migraine

Migraine is a debilitating neurological disorder affecting around one in seven people worldwide, but its molecular mechanisms remain poorly understood. There is some debate about whether migraine is a disease of vascular dysfunction or a result of neuronal dysfunction with secondary vascular changes. Genome-wide association (GWA) studies have thus far identified 13 independent loci associated with migraine. To identify new susceptibility loci, we carried out a genetic study of migraine on 59,674 affected subjects and 316,078 controls from 22 GWA studies. We identified 44 independent single-nucleotide polymorphisms (SNPs) significantly associated with migraine risk (P < 5 × 10−8) that mapped to 38 distinct genomic loci, including 28 loci not previously reported and a locus that to our knowledge is the first to be identified on chromosome X. In subsequent computational analyses, the identified loci showed enrichment for genes expressed in vascular and smooth muscle tissues, consistent with a predominant theory of migraine that highlights vascular etiologies