XCS Classifier System with Experience Replay

XCS constitutes the most deeply investigated classifier system today. It bears strong potentials and comes with inherent capabilities for mastering a variety of different learning tasks. Besides outstanding successes in various classification and regression tasks, XCS also proved very effective in certain multi-step environments from the domain of reinforcement learning. Especially in the latter domain, recent advances have been mainly driven by algorithms which model their policies based on deep neural networks -- among which the Deep-Q-Network (DQN) is a prominent representative. Experience Replay (ER) constitutes one of the crucial factors for the DQN's successes, since it facilitates stabilized training of the neural network-based Q-function approximators. Surprisingly, XCS barely takes advantage of similar mechanisms that leverage stored raw experiences encountered so far. To bridge this gap, this paper investigates the benefits of extending XCS with ER. On the one hand, we demonstrate that for single-step tasks ER bears massive potential for improvements in terms of sample efficiency. On the shady side, however, we reveal that the use of ER might further aggravate well-studied issues not yet solved for XCS when applied to sequential decision problems demanding for long-action-chains

Bootstrapping a DQN Replay Memory with Synthetic Experiences

An important component of many Deep Reinforcement Learning algorithms is the Experience Replay which serves as a storage mechanism or memory of made experiences. These experiences are used for training and help the agent to stably find the perfect trajectory through the problem space. The classic Experience Replay however makes only use of the experiences it actually made, but the stored samples bear great potential in form of knowledge about the problem that can be extracted. We present an algorithm that creates synthetic experiences in a nondeterministic discrete environment to assist the learner. The Interpolated Experience Replay is evaluated on the FrozenLake environment and we show that it can support the agent to learn faster and even better than the classic version

The connection between BRG1, CTCF and topoisomerases at TAD boundaries

The eukaryotic genome is partitioned into topologically associating domains (TADs). Despite recent advances characterizing TADs and TAD boundaries, the organization of these structures is an important dimension of genome architecture and function that is not well understood. Recently, we demonstrated that knockdown of BRG1, an ATPase driving the chromatin remodeling activity of mammalian SWI/SNF enzymes, globally alters long-range genomic interactions and results in a reduction of TAD boundary strength. We provided evidence suggesting that this effect may be due to BRG1 affecting nucleosome occupancy around CTCF sites present at TAD boundaries. In this review, we elaborate on our findings and speculate that BRG1 may contribute to the regulation of the structural and functional properties of chromatin at TAD boundaries by affecting the function or the recruitment of CTCF and DNA topoisomerase complexes

Short Term and Long-Term Efficacy of Calcipotriene/ Betamethasone Dipropionate Foam Combination

Psoriasis is a well-known chronic disease characterized by the development of erythematous, indurated, scaly, pruritic plaques on the skin with cycles of remission and symptom flare-ups. The management of patients with chronic plaque psoriasis has been more challenging since the Covid-19 pandemic as health care professionals have had to adapt to remote consultations for some patients, and patients have had to adapt to the changing health landscape. The rapid resolution of psoriasis symptoms especially those with a substantial impact on quality of life can improve patient satisfaction and adherence, making it an important factor in successful treatment. Cal/BD foam contributes to improved patient adherence and treatment outcome through its rapid action and superior efficacy versus Cal or BD monotherapy, Cal/BD ointment and gel and clobetasol cream in the short-term flare treatment of psoriasis. Moreover, the benefits of proactive long-term management of psoriasis compared to reactive management and its favourable safety profile are higher efficacy and a better health-related quality of life. Cal/BD foam should be considered an effective topical treatment for short-term flare treatment and long-term control of adult psoriatic patients

Target mass corrections and twist-3 in the nucleon spin structure functions

The Nachtmann moment is employed to study the contribution of twist-3 operator to the nucleon spin structure functions. Target mass corrections to the Cornwall-Norton moments of the spin structure functions $g_{1,2}$ are discussed. It is found that the corrections play a sizeable role to the contribution of the twist-3 $\tilde d_2$ extracted from the Cornwall-Norton moments.Comment: 11 pages, 1 figure

Parton Sum Rules and Improved Scaling Variable

The effect from quark masses and transversal motion on the Gottfried, Bjorken, and Ellis-Jaffe sum rules is examined by using a quark-parton model of nucleon structure functions based on an improved scaling variable. Its use results in corrections to the Gottfried, Bjorken, and Ellis-Jaffe sum rules. We use the Brodsky-Huang-Lepage prescription of light-cone wavefunctions to estimate the size of the corrections. We constrain our choice of parameters by the roughly known higher twist corrections to the Bjorken sum rule and find that the resulting corrections to the Gottfried and Ellis-Jaffe sum rules are relevant, though not large enough to explain the observed sum rule violations.Comment: latex, with 1 postscript figure, to be published in Phys.Lett.

Increased markers of cardiac vagal activity in leucine-rich repeat kinase 2-associated Parkinson's disease.

PurposeCardiac autonomic dysfunction manifests as reduced heart rate variability (HRV) in idiopathic Parkinson's disease (PD), but no significant reduction has been found in PD patients who carry the LRRK2 mutation. Novel HRV features have not been investigated in these individuals. We aimed to assess cardiac autonomic modulation through standard and novel approaches to HRV analysis in individuals who carry the LRRK2 G2019S mutation.MethodsShort-term electrocardiograms were recorded in 14 LRRK2-associated PD patients, 25 LRRK2-non-manifesting carriers, 32 related non-carriers, 20 idiopathic PD patients, and 27 healthy controls. HRV measures were compared using regression modeling, controlling for age, sex, mean heart rate, and disease duration. Discriminant analysis highlighted the feature combination that best distinguished LRRK2-associated PD from controls.ResultsBeat-to-beat and global HRV measures were significantly increased in LRRK2-associated PD patients compared with controls (e.g., deceleration capacity of heart rate: p = 0.006) and idiopathic PD patients (e.g., 8th standardized moment of the interbeat interval distribution: p = 0.0003), respectively. LRRK2-associated PD patients also showed significantly increased irregularity of heart rate dynamics, as quantified by Rényi entropy, when compared with controls (p = 0.002) and idiopathic PD patients (p = 0.0004). Ordinal pattern statistics permitted the identification of LRRK2-associated PD individuals with 93% sensitivity and 93% specificity. Consistent results were found in a subgroup of LRRK2-non-manifesting carriers when compared with controls.ConclusionsIncreased beat-to-beat HRV in LRRK2 G2019S mutation carriers compared with controls and idiopathic PD patients may indicate augmented cardiac autonomic cholinergic activity, suggesting early impairment of central vagal feedback loops in LRRK2-associated PD

An Analysis of the Next-to-Leading Order Corrections to the g_T(=g_1+g_2) Scaling Function

We present a general method for obtaining the quantum chromodynamical radiative corrections to the higher-twist (power-suppressed) contributions to inclusive deep-inelastic scattering in terms of light-cone correlation functions of the fundamental fields of quantum chromodynamics. Using this procedure, we calculate the previously unknown ${\cal O}(\alpha_s)$ corrections to the twist-three part of the spin scaling function $g_T(x_B,Q^2) (=g_1(x_B,Q^2)+g_2(x_B,Q^2))$ and the corresponding forward Compton amplitude $S_T(\nu,Q^2)$. Expanding our result about the unphysical point $x_B=\infty$, we arrive at an operator product expansion of the nonlocal product of two electromagnetic current operators involving twist-two and -three operators valid to ${\cal O}(\alpha_s)$ for forward matrix elements. We find that the Wandzura-Wilczek relation between $g_1(x_B,Q^2)$ and the twist-two part of $g_T(x_B,Q^2)$ is respected in both the singlet and non-singlet sectors at this order, and argue its validity to all orders. The large-$N_c$ limit does not appreciably simplify the twist-three Wilson coefficients.Comment: 41 pages, 9 figures, corrected minor erro