Leveraging Prior Knowledge in Reinforcement Learning via Double-Sided Bounds on the Value Function

An agent's ability to leverage past experience is critical for efficiently solving new tasks. Approximate solutions for new tasks can be obtained from previously derived value functions, as demonstrated by research on transfer learning, curriculum learning, and compositionality. However, prior work has primarily focused on using value functions to obtain zero-shot approximations for solutions to a new task. In this work, we show how an arbitrary approximation for the value function can be used to derive double-sided bounds on the optimal value function of interest. We further extend the framework with error analysis for continuous state and action spaces. The derived results lead to new approaches for clipping during training which we validate numerically in simple domains

Effect of Tibial Nerve Stimulation on Gastrosoleus Spasticity: A Randomized Controlled Trial

OBJECTIVES: To compare the effect of repetitive low-threshhold afferent electrical stimulation of the Posterior Tibial Nerve against the standard treatment in the management of gastrosoleus spasticity in patients surviving cerebrovascular accidents. Objective measurement of the change in gastrosoleus spasticity using an electrodiagnostic technique, namely the H:M Ratio and comparing the same with the Modified Ashworth Scale. METHODOLOGY: Patients attending the weekly stroke clinic were screened and out of the 24 who were enrolled in the study, 4 dropped out, leaving 11 patients in the sham stimulation arm and 9 in the active intervention arm. Baseline values of the levels of spasticity as per the Modified Ashworth Scale (MAS) and H-max/M-max ratio were recorded. A trained physiotherapist administered the electrical stimulation on a daily basis for 2 weeks, with each session lasting 30 minutes. The electrical stimulation was administered over the Posterior Tibial Nerve at the medial malleolus, on the paretic lower limb. Following 2 weeks of electrical stimulation, levels of spasticity were re-assessed using the MAS and H-max/M-max ratio. RESULTS: The change in the median H/M ratio value from pre-intervention (0.72) to post-intervention (0.56) within the sham arm was not statistically significant (p=0.62). Likewise the change in the median H/M ratio value from pre-intervention (0.52) to post-intervention (0.53) in the active intervention arm was also not statistically significant (p=0.10). For the sham arm, the median percentage increase after the intervention is 7.46% while in the active intervention group, it is 27.87%, where the difference between the two arms is not statistically significant (p=0.38). With regard to the Modified Ashworth Scale scores, clonus which was observed prior to starting the intervention was observed even after the two weeks of electrical stimulation

Reliability of Accelerometer-Based Reaction Time Tests

Concussions are traumatic brain injuries that affect the function of the brain. One of the primary symptoms of a concussion is a lack of reaction time. The people that are most susceptible to concussions are athletes; Laker’s (2011) study found that 135,000 patients that suffer concussions from playing sports are expected to be hospitalized each year, with football making up 75% of concussions at high school and college levels. Honda et al. (2018) suggested reaction time as an important biomarker of concussion. Laboratory camera-based motion capture data, while reliable, is not a realistic tool to use outside of a laboratory and requires hardware that makes it impractical in non-laboratory settings. Accelerometers measure linear triaxial accelerations are wireless miniature MEMS devices and can be easily affixed on the hand. If accelerometers were proven to be as accurate as motion capture systems in obtaining reaction time, accelerometer-dependent devices, such as a drop stick, could serve as a reliable and portable tool to diagnose a concussion quickly and easily. Our study, which aims to compare the accuracy of laboratory motion capture to accelerometer data, uses a drop stick test, in which participants had an accelerometer strapped to their wrist and infrared markers placed on three bony landmarks of the hand, as well as on an accelerometer situated on the base of the drop stick. In multiple series of tests, participants stood on both solid ground and a foam pad to measure differences in reaction time. Analysis of data will compare the accuracy of accelerometers to motion capture. If proven accurate, this method of measuring reaction time should yield data that is just as reliable as laboratory camera-based motion capture, while simultaneously being more convenient to use as sideline assessment during play

Dose and Time-dependent Alterations in Various Cholinergic and Non-cholinergic Markers after Organophosphate Poisoning: Possible Role in Diagnosis

Toxicity of organophosphates (OP) is mainly ascribed to inhibition of acetylcholinesterase (AChE) enzyme at the cholinergic synapses. This results in cholinergic crisis leading to various muscarinic, nicotinic and central effects. Additionally, there are several non-cholinergic effects of OP which are likely to exacerbate the toxicity and complicate diagnosis. The present study reports the dose (0.125 - 4.0 LD50) and time (1 h - 14 d)- dependent acute effect of diisopropyl phosphorofluoridate (DFP) on mice body weight, organ-body weight index (brain), butyrylcholinesterase (BChE) and β-glucoronidase (BG) activity in plasma, AChE activity, reduced glutathione (GSH) and malondialdehyde (MDA) levels in brain, and DNA damage in brain (agarose gel electrophoresis) and blood (comet assay). The study reveals a dose and time- dependent BChE inhibition up to 24 h and AChE inhibition up to 7 d. However, elevated BG levels were observed up to 1 h only after 1.0, 2.0, and 4.0 LD50 DFP. Diminished GSH levels up to 24 h and increased MDA levels at 4 h indicated oxidative stress. None of the treatments produced any DNA damage in soft tissues. In addition to cholinesterase, the study suggests possible relevance of measuring BG levels (non-cholinergic marker) in the diagnosis of OP poisoning

Navigating the Web of Misinformation: A Framework for Misinformation Domain Detection Using Browser Traffic

The proliferation of misinformation and propaganda is a global challenge, with profound effects during major crises such as the COVID-19 pandemic and the Russian invasion of Ukraine. Understanding the spread of misinformation and its social impacts requires identifying the news sources spreading false information. While machine learning (ML) techniques have been proposed to address this issue, ML models have failed to provide an efficient implementation scenario that yields useful results. In prior research, the precision of deployment in real traffic deteriorates significantly, experiencing a decrement up to ten times compared to the results derived from benchmark data sets. Our research addresses this gap by proposing a graph-based approach to capture navigational patterns and generate traffic-based features which are used to train a classification model. These navigational and traffic-based features result in classifiers that present outstanding performance when evaluated against real traffic. Moreover, we also propose graph-based filtering techniques to filter out models to be classified by our framework. These filtering techniques increase the signal-to-noise ratio of the models to be classified, greatly reducing false positives and the computational cost of deploying the model. Our proposed framework for the detection of misinformation domains achieves a precision of 0.78 when evaluated in real traffic. This outcome represents an improvement factor of over ten times over those achieved in previous studies

Ultracold polar molecules as qudits

We discuss how the internal structure of ultracold molecules, trapped in the motional ground state of optical tweezers, can be used to implement qudits. We explore the rotational, fine and hyperfine structure of $^{40}$Ca$^{19}$F and $^{87}$Rb$^{133}$Cs, which are examples of molecules with $^2\Sigma$ and $^1\Sigma$ electronic ground states, respectively. In each case we identify a subset of levels within a single rotational manifold suitable to implement a 4-level qudit. Quantum gates can be implemented using two-photon microwave transitions via levels in a neighboring rotational manifold. We discuss limitations to the usefulness of molecular qudits, arising from off-resonant excitation and decoherence. As an example, we present a protocol for using a molecular qudit of dimension $d=4$ to perform the Deutsch algorithm

A Multi-hop Mobile Networking Test-bed for Telematics

An onboard vehicle-to-vehicle multi-hop wireless networking system has been developed to test the realworld performance of telematics applications. The system targets emergency and safety messaging, traffic updates, audio/video streaming and commercial announcements. The test-bed includes a Differential GPS receiver, an IEEE 802.11a radio card modified to emulate the DSRC standard, a 1xRTT cellular-data connection, an onboard computer and audio-visual equipment. Vehicles exchange data directly or via intermediate vehicles using a multi-hop routing protocol. The focus of the test-bed is to (a) evaluate the feasibility of high-speed inter-vehicular networking, (b) characterize 5.8GHz signal propagation within a dynamic mobile ad hoc environment, and (c) develop routing protocols for highly mobile networks. The test-bed has been deployed across five vehicles and tested over 400 miles on the road

Using low levels of stochastic vestibular stimulation to improve locomotor stability

Low levels of bipolar binaural white noise based imperceptible stochastic electrical stimulation to the vestibular system (stochastic vestibular stimulation, SVS) have been shown to improve stability during balance tasks in normal, healthy subjects by facilitating enhanced information transfer using stochastic resonance (SR) principles. We hypothesize that detection of time-critical sub-threshold sensory signals using low levels of bipolar binaural SVS based on SR principles will help improve stability of walking during support surface perturbations. In the current study 13 healthy subjects were exposed to short continuous support surface perturbations for 60 s while walking on a treadmill and simultaneously viewing perceptually matched linear optic flow. Low levels of bipolar binaural white noise based SVS were applied to the vestibular organs. Multiple trials of the treadmill locomotion test were performed with stimulation current levels varying in the range of 0–1500 μA, randomized across trials. The results show that subjects significantly improved their walking stability during support surface perturbations at stimulation levels with peak amplitude predominantly in the range of 100–500 μA consistent with the SR phenomenon. Additionally, objective perceptual motion thresholds were measured separately as estimates of internal noise while subjects sat on a chair with their eyes closed and received 1 Hz bipolar binaural sinusoidal electrical stimuli. The optimal improvement in walking stability was achieved on average with peak stimulation amplitudes of approximately 35% of perceptual motion threshold. This study shows the effectiveness of using low imperceptible levels of SVS to improve dynamic stability during walking on a laterally oscillating treadmill via the SR phenomenon