Metrics and continuity in reinforcement learning

In most practical applications of reinforcement learning, it is untenable to maintain direct estimates for individual states; in continuous-state systems, it is impossible. Instead, researchers often leverage state similarity (whether explicitly or implicitly) to build models that can generalize well from a limited set of samples. The notion of state similarity used, and the neighbourhoods and topologies they induce, is thus of crucial importance, as it will directly affect the performance of the algorithms. Indeed, a number of recent works introduce algorithms assuming the existence of "well-behaved" neighbourhoods, but leave the full specification of such topologies for future work. In this paper we introduce a unified formalism for defining these topologies through the lens of metrics. We establish a hierarchy amongst these metrics and demonstrate their theoretical implications on the Markov Decision Process specifying the reinforcement learning problem. We complement our theoretical results with empirical evaluations showcasing the differences between the metrics considered.Comment: Accepted at AAAI 202

Harnessing AI for Speech Reconstruction using Multi-view Silent Video Feed

Speechreading or lipreading is the technique of understanding and getting phonetic features from a speaker's visual features such as movement of lips, face, teeth and tongue. It has a wide range of multimedia applications such as in surveillance, Internet telephony, and as an aid to a person with hearing impairments. However, most of the work in speechreading has been limited to text generation from silent videos. Recently, research has started venturing into generating (audio) speech from silent video sequences but there have been no developments thus far in dealing with divergent views and poses of a speaker. Thus although, we have multiple camera feeds for the speech of a user, but we have failed in using these multiple video feeds for dealing with the different poses. To this end, this paper presents the world's first ever multi-view speech reading and reconstruction system. This work encompasses the boundaries of multimedia research by putting forth a model which leverages silent video feeds from multiple cameras recording the same subject to generate intelligent speech for a speaker. Initial results confirm the usefulness of exploiting multiple camera views in building an efficient speech reading and reconstruction system. It further shows the optimal placement of cameras which would lead to the maximum intelligibility of speech. Next, it lays out various innovative applications for the proposed system focusing on its potential prodigious impact in not just security arena but in many other multimedia analytics problems.Comment: 2018 ACM Multimedia Conference (MM '18), October 22--26, 2018, Seoul, Republic of Kore

Bootstrapped Representations in Reinforcement Learning

In reinforcement learning (RL), state representations are key to dealing with large or continuous state spaces. While one of the promises of deep learning algorithms is to automatically construct features well-tuned for the task they try to solve, such a representation might not emerge from end-to-end training of deep RL agents. To mitigate this issue, auxiliary objectives are often incorporated into the learning process and help shape the learnt state representation. Bootstrapping methods are today's method of choice to make these additional predictions. Yet, it is unclear which features these algorithms capture and how they relate to those from other auxiliary-task-based approaches. In this paper, we address this gap and provide a theoretical characterization of the state representation learnt by temporal difference learning (Sutton, 1988). Surprisingly, we find that this representation differs from the features learned by Monte Carlo and residual gradient algorithms for most transition structures of the environment in the policy evaluation setting. We describe the efficacy of these representations for policy evaluation, and use our theoretical analysis to design new auxiliary learning rules. We complement our theoretical results with an empirical comparison of these learning rules for different cumulant functions on classic domains such as the four-room domain (Sutton et al, 1999) and Mountain Car (Moore, 1990).Comment: ICML 202

Controlling soliton excitations in Heisenberg spin chain through magic angle

We study the nonlinear dynamics of collective excitation in a $N$-site $XXZ$ quantum spin chain, which is manipulated by an oblique magnetic field. We show that, when the tilted field is applied along the magic angle $\theta_{0} =\pm\arccos \sqrt{1/3}$, the anisotropic Heisenberg spin chain becomes isotropic and thus an free propagating spin wave is stimulated. And in the regime of the tilted angle larger and smaller then the magic angle, two types of nonlinear excitations appear, which are bright soliton and dark soliton.Comment: 7 pages 4 figure

Genetical genomics: spotlight on QTL hotspots

No abstract available

Formation of ionospheric irregularities over Southeast Asia during the 2015 St. Patrickˈs Day storm

We investigate the geospace response to the 2015 St. Patrickˈs Day storm leveraging on instruments spread over Southeast Asia (SEA), covering a wide longitudinal sector of the low-latitude ionosphere. A regional characterization of the storm is provided, identifying the peculiarities of ionospheric irregularity formation. The novelties of this work are the characterization in a broad longitudinal range and the methodology relying on the integration of data acquired by Global Navigation Satellite System (GNSS) receivers, magnetometers, ionosondes, and Swarm satellites. This work is a legacy of the project EquatoRial Ionosphere Characterization in Asia (ERICA). ERICA aimed to capture the features of both crests of the equatorial ionospheric anomaly (EIA) and trough (EIT) by means of a dedicated measurement campaign. The campaign lasted from March to October 2015 and was able to observe the ionospheric variability causing effects on radio systems, GNSS in particular. The multiinstrumental and multiparametric observations of the region enabled an in-depth investigation of the response to the largest geomagnetic storm of the current solar cycle in a region scarcely reported in literature. Our work discusses the comparison between northern and southern crests of the EIA in the SEA region. The observations recorded positive and negative ionospheric storms, spread F conditions, scintillation enhancement and inhibition, and total electron content variability. The ancillary information on the local magnetic field highlights the variety of ionospheric perturbations during the different storm phases. The combined use of ionospheric bottomside, topside, and integrated information points out how the storm affects the F layer altitude and the consequent enhancement/suppression of scintillations.Published12211–122331A. Geomagnetismo e Paleomagnetismo2A. Fisica dell'alta atmosfera1IT. Reti di monitoraggio e Osservazioni5IT. Osservazioni satellitariJCR Journalope

Search for the glueball candidates f0(1500) and fJ(1710) in gamma gamma collisions

Data taken with the ALEPH detector at LEP1 have been used to search for gamma gamma production of the glueball candidates f0(1500) and fJ(1710) via their decay to pi+pi-. No signal is observed and upper limits to the product of gamma gamma width and pi+pi- branching ratio of the f0(1500) and the fJ(1710) have been measured to be Gamma_(gamma gamma -> f0(1500)). BR(f0(1500)->pi+pi-) < 0.31 keV and Gamma_(gamma gamma -> fJ(1710)). BR(fJ(1710)->pi+pi-) < 0.55 keV at 95% confidence level.Comment: 10 pages, 3 figure

Search for CP Violation in the Decay Z -> b (b bar) g

About three million hadronic decays of the Z collected by ALEPH in the years 1991-1994 are used to search for anomalous CP violation beyond the Standard Model in the decay Z -> b \bar{b} g. The study is performed by analyzing angular correlations between the two quarks and the gluon in three-jet events and by measuring the differential two-jet rate. No signal of CP violation is found. For the combinations of anomalous CP violating couplings, ${\hat{h}}_b = {\hat{h}}_{Ab}g_{Vb}-{\hat{h}}_{Vb}g_{Ab}$ and $h^{\ast}_b = \sqrt{\hat{h}_{Vb}^{2}+\hat{h}_{Ab}^{2}}$, limits of \hat{h}_b < 0.59$and$h^{\ast}_{b} < 3.02$ are given at 95\% CL.Comment: 8 pages, 1 postscript figure, uses here.sty, epsfig.st

Search for direct stau production in events with two hadronic tau-leptons in root s=13 TeV pp collisions with the ATLAS detector

A search for the direct production of the supersymmetric partners ofτ-leptons (staus) in final stateswith two hadronically decayingτ-leptons is presented. The analysis uses a dataset of pp collisions corresponding to an integrated luminosity of139fb−1, recorded with the ATLAS detector at the LargeHadron Collider at a center-of-mass energy of 13 TeV. No significant deviation from the expected StandardModel background is observed. Limits are derived in scenarios of direct production of stau pairs with eachstau decaying into the stable lightest neutralino and oneτ-lepton in simplified models where the two staumass eigenstates are degenerate. Stau masses from 120 GeV to 390 GeV are excluded at 95% confidencelevel for a massless lightest neutralino

Invasive Non-typhoidal Salmonella Infections in Asia: Clinical Observations, Disease Outcome and Dominant Serovars from an Infectious Disease Hospital in Vietnam.

Invasive non-typhoidal Salmonella (iNTS) infections are now a well-described cause of morbidity and mortality in children and HIV-infected adults in sub-Saharan Africa. In contrast, the epidemiology and clinical manifestations of iNTS disease in Asia are not well documented. We retrospectively identified >100 cases of iNTS infections in an infectious disease hospital in Southern Vietnam between 2008 and 2013. Clinical records were accessed to evaluate demographic and clinical factors associated with iNTS infection and to identify risk factors associated with death. Multi-locus sequence typing and antimicrobial susceptibility testing was performed on all organisms. Of 102 iNTS patients, 71% were HIV-infected, >90% were adults, 71% were male and 33% reported intravenous drug use. Twenty-six/92 (28%) patients with a known outcome died; HIV infection was significantly associated with death (p = 0.039). S. Enteritidis (Sequence Types (ST)11) (48%, 43/89) and S. Typhimurium (ST19, 34 and 1544) (26%, 23/89) were the most commonly identified serovars; S. Typhimurium was significantly more common in HIV-infected individuals (p = 0.003). Isolates from HIV-infected patients were more likely to exhibit reduced susceptibility against trimethoprim-sulfamethoxazole than HIV-negative patients (p = 0.037). We conclude that iNTS disease is a severe infection in Vietnam with a high mortality rate. As in sub-Saharan Africa, HIV infection was a risk factor for death, with the majority of the burden in this population found in HIV-infected adult men