Delayed feedback in kernel bandits

Black box optimisation of an unknown function from expensive and noisy evaluations is a ubiquitous problem in machine learning, academic research and industrial production. An abstraction of the problem can be formulated as a kernel based bandit problem (also known as Bayesian optimisation), where a learner aims at optimising a kernelized function through sequential noisy observations. The existing work predominantly assumes feedback is immediately available; an assumption which fails in many real world situations, including recommendation systems, clinical trials and hyperparameter tuning. We consider a kernel bandit problem under stochastically delayed feedback, and propose an algorithm with O~(Γk(T)T−−−−−−√+E[τ]) regret, where T is the number of time steps, Γk(T) is the maximum information gain of the kernel with T observations, and τ is the delay random variable. This represents a significant improvement over the state of the art regret bound of O~(Γk(T)T−−√+E[τ]Γk(T)) reported in (Verma et al., 2022). In particular, for very non-smooth kernels, the information gain grows almost linearly in time, trivializing the existing results. We also validate our theoretical results with simulations

Self-reported symptoms of SARS-CoV-2 infection in a non-hospitalized population : results from the large Italian web-based EPICOVID19 cross-sectional survey. (Preprint)

Background: Understanding the occurrence of Severe Acute Respiratory Syndrome-Coronavirus-2 (SARS-CoV-2)-like symptoms in a large non-hospitalized population, when the epidemic peak was occurring in Italy, is of paramount importance but data are scarce. Objective: Aims of this study were to evaluate the association of self-reported symptoms with SARS-CoV-2 nasopharyngeal swab (NPS) test in non-hospitalized individuals and to estimate the occurrence of COVID-19-like symptoms in a larger non-tested population. Methods: This is an Italian countrywide self-administered cross-sectional web-based survey on voluntary adults who completed an anonymous questionnaire in the period 13-21 April 2020. The associations between symptoms potentially related to SARS-CoV-2 infection and NPS results were calculated as adjusted odds ratios with 95% confidence intervals (aOR, 95%CI) by means of multiple logistic regression analysis controlling for age, sex, education, smoking habits, and the number of co-morbidities. Thereafter, for each symptom and for their combination, we calculated sensitivity, specificity, accuracy and AUC in a ROC analysis to estimate the occurrence of COVID-19-like infections in the non-tested population. Results: A total of 171,310 responded to the survey (59.9% females, mean age 47.4 years). Out of the 4,785 respondents with known NPS test result, 4,392 were not hospitalized. Among them, the NPS positive respondents (n=856) most frequently reported myalgia (61.6%), olfactory and/or taste disorders (OTDs, 59.2%), cough (54.4%), and fever (51.9%) whereas 7.7% were asymptomatic. Multiple regression analysis showed that OTDs (aOR 10.3, [95%CI 8.4-12.7]), fever (2.5, 95%CI 2.0-3.1), myalgia (1.5, 95%CI 1.2-1.8), and cough (1.3, 95%CI 1.0-1.6) were associated with NPS positivity. Having two to four of these symptoms increased the aOR from 7.4 (95%CI, 5.6-9.7) to 35.5 (95%CI, 24.6-52.2). The combination of the four symptoms showed an AUC of 0.810 (95%CI 0.795-0.825) in classifying NPS-P, and was applied to the non-hospitalized and non-tested sample (n=165,782). We found that from 4.4% to 12.1% of respondents had experienced symptoms suggestive of COVID-19 infection. Conclusions: Our results suggest that self-reported symptoms may be reliable indicators of SARS-CoV-2 infection in a pandemic context. A not negligible part (up to 12.1%) of the symptomatic respondents were left undiagnosed and potentially contributed to the spread of the infection

30-day mortality in patients hospitalized with COVID-19 during the first wave of the Italian epidemic: a prospective cohort study

Italy was the first European country hit by the COVID-19 pandemic and has the highest number of recorded COVID-19 deaths in Europe. This prospective cohort study of the correlates of the risk of death in COVID-19 patients was conducted at the Infectious Diseases and Intensive Care units of Luigi Sacco Hospital, Milan, Italy. The clinical characteristics of all the COVID-19 patients hospitalised in the early days of the epidemic (21 February -19 March 2020) were recorded upon admission, and the time-dependent probability of death was evaluated using the Kaplan-Meier method (censored as of 20 April 2020). Cox proportional hazard models were used to assess the factors independently associated with the risk of death. Forty-eight (20.6%) of the 233 patients followed up for a median of 40 days (interquartile range 33-47) died during the follow-up. Most were males (69.1%) and their median age was 61 years (IQR 50-72). The time-dependent probability of death was 19.7% (95% CI 14.6-24.9%) 30 days after hospital admission. Age (adjusted hazard ratio [aHR] 2.08, 95% CI 1.48-2.92 per ten years more) and obesity (aHR 3.04, 95% CI 1.42-6.49) were independently associated with an increased risk of death, which was also associated with critical disease (aHR 8.26, 95% CI 1.41-48.29), C-reactive protein levels (aHR 1.17, 95% CI 1.02-1.35 per 50\u2009mg/L more) and creatinine kinase levels above 185 U/L (aHR 2.58, 95% CI 1.37-4.87) upon admission. Case-fatality rate of patients hospitalized with COVID-19 in the early days of the Italian epidemic was about 20%. Our study adds evidence to the notion that older age, obesity and more advanced illness are factors associated to an increased risk of death among patients hospitalized with COVID-19

Continuous Attractors with Morphed/Correlated Maps

Continuous attractor networks are used to model the storage and representation of analog quantities, such as position of a visual stimulus. The storage of multiple continuous attractors in the same network has previously been studied in the context of self-position coding. Several uncorrelated maps of environments are stored in the synaptic connections, and a position in a given environment is represented by a localized pattern of neural activity in the corresponding map, driven by a spatially tuned input. Here we analyze networks storing a pair of correlated maps, or a morph sequence between two uncorrelated maps. We find a novel state in which the network activity is simultaneously localized in both maps. In this state, a fixed cue presented to the network does not determine uniquely the location of the bump, i.e. the response is unreliable, with neurons not always responding when their preferred input is present. When the tuned input varies smoothly in time, the neuronal responses become reliable and selective for the environment: the subset of neurons responsive to a moving input in one map changes almost completely in the other map. This form of remapping is a non-trivial transformation between the tuned input to the network and the resulting tuning curves of the neurons. The new state of the network could be related to the formation of direction selectivity in one-dimensional environments and hippocampal remapping. The applicability of the model is not confined to self-position representations; we show an instance of the network solving a simple delayed discrimination task

Polarity of uncertainty representation during exploration and exploitation in ventromedial prefrontal cortex

Environments furnish multiple information sources for making predictions about future events. Here we use behavioural modelling and functional magnetic resonance imaging to describe how humans select predictors that might be most relevant. First, during early encounters with potential predictors, participants’ selections were explorative and directed towards subjectively uncertain predictors (positive uncertainty effect). This was particularly the case when many future opportunities remained to exploit knowledge gained. Then, preferences for accurate predictors increased over time, while uncertain predictors were avoided (negative uncertainty effect). The behavioural transition from positive to negative uncertainty-driven selections was accompanied by changes in the representations of belief uncertainty in ventromedial prefrontal cortex (vmPFC). The polarity of uncertainty representations (positive or negative encoding of uncertainty) changed between exploration and exploitation periods. Moreover, the two periods were separated by a third transitional period in which beliefs about predictors’ accuracy predominated. The vmPFC signals a multiplicity of decision variables, the strength and polarity of which vary with behavioural context

Global reward state affects learning and activity in raphe nucleus and anterior insula in monkeys

202009 bcrcVersion of RecordPublishe

Sympathetic involvement in time-constrained sequential foraging

Appraising sequential offers relative to an unknown future opportunity and a time cost requires an optimization policy that draws on a learned estimate of an environment’s richness. Converging evidence points to a learning asymmetry, whereby estimates of this richness update with a bias toward integrating positive information. We replicate this bias in a sequential foraging (prey selection) task and probe associated activation within the sympathetic branch of the autonomic system, using trial-by-trial measures of simultaneously recorded cardiac autonomic physiology. We reveal a unique adaptive role for the sympathetic branch in learning. It was specifically associated with adaptation to a deteriorating environment: it correlated with both the rate of negative information integration in belief estimates and downward changes in moment-to-moment environmental richness, and was predictive of optimal performance on the task. The findings are consistent with a framework whereby autonomic function supports the learning demands of prey selection