A Finite Time Analysis of Two Time-Scale Actor Critic Methods

Actor-critic (AC) methods have exhibited great empirical success compared with other reinforcement learning algorithms, where the actor uses the policy gradient to improve the learning policy and the critic uses temporal difference learning to estimate the policy gradient. Under the two time-scale learning rate schedule, the asymptotic convergence of AC has been well studied in the literature. However, the non-asymptotic convergence and finite sample complexity of actor-critic methods are largely open. In this work, we provide a non-asymptotic analysis for two time-scale actor-critic methods under non-i.i.d. setting. We prove that the actor-critic method is guaranteed to find a first-order stationary point (i.e., $\|\nabla J(\boldsymbol{\theta})\|_2^2 \le \epsilon$) of the non-concave performance function $J(\boldsymbol{\theta})$, with $\mathcal{\tilde{O}}(\epsilon^{-2.5})$ sample complexity. To the best of our knowledge, this is the first work providing finite-time analysis and sample complexity bound for two time-scale actor-critic methods.Comment: 45 page

Aplicación de la técnica mapas mentales para desarrollar la compresión lectora en los estudiantes del primer ciclo de la Facultad de Ciencias Empresariales de la Universidad Técnica de Machala - Ecuador - 2013

Publicación a texto completo no autorizada por el autorAplica la técnica de mapas mentales para desarrollar la comprensión lectora en los estudiantes del primer ciclo de la Facultad De Ciencias Empresariales de la Universidad Técnica de Machala. El diseño empleado fue el cuasi experimental. El estudio se desarrolló durante el segundo semestre del año 201con una muestra de 144 estudiantes de ambos sexos pertenecientes a la Facultad de Ciencias empresariales de la Universidad Técnica de Machala. Los niveles de comprensión lectora fueron evaluados antes y después de la aplicación de la técnica de mapas mentales mediante la prueba de comprensión lectora. Se utilizó también el instrumento para evaluar mapas mentales, de Suárez y García. Los resultados permiten concluir que la aplicación de la técnica de mapas mentales ha producido un incremento significativo en los niveles de comprensión lectora de la población estudiada.Tesi

Efficient traffic congestion estimation using multiple spatio-temporal properties

Traffic estimation is an important issue to analyze the traffic congestion in large-scale urban traffic situations. Recently, many researchers have used GPS data to estimate traffic congestion. However, how to fuse the multiple data reasonably and guarantee the accuracy and efficiency of these methods are still challenging problems. In this paper, we propose a novel method Multiple Data Estimation (MDE) to estimate the congestion status in urban environment with GPS trajectory data efficiently, where we estimate the congestion status of the area through utilizing multiple properties, including density, velocity, inflow and previous status. Among them, traffic inflow and previous status (combination of time and space factors) are not both used in other existing methods. In order to ensure the accuracy and efficiency, we apply dynamic weights of data and parameters in MDE method. To evaluate our methods, we apply it on large-scale taxi GPS data of Beijing and Shanghai. Extensive experiments on these two real-world datasets demonstrate the significant improvements of our method over several state-of-the-art methods

Current situation and factors influencing elderly care in community day care centers: a cross-sectional study

BackgroundThe latest census data show that people over 60 years of age account for about 18.7% of the total population in China, and the aging of the population has become an irreversible trend in the 21st century. This study aimed to investigate the current status and factors influencing the care of the elderly in community day care centers in order to lay the foundation for the development of better services in community day care centers.MethodsThis study was a cross-sectional survey using convenience sampling in Nanjing, China. The survey instrument was the Day care and Elderly Care Service Needs Questionnaire, which included the Ability of Daily Living Assessment (ADL), the Xiao Shuiyuan Social Support Rating Scale (SSRS) and the Day care Elderly Care Service Needs Survey Form, and a general information survey.ResultsA total of 450 elderly people in day care centers were surveyed. The elderly had different levels of demand for day care services, especially regarding daily care. Correlation analyses indicated that age (r = 0.619), education level (r = 0.616), source of income (r = 0.582), caregiver (r = 0.557), satisfaction with care service (r = 0.603), and degree of ADL (r = 0.629) were correlated with the need for elderly day care services (all p < 0.05). The factors influencing the demand for day care services encompassed age, education level, income source, caregiver, satisfaction with service, and ADL (all p < 0.05).ConclusionElderly care services in community day care centers are mainly based on daily and spiritual comfort, and the needs of the elderly are influenced by many factors. Timely nursing care policies and measures that target these factors are needed to improve elderly care

LncRNA DANCR restrained the survival of mycobacterium tuberculosis H37Ra by sponging miR-1301-3p/miR-5194

Tuberculosis is a worldwide contagion caused by Mycobacterium tuberculosis (MTB). MTB is characterized by intracellular parasitism and is semi-dormant inside host cells. The persistent inflammation caused by MTB can form a granuloma in lesion regions and intensify the latency of bacteria. In recent years, several studies have proven that long non-coding RNAs (lncRNAs) play critical roles in modulating autophagy. In our study, the Gene Expression Omnibus (GEO) databases were searched for lncRNAs that are associated with tuberculosis. We found that lncRNA differentiation antagonizing non-protein coding RNA (DANCR) increased in the peripheral blood samples collected from 54 pulmonary tuberculosis patients compared to 23 healthy donors. By constructing DANCR overexpression cells, we analyzed the possible cellular function of DANCR. After analyzing our experiments, it was found that the data revealed that upregulation of DANCR facilitated the expression of signal transducer and activator of transcription 3, autophagy-related 4D cysteine peptides, autophagy-related 5, Ras homolog enriched in the brain, and microtubule-associated protein 1A/1B light chain 3 (STAT3, ATG4D, ATG5, RHEB, and LC3, respectively) by sponging miR-1301-3p and miR-5194. Immunofluorescence analysis indicated that DANCR played a positive role in both autophagosome formation and fusion of autolysosomes in macrophages. The colony-forming unit (CFU) assay data also showed that the cells overexpressing DANCR were more efficient in eliminating the intracellular H37Ra strain. Consequently, these data suggest that DANCR restrained intracellular survival of M. tuberculosis by promoting autophagy via miR-1301-3p and miR-5194

Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States

Short-term probabilistic forecasts of the trajectory of the COVID-19 pandemic in the United States have served as a visible and important communication channel between the scientific modeling community and both the general public and decision-makers. Forecasting models provide specific, quantitative, and evaluable predictions that inform short-term decisions such as healthcare staffing needs, school closures, and allocation of medical supplies. Starting in April 2020, the US COVID-19 Forecast Hub (https://covid19forecasthub.org/) collected, disseminated, and synthesized tens of millions of specific predictions from more than 90 different academic, industry, and independent research groups. A multimodel ensemble forecast that combined predictions from dozens of groups every week provided the most consistently accurate probabilistic forecasts of incident deaths due to COVID-19 at the state and national level from April 2020 through October 2021. The performance of 27 individual models that submitted complete forecasts of COVID-19 deaths consistently throughout this year showed high variability in forecast skill across time, geospatial units, and forecast horizons. Two-thirds of the models evaluated showed better accuracy than a naïve baseline model. Forecast accuracy degraded as models made predictions further into the future, with probabilistic error at a 20-wk horizon three to five times larger than when predicting at a 1-wk horizon. This project underscores the role that collaboration and active coordination between governmental public-health agencies, academic modeling teams, and industry partners can play in developing modern modeling capabilities to support local, state, and federal response to outbreaks

The United States COVID-19 Forecast Hub dataset

Academic researchers, government agencies, industry groups, and individuals have produced forecasts at an unprecedented scale during the COVID-19 pandemic. To leverage these forecasts, the United States Centers for Disease Control and Prevention (CDC) partnered with an academic research lab at the University of Massachusetts Amherst to create the US COVID-19 Forecast Hub. Launched in April 2020, the Forecast Hub is a dataset with point and probabilistic forecasts of incident cases, incident hospitalizations, incident deaths, and cumulative deaths due to COVID-19 at county, state, and national, levels in the United States. Included forecasts represent a variety of modeling approaches, data sources, and assumptions regarding the spread of COVID-19. The goal of this dataset is to establish a standardized and comparable set of short-term forecasts from modeling teams. These data can be used to develop ensemble models, communicate forecasts to the public, create visualizations, compare models, and inform policies regarding COVID-19 mitigation. These open-source data are available via download from GitHub, through an online API, and through R packages

Metal cofactor modulated folding and target recognition of HIV-1 NCp7.

The HIV-1 nucleocapsid 7 (NCp7) plays crucial roles in multiple stages of HIV-1 life cycle, and its biological functions rely on the binding of zinc ions. Understanding the molecular mechanism of how the zinc ions modulate the conformational dynamics and functions of the NCp7 is essential for the drug development and HIV-1 treatment. In this work, using a structure-based coarse-grained model, we studied the effects of zinc cofactors on the folding and target RNA(SL3) recognition of the NCp7 by molecular dynamics simulations. After reproducing some key properties of the zinc binding and folding of the NCp7 observed in previous experiments, our simulations revealed several interesting features in the metal ion modulated folding and target recognition. Firstly, we showed that the zinc binding makes the folding transition states of the two zinc fingers less structured, which is in line with the Hammond effect observed typically in mutation, temperature or denaturant induced perturbations to protein structure and stability. Secondly, We showed that there exists mutual interplay between the zinc ion binding and NCp7-target recognition. Binding of zinc ions enhances the affinity between the NCp7 and the target RNA, whereas the formation of the NCp7-RNA complex reshapes the intrinsic energy landscape of the NCp7 and increases the stability and zinc affinity of the two zinc fingers. Thirdly, by characterizing the effects of salt concentrations on the target RNA recognition, we showed that the NCp7 achieves optimal balance between the affinity and binding kinetics near the physiologically relevant salt concentrations. In addition, the effects of zinc binding on the inter-domain conformational flexibility and folding cooperativity of the NCp7 were also discussed

High Spatial and Temporal Variations of Microbial Community along the Southern Catfish Gastrointestinal Tract: Insights into Dynamic Food Digestion

The fish intestinal microbiota is affected by dietary shifts or diet-related seasonal fluctuations making it highly variable and dynamic. It assists with the digestion and absorption of food that is a common, yet dynamic process. However, fundamental dynamics of microbial ecology associated with food digestion in intestine and stomach are poorly understood in fish. We selected the southern catfish, Silurus meridionalis, as the targeted species, owing to its foraging behavior with a large meal that can assure clear periodic rhythms in food digestion, to study spatial variations of the microbial community along the gastrointestinal (GI) tract. We further evaluated temporal microbial dynamics by collecting GI tract samples at time intervals 03, 12, and 24h after feeding. High-throughput sequencing results showed higher microbial diversity in the stomach than in the intestine and distinguishable community structures between stomach and intestine. Firmicutes were dominated by both Clostridium and unclassified Clostridiaceae, which was the most abundant taxon in the stomach, whereas Fusobacteria were dominated by Cetobacterium, which prevailed in the intestine. Firmicutes was significantly increased and Fusobacteria was decreased after feeding. Furthermore, inter-stomach microbial variability was greater than inter-intestine microbial variability. These results demonstrate that GI microbial assemblies are specific per anatomical site and are highly dynamic during food digestion, indicating that digestive status and/or sampling time are factors potentially influencing the microbial compositions. Furthermore, the finding of high spatial and temporal variations of the microbial community along the GI tract suggests limitations of single sampling regime to study food-derived microbial ecology