From Optimization to Control: Quasi Policy Iteration

Recent control algorithms for Markov decision processes (MDPs) have been designed using an implicit analogy with well-established optimization algorithms. In this paper, we make this analogy explicit across four problem classes with a unified solution characterization. This novel framework, in turn, allows for a systematic transformation of algorithms from one domain to the other. In particular, we identify equivalent optimization and control algorithms that have already been pointed out in the existing literature, but mostly in a scattered way. With this unifying framework in mind, we then exploit two linear structural constraints specific to MDPs for approximating the Hessian in a second-order-type algorithm from optimization, namely, Anderson mixing. This leads to a novel first-order control algorithm that modifies the standard value iteration (VI) algorithm by incorporating two new directions and adaptive step sizes. While the proposed algorithm, coined as quasi-policy iteration, has the same computational complexity as VI, it interestingly exhibits an empirical convergence behavior similar to policy iteration with a very low sensitivity to the discount factor

Optimizing Hardware Simulation and Realization of Discrete Cosine Transform Using VHDL Hardware Description Language

Discrete cosine transform (DCT) is the fundamental part of JPEG compressor and is one of the most widely used conversion technique in digital signal processing (DSP) and image compression. Due to importance of the discrete cosine transform in JPEG standard, an algorithm is proposed that is in parallel structure thus intensify hardware implementation speed of discrete cosine transform and JPEG compression procedure. The proposed method is implemented by utilizing VHDL hardware description language in structural format and follows optimal programming tips by which, low hardware resource utilization, low latency, high throughput and high clock rate are achieved. Inputs are 8-bit long, 4 separate units are considered and CSA and CLA adders are used to realize discrete cosine transform. Working frequency for this implementation is 100 MHz and each stage delay is 10ns which is optimum in comparison with other methods. This proposed method can be easily utilized in any hardware applications such as JPEG compressor, image/signal processing and etc. by minimum change in design parameters. Also, it can be used as a hard-core in embedded systems, system on chips (SOC), system on programmable chips (SOPC) and network on chips (NOC)

Condicionamento osmótico das sementes e disponibilidade de enxofre na estabilidade da membrana celular e produtividade de soja em solo salino

The objective of this work was to determine the effects of seed priming and sulfur application on cell membrane characteristics, seedling emergence, chlorophyll content and grain yield of soybean (Glycine max) in saline soil. A complete-block design in 4x3 factorial arrangement with three replicates was used to test four types of seed priming (water, auxin, gibberellin and non-priming) and three levels of sulfate availability (0, 70 and 140 kg ha-1 K2SO4). The soil had a silty loam texture with an electrical conductivity of 3.61 ds m-1, a pH of 8.2 and a saturation percentage of about 46%. Seed priming had significant effects on mean emergence rate (MER), emergence percentage, relative water content (RWC) of leaves, relative chlorophyll content, time of maturity, shoot length and grain yield. The highest values for these variables were observed in the priming treatments, except for the time of maturity. Sulfur application had significant effects on MER, shoot length, RWC, membrane injury index and grain yield. Priming treatments provide greater emergence rates and grain yields and interact sinergicaly with sulfur rates.O objetivo deste trabalho foi determinar os efeitos do condicionamento osmótico das sementes e da disponibilidade de enxofre sobre características da membrana celular, emergência de plântulas, conteúdo relativo de clorofila e produtividade de soja (Glycine max) em solo salino. O experimento foi conduzido em delineamento de blocos ao acaso, em arranjo fatorial 4x3, com três repetições, para avaliar quatro tipos de condicionamento osmótico das sementes (água, auxina, giberelina e ausência de condicionamento) e três níveis de disponibilidade de enxofre para as plantas (aplicação ao solo de 0, 70 e 140 kg ha-1 de K2SO4). O solo utilizado apresentava textura média, com condutividade elétrica de aproximadamente 3,61 ds m-1, pH de 8,2 e percentagem de saturação em torno de 46%. O condicionamento osmótico das sementes teve efeito significativo sobre a taxa média de emergência (MER), percentagem de emergência, conteúdo relativo de água das folhas (RWC), conteúdo relativo de clorofila, época de maturação, comprimento da parte aérea e produtividade de grãos. Os maiores valores para essas variáveis, exceto época de maturação, foram observados nos tratamentos com condicionamento osmótico. A aplicação de enxofre teve efeito significativo sobre MER, comprimento da parte aérea, RWC, índice de dano à membrana celular, e produtividade de grãos. O condicionamento osmótico proporciona maiores taxas de emergência e produtividade de grãos e interage significativamente com a aplicação de enxofre

Modulation of Drug Craving in Crystalline-Heroin Users by Transcranial Direct Current Stimulation of Dorsolateral Prefrontal Cortex

Background: Drug craving, the main cause of relapse and a major motivator for drug use, is a challenging obstacle in substance use treatment. Transcranial direct current stimulation (tDCS), a non-invasive neuromodulatory technique, has shown promising outcomes in treating different neuropsychiatric disorders such as drug addiction, more specifically on drug craving. The aim in the current study was to examine the effects of applying tDCS on dorsolateral prefrontal cortex (DLPFC) in reducing drug cravings in former crystalline-heroin users enrolled in methadone maintenance (MMT) programs. Methods: The present study was a semi-experimental, crossover study with pre/post-test, and a control group. 40 right-handed men were selected from former crystalline-heroin users enrolled in MMT programs in Tehran, Iran. They were then divided into two matched groups based on age, education, and age of onset crystalline-heroin abuse. Desire for Drug Questionnaire (DDQ) was administered two times to all of the subjects, before first brain stimulation, and at the end of the last session. Experimental group received TDCS on DLPFC, and sham stimulation was applied on control subjects. The data were analyzed by analysis of covariance (ANCOVA) method using SPSS software. Findings: The study results indicated anodal tDCS over right and cathodal TDCS over left DLPFC, and in parallel with sham, significantly decreased drug cravings among former crystalline-heroin users (P < 0.050). Conclusion: This study showed that applying TDCS on DLPFC of former crystalline-heroin users reduces drug craving. The findings of this study expanded the results of previous studies on effects of this neuromodulatory technique for drug craving reduction in other drug type settings

Global age-sex-specific mortality, life expectancy, and population estimates in 204 countries and territories and 811 subnational locations, 1950–2021, and the impact of the COVID-19 pandemic: a comprehensive demographic analysis for the Global Burden of Disease Study 2021

Background: Estimates of demographic metrics are crucial to assess levels and trends of population health outcomes. The profound impact of the COVID-19 pandemic on populations worldwide has underscored the need for timely estimates to understand this unprecedented event within the context of long-term population health trends. The Global Burden of Diseases, Injuries, and Risk Factors Study (GBD) 2021 provides new demographic estimates for 204 countries and territories and 811 additional subnational locations from 1950 to 2021, with a particular emphasis on changes in mortality and life expectancy that occurred during the 2020–21 COVID-19 pandemic period. Methods: 22 223 data sources from vital registration, sample registration, surveys, censuses, and other sources were used to estimate mortality, with a subset of these sources used exclusively to estimate excess mortality due to the COVID-19 pandemic. 2026 data sources were used for population estimation. Additional sources were used to estimate migration; the effects of the HIV epidemic; and demographic discontinuities due to conflicts, famines, natural disasters, and pandemics, which are used as inputs for estimating mortality and population. Spatiotemporal Gaussian process regression (ST-GPR) was used to generate under-5 mortality rates, which synthesised 30 763 location-years of vital registration and sample registration data, 1365 surveys and censuses, and 80 other sources. ST-GPR was also used to estimate adult mortality (between ages 15 and 59 years) based on information from 31 642 location-years of vital registration and sample registration data, 355 surveys and censuses, and 24 other sources. Estimates of child and adult mortality rates were then used to generate life tables with a relational model life table system. For countries with large HIV epidemics, life tables were adjusted using independent estimates of HIV-specific mortality generated via an epidemiological analysis of HIV prevalence surveys, antenatal clinic serosurveillance, and other data sources. Excess mortality due to the COVID-19 pandemic in 2020 and 2021 was determined by subtracting observed all-cause mortality (adjusted for late registration and mortality anomalies) from the mortality expected in the absence of the pandemic. Expected mortality was calculated based on historical trends using an ensemble of models. In location-years where all-cause mortality data were unavailable, we estimated excess mortality rates using a regression model with covariates pertaining to the pandemic. Population size was computed using a Bayesian hierarchical cohort component model. Life expectancy was calculated using age-specific mortality rates and standard demographic methods. Uncertainty intervals (UIs) were calculated for every metric using the 25th and 975th ordered values from a 1000-draw posterior distribution. Findings: Global all-cause mortality followed two distinct patterns over the study period: age-standardised mortality rates declined between 1950 and 2019 (a 62·8% [95% UI 60·5–65·1] decline), and increased during the COVID-19 pandemic period (2020–21; 5·1% [0·9–9·6] increase). In contrast with the overall reverse in mortality trends during the pandemic period, child mortality continued to decline, with 4·66 million (3·98–5·50) global deaths in children younger than 5 years in 2021 compared with 5·21 million (4·50–6·01) in 2019. An estimated 131 million (126–137) people died globally from all causes in 2020 and 2021 combined, of which 15·9 million (14·7–17·2) were due to the COVID-19 pandemic (measured by excess mortality, which includes deaths directly due to SARS-CoV-2 infection and those indirectly due to other social, economic, or behavioural changes associated with the pandemic). Excess mortality rates exceeded 150 deaths per 100 000 population during at least one year of the pandemic in 80 countries and territories, whereas 20 nations had a negative excess mortality rate in 2020 or 2021, indicating that all-cause mortality in these countries was lower during the pandemic than expected based on historical trends. Between 1950 and 2021, global life expectancy at birth increased by 22·7 years (20·8–24·8), from 49·0 years (46·7–51·3) to 71·7 years (70·9–72·5). Global life expectancy at birth declined by 1·6 years (1·0–2·2) between 2019 and 2021, reversing historical trends. An increase in life expectancy was only observed in 32 (15·7%) of 204 countries and territories between 2019 and 2021. The global population reached 7·89 billion (7·67–8·13) people in 2021, by which time 56 of 204 countries and territories had peaked and subsequently populations have declined. The largest proportion of population growth between 2020 and 2021 was in sub-Saharan Africa (39·5% [28·4–52·7]) and south Asia (26·3% [9·0–44·7]). From 2000 to 2021, the ratio of the population aged 65 years and older to the population aged younger than 15 years increased in 188 (92·2%) of 204 nations. Interpretation: Global adult mortality rates markedly increased during the COVID-19 pandemic in 2020 and 2021, reversing past decreasing trends, while child mortality rates continued to decline, albeit more slowly than in earlier years. Although COVID-19 had a substantial impact on many demographic indicators during the first 2 years of the pandemic, overall global health progress over the 72 years evaluated has been profound, with considerable improvements in mortality and life expectancy. Additionally, we observed a deceleration of global population growth since 2017, despite steady or increasing growth in lower-income countries, combined with a continued global shift of population age structures towards older ages. These demographic changes will likely present future challenges to health systems, economies, and societies. The comprehensive demographic estimates reported here will enable researchers, policy makers, health practitioners, and other key stakeholders to better understand and address the profound changes that have occurred in the global health landscape following the first 2 years of the COVID-19 pandemic, and longer-term trends beyond the pandemic

Global incidence, prevalence, years lived with disability (YLDs), disability-adjusted life-years (DALYs), and healthy life expectancy (HALE) for 371 diseases and injuries in 204 countries and territories and 811 subnational locations, 1990–2021: a systematic analysis for the Global Burden of Disease Study 2021

Background: Detailed, comprehensive, and timely reporting on population health by underlying causes of disability and premature death is crucial to understanding and responding to complex patterns of disease and injury burden over time and across age groups, sexes, and locations. The availability of disease burden estimates can promote evidence-based interventions that enable public health researchers, policy makers, and other professionals to implement strategies that can mitigate diseases. It can also facilitate more rigorous monitoring of progress towards national and international health targets, such as the Sustainable Development Goals. For three decades, the Global Burden of Diseases, Injuries, and Risk Factors Study (GBD) has filled that need. A global network of collaborators contributed to the production of GBD 2021 by providing, reviewing, and analysing all available data. GBD estimates are updated routinely with additional data and refined analytical methods. GBD 2021 presents, for the first time, estimates of health loss due to the COVID-19 pandemic. Methods: The GBD 2021 disease and injury burden analysis estimated years lived with disability (YLDs), years of life lost (YLLs), disability-adjusted life-years (DALYs), and healthy life expectancy (HALE) for 371 diseases and injuries using 100 983 data sources. Data were extracted from vital registration systems, verbal autopsies, censuses, household surveys, disease-specific registries, health service contact data, and other sources. YLDs were calculated by multiplying cause-age-sex-location-year-specific prevalence of sequelae by their respective disability weights, for each disease and injury. YLLs were calculated by multiplying cause-age-sex-location-year-specific deaths by the standard life expectancy at the age that death occurred. DALYs were calculated by summing YLDs and YLLs. HALE estimates were produced using YLDs per capita and age-specific mortality rates by location, age, sex, year, and cause. 95% uncertainty intervals (UIs) were generated for all final estimates as the 2·5th and 97·5th percentiles values of 500 draws. Uncertainty was propagated at each step of the estimation process. Counts and age-standardised rates were calculated globally, for seven super-regions, 21 regions, 204 countries and territories (including 21 countries with subnational locations), and 811 subnational locations, from 1990 to 2021. Here we report data for 2010 to 2021 to highlight trends in disease burden over the past decade and through the first 2 years of the COVID-19 pandemic. Findings: Global DALYs increased from 2·63 billion (95% UI 2·44–2·85) in 2010 to 2·88 billion (2·64–3·15) in 2021 for all causes combined. Much of this increase in the number of DALYs was due to population growth and ageing, as indicated by a decrease in global age-standardised all-cause DALY rates of 14·2% (95% UI 10·7–17·3) between 2010 and 2019. Notably, however, this decrease in rates reversed during the first 2 years of the COVID-19 pandemic, with increases in global age-standardised all-cause DALY rates since 2019 of 4·1% (1·8–6·3) in 2020 and 7·2% (4·7–10·0) in 2021. In 2021, COVID-19 was the leading cause of DALYs globally (212·0 million [198·0–234·5] DALYs), followed by ischaemic heart disease (188·3 million [176·7–198·3]), neonatal disorders (186·3 million [162·3–214·9]), and stroke (160·4 million [148·0–171·7]). However, notable health gains were seen among other leading communicable, maternal, neonatal, and nutritional (CMNN) diseases. Globally between 2010 and 2021, the age-standardised DALY rates for HIV/AIDS decreased by 47·8% (43·3–51·7) and for diarrhoeal diseases decreased by 47·0% (39·9–52·9). Non-communicable diseases contributed 1·73 billion (95% UI 1·54–1·94) DALYs in 2021, with a decrease in age-standardised DALY rates since 2010 of 6·4% (95% UI 3·5–9·5). Between 2010 and 2021, among the 25 leading Level 3 causes, age-standardised DALY rates increased most substantially for anxiety disorders (16·7% [14·0–19·8]), depressive disorders (16·4% [11·9–21·3]), and diabetes (14·0% [10·0–17·4]). Age-standardised DALY rates due to injuries decreased globally by 24·0% (20·7–27·2) between 2010 and 2021, although improvements were not uniform across locations, ages, and sexes. Globally, HALE at birth improved slightly, from 61·3 years (58·6–63·6) in 2010 to 62·2 years (59·4–64·7) in 2021. However, despite this overall increase, HALE decreased by 2·2% (1·6–2·9) between 2019 and 2021. Interpretation: Putting the COVID-19 pandemic in the context of a mutually exclusive and collectively exhaustive list of causes of health loss is crucial to understanding its impact and ensuring that health funding and policy address needs at both local and global levels through cost-effective and evidence-based interventions. A global epidemiological transition remains underway. Our findings suggest that prioritising non-communicable disease prevention and treatment policies, as well as strengthening health systems, continues to be crucially important. The progress on reducing the burden of CMNN diseases must not stall; although global trends are improving, the burden of CMNN diseases remains unacceptably high. Evidence-based interventions will help save the lives of young children and mothers and improve the overall health and economic conditions of societies across the world. Governments and multilateral organisations should prioritise pandemic preparedness planning alongside efforts to reduce the burden of diseases and injuries that will strain resources in the coming decades. Funding: Bill &amp; Melinda Gates Foundation