Revisiting Maximum Entropy Inverse Reinforcement Learning: New Perspectives and Algorithms

We provide new perspectives and inference algorithms for Maximum Entropy (MaxEnt) Inverse Reinforcement Learning (IRL), which provides a principled method to find a most non-committal reward function consistent with given expert demonstrations, among many consistent reward functions. We first present a generalized MaxEnt formulation based on minimizing a KL-divergence instead of maximizing an entropy. This improves the previous heuristic derivation of the MaxEnt IRL model (for stochastic MDPs), allows a unified view of MaxEnt IRL and Relative Entropy IRL, and leads to a model-free learning algorithm for the MaxEnt IRL model. Second, a careful review of existing inference algorithms and implementations showed that they approximately compute the marginals required for learning the model. We provide examples to illustrate this, and present an efficient and exact inference algorithm. Our algorithm can handle variable length demonstrations; in addition, while a basic version takes time quadratic in the maximum demonstration length L, an improved version of this algorithm reduces this to linear using a padding trick. Experiments show that our exact algorithm improves reward learning as compared to the approximate ones. Furthermore, our algorithm scales up to a large, real-world dataset involving driver behaviour forecasting. We provide an optimized implementation compatible with the OpenAI Gym interface. Our new insight and algorithms could possibly lead to further interest and exploration of the original MaxEnt IRL model.Comment: Published as a conference paper at the 2020 IEEE Symposium Series on Computational Intelligence (SSCI

Wireless Power Transmission

Wireless Power Transmission through inductive coupling is one of the new emerging technologies that will bring tremendous change in human life. Due to shortage of time and fast running life style it is difficult to carry the complete charging set which increases the demand of the wirelessly charged products. Wireless power transfer is one of the simplest and inexpensive ways of charging as it eliminate the use of conventional copper cables and current carrying wires. In this paper, a technique is devised for a wireless power transfer through induction, and a feasible design is modeled accordingly. The technique used in this paper is the inductive coupling as it the easiest method of high efficiency power transfer without using wired medium (eg, transformer). In this paper the result of experiment is given which is done to check wireless working of a simple application by glowing LED, and charging a mobile. Wireless power transfer is not much affected by placing hurdles likes books, hands and plastic between transceiver and receiver. This research work focuses on the study of wireless power transfer for the purpose of transferring cut and dried amount of energy at maximum efficiency

Towards Reduced-Order Models for Online Motion Planning and Control of UAVs in the Presence of Wind

Abstract This paper describes a model reduction strategy for obtaining a computationally efficient prediction of a fixed-wing UAV performing waypoint navigation under steady wind conditions. The strategy relies on the off-line generation of time parametrized trajectory libraries for a set of flight conditions and reduced order basis functions functions for determining intermediate locations. It is assumed that the UAV has independent bounded control over the airspeed and altitude, and consider a 2D slice of the operating environment. We found that the reduced-order model finds intermediate positions within 10% and at speeds of 10x faster than clock-time (even in wind conditions in excess of 50% of the UAV's forward airspeed) when compared against simulation results using a medium-fidelity flight dynamics model. The potential of this strategy for online planning operations is highlighted

Minimal information for studies of extracellular vesicles (MISEV2023): From basic to advanced approaches

Extracellular vesicles (EVs), through their complex cargo, can reflect the state of their cell of origin and change the functions and phenotypes of other cells. These features indicate strong biomarker and therapeutic potential and have generated broad interest, as evidenced by the steady year-on-year increase in the numbers of scientific publications about EVs. Important advances have been made in EV metrology and in understanding and applying EV biology. However, hurdles remain to realising the potential of EVs in domains ranging from basic biology to clinical applications due to challenges in EV nomenclature, separation from non-vesicular extracellular particles, characterisation and functional studies. To address the challenges and opportunities in this rapidly evolving field, the International Society for Extracellular Vesicles (ISEV) updates its 'Minimal Information for Studies of Extracellular Vesicles', which was first published in 2014 and then in 2018 as MISEV2014 and MISEV2018, respectively. The goal of the current document, MISEV2023, is to provide researchers with an updated snapshot of available approaches and their advantages and limitations for production, separation and characterisation of EVs from multiple sources, including cell culture, body fluids and solid tissues. In addition to presenting the latest state of the art in basic principles of EV research, this document also covers advanced techniques and approaches that are currently expanding the boundaries of the field. MISEV2023 also includes new sections on EV release and uptake and a brief discussion of in vivo approaches to study EVs. Compiling feedback from ISEV expert task forces and more than 1000 researchers, this document conveys the current state of EV research to facilitate robust scientific discoveries and move the field forward even more rapidly

Global, regional, and national comparative risk assessment of 84 behavioural, environmental and occupational, and metabolic risks or clusters of risks for 195 countries and territories, 1990-2017: a systematic analysis for the Global Burden of Disease Study 2017

Background The Global Burden of Diseases, Injuries, and Risk Factors Study (GBD) 2017 comparative risk assessment (CRA) is a comprehensive approach to risk factor quantification that offers a useful tool for synthesising evidence on risks and risk–outcome associations. With each annual GBD study, we update the GBD CRA to incorporate improved methods, new risks and risk–outcome pairs, and new data on risk exposure levels and risk–outcome associations. Methods We used the CRA framework developed for previous iterations of GBD to estimate levels and trends in exposure, attributable deaths, and attributable disability-adjusted life-years (DALYs), by age group, sex, year, and location for 84 behavioural, environmental and occupational, and metabolic risks or groups of risks from 1990 to 2017. This study included 476 risk–outcome pairs that met the GBD study criteria for convincing or probable evidence of causation. We extracted relative risk and exposure estimates from 46 749 randomised controlled trials, cohort studies, household surveys, census data, satellite data, and other sources. We used statistical models to pool data, adjust for bias, and incorporate covariates. Using the counterfactual scenario of theoretical minimum risk exposure level (TMREL), we estimated the portion of deaths and DALYs that could be attributed to a given risk. We explored the relationship between development and risk exposure by modelling the relationship between the Socio-demographic Index (SDI) and risk-weighted exposure prevalence and estimated expected levels of exposure and risk-attributable burden by SDI. Finally, we explored temporal changes in risk-attributable DALYs by decomposing those changes into six main component drivers of change as follows: (1) population growth; (2) changes in population age structures; (3) changes in exposure to environmental and occupational risks; (4) changes in exposure to behavioural risks; (5) changes in exposure to metabolic risks; and (6) changes due to all other factors, approximated as the risk-deleted death and DALY rates, where the risk-deleted rate is the rate that would be observed had we reduced the exposure levels to the TMREL for all risk factors included in GBD 2017. Findings In 2017, 34·1 million (95% uncertainty interval [UI] 33·3–35·0) deaths and 1·21 billion (1·14–1·28) DALYs were attributable to GBD risk factors. Globally, 61·0% (59·6–62·4) of deaths and 48·3% (46·3–50·2) of DALYs were attributed to the GBD 2017 risk factors. When ranked by risk-attributable DALYs, high systolic blood pressure (SBP) was the leading risk factor, accounting for 10·4 million (9·39–11·5) deaths and 218 million (198–237) DALYs, followed by smoking (7·10 million [6·83–7·37] deaths and 182 million [173–193] DALYs), high fasting plasma glucose (6·53 million [5·23–8·23] deaths and 171 million [144–201] DALYs), high body-mass index (BMI; 4·72 million [2·99–6·70] deaths and 148 million [98·6–202] DALYs), and short gestation for birthweight (1·43 million [1·36–1·51] deaths and 139 million [131–147] DALYs). In total, risk-attributable DALYs declined by 4·9% (3·3–6·5) between 2007 and 2017. In the absence of demographic changes (ie, population growth and ageing), changes in risk exposure and risk-deleted DALYs would have led to a 23·5% decline in DALYs during that period. Conversely, in the absence of changes in risk exposure and risk-deleted DALYs, demographic changes would have led to an 18·6% increase in DALYs during that period. The ratios of observed risk exposure levels to exposure levels expected based on SDI (O/E ratios) increased globally for unsafe drinking water and household air pollution between 1990 and 2017. This result suggests that development is occurring more rapidly than are changes in the underlying risk structure in a population. Conversely, nearly universal declines in O/E ratios for smoking and alcohol use indicate that, for a given SDI, exposure to these risks is declining. In 2017, the leading Level 4 risk factor for age-standardised DALY rates was high SBP in four super-regions: central Europe, eastern Europe, and central Asia; north Africa and Middle East; south Asia; and southeast Asia, east Asia, and Oceania. The leading risk factor in the high-income super-region was smoking, in Latin America and Caribbean was high BMI, and in sub-Saharan Africa was unsafe sex. O/E ratios for unsafe sex in sub-Saharan Africa were notably high, and those for alcohol use in north Africa and the Middle East were notably low. Interpretation By quantifying levels and trends in exposures to risk factors and the resulting disease burden, this assessment offers insight into where past policy and programme efforts might have been successful and highlights current priorities for public health action. Decreases in behavioural, environmental, and occupational risks have largely offset the effects of population growth and ageing, in relation to trends in absolute burden. Conversely, the combination of increasing metabolic risks and population ageing will probably continue to drive the increasing trends in non-communicable diseases at the global level, which presents both a public health challenge and opportunity. We see considerable spatiotemporal heterogeneity in levels of risk exposure and risk-attributable burden. Although levels of development underlie some of this heterogeneity, O/E ratios show risks for which countries are overperforming or underperforming relative to their level of development. As such, these ratios provide a benchmarking tool to help to focus local decision making. Our findings reinforce the importance of both risk exposure monitoring and epidemiological research to assess causal connections between risks and health outcomes, and they highlight the usefulness of the GBD study in synthesising data to draw comprehensive and robust conclusions that help to inform good policy and strategic health planning

Erratum: Global, regional, and national comparative risk assessment of 84 behavioural, environmental and occupational, and metabolic risks or clusters of risks for 195 countries and territories, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017

Interpretation: By quantifying levels and trends in exposures to risk factors and the resulting disease burden, this assessment offers insight into where past policy and programme efforts might have been successful and highlights current priorities for public health action. Decreases in behavioural, environmental, and occupational risks have largely offset the effects of population growth and ageing, in relation to trends in absolute burden. Conversely, the combination of increasing metabolic risks and population ageing will probably continue to drive the increasing trends in non-communicable diseases at the global level, which presents both a public health challenge and opportunity. We see considerable spatiotemporal heterogeneity in levels of risk exposure and risk-attributable burden. Although levels of development underlie some of this heterogeneity, O/E ratios show risks for which countries are overperforming or underperforming relative to their level of development. As such, these ratios provide a benchmarking tool to help to focus local decision making. Our findings reinforce the importance of both risk exposure monitoring and epidemiological research to assess causal connections between risks and health outcomes, and they highlight the usefulness of the GBD study in synthesising data to draw comprehensive and robust conclusions that help to inform good policy and strategic health planning

Reducing the environmental impact of surgery on a global scale: systematic review and co-prioritization with healthcare workers in 132 countries

Background Healthcare cannot achieve net-zero carbon without addressing operating theatres. The aim of this study was to prioritize feasible interventions to reduce the environmental impact of operating theatres. Methods This study adopted a four-phase Delphi consensus co-prioritization methodology. In phase 1, a systematic review of published interventions and global consultation of perioperative healthcare professionals were used to longlist interventions. In phase 2, iterative thematic analysis consolidated comparable interventions into a shortlist. In phase 3, the shortlist was co-prioritized based on patient and clinician views on acceptability, feasibility, and safety. In phase 4, ranked lists of interventions were presented by their relevance to high-income countries and low–middle-income countries. Results In phase 1, 43 interventions were identified, which had low uptake in practice according to 3042 professionals globally. In phase 2, a shortlist of 15 intervention domains was generated. In phase 3, interventions were deemed acceptable for more than 90 per cent of patients except for reducing general anaesthesia (84 per cent) and re-sterilization of ‘single-use’ consumables (86 per cent). In phase 4, the top three shortlisted interventions for high-income countries were: introducing recycling; reducing use of anaesthetic gases; and appropriate clinical waste processing. In phase 4, the top three shortlisted interventions for low–middle-income countries were: introducing reusable surgical devices; reducing use of consumables; and reducing the use of general anaesthesia. Conclusion This is a step toward environmentally sustainable operating environments with actionable interventions applicable to both high– and low–middle–income countries