Leveraging Optimal Transport for Enhanced Offline Reinforcement Learning in Surgical Robotic Environments

Most Reinforcement Learning (RL) methods are traditionally studied in an active learning setting, where agents directly interact with their environments, observe action outcomes, and learn through trial and error. However, allowing partially trained agents to interact with real physical systems poses significant challenges, including high costs, safety risks, and the need for constant supervision. Offline RL addresses these cost and safety concerns by leveraging existing datasets and reducing the need for resource-intensive real-time interactions. Nevertheless, a substantial challenge lies in the demand for these datasets to be meticulously annotated with rewards. In this paper, we introduce Optimal Transport Reward (OTR) labelling, an innovative algorithm designed to assign rewards to offline trajectories, using a small number of high-quality expert demonstrations. The core principle of OTR involves employing Optimal Transport (OT) to calculate an optimal alignment between an unlabeled trajectory from the dataset and an expert demonstration. This alignment yields a similarity measure that is effectively interpreted as a reward signal. An offline RL algorithm can then utilize these reward signals to learn a policy. This approach circumvents the need for handcrafted rewards, unlocking the potential to harness vast datasets for policy learning. Leveraging the SurRoL simulation platform tailored for surgical robot learning, we generate datasets and employ them to train policies using the OTR algorithm. By demonstrating the efficacy of OTR in a different domain, we emphasize its versatility and its potential to expedite RL deployment across a wide range of fields.Comment: Preprin

A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges

In recent years, the development of robotics and artificial intelligence (AI) systems has been nothing short of remarkable. As these systems continue to evolve, they are being utilized in increasingly complex and unstructured environments, such as autonomous driving, aerial robotics, and natural language processing. As a consequence, programming their behaviors manually or defining their behavior through reward functions (as done in reinforcement learning (RL)) has become exceedingly difficult. This is because such environments require a high degree of flexibility and adaptability, making it challenging to specify an optimal set of rules or reward signals that can account for all possible situations. In such environments, learning from an expert's behavior through imitation is often more appealing. This is where imitation learning (IL) comes into play - a process where desired behavior is learned by imitating an expert's behavior, which is provided through demonstrations. This paper aims to provide an introduction to IL and an overview of its underlying assumptions and approaches. It also offers a detailed description of recent advances and emerging areas of research in the field. Additionally, the paper discusses how researchers have addressed common challenges associated with IL and provides potential directions for future research. Overall, the goal of the paper is to provide a comprehensive guide to the growing field of IL in robotics and AI.Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibl

Inferior Spear-like Lens Opacity as a Sign of Keratoconus

Purpose: To report 21 cases of typical inferior feather-shape lens opacity associated with keratoconus. Methods: In this cross-sectional study, we evaluated the association of keratoconus with inferior feather-shape lens opacity in refractive surgery candidates. Visual acuity, demographic, refractive, and topographic characteristics of 26 eyes of 21 patients with inferior feather-shape lens opacity were evaluated in detail. Pedigree analysis was also performed to assess possible inheritance. Results: Overall, 2122 out of 33,368 cases (6.4%) without lens opacity had keratoconus, while 20 out of 21 patients (95.2%) with peculiar lens opacity had definite keratoconus (P < 0.001). Lens opacity was bilateral in 5 cases (24%), and keratoconus was bilateral in all 20 patients with lens opacity. Nine eyes out of thirty-six with a complete data record (25%) had severe keratoconus and underwent deep lamellar keratoplasty, while 11 (31%) had forme fruste keratoconus. Pedigrees were drawn for eight patients, most families of whom suggested an X-linked recessive inheritance. Conclusion: The present study was the first to investigate patients with a peculiar inferior feather-shape lens opacity accompanied by bilateral keratoconus, which was observed in 95% of the patients. This finding should raise awareness as to the possibility of diagnosing keratoconus in the eyes of the patients with these characteristics

Design and conduct of 'Xtreme Alps' : a double-blind, randomised controlled study of the effects of dietary nitrate supplementation on acclimatisation to high altitude

The study of healthy human volunteers ascending to high altitude provides a robust model of the complex physiological interplay that emulates human adaptation to hypoxaemia in clinical conditions. Nitric oxide (NO) metabolism may play an important role in both adaptation to high altitude and response to hypoxaemia during critical illness at sea level. Circulating nitrate and nitrite concentrations can be augmented by dietary supplementation and this is associated with improved exercise performance and mitochondrial efficiency. We hypothesised that the administration of a dietary substance (beetroot juice) rich in nitrate would improve oxygen efficiency during exercise at high altitude by enhancing tissue microcirculatory blood flow and oxygenation. Furthermore, nitrate supplementation would lead to measurable increases in NO bioactivity throughout the body. This methodological manuscript describes the design and conduct of the ‘Xtreme Alps’ expedition, a double-blind randomised controlled trial investigating the effects of dietary nitrate supplementation on acclimatisation to hypobaric hypoxia at high altitude in healthy human volunteers. The primary outcome measure was the change in oxygen efficiency during exercise at high altitude between participants allocated to receive nitrate supplementation and those receiving a placebo. A number of secondary measures were recorded, including exercise capacity, peripheral and microcirculatory blood flow and tissue oxygenation. Results from this study will further elucidate the role of NO in adaption to hypoxaemia and guide clinical trials in critically ill patients. Improved understanding of hypoxaemia in critical illness may provide new therapeutic avenues for interventions that will improve survival in critically ill patients

Disability-adjusted life-years (DALYs) for 315 diseases and injuries and healthy life expectancy (HALE) in Iran and its neighboring countries, 1990–2015

BACKGROUND: Summary measures of health are essential in making estimates of health status that are comparable across time and place. They can be used for assessing the performance of health systems, informing effective policy making, and monitoring the progress of nations toward achievement of sustainable development goals. The Global Burden of Diseases, Injuries, and Risk Factors Study 2015 (GBD 2015) provides disability-adjusted life-years (DALYs) and healthy life expectancy (HALE) as main summary measures of health. We assessed the trends of health status in Iran and 15 neighboring countries using these summary measures. METHODS: We used the results of GBD 2015 to present the levels and trends of DALYs, life expectancy (LE), and HALE in Iran and its 15 neighboring countries from 1990 to 2015. For each country, we assessed the ratio of observed levels of DALYs and HALE to those expected based on socio-demographic index (SDI), an indicator composed of measures of total fertility rate, income per capita, and average years of schooling. RESULTS: All-age numbers of DALYs reached over 19 million years in Iran in 2015. The all-age number of DALYs has remained stable during the past two decades in Iran, despite the decreasing trends in all-age and age-standardized rates. The all-cause DALY rates decreased from 47,200 in 1990 to 28,400 per 100,000 in 2015. The share of non-communicable diseases in DALYs increased in Iran (from 42% to 74%) and all of its neighbors between 1990 and 2015; the pattern of change is similar in almost all 16 countries. The DALY rates for NCDs and injuries in Iran were higher than global rates and the average rate in High Middle SDI countries, while those for communicable, maternal, neonatal, and nutritional disorders were much lower in Iran. Among men, cardiovascular diseases ranked first in all countries of the region except for Bahrain. Among women, they ranked first in 13 countries. Life expectancy and HALE show a consistent increase in all countries. Still, there are dissimilarities indicating a generally low LE and HALE in Afghanistan and Pakistan and high expectancy in Qatar, Kuwait, and Saudi Arabia. Iran ranked 11th in terms of LE at birth and 12th in terms of HALE at birth in 1990 which improved to 9th for both metrics in 2015. Turkey and Iran had the highest increase in LE and HALE from 1990 to 2015 while the lowest increase was observed in Armenia, Pakistan, Kuwait, Kazakhstan, Russia, and Iraq. CONCLUSIONS: The levels and trends in causes of DALYs, life expectancy, and HALE generally show similarities between the 16 countries, although differences exist. The differences observed between countries can be attributed to a myriad of determinants, including social, cultural, ethnic, religious, political, economic, and environmental factors as well as the performance of the health system. Investigating the differences between countries can inform more effective health policy and resource allocation. Concerted efforts at national and regional levels are required to tackle the emerging burden of non-communicable diseases and injuries in Iran and its neighbors

Transport injuries and deaths in the Eastern Mediterranean Region : findings from the Global Burden of Disease 2015 Study

Transport injuries (TI) are ranked as one of the leading causes of death, disability, and property loss worldwide. This paper provides an overview of the burden of TI in the Eastern Mediterranean Region (EMR) by age and sex from 1990 to 2015. Transport injuries mortality in the EMR was estimated using the Global Burden of Disease mortality database, with corrections for ill-defined causes of death, using the cause of death ensemble modeling tool. Morbidity estimation was based on inpatient and outpatient datasets, 26 cause-of-injury and 47 nature-of-injury categories. In 2015, 152,855 (95% uncertainty interval: 137,900-168,100) people died from TI in the EMR countries. Between 1990 and 2015, the years of life lost (YLL) rate per 100,000 due to TI decreased by 15.5%, while the years lived with disability (YLD) rate decreased by 10%, and the age-standardized disability-adjusted life years (DALYs) rate decreased by 16%. Although the burden of TI mortality and morbidity decreased over the last two decades, there is still a considerable burden that needs to be addressed by increasing awareness, enforcing laws, and improving road conditions.Peer reviewe

Maternal mortality and morbidity burden in the Eastern Mediterranean region : findings from the Global Burden of Disease 2015 study

Assessing the burden of maternal mortality is important for tracking progress and identifying public health gaps. This paper provides an overview of the burden of maternal mortality in the Eastern Mediterranean Region (EMR) by underlying cause and age from 1990 to 2015. We used the results of the Global Burden of Disease 2015 study to explore maternal mortality in the EMR countries. The maternal mortality ratio in the EMR decreased 16.3% from 283 (241-328) maternal deaths per 100,000 live births in 1990 to 237 (188-293) in 2015. Maternal mortality ratio was strongly correlated with socio-demographic status, where the lowest-income countries contributed the most to the burden of maternal mortality in the region. Progress in reducing maternal mortality in the EMR has accelerated in the past 15 years, but the burden remains high. Coordinated and rigorous efforts are needed to make sure that adequate and timely services and interventions are available for women at each stage of reproductive life

Mapping geographical inequalities in access to drinking water and sanitation facilities in low-income and middle-income countries, 2000-17

Background: Universal access to safe drinking water and sanitation facilities is an essential human right, recognised in the Sustainable Development Goals as crucial for preventing disease and improving human wellbeing. Comprehensive, high-resolution estimates are important to inform progress towards achieving this goal. We aimed to produce high-resolution geospatial estimates of access to drinking water and sanitation facilities. Methods: We used a Bayesian geostatistical model and data from 600 sources across more than 88 low-income and middle-income countries (LMICs) to estimate access to drinking water and sanitation facilities on continuous continent-wide surfaces from 2000 to 2017, and aggregated results to policy-relevant administrative units. We estimated mutually exclusive and collectively exhaustive subcategories of facilities for drinking water (piped water on or off premises, other improved facilities, unimproved, and surface water) and sanitation facilities (septic or sewer sanitation, other improved, unimproved, and open defecation) with use of ordinal regression. We also estimated the number of diarrhoeal deaths in children younger than 5 years attributed to unsafe facilities and estimated deaths that were averted by increased access to safe facilities in 2017, and analysed geographical inequality in access within LMICs. Findings: Across LMICs, access to both piped water and improved water overall increased between 2000 and 2017, with progress varying spatially. For piped water, the safest water facility type, access increased from 40·0% (95% uncertainty interval [UI] 39·4–40·7) to 50·3% (50·0–50·5), but was lowest in sub-Saharan Africa, where access to piped water was mostly concentrated in urban centres. Access to both sewer or septic sanitation and improved sanitation overall also increased across all LMICs during the study period. For sewer or septic sanitation, access was 46·3% (95% UI 46·1–46·5) in 2017, compared with 28·7% (28·5–29·0) in 2000. Although some units improved access to the safest drinking water or sanitation facilities since 2000, a large absolute number of people continued to not have access in several units with high access to such facilities (>80%) in 2017. More than 253 000 people did not have access to sewer or septic sanitation facilities in the city of Harare, Zimbabwe, despite 88·6% (95% UI 87·2–89·7) access overall. Many units were able to transition from the least safe facilities in 2000 to safe facilities by 2017; for units in which populations primarily practised open defecation in 2000, 686 (95% UI 664–711) of the 1830 (1797–1863) units transitioned to the use of improved sanitation. Geographical disparities in access to improved water across units decreased in 76·1% (95% UI 71·6–80·7) of countries from 2000 to 2017, and in 53·9% (50·6–59·6) of countries for access to improved sanitation, but remained evident subnationally in most countries in 2017. Interpretation: Our estimates, combined with geospatial trends in diarrhoeal burden, identify where efforts to increase access to safe drinking water and sanitation facilities are most needed. By highlighting areas with successful approaches or in need of targeted interventions, our estimates can enable precision public health to effectively progress towards universal access to safe water and sanitation

Mapping local patterns of childhood overweight and wasting in low- and middle-income countries between 2000 and 2017

A double burden of malnutrition occurs when individuals, household members or communities experience both undernutrition and overweight. Here, we show geospatial estimates of overweight and wasting prevalence among children under 5 years of age in 105 low- and middle-income countries (LMICs) from 2000 to 2017 and aggregate these to policy-relevant administrative units. Wasting decreased overall across LMICs between 2000 and 2017, from 8.4% (62.3 (55.1–70.8) million) to 6.4% (58.3 (47.6–70.7) million), but is predicted to remain above the World Health Organization’s Global Nutrition Target of <5% in over half of LMICs by 2025. Prevalence of overweight increased from 5.2% (30 (22.8–38.5) million) in 2000 to 6.0% (55.5 (44.8–67.9) million) children aged under 5 years in 2017. Areas most affected by double burden of malnutrition were located in Indonesia, Thailand, southeastern China, Botswana, Cameroon and central Nigeria. Our estimates provide a new perspective to researchers, policy makers and public health agencies in their efforts to address this global childhood syndemic

Global, regional, and national cancer incidence, mortality, years of life lost, years lived with disability, and disability-Adjusted life-years for 29 cancer groups, 1990 to 2017 : A systematic analysis for the global burden of disease study

Importance: Cancer and other noncommunicable diseases (NCDs) are now widely recognized as a threat to global development. The latest United Nations high-level meeting on NCDs reaffirmed this observation and also highlighted the slow progress in meeting the 2011 Political Declaration on the Prevention and Control of Noncommunicable Diseases and the third Sustainable Development Goal. Lack of situational analyses, priority setting, and budgeting have been identified as major obstacles in achieving these goals. All of these have in common that they require information on the local cancer epidemiology. The Global Burden of Disease (GBD) study is uniquely poised to provide these crucial data. Objective: To describe cancer burden for 29 cancer groups in 195 countries from 1990 through 2017 to provide data needed for cancer control planning. Evidence Review: We used the GBD study estimation methods to describe cancer incidence, mortality, years lived with disability, years of life lost, and disability-Adjusted life-years (DALYs). Results are presented at the national level as well as by Socio-demographic Index (SDI), a composite indicator of income, educational attainment, and total fertility rate. We also analyzed the influence of the epidemiological vs the demographic transition on cancer incidence. Findings: In 2017, there were 24.5 million incident cancer cases worldwide (16.8 million without nonmelanoma skin cancer [NMSC]) and 9.6 million cancer deaths. The majority of cancer DALYs came from years of life lost (97%), and only 3% came from years lived with disability. The odds of developing cancer were the lowest in the low SDI quintile (1 in 7) and the highest in the high SDI quintile (1 in 2) for both sexes. In 2017, the most common incident cancers in men were NMSC (4.3 million incident cases); tracheal, bronchus, and lung (TBL) cancer (1.5 million incident cases); and prostate cancer (1.3 million incident cases). The most common causes of cancer deaths and DALYs for men were TBL cancer (1.3 million deaths and 28.4 million DALYs), liver cancer (572000 deaths and 15.2 million DALYs), and stomach cancer (542000 deaths and 12.2 million DALYs). For women in 2017, the most common incident cancers were NMSC (3.3 million incident cases), breast cancer (1.9 million incident cases), and colorectal cancer (819000 incident cases). The leading causes of cancer deaths and DALYs for women were breast cancer (601000 deaths and 17.4 million DALYs), TBL cancer (596000 deaths and 12.6 million DALYs), and colorectal cancer (414000 deaths and 8.3 million DALYs). Conclusions and Relevance: The national epidemiological profiles of cancer burden in the GBD study show large heterogeneities, which are a reflection of different exposures to risk factors, economic settings, lifestyles, and access to care and screening. The GBD study can be used by policy makers and other stakeholders to develop and improve national and local cancer control in order to achieve the global targets and improve equity in cancer care. © 2019 American Medical Association. All rights reserved.Peer reviewe