Improving Multimodal Interactive Agents with Reinforcement Learning from Human Feedback

An important goal in artificial intelligence is to create agents that can both interact naturally with humans and learn from their feedback. Here we demonstrate how to use reinforcement learning from human feedback (RLHF) to improve upon simulated, embodied agents trained to a base level of competency with imitation learning. First, we collected data of humans interacting with agents in a simulated 3D world. We then asked annotators to record moments where they believed that agents either progressed toward or regressed from their human-instructed goal. Using this annotation data we leveraged a novel method - which we call "Inter-temporal Bradley-Terry" (IBT) modelling - to build a reward model that captures human judgments. Agents trained to optimise rewards delivered from IBT reward models improved with respect to all of our metrics, including subsequent human judgment during live interactions with agents. Altogether our results demonstrate how one can successfully leverage human judgments to improve agent behaviour, allowing us to use reinforcement learning in complex, embodied domains without programmatic reward functions. Videos of agent behaviour may be found at https://youtu.be/v_Z9F2_eKk4

What and how can we learn from complex global problems for antimicrobial resistance policy? A comparative study combining historical and foresight approaches

Objectives: To (i) develop a methodology for using historical and comparative perspectives to inform policy and (ii) provide evidence for antimicrobial-resistance (AMR) policymaking by drawing on lessons from climate change and tobacco control. Methods: Using a qualitative design, we systematically examined two other complex, large-scale policy issues—climate change and tobacco control—to identify what relevance to AMR can be learned from how these issues have evolved over time. During 2018–2020, we employed a five-stage approach to conducting an exploratory study involving a review of secondary historical analysis, identification of drivers of change, prioritisation of the identified drivers, scenario generation and elicitation of possible policy responses. We sought to disrupt more ‘traditional’ policy and research spaces to create an alternative where, stimulated by historical analysis, academics (including historians) and policymakers could come together to challenge norms and practices and think creatively about AMR policy design. Results: An iterative process of analysis and engagement resulted in lessons for AMR policy concerning persistent evidence gaps and uncertainty, the need for cross-sector involvement and a collective effort through global governance, the demand for new interventions through more investment in research and innovation, and recognising the dynamic relationship between social change and policy to change people's attitudes and behaviours are crucial towards tackling AMR. Conclusion: We draw on new methodological lessons around the pragmatism of future- and policy-oriented approaches incorporating robust historical and comparative analysis. The study demonstrates proof of concept and offers a reproducible method to advance further methodology, including transferrable policies that could tackle health problems, such as AMR

Economy matters to fight against malnutrition: results from a multicenter survey

Background and Aim: Malnutrition represents a serious health care threat, as it increases morbidity, mortality and health care cost. The effective screening and treatment with enteral (EN) or parenteral (PN) nutrition are the key elements of the policy called Optimal Nutrition Care for All (ONCA). The study tried to analyze the impact of the state’s economy on the implementation of EN and PN to define its role in ONCA. Material and Methods: an international survey in twenty two European countries was performed between January and December 2014. An electronic questionnaire was distributed to 22 representatives of clinical nutrition (PEN) societies. The questionnaire comprised questions regarding country economy, reimbursement, education and the use EN and PN. Return rate was 90.1% (n=20) Results: EN and PN were used in all countries surveyed (100%), but to different extent. The country’s income significantly influenced the reimbursement for EN and PN (p0.05). Education was actively carried out in all countries, however the teaching at the pre-graduate level was the least widespread, and also correlated with the country income (p=0.042). Conclusions: Results indicated that economic situation influences all aspects of ONCA, including education and treatment. The reimbursement for EN and PN seemed to be the key factor of effective campaign against malnutrition

Evolving and sustaining ocean best practices and standards for the next decade

The oceans play a key role in global issues such as climate change, food security, and human health. Given their vast dimensions and internal complexity, efficient monitoring and predicting of the planet’s ocean must be a collaborative effort of both regional and global scale. A first and foremost requirement for such collaborative ocean observing is the need to follow well-defined and reproducible methods across activities: from strategies for structuring observing systems, sensor deployment and usage, and the generation of data and information products, to ethical and governance aspects when executing ocean observing. To meet the urgent, planet-wide challenges we face, methods across all aspects of ocean observing should be broadly adopted by the ocean community and, where appropriate, should evolve into “Ocean Best Practices.” While many groups have created best practices, they are scattered across the Web or buried in local repositories and many have yet to be digitized. To reduce this fragmentation, we introduce a new open access, permanent, digital repository of best practices documentation (oceanbestpractices.org) that is part of the Ocean Best Practices System (OBPS). The new OBPS provides an opportunity space for the centralized and coordinated improvement of ocean observing methods. The OBPS repository employs user-friendly software to significantly improve discovery and access to methods. The software includes advanced semantic technologies for search capabilities to enhance repository operations. In addition to the repository, the OBPS also includes a peer reviewed journal research topic, a forum for community discussion and a training activity for use of best practices. Together, these components serve to realize a core objective of the OBPS, which is to enable the ocean community to create superior methods for every activity in ocean observing from research to operations to applications that are agreed upon and broadly adopted across communities. Using selected ocean observing examples, we show how the OBPS supports this objective. This paper lays out a future vision of ocean best practices and how OBPS will contribute to improving ocean observing in the decade to come

Controversy: Use of semi-elemental mixtures in enteral nutrition

Semi-elemental enteral nutrition mixtures are defined, by comparison with polymeric mixtures, by their content of small peptides and/or hydrolyzed protein (vs. intact protein), and a high proportion of medium-chain triglycerides (instead of long-chain triglycerides). The easier digestion is presumed to faVor intestinal absorption and improve the efficacy of enteral nutrition in clinical settings when protein hydrolysis is impaired: exocrine pancreatic insufficiency, acute pancreatitis, enteral nutrition administered into a jejunum, duodeno-pancreatectomy, interruption of biliary flow, short bowel syndrome. Even though a few small randomized trials suggest satisfactory tolerance of semi-elemental mixtures in acute pancreatitis, and improved nitrogen bioavailability in jejunal nutrition or in short bowel syndrome, hard evidence and controlled trials are scarce. The use of semi-elemental mixtures with low long-chain triglycerides to reduce lymph secretion is warranted in chylothorax or chylous ascites. There is, however, no strong evidence for an improved trophic or anti-inflammatory effect of semi-elemental, vs. polymeric mixtures in chronic inflammatory bowel disease. The higher osmolarity of semi-elemental mixtures exposes to a higher risk of poor gastrointestinal tolerance, and the nature of nitrogen supply, to a risk of impaired net protein accretion. In summary, although semi-elemental mixtures emerged over 30 years ago, there is no decisive proof of their superiority over polymeric mixtures, so the prescription of semi-elemental mixtures should probably be restricted to specific clinical settings such as chylous ascites, and in some cases of severe malabsorption, or as a second line treatment when polymeric mixtures fail. Yet such indications only rely on expert opinions, rather than strong evidence. (C) 2017 Association pour le developpement de la recherche en nutrition (ADREN)

Subcarrier Allocation for Variable Bit Rate Video Streams in Wireless OFDM Systems

Wireless OFDM systems have attractive means for adapting wireless transmission to a given situation: one possibility is to assign a varying number of subcarriers to wireless terminals for downlink communication. Deciding how many and which subcarriers to assign to a given terminal is a difficult problem. This paper concentrates on deciding how many: we use the relative length of a terminal's queue in an access point to determine this number. Applying this scheme to the transmission of homogeneous MPEG-4 videos, we obtain a significant capacity increase compared to nonadaptive subcarrier allocation schemes.QC 20140103</p

Semantic-Aware Link Layer Scheduling of MPEG-4 Video Streams in Wireless Systems

Delivering video streams to terminals via a wireless last hop is a challengingtask due to the varying nature of the wireless link. While a common approach suggeststo exploit the variations of the wireless channel, an alternative is to exploitcharacteristics of the video stream to improve the transmission. In this paper weshow how semantic stream variations of MPEG-4 coded video can be used to increasethe number of terminals which can be supported in a wireless cell at a givenminimum video quality. As example system we consider the performance of anOFDM system with different video sources. Simulations show that the number ofsupportable terminals can be increased on average by fifty percent.Qc 20131212</p