Accelerating Reinforcement Learning through the Discovery of Useful Subgoals

An ability to adjust to changing environments and unforeseen circumstances is likely to be an important component of a successful autonomous space robot. This paper shows how to augment reinforcement learning algorithms with a method for automatically discovering certain types of subgoals online. By creating useful new subgoals while learning, the agent is able to accelerate learning on a current task and to transfer its expertise to related tasks through the reuse of its ability to attain subgoals. Subgoals are created based on commonalities across multiple paths to a solution. We cast the task of finding these commonalities as a multiple-instance learning problem and use the concept of diverse density to find solutions. We introduced this approach in [10] and here we present additional results for a simulated mobile robot task

Automatic Discovery of Subgoals in Reinforcement Learning using Diverse Density

This paper presents a method by which a reinforcement learning agent can automatically discover certain types of subgoals online. By creating useful new subgoals while learning, the agent is able to accelerate learning on the current task and to transfer its expertise to other, related tasks through the reuse of its ability to attain subgoals. The agent discovers subgoals based on commonalities across multiple paths to a solution. We cast the task of finding these commonalities as a multiple-instance learning problem and use the concept of diverse density to find solutions. We illustrate this approach using several gridworld tasks

PROPOSAL DEVELOPMENT PATTERNS AND PRACTICES IN THE ACADEMIC RESEARCH COMMUNITY

A disciplined and well-structured proposal development process can increase the probability of winning an opportunity and securing funding. Many industries have established best practices for proposal development that include maintaining a rigorous schedule; conducting regular in-progress reviews; and assigning a dedicated accountable person. However, within the academic research community best practices have not been identified and common beneficial tools and techniques are used infrequently. This research thesis examined the proposal practices used within colleges and universities for academic research funding opportunities. Research for this project included a literature review; an anonymous survey with professional proposal practitioners; and interviews with subject matter experts (SMEs). The data was then reviewed, cleaned, and analyzed. In total, 55 participants participated in the research survey; however, the data from up to four respondents was excluded in some areas due to incomplete survey responses. The research showed that the academic research community may sometimes use internally developed processes for their proposals, but that these processes are widely variant and frequently do not conform to known best practices or take advantage of common tools. Further research should be conducted to: (1) identify best practices unique to the academic research community based on outcome-based criteria; (2) quantify the impact of the adoption of best practices; and (3) determine if any factors, such as funding opportunity size, university/department size, and research activity level warrant different processes, tools, and levels of oversight

Making In-Class Competitions Desirable For Marginalized Groups

Abstract-Inspired by research that indicates that direct competition is not always comfortable for female students, we redesigned an existing class competition to permit students to choose whether they wished to participate in either direct or indirect competition. We pilot tested it in the Spring of 2013 in a undergraduate/graduate class on introductory artificial intelligence at the University of Oklahoma. Although the results for female students are inconclusive due to their small number, we observed that international students embraced the indirect competitions. This suggests that allowing the option of indirect competition may also appeal other groups of students who can be marginalized in engineering. Our results indicate the international students prefer the less risky option of indirect competition

A Machine Learning Tutorial for Operational Meteorology, Part II: Neural Networks and Deep Learning

Over the past decade the use of machine learning in meteorology has grown rapidly. Specifically neural networks and deep learning have been used at an unprecedented rate. In order to fill the dearth of resources covering neural networks with a meteorological lens, this paper discusses machine learning methods in a plain language format that is targeted for the operational meteorological community. This is the second paper in a pair that aim to serve as a machine learning resource for meteorologists. While the first paper focused on traditional machine learning methods (e.g., random forest), here a broad spectrum of neural networks and deep learning methods are discussed. Specifically this paper covers perceptrons, artificial neural networks, convolutional neural networks and U-networks. Like the part 1 paper, this manuscript discusses the terms associated with neural networks and their training. Then the manuscript provides some intuition behind every method and concludes by showing each method used in a meteorological example of diagnosing thunderstorms from satellite images (e.g., lightning flashes). This paper is accompanied with an open-source code repository to allow readers to explore neural networks using either the dataset provided (which is used in the paper) or as a template for alternate datasets

Electrochemical AFM : understanding the electromaterial-cellular interface

Organic conducting polymers are emerging as an exciting new class of biomaterial that can be used to enhance and control the growth of mammalian cells for tissue regeneration and engineering application

No Thanks! A Mixed-Methods Exploration of the Social Processes Shaping Persistent Non-Initiation of Amphetamine-Type Stimulants

Amphetamine-Type Stimulants (ATS), such as amphetamines, MDMA, and methamphetamine are a commonly used class of illicit drugs in Europe. There is a large existing literature on motives for the use of illicit drugs, often focusing on initiation. However, few studies have explored the reasons why some people choose not to use drugs (non-use), and even fewer focus on the social processes influencing non-use of ATS specifically. We explored social processes related to normalization, and how persistent non-users negotiate their non-use in social contexts where ATS is used, using qualitative interview (n = 21) and survey questionnaire (n = 126) data from a mixed-method study conducted in the Netherlands and England. Our findings showed that in both countries, most participants were repeatedly exposed to ATS use, often in social or nightlife settings. Participants abstained from use for a number of reasons, including: lack of interest in illicit drug use in general; desire to maintain control over their own behavior and environment; and to avoid the associated health risks. Social processes also shaped persistent non-use of ATS, via conscious socialization with, and selection of, other non-using peers over time. Our findings contribute to the literature on the normalization thesis, showing that recreational ATS use is only partly socially accommodated and normalized among persistent non-users, suggesting differentiated normalization