1,887 research outputs found
Human-Machine Collaborative Optimization via Apprenticeship Scheduling
Coordinating agents to complete a set of tasks with intercoupled temporal and
resource constraints is computationally challenging, yet human domain experts
can solve these difficult scheduling problems using paradigms learned through
years of apprenticeship. A process for manually codifying this domain knowledge
within a computational framework is necessary to scale beyond the
``single-expert, single-trainee" apprenticeship model. However, human domain
experts often have difficulty describing their decision-making processes,
causing the codification of this knowledge to become laborious. We propose a
new approach for capturing domain-expert heuristics through a pairwise ranking
formulation. Our approach is model-free and does not require enumerating or
iterating through a large state space. We empirically demonstrate that this
approach accurately learns multifaceted heuristics on a synthetic data set
incorporating job-shop scheduling and vehicle routing problems, as well as on
two real-world data sets consisting of demonstrations of experts solving a
weapon-to-target assignment problem and a hospital resource allocation problem.
We also demonstrate that policies learned from human scheduling demonstration
via apprenticeship learning can substantially improve the efficiency of a
branch-and-bound search for an optimal schedule. We employ this human-machine
collaborative optimization technique on a variant of the weapon-to-target
assignment problem. We demonstrate that this technique generates solutions
substantially superior to those produced by human domain experts at a rate up
to 9.5 times faster than an optimization approach and can be applied to
optimally solve problems twice as complex as those solved by a human
demonstrator.Comment: Portions of this paper were published in the Proceedings of the
International Joint Conference on Artificial Intelligence (IJCAI) in 2016 and
in the Proceedings of Robotics: Science and Systems (RSS) in 2016. The paper
consists of 50 pages with 11 figures and 4 table
PhD Thesis Proposal: Human-Machine Collaborative Optimization via Apprenticeship Scheduling
Resource optimization in health care, manufacturing, and military operations requires the careful choreography of people and equipment to effectively fulfill the responsibilities of the profession. However, resource optimization is a computationally challenging problem, and poorly utilizing resources can have drastic consequences. Within these professions, there are human domain experts who are able to learn from experience to develop strategies, heuristics, and rules-of-thumb to effectively utilize the resources at their disposal. Manually codifying these heuristics within a computational tool is a laborious process and leaves much to be desired. Even with a codified set of heuristics, it is not clear how to best insert an autonomous decision-support system into the human decision-making process. The aim of this thesis is to develop an autonomous computational method for learning domain-expert heuristics from demonstration that can support the human decision-making process. We propose a new framework, called apprenticeship scheduling, which learns and embeds these heuristics within a scalable resource optimization algorithm for real-time decision-support. Our initial investigation, comprised of developing scalable methods for scheduling and studying shared control in human-machine collaborative resource optimization, inspires the development of our apprenticeship scheduling approach. We present a promising, initial prototype for learning heuristics from demonstration and outline a plan for our continuing work
Joint Goal and Strategy Inference across Heterogeneous Demonstrators via Reward Network Distillation
Reinforcement learning (RL) has achieved tremendous success as a general
framework for learning how to make decisions. However, this success relies on
the interactive hand-tuning of a reward function by RL experts. On the other
hand, inverse reinforcement learning (IRL) seeks to learn a reward function
from readily-obtained human demonstrations. Yet, IRL suffers from two major
limitations: 1) reward ambiguity - there are an infinite number of possible
reward functions that could explain an expert's demonstration and 2)
heterogeneity - human experts adopt varying strategies and preferences, which
makes learning from multiple demonstrators difficult due to the common
assumption that demonstrators seeks to maximize the same reward. In this work,
we propose a method to jointly infer a task goal and humans' strategic
preferences via network distillation. This approach enables us to distill a
robust task reward (addressing reward ambiguity) and to model each strategy's
objective (handling heterogeneity). We demonstrate our algorithm can better
recover task reward and strategy rewards and imitate the strategies in two
simulated tasks and a real-world table tennis task.Comment: In Proceedings of the 2020 ACM/IEEE In-ternational Conference on
Human-Robot Interaction (HRI '20), March 23 to 26, 2020, Cambridge, United
Kingdom.ACM, New York, NY, USA, 10 page
Southern California Regional Workforce Development Needs Assessment for the Transportation and Supply Chain Industry Sectors
COVID-19 brought the public’s attention to the critical value of transportation and supply chain workers as lifelines to access food and other supplies. This report examines essential job skills required of the middle-skill workforce (workers with more than a high school degree, but less than a four-year college degree). Many of these middle-skill transportation and supply chain jobs are what the Federal Reserve Bank defines as “opportunity occupations” -- jobs that pay above median wages and can be accessible to those without a four-year college degree. This report lays out the complex landscape of selected technological disruptions of the supply chain to understand the new workforce needs of these middle-skill workers, followed by competencies identified by industry. With workplace social distancing policies, logistics organizations now rely heavily on data management and analysis for their operations. All rungs of employees, including warehouse workers and truck drivers, require digital skills to use mobile devices, sensors, and dashboards, among other applications. Workforce training requires a focus on data, problem solving, connectivity, and collaboration. Industry partners identified key workforce competencies required in digital literacy, data management, front/back office jobs, and in operations and maintenance. Education and training providers identified strategies to effectively develop workforce development programs. This report concludes with an exploration of the role of Institutes of Higher Education in delivering effective workforce education and training programs that reimagine how to frame programs to be customizable, easily accessible, and relevant
- …