52,099 research outputs found
Discovering Blind Spots in Reinforcement Learning
Agents trained in simulation may make errors in the real world due to
mismatches between training and execution environments. These mistakes can be
dangerous and difficult to discover because the agent cannot predict them a
priori. We propose using oracle feedback to learn a predictive model of these
blind spots to reduce costly errors in real-world applications. We focus on
blind spots in reinforcement learning (RL) that occur due to incomplete state
representation: The agent does not have the appropriate features to represent
the true state of the world and thus cannot distinguish among numerous states.
We formalize the problem of discovering blind spots in RL as a noisy supervised
learning problem with class imbalance. We learn models to predict blind spots
in unseen regions of the state space by combining techniques for label
aggregation, calibration, and supervised learning. The models take into
consideration noise emerging from different forms of oracle feedback, including
demonstrations and corrections. We evaluate our approach on two domains and
show that it achieves higher predictive performance than baseline methods, and
that the learned model can be used to selectively query an oracle at execution
time to prevent errors. We also empirically analyze the biases of various
feedback types and how they influence the discovery of blind spots.Comment: To appear at AAMAS 201
Grand Challenges of Traceability: The Next Ten Years
In 2007, the software and systems traceability community met at the first
Natural Bridge symposium on the Grand Challenges of Traceability to establish
and address research goals for achieving effective, trustworthy, and ubiquitous
traceability. Ten years later, in 2017, the community came together to evaluate
a decade of progress towards achieving these goals. These proceedings document
some of that progress. They include a series of short position papers,
representing current work in the community organized across four process axes
of traceability practice. The sessions covered topics from Trace Strategizing,
Trace Link Creation and Evolution, Trace Link Usage, real-world applications of
Traceability, and Traceability Datasets and benchmarks. Two breakout groups
focused on the importance of creating and sharing traceability datasets within
the research community, and discussed challenges related to the adoption of
tracing techniques in industrial practice. Members of the research community
are engaged in many active, ongoing, and impactful research projects. Our hope
is that ten years from now we will be able to look back at a productive decade
of research and claim that we have achieved the overarching Grand Challenge of
Traceability, which seeks for traceability to be always present, built into the
engineering process, and for it to have "effectively disappeared without a
trace". We hope that others will see the potential that traceability has for
empowering software and systems engineers to develop higher-quality products at
increasing levels of complexity and scale, and that they will join the active
community of Software and Systems traceability researchers as we move forward
into the next decade of research
Grand Challenges of Traceability: The Next Ten Years
In 2007, the software and systems traceability community met at the first
Natural Bridge symposium on the Grand Challenges of Traceability to establish
and address research goals for achieving effective, trustworthy, and ubiquitous
traceability. Ten years later, in 2017, the community came together to evaluate
a decade of progress towards achieving these goals. These proceedings document
some of that progress. They include a series of short position papers,
representing current work in the community organized across four process axes
of traceability practice. The sessions covered topics from Trace Strategizing,
Trace Link Creation and Evolution, Trace Link Usage, real-world applications of
Traceability, and Traceability Datasets and benchmarks. Two breakout groups
focused on the importance of creating and sharing traceability datasets within
the research community, and discussed challenges related to the adoption of
tracing techniques in industrial practice. Members of the research community
are engaged in many active, ongoing, and impactful research projects. Our hope
is that ten years from now we will be able to look back at a productive decade
of research and claim that we have achieved the overarching Grand Challenge of
Traceability, which seeks for traceability to be always present, built into the
engineering process, and for it to have "effectively disappeared without a
trace". We hope that others will see the potential that traceability has for
empowering software and systems engineers to develop higher-quality products at
increasing levels of complexity and scale, and that they will join the active
community of Software and Systems traceability researchers as we move forward
into the next decade of research
- …