507 research outputs found
Resilience of multi-robot systems to physical masquerade attacks
The advent of autonomous mobile multi-robot systems has driven innovation in both the industrial and defense sectors. The integration of such systems in safety-and security-critical applications has raised concern over their resilience to attack. In this work, we investigate the security problem of a stealthy adversary masquerading as a properly functioning agent. We show that conventional multi-agent pathfinding solutions are vulnerable to these physical masquerade attacks. Furthermore, we provide a constraint-based formulation of multi-agent pathfinding that yields multi-agent plans that are provably resilient to physical masquerade attacks. This formalization leverages inter-agent observations to facilitate introspective monitoring to guarantee resilience.Accepted manuscrip
When Humans Aren't Optimal: Robots that Collaborate with Risk-Aware Humans
In order to collaborate safely and efficiently, robots need to anticipate how
their human partners will behave. Some of today's robots model humans as if
they were also robots, and assume users are always optimal. Other robots
account for human limitations, and relax this assumption so that the human is
noisily rational. Both of these models make sense when the human receives
deterministic rewards: i.e., gaining either 130 with certainty. But in
real world scenarios, rewards are rarely deterministic. Instead, we must make
choices subject to risk and uncertainty--and in these settings, humans exhibit
a cognitive bias towards suboptimal behavior. For example, when deciding
between gaining 130 only 80% of the time, people tend
to make the risk-averse choice--even though it leads to a lower expected gain!
In this paper, we adopt a well-known Risk-Aware human model from behavioral
economics called Cumulative Prospect Theory and enable robots to leverage this
model during human-robot interaction (HRI). In our user studies, we offer
supporting evidence that the Risk-Aware model more accurately predicts
suboptimal human behavior. We find that this increased modeling accuracy
results in safer and more efficient human-robot collaboration. Overall, we
extend existing rational human models so that collaborative robots can
anticipate and plan around suboptimal human behavior during HRI.Comment: ACM/IEEE International Conference on Human-Robot Interactio
Modeling Operator Behavior in the Safety Analysis of Collaborative Robotic Applications
Human-Robot Collaboration is increasingly prominent in peo-
ple's lives and in the industrial domain, for example in manufacturing
applications. The close proximity and frequent physical contacts between
humans and robots in such applications make guaranteeing suitable levels
of safety for human operators of the utmost importance. Formal veri-
cation techniques can help in this regard through the exhaustive explo-
ration of system models, which can identify unwanted situations early in
the development process. This work extends our SAFER-HRC method-
ology with a rich non-deterministic formal model of operator behaviors,
which captures the hazardous situations resulting from human errors.
The model allows safety engineers to rene their designs until all plausi-
ble erroneous behaviors are considered and mitigated
On Specifying for Trustworthiness
As autonomous systems (AS) increasingly become part of our daily lives,
ensuring their trustworthiness is crucial. In order to demonstrate the
trustworthiness of an AS, we first need to specify what is required for an AS
to be considered trustworthy. This roadmap paper identifies key challenges for
specifying for trustworthiness in AS, as identified during the "Specifying for
Trustworthiness" workshop held as part of the UK Research and Innovation (UKRI)
Trustworthy Autonomous Systems (TAS) programme. We look across a range of AS
domains with consideration of the resilience, trust, functionality,
verifiability, security, and governance and regulation of AS and identify some
of the key specification challenges in these domains. We then highlight the
intellectual challenges that are involved with specifying for trustworthiness
in AS that cut across domains and are exacerbated by the inherent uncertainty
involved with the environments in which AS need to operate.Comment: Accepted version of paper. 13 pages, 1 table, 1 figur
Large Language Models for Robotics: A Survey
The human ability to learn, generalize, and control complex manipulation
tasks through multi-modality feedback suggests a unique capability, which we
refer to as dexterity intelligence. Understanding and assessing this
intelligence is a complex task. Amidst the swift progress and extensive
proliferation of large language models (LLMs), their applications in the field
of robotics have garnered increasing attention. LLMs possess the ability to
process and generate natural language, facilitating efficient interaction and
collaboration with robots. Researchers and engineers in the field of robotics
have recognized the immense potential of LLMs in enhancing robot intelligence,
human-robot interaction, and autonomy. Therefore, this comprehensive review
aims to summarize the applications of LLMs in robotics, delving into their
impact and contributions to key areas such as robot control, perception,
decision-making, and path planning. We first provide an overview of the
background and development of LLMs for robotics, followed by a description of
the benefits of LLMs for robotics and recent advancements in robotics models
based on LLMs. We then delve into the various techniques used in the model,
including those employed in perception, decision-making, control, and
interaction. Finally, we explore the applications of LLMs in robotics and some
potential challenges they may face in the near future. Embodied intelligence is
the future of intelligent science, and LLMs-based robotics is one of the
promising but challenging paths to achieve this.Comment: Preprint. 4 figures, 3 table
Designing Trustworthy Autonomous Systems
The design of autonomous systems is challenging and ensuring their trustworthiness can have different meanings, such as i) ensuring consistency and completeness of the requirements by a correct elicitation and formalization process; ii) ensuring that requirements are correctly mapped to system implementations so that any system behaviors never violate its requirements; iii) maximizing the reuse of available components and subsystems in order to cope with the design complexity; and iv) ensuring correct coordination of the system with its environment.Several techniques have been proposed over the years to cope with specific problems. However, a holistic design framework that, leveraging on existing tools and methodologies, practically helps the analysis and design of autonomous systems is still missing. This thesis explores the problem of building trustworthy autonomous systems from different angles. We have analyzed how current approaches of formal verification can provide assurances: 1) to the requirement corpora itself by formalizing requirements with assume/guarantee contracts to detect incompleteness and conflicts; 2) to the reward function used to then train the system so that the requirements do not get misinterpreted; 3) to the execution of the system by run-time monitoring and enforcing certain invariants; 4) to the coordination of the system with other external entities in a system of system scenario and 5) to system behaviors by automatically synthesize a policy which is correct
- …