4,625 research outputs found
Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
Learning-based pattern classifiers, including deep networks, have shown
impressive performance in several application domains, ranging from computer
vision to cybersecurity. However, it has also been shown that adversarial input
perturbations carefully crafted either at training or at test time can easily
subvert their predictions. The vulnerability of machine learning to such wild
patterns (also referred to as adversarial examples), along with the design of
suitable countermeasures, have been investigated in the research field of
adversarial machine learning. In this work, we provide a thorough overview of
the evolution of this research area over the last ten years and beyond,
starting from pioneering, earlier work on the security of non-deep learning
algorithms up to more recent work aimed to understand the security properties
of deep learning algorithms, in the context of computer vision and
cybersecurity tasks. We report interesting connections between these
apparently-different lines of work, highlighting common misconceptions related
to the security evaluation of machine-learning algorithms. We review the main
threat models and attacks defined to this end, and discuss the main limitations
of current work, along with the corresponding future challenges towards the
design of more secure learning algorithms.Comment: Accepted for publication on Pattern Recognition, 201
A Survey on ChatGPT: AI-Generated Contents, Challenges, and Solutions
With the widespread use of large artificial intelligence (AI) models such as
ChatGPT, AI-generated content (AIGC) has garnered increasing attention and is
leading a paradigm shift in content creation and knowledge representation. AIGC
uses generative large AI algorithms to assist or replace humans in creating
massive, high-quality, and human-like content at a faster pace and lower cost,
based on user-provided prompts. Despite the recent significant progress in
AIGC, security, privacy, ethical, and legal challenges still need to be
addressed. This paper presents an in-depth survey of working principles,
security and privacy threats, state-of-the-art solutions, and future challenges
of the AIGC paradigm. Specifically, we first explore the enabling technologies,
general architecture of AIGC, and discuss its working modes and key
characteristics. Then, we investigate the taxonomy of security and privacy
threats to AIGC and highlight the ethical and societal implications of GPT and
AIGC technologies. Furthermore, we review the state-of-the-art AIGC
watermarking approaches for regulatable AIGC paradigms regarding the AIGC model
and its produced content. Finally, we identify future challenges and open
research directions related to AIGC.Comment: 20 pages, 6 figures, 4 table
Adv3D: Generating Safety-Critical 3D Objects through Closed-Loop Simulation
Self-driving vehicles (SDVs) must be rigorously tested on a wide range of
scenarios to ensure safe deployment. The industry typically relies on
closed-loop simulation to evaluate how the SDV interacts on a corpus of
synthetic and real scenarios and verify it performs properly. However, they
primarily only test the system's motion planning module, and only consider
behavior variations. It is key to evaluate the full autonomy system in
closed-loop, and to understand how variations in sensor data based on scene
appearance, such as the shape of actors, affect system performance. In this
paper, we propose a framework, Adv3D, that takes real world scenarios and
performs closed-loop sensor simulation to evaluate autonomy performance, and
finds vehicle shapes that make the scenario more challenging, resulting in
autonomy failures and uncomfortable SDV maneuvers. Unlike prior works that add
contrived adversarial shapes to vehicle roof-tops or roadside to harm
perception only, we optimize a low-dimensional shape representation to modify
the vehicle shape itself in a realistic manner to degrade autonomy performance
(e.g., perception, prediction, and motion planning). Moreover, we find that the
shape variations found with Adv3D optimized in closed-loop are much more
effective than those in open-loop, demonstrating the importance of finding
scene appearance variations that affect autonomy in the interactive setting.Comment: CoRL 2023. Project page: https://waabi.ai/adv3d
AI Security Threats against Pervasive Robotic Systems: A Course for Next Generation Cybersecurity Workforce
Robotics, automation, and related Artificial Intelligence (AI) systems have
become pervasive bringing in concerns related to security, safety, accuracy,
and trust. With growing dependency on physical robots that work in close
proximity to humans, the security of these systems is becoming increasingly
important to prevent cyber-attacks that could lead to privacy invasion,
critical operations sabotage, and bodily harm. The current shortfall of
professionals who can defend such systems demands development and integration
of such a curriculum. This course description includes details about seven
self-contained and adaptive modules on "AI security threats against pervasive
robotic systems". Topics include: 1) Introduction, examples of attacks, and
motivation; 2) - Robotic AI attack surfaces and penetration testing; 3) -
Attack patterns and security strategies for input sensors; 4) - Training
attacks and associated security strategies; 5) - Inference attacks and
associated security strategies; 6) - Actuator attacks and associated security
strategies; and 7) - Ethics of AI, robotics, and cybersecurity
Security Considerations in AI-Robotics: A Survey of Current Methods, Challenges, and Opportunities
Robotics and Artificial Intelligence (AI) have been inextricably intertwined
since their inception. Today, AI-Robotics systems have become an integral part
of our daily lives, from robotic vacuum cleaners to semi-autonomous cars. These
systems are built upon three fundamental architectural elements: perception,
navigation and planning, and control. However, while the integration of
AI-Robotics systems has enhanced the quality our lives, it has also presented a
serious problem - these systems are vulnerable to security attacks. The
physical components, algorithms, and data that make up AI-Robotics systems can
be exploited by malicious actors, potentially leading to dire consequences.
Motivated by the need to address the security concerns in AI-Robotics systems,
this paper presents a comprehensive survey and taxonomy across three
dimensions: attack surfaces, ethical and legal concerns, and Human-Robot
Interaction (HRI) security. Our goal is to provide users, developers and other
stakeholders with a holistic understanding of these areas to enhance the
overall AI-Robotics system security. We begin by surveying potential attack
surfaces and provide mitigating defensive strategies. We then delve into
ethical issues, such as dependency and psychological impact, as well as the
legal concerns regarding accountability for these systems. Besides, emerging
trends such as HRI are discussed, considering privacy, integrity, safety,
trustworthiness, and explainability concerns. Finally, we present our vision
for future research directions in this dynamic and promising field
- …