5,379 research outputs found
Applications of Machine Learning to Threat Intelligence, Intrusion Detection and Malware
Artificial Intelligence (AI) and Machine Learning (ML) are emerging technologies with applications to many fields. This paper is a survey of use cases of ML for threat intelligence, intrusion detection, and malware analysis and detection. Threat intelligence, especially attack attribution, can benefit from the use of ML classification. False positives from rule-based intrusion detection systems can be reduced with the use of ML models. Malware analysis and classification can be made easier by developing ML frameworks to distill similarities between the malicious programs. Adversarial machine learning will also be discussed, because while ML can be used to solve problems or reduce analyst workload, it also introduces new attack surfaces
Text Processing Like Humans Do: Visually Attacking and Shielding NLP Systems
Visual modifications to text are often used to obfuscate offensive comments
in social media (e.g., "!d10t") or as a writing style ("1337" in "leet speak"),
among other scenarios. We consider this as a new type of adversarial attack in
NLP, a setting to which humans are very robust, as our experiments with both
simple and more difficult visual input perturbations demonstrate. We then
investigate the impact of visual adversarial attacks on current NLP systems on
character-, word-, and sentence-level tasks, showing that both neural and
non-neural models are, in contrast to humans, extremely sensitive to such
attacks, suffering performance decreases of up to 82\%. We then explore three
shielding methods---visual character embeddings, adversarial training, and
rule-based recovery---which substantially improve the robustness of the models.
However, the shielding methods still fall behind performances achieved in
non-attack scenarios, which demonstrates the difficulty of dealing with visual
attacks.Comment: Accepted as long paper at NAACL-2019; fixed one ungrammatical
sentenc
Are aligned neural networks adversarially aligned?
Large language models are now tuned to align with the goals of their
creators, namely to be "helpful and harmless." These models should respond
helpfully to user questions, but refuse to answer requests that could cause
harm. However, adversarial users can construct inputs which circumvent attempts
at alignment. In this work, we study to what extent these models remain
aligned, even when interacting with an adversarial user who constructs
worst-case inputs (adversarial examples). These inputs are designed to cause
the model to emit harmful content that would otherwise be prohibited. We show
that existing NLP-based optimization attacks are insufficiently powerful to
reliably attack aligned text models: even when current NLP-based attacks fail,
we can find adversarial inputs with brute force. As a result, the failure of
current attacks should not be seen as proof that aligned text models remain
aligned under adversarial inputs.
However the recent trend in large-scale ML models is multimodal models that
allow users to provide images that influence the text that is generated. We
show these models can be easily attacked, i.e., induced to perform arbitrary
un-aligned behavior through adversarial perturbation of the input image. We
conjecture that improved NLP attacks may demonstrate this same level of
adversarial control over text-only models
Are AlphaZero-like Agents Robust to Adversarial Perturbations?
The success of AlphaZero (AZ) has demonstrated that neural-network-based Go
AIs can surpass human performance by a large margin. Given that the state space
of Go is extremely large and a human player can play the game from any legal
state, we ask whether adversarial states exist for Go AIs that may lead them to
play surprisingly wrong actions. In this paper, we first extend the concept of
adversarial examples to the game of Go: we generate perturbed states that are
``semantically'' equivalent to the original state by adding meaningless moves
to the game, and an adversarial state is a perturbed state leading to an
undoubtedly inferior action that is obvious even for Go beginners. However,
searching the adversarial state is challenging due to the large, discrete, and
non-differentiable search space. To tackle this challenge, we develop the first
adversarial attack on Go AIs that can efficiently search for adversarial states
by strategically reducing the search space. This method can also be extended to
other board games such as NoGo. Experimentally, we show that the actions taken
by both Policy-Value neural network (PV-NN) and Monte Carlo tree search (MCTS)
can be misled by adding one or two meaningless stones; for example, on 58\% of
the AlphaGo Zero self-play games, our method can make the widely used KataGo
agent with 50 simulations of MCTS plays a losing action by adding two
meaningless stones. We additionally evaluated the adversarial examples found by
our algorithm with amateur human Go players and 90\% of examples indeed lead
the Go agent to play an obviously inferior action. Our code is available at
\url{https://PaperCode.cc/GoAttack}.Comment: Accepted by Neurips 202
Impacts and Risk of Generative AI Technology on Cyber Defense
Generative Artificial Intelligence (GenAI) has emerged as a powerful
technology capable of autonomously producing highly realistic content in
various domains, such as text, images, audio, and videos. With its potential
for positive applications in creative arts, content generation, virtual
assistants, and data synthesis, GenAI has garnered significant attention and
adoption. However, the increasing adoption of GenAI raises concerns about its
potential misuse for crafting convincing phishing emails, generating
disinformation through deepfake videos, and spreading misinformation via
authentic-looking social media posts, posing a new set of challenges and risks
in the realm of cybersecurity. To combat the threats posed by GenAI, we propose
leveraging the Cyber Kill Chain (CKC) to understand the lifecycle of
cyberattacks, as a foundational model for cyber defense. This paper aims to
provide a comprehensive analysis of the risk areas introduced by the offensive
use of GenAI techniques in each phase of the CKC framework. We also analyze the
strategies employed by threat actors and examine their utilization throughout
different phases of the CKC, highlighting the implications for cyber defense.
Additionally, we propose GenAI-enabled defense strategies that are both
attack-aware and adaptive. These strategies encompass various techniques such
as detection, deception, and adversarial training, among others, aiming to
effectively mitigate the risks posed by GenAI-induced cyber threats
- …