2,437 research outputs found
Applications of Machine Learning to Threat Intelligence, Intrusion Detection and Malware
Artificial Intelligence (AI) and Machine Learning (ML) are emerging technologies with applications to many fields. This paper is a survey of use cases of ML for threat intelligence, intrusion detection, and malware analysis and detection. Threat intelligence, especially attack attribution, can benefit from the use of ML classification. False positives from rule-based intrusion detection systems can be reduced with the use of ML models. Malware analysis and classification can be made easier by developing ML frameworks to distill similarities between the malicious programs. Adversarial machine learning will also be discussed, because while ML can be used to solve problems or reduce analyst workload, it also introduces new attack surfaces
Automating endoscopic camera motion for teleoperated minimally invasive surgery using inverse reinforcement learning
During a laparoscopic surgery, an endoscopic camera is used to provide visual feedback of the surgery to the surgeon and is controlled by a skilled assisting surgeon or a nurse. However, in robot-assisted teleoperated systems such as the daVinci surgical system, the same control lies with the operating surgeons. This results in an added task of constantly changing view point of the endoscope which can be disruptive and also increase the cognitive load on the surgeons. The work presented in this thesis aims to provide an approach that results in an intelligent camera control for such systems using machine learning algorithms. A particular task of pick and place was selected to demonstrate this approach. To add a layer of intelligence to the endoscope, the task was classified into subtasks representing the intent of the user. Neural networks with long short term memory cells (LSTMs) were trained to classify the motion of the instruments in the subtasks and a policy was calculated for each subtask using inverse reinforcement learning (IRL). Since current surgical robots do not enable the movement of the camera and instruments simultaneously, an expert data set was unavailable that could be used to train the models. Hence, a user study was conducted in which the participants were asked to complete the task of picking and placing a ring on a peg in a 3-D immersive simulation environment created using CHAI libraries. A virtual reality headset, Oculus Rift, was used during the study to track the head movements of the users to obtain their view points while they performed the task. This was considered to be expert data and was used to train the algorithm to automate the endoscope motion. A 71.3% accuracy was obtained for the classification of the task into 4 subtasks and the inverse reinforcement learning resulted in an automated trajectory of the endoscope which was 94.7% similar to the human trajectories collected demonstrating that the approach provided in thesis can be used to automate endoscopic motion similar to a skilled assisting surgeon
An Ensemble Method of Deep Reinforcement Learning for Automated Cryptocurrency Trading
We propose an ensemble method to improve the generalization performance of
trading strategies trained by deep reinforcement learning algorithms in a
highly stochastic environment of intraday cryptocurrency portfolio trading. We
adopt a model selection method that evaluates on multiple validation periods,
and propose a novel mixture distribution policy to effectively ensemble the
selected models. We provide a distributional view of the out-of-sample
performance on granular test periods to demonstrate the robustness of the
strategies in evolving market conditions, and retrain the models periodically
to address non-stationarity of financial data. Our proposed ensemble method
improves the out-of-sample performance compared with the benchmarks of a deep
reinforcement learning strategy and a passive investment strategy
Multi-objective evolution for Generalizable Policy Gradient Algorithms
Performance, generalizability, and stability are three Reinforcement Learning
(RL) challenges relevant to many practical applications in which they present
themselves in combination. Still, state-of-the-art RL algorithms fall short
when addressing multiple RL objectives simultaneously and current human-driven
design practices might not be well-suited for multi-objective RL. In this paper
we present MetaPG, an evolutionary method that discovers new RL algorithms
represented as graphs, following a multi-objective search criteria in which
different RL objectives are encoded in separate fitness scores. Our findings
show that, when using a graph-based implementation of Soft Actor-Critic (SAC)
to initialize the population, our method is able to find new algorithms that
improve upon SAC's performance and generalizability by 3% and 17%,
respectively, and reduce instability up to 65%. In addition, we analyze the
graph structure of the best algorithms in the population and offer an
interpretation of specific elements that help trading performance for
generalizability and vice versa. We validate our findings in three different
continuous control tasks: RWRL Cartpole, RWRL Walker, and Gym Pendulum.Comment: 23 pages, 12 figures, 10 table
- …