Neural Field Representations of Articulated Objects for Robotic Manipulation Planning

Traditional approaches for manipulation planning rely on an explicit geometric model of the environment to formulate a given task as an optimization problem. However, inferring an accurate model from raw sensor input is a hard problem in itself, in particular for articulated objects (e.g., closets, drawers). In this paper, we propose a Neural Field Representation (NFR) of articulated objects that enables manipulation planning directly from images. Specifically, after taking a few pictures of a new articulated object, we can forward simulate its possible movements, and, therefore, use this neural model directly for planning with trajectory optimization. Additionally, this representation can be used for shape reconstruction, semantic segmentation and image rendering, which provides a strong supervision signal during training and generalization. We show that our model, which was trained only on synthetic images, is able to extract a meaningful representation for unseen objects of the same class, both in simulation and with real images. Furthermore, we demonstrate that the representation enables robotic manipulation of an articulated object in the real world directly from images

Self-Supervised Learning of Action Affordances as Interaction Modes

When humans perform a task with an articulated object, they interact with the object only in a handful of ways, while the space of all possible interactions is nearly endless. This is because humans have prior knowledge about what interactions are likely to be successful, i.e., to open a new door we first try the handle. While learning such priors without supervision is easy for humans, it is notoriously hard for machines. In this work, we tackle unsupervised learning of priors of useful interactions with articulated objects, which we call interaction modes. In contrast to the prior art, we use no supervision or privileged information; we only assume access to the depth sensor in the simulator to learn the interaction modes. More precisely, we define a successful interaction as the one changing the visual environment substantially and learn a generative model of such interactions, that can be conditioned on the desired goal state of the object. In our experiments, we show that our model covers most of the human interaction modes, outperforms existing state-of-the-art methods for affordance learning, and can generalize to objects never seen during training. Additionally, we show promising results in the goal-conditional setup, where our model can be quickly fine-tuned to perform a given task. We show in the experiments that such affordance learning predicts interaction which covers most modes of interaction for the querying articulated object and can be fine-tuned to a goal-conditional model. For supplementary: https://actaim.github.io

New product development resource forecasting

Forecasting resource requirements for new product development (NPD) projects is essential for both strategic and tactical planning. Sophisticated, elegant planning tools to present data and inform decision-making do exist. However, in NPD, such tools run on unreliable, estimation-based resource information derived through undefined processes. This paper establishes that existing methods do not provide transparent, consistent, timely or accurate resource planning information, highlighting the need for a new approach to resource forecasting, specifically in the field of NPD. The gap between the practical issues and available methods highlights the possibility of developing a novel design of experiments approach to create resource forecasting models

Hierarchical reinforcement learning for efficient and effective automated penetration testing of large networks

Penetration testing (PT) is a method for assessing and evaluating the security of digital assets by planning, generating, and executing possible attacks that aim to discover and exploit vulnerabilities. In large networks, penetration testing becomes repetitive, complex and resource consuming despite the use of automated tools. This paper investigates reinforcement learning (RL) to make penetration testing more intelligent, targeted, and efficient. The proposed approach called Intelligent Automated Penetration Testing Framework (IAPTF) utilizes model-based RL to automate sequential decision making. Penetration testing tasks are treated as a partially observed Markov decision process (POMDP) which is solved with an external POMDP-solver using different algorithms to identify the most efficient options. A major difficulty encountered was solving large POMDPs resulting from large networks. This was overcome by representing networks hierarchically as a group of clusters and treating each cluster separately. This approach is tested through simulations of networks of various sizes. The results show that IAPTF with hierarchical network modeling outperforms previous approaches as well as human performance in terms of time, number of tested vectors and accuracy, and the advantage increases with the network size. Another advantage of IAPTF is the ease of repetition for retesting similar networks, which is often encountered in real PT. The results suggest that IAPTF is a promising approach to offload work from and ultimately replace human pen testing

Hierarchical reinforcement learning for efficient and effective automated penetration testing of large networks

Penetration testing (PT) is a method for assessing and evaluating the security of digital assets by planning, generating, and executing possible attacks that aim to discover and exploit vulnerabilities. In large networks, penetration testing becomes repetitive, complex and resource consuming despite the use of automated tools. This paper investigates reinforcement learning (RL) to make penetration testing more intelligent, targeted, and efficient. The proposed approach called Intelligent Automated Penetration Testing Framework (IAPTF) utilizes model-based RL to automate sequential decision making. Penetration testing tasks are treated as a partially observed Markov decision process (POMDP) which is solved with an external POMDP-solver using different algorithms to identify the most efficient options. A major difficulty encountered was solving large POMDPs resulting from large networks. This was overcome by representing networks hierarchically as a group of clusters and treating each cluster separately. This approach is tested through simulations of networks of various sizes. The results show that IAPTF with hierarchical network modeling outperforms previous approaches as well as human performance in terms of time, number of tested vectors and accuracy, and the advantage increases with the network size. Another advantage of IAPTF is the ease of repetition for retesting similar networks, which is often encountered in real PT. The results suggest that IAPTF is a promising approach to offload work from and ultimately replace human pen testing

Hierarchical reinforcement learning for efficient and effective automated penetration testing of large networks

Penetration testing (PT) is a method for assessing and evaluating the security of digital assets by planning, generating, and executing possible attacks that aim to discover and exploit vulnerabilities. In large networks, penetration testing becomes repetitive, complex and resource consuming despite the use of automated tools. This paper investigates reinforcement learning (RL) to make penetration testing more intelligent, targeted, and efficient. The proposed approach called Intelligent Automated Penetration Testing Framework (IAPTF) utilizes model-based RL to automate sequential decision making. Penetration testing tasks are treated as a partially observed Markov decision process (POMDP) which is solved with an external POMDP-solver using different algorithms to identify the most efficient options. A major difficulty encountered was solving large POMDPs resulting from large networks. This was overcome by representing networks hierarchically as a group of clusters and treating each cluster separately. This approach is tested through simulations of networks of various sizes. The results show that IAPTF with hierarchical network modeling outperforms previous approaches as well as human performance in terms of time, number of tested vectors and accuracy, and the advantage increases with the network size. Another advantage of IAPTF is the ease of repetition for retesting similar networks, which is often encountered in real PT. The results suggest that IAPTF is a promising approach to offload work from and ultimately replace human pen testing

Towards an efficient automation of network penetration testing using model-based reinforcement learning

Penetration Testing (PT) is an offensive method for assessing and evaluating the security of digital asset by planning, generating, and executing all or some of the possible attacks that aim to exploit its vulnerabilities. In large networks, penetration testing become repetitive, complex and resources consuming despite the use of autonomous tools. To maintain the consistency and efficiency of PT in medium and large network context. it is imperative to go through making it intelligent and optimized which will allow regular and systematic testing without having to provide a prohibitive amount of human labor in one hand and reducing the precious consumed time and tested system downtime in another hand. Reinforcement Learning (RL) led testing will unburden human experts from the heavy repetitive tasks and unveil special and complex situations such as unusual vulnerabilities or combined non-obvious combinations which are often ignored in manual testing. In this research, we are concerned with the specific context of improving current automated testing systems and making them intelligent, targeted, and efficient by embedding reinforcement learning techniques where it is relevant. The proposed Intelligent Automated Penetration Testing Framework (IAPTF) utilizes RL because of its relevance to sequential decision-making problems, it relies on a model based RL where planning and learning are combined and decomposed tasks to represent it as POMDP domain accounting for major PT features, tasks and information flowchart to realistically reflect the real-world context. The problem is then solved on an external POMDP-solver using different algorithms to identify most efficient options. As we encountered a huge scaling-up challenges in solving large POMDP which reflect the regular representation of PT on large networks, we propose thus a Hierarchical representation on which we divided large networks into security clusters and enabling IAPTF to deal with each cluster separately as small networks (intra-clusters), later we proceed to the testing of the network of clusters heads to ensure covering all possible complex and multistep attacking vectors largely adopted by nowadays hackers. The obtained results are unanimous and defeat both previous results and any human performances in term of consumed time, number tested vectors and accuracy especially in large networks. The learning is the second strength of our new model, as the generalization of the extracted knowledge become easier and allowing therefore the re-usability notably in the case of retesting the same network with few changes which is often the real-world context in PT. The performance enhancement and the knowledge extracted, and reuse confirm the efficiency, accuracy, and suitability of our proposed framework. Finally, IAPTF is designed to offload and ultimately replace human expert and to be independent, comprehensive, and versatile so it can integrate any automated PT platform or toolkit. Initially, the framework connects directly with Metasploit and Nessus APIs as both free versions coding architecture allows to perform such utilization

Solving large stochastic planning problems using multiple dynamic abstractions

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2005.Includes bibliographical references (p. 165-172).One of the goals of AI is to produce a computer system that can plan and act intelligently in the real world. It is difficult to do so, in part because real-world domains are very large. Existing research generally deals with the large domain size using a static representation and exploiting a single type of domain structure. This leads either to an inability to complete planning on larger domains or to poor solution quality because pertinent information is discarded. This thesis creates a framework that encapsulates existing and new abstraction and approximation methods into modules and combines arbitrary modules into a 'hierarchy that allows for dynamic representation changes. The combination of different abstraction methods allows many qualitatively different types of structure in the domain to be exploited simultaneously. The ability to change the representation dynamically allows the framework to take advantage of how different domain subparts are relevant in different ways at different times. Since the current plan tracks the current representation, choosing to simplify (or omit) distant or improbable areas of the domain sacrifices little in the way of solution quality while making the planning problem considerably easier.(cont.) The module hierarchy approach leads to greater abstraction that is tailored to the domain and therefore need not give up hope of creating reasonable solutions. While there are no optimality guarantees, experimental results show that suitable module choices gain computational tractability at little cost to behavioral optimality and allow the module hierarchy to solve larger and more interesting domains than previously possible.by Kurt Alan Steinkraus.Ph.D