142 research outputs found

    Behavior policy learning: Learning multi-stage tasks via solution sketches and model-based controllers

    Get PDF
    Multi-stage tasks are a challenge for reinforcement learning methods, and require either specific task knowledge (e.g., task segmentation) or big amount of interaction times to be learned. In this paper, we propose Behavior Policy Learning (BPL) that effectively combines 1) only few solution sketches, that is demonstrations without the actions, but only the states, 2) model-based controllers, and 3) simulations to effectively solve multi-stage tasks without strong knowledge about the underlying task. Our main intuition is that solution sketches alone can provide strong data for learning a high-level trajectory by imitation, and model-based controllers can be used to follow this trajectory (we call it behavior) effectively. Finally, we utilize robotic simulations to further improve the policy and make it robust in a Sim2Real style. We evaluate our method in simulation with a robotic manipulator that has to perform two tasks with variations: 1) grasp a box and place it in a basket, and 2) re-place a book on a different level within a bookcase. We also validate the Sim2Real capabilities of our method by performing real-world experiments and realistic simulated experiments where the objects are tracked through an RGB-D camera for the first task

    A Study on the Evolution of Ransomware Detection Using Machine Learning and Deep Learning Techniques

    Get PDF
    This survey investigates the contributions of research into the detection of ransomware malware using machine learning and deep learning algorithms. The main motivations for this study are the destructive nature of ransomware, the difficulty of reversing a ransomware infection, and how important it is to detect it before infecting a system. Machine learning is coming to the forefront of combatting ransomware, so we attempted to identify weaknesses in machine learning approaches and how they can be strengthened. The threat posed by ransomware is exceptionally high, with new variants and families continually being found on the internet and dark web. Recovering from ransomware infections is difficult, given the nature of the encryption schemes used by them. The increase in the use of artificial intelligence also coincides with this boom in ransomware. The exploration into machine learning and deep learning approaches when it comes to detecting ransomware poses high interest because machine learning and deep learning can detect zero-day threats. These techniques can generate predictive models that can learn the behaviour of ransomware and use this knowledge to detect variants and families which have not yet been seen. In this survey, we review prominent research studies which all showcase a machine learning or deep learning approach when detecting ransomware malware. These studies were chosen based on the number of citations they had by other research. We carried out experiments to investigate how the discussed research studies are impacted by malware evolution. We also explored the new directions of ransomware and how we expect it to evolve in the coming years, such as expansion into IoT (Internet of Things), with IoT being integrated more into infrastructures and into homes

    Anatomical variations of the pelvis during abdominal hysterectomy for benign conditions

    Get PDF
    Background: Anatomical variations are defined as atypical morphologic and positional presentations of anatomical entities. Pelvic anatomical variations encountered during abdominal hysterectomy can be of clinical interest, given that misidentification of certain structures can lead to iatrogenic injuries and postoperative sequelae. The aim of the present study was to detect and highlight the anatomical structures of interest and their variations to the surgeon performing abdominal hysterectomy for benign conditions. Materials and methods: A narrative review of the literature was performed including reports of anatomical variations encountered in cadavers, by surgeons during abdominal hysterectomy and radiologists on computed tomography angiography, searching within a 10-year span on Pubmed database. Studies regarding the treatment of malignant conditions requiring lymphadenectomy and different modes of surgical approach were reviewed with regards to the aspects relevant to benign conditions. The search was extended to the reference lists of all retrieved articles. Results: Ureters and the uterine arteries, due to anatomical variations, are the anatomical structures most vulnerable during abdominal hysterectomy. Specifically, the ureters can present multiplications, retroiliac positionings and ureteric diverticula, whereas, the uterine arteries can present notable variability in their origins. Such variations can be detected preoperatively or intraoperatively. Conclusions: Although rare, the presence of anatomical variations of the uterine arteries and ureters can increase the posibility of complications should they escape detection. Intraoperative misidentification could lead to improper dissection or ligation of the affected structures. Knowledge of these variations, coupled with extensive preoperative investigation and intraoperative vigilance can minimize the risk of complications
    • …
    corecore