The MVP sensor planning system for robotic vision tasks

The MVP (machine vision planner) model-based sensor planning system for robotic vision is presented. MVP automatically synthesizes desirable camera views of a scene based on geometric models of the environment, optical models of the vision sensors, and models of the task to be achieved. The generic task of feature detectability has been chosen since it is applicable to many robot-controlled vision systems. For such a task, features of interest in the environment are required to simultaneously be visible, inside the field of view, in focus, and magnified as required. In this paper, we present a technique that poses the vision sensor planning problem in an optimization setting and determines viewpoints that satisfy all previous requirements simultaneously and with a margin. In addition, we present experimental results of this technique when applied to a robotic vision system that consists of a camera mounted on a robot manipulator in a hand-eye configuration

Multi-View Picking: Next-best-view Reaching for Improved Grasping in Clutter

Camera viewpoint selection is an important aspect of visual grasp detection, especially in clutter where many occlusions are present. Where other approaches use a static camera position or fixed data collection routines, our Multi-View Picking (MVP) controller uses an active perception approach to choose informative viewpoints based directly on a distribution of grasp pose estimates in real time, reducing uncertainty in the grasp poses caused by clutter and occlusions. In trials of grasping 20 objects from clutter, our MVP controller achieves 80% grasp success, outperforming a single-viewpoint grasp detector by 12%. We also show that our approach is both more accurate and more efficient than approaches which consider multiple fixed viewpoints.Comment: ICRA 2019 Video: https://youtu.be/Vn3vSPKlaEk Code: https://github.com/dougsm/mvp_gras

Swept volumes and their use in viewpoint computation in robot work-cells

This paper discusses the automatic computation of viewpoints for monitoring objects and features in an active robot work-cell. An important step in the authors' algorithm for finding viewpoints is the computation of the volumes swept by polyhedral objects moving through space. A method for approximating these volumes for arbitrarily moving polyhedra is presented. Some swept volume results are presented and methods for integrating these results into the authors' automated machine vision planning (MVP) system are discussed

Computing camera viewpoints in a robot work-cell

Automatically planning a camera viewpoint for tasks such as inspection in an active robot work-cell is a difficult problem. This paper discusses new methods for computing viewpoints which meet the feature detectability constraints of focus, field-of-view, visibility, and resolution. A theoretical outline of the method is presented, followed by experimental results and a discussion of future work

Automated sensor planning for robotic vision tasks

A method is presented to determine viewpoints for a robotic vision system for which object features of interest will simultaneously by visible, inside the field-of-view, in-focus, and magnified as required. A technique that poses the problem in an optimization setting in order to determine viewpoints that satisfy all requirements simultaneously and with a margin is presented. The formulation and results of the optimization are shown, as well as experimental results in which a robot vision system is positioned and its lens is set according to this method. Camera views are taken from the computed viewpoints in order to verify that all feature detectability requirements are satisfied

Design of a polishing tool for collaborative robotics using minimum viable product approach

This is an Author's Accepted Manuscript of an article published in Carlos Perez-Vidal, Luis Gracia, Samuel Sanchez-Caballero, J. Ernesto Solanes, Alessandro Saccon & Josep Tornero (2019) Design of a polishing tool for collaborative robotics using minimum viable product approach, International Journal of Computer Integrated Manufacturing, 32:9, 848-857, DOI: 10.1080/0951192X.2019.1637026 [copyright Taylor & Francis], available online at: http://www.tandfonline.com/10.1080/0951192X.2019.1637026[EN] A collaborative tool for robotic polishing is developed in this work in order to allow the simultaneous operation of the robot system and human operator to cooperatively carry out the polishing task. For this purpose, the collaborative environment is detailed and the polishing application is designed. Moreover, the polishing tool is developed and its implementation using the minimum viable product approach is obtained. Furthermore, a robust hybrid position-force control is proposed to use the developed tool attached to a robot system and some experiments are given to show its performance.This work was supported in part by the Ministerio de Ciencia e Innovacion (Spanish Government) under project [DPI2017-87656-C2-1-R] and the Generalitat Valenciana under Grant [VALi+ d APOSTD/2016/044].Perez-Vidal, C.; Gracia Calandin, LI.; Sanchez-Caballero, S.; Solanes Galbis, JE.; Saccon, A.; Tornero Montserrat, J. (2019). Design of a polishing tool for collaborative robotics using minimum viable product approach. International Journal of Computer Integrated Manufacturing. 32(9):848-857. https://doi.org/10.1080/0951192X.2019.1637026S848857329Alders, K., M. Lehe, and G. Wan. 2001. “Method for the Automatic Recognition of Surface Defects in Body Shells and Device for Carrying Out Said Method” US Patent 6,320,654, Accessed 2001 November. https://www.google.ch/patents/US6320654Alexopoulos, K., Mavrikios, D., & Chryssolouris, G. (2013). ErgoToolkit: an ergonomic analysis tool in a virtual manufacturing environment. International Journal of Computer Integrated Manufacturing, 26(5), 440-452. doi:10.1080/0951192x.2012.731610Andres, J., Gracia, L., & Tornero, J. (2011). Calibration and control of a redundant robotic workcell for milling tasks. International Journal of Computer Integrated Manufacturing, 24(6), 561-573. doi:10.1080/0951192x.2011.566284Arnal, L., Solanes, J. E., Molina, J., & Tornero, J. (2017). Detecting dings and dents on specular car body surfaces based on optical flow. Journal of Manufacturing Systems, 45, 306-321. doi:10.1016/j.jmsy.2017.07.006Blank, S. 2010. “Perfection By Subtraction - The Minimum Feature Set”. Accessed 2018 August. http://steveblank.com/2010/03/04/perfection-by-subtraction-the-minimum-feature-set/Dimeas, F., & Aspragathos, N. (2016). Online Stability in Human-Robot Cooperation with Admittance Control. IEEE Transactions on Haptics, 9(2), 267-278. doi:10.1109/toh.2016.2518670Fitzgerald, C. “Developing Baxter, A new industrial robot with common sense for U.S. manufacturing.” 2013.Gracia, L., Sala, A., & Garelli, F. (2012). A supervisory loop approach to fulfill workspace constraints in redundant robots. Robotics and Autonomous Systems, 60(1), 1-15. doi:10.1016/j.robot.2011.07.008Gracia, L., Sala, A., & Garelli, F. (2014). Robot coordination using task-priority and sliding-mode techniques. Robotics and Computer-Integrated Manufacturing, 30(1), 74-89. doi:10.1016/j.rcim.2013.08.003Gracia, L., Solanes, J. E., Muñoz-Benavent, P., Valls Miro, J., Perez-Vidal, C., & Tornero, J. (2018). Adaptive Sliding Mode Control for Robotic Surface Treatment Using Force Feedback. Mechatronics, 52, 102-118. doi:10.1016/j.mechatronics.2018.04.008Julius, R., Schürenberg, M., Schumacher, F., & Fay, A. (2017). Transformation of GRAFCET to PLC code including hierarchical structures. Control Engineering Practice, 64, 173-194. doi:10.1016/j.conengprac.2017.03.012. E. K. (2016). TOWARDS AN AUTOMATED POLISHING SYSTEM - CAPTURING MANUAL POLISHING OPERATIONS. International Journal of Research in Engineering and Technology, 05(07), 182-192. doi:10.15623/ijret.2016.0507030Khan, A. M., Yun, D., Zuhaib, K. M., Iqbal, J., Yan, R.-J., Khan, F., & Han, C. (2017). Estimation of Desired Motion Intention and compliance control for upper limb assist exoskeleton. International Journal of Control, Automation and Systems, 15(2), 802-814. doi:10.1007/s12555-015-0151-7Kirschner, D., Velik, R., Yahyanejad, S., Brandstötter, M., & Hofbaur, M. (2016). YuMi, Come and Play with Me! A Collaborative Robot for Piecing Together a Tangram Puzzle. Interactive Collaborative Robotics, 243-251. doi:10.1007/978-3-319-43955-6_29Mohammad, A. E. K., Hong, J., & Wang, D. (2018). Design of a force-controlled end-effector with low-inertia effect for robotic polishing using macro-mini robot approach. Robotics and Computer-Integrated Manufacturing, 49, 54-65. doi:10.1016/j.rcim.2017.05.011Nagata, F., Hase, T., Haga, Z., Omoto, M., & Watanabe, K. (2007). CAD/CAM-based position/force controller for a mold polishing robot. Mechatronics, 17(4-5), 207-216. doi:10.1016/j.mechatronics.2007.01.003Nakamura, Y., Hanafusa, H., & Yoshikawa, T. (1987). Task-Priority Based Redundancy Control of Robot Manipulators. The International Journal of Robotics Research, 6(2), 3-15. doi:10.1177/027836498700600201Ries, E. 2009. “What is the Minimum Viable Product”. March. Accessed 2018 August. http://venturehacks.com/articles/minimum-viable-productRobinson, F. 2001 “A Proven Methodology to Maximize Return on Risk”. Accessed 2018 August. http://www.syncdev.com/minimum-viable-productShepherd, S., & Buchstab, A. (2014). KUKA Robots On-Site. Robotic Fabrication in Architecture, Art and Design 2014, 373-380. doi:10.1007/978-3-319-04663-1_26SYMPLEXITY. “Symbiotic Human-Robot Solutions for Complex Surface Finishing Operations.” European project funded by E.U. through the H2020. Project no. 637080. Call: H2020-FoF-2014. Topic: FoF-06-2014. Starting date: 01/ 01/2015.Duration: 48 months. Accessed 2019 March. https://www.symplexity.eu/Vihlborg, P., I. Bryngelsson, B. Lindgren, L. G. Gunnarsson, and P. Graff. 2017. “Associatio between vibration exposure and hand-arm vibration symptoms in a Swedish mechanical industry.” February 2017.Vogel, J., Haddadin, S., Jarosiewicz, B., Simeral, J. D., Bacher, D., Hochberg, L. R., … van der Smagt, P. (2015). An assistive decision-and-control architecture for force-sensitive hand–arm systems driven by human–machine interfaces. The International Journal of Robotics Research, 34(6), 763-780. doi:10.1177/027836491456153

Dynamic sensor planning

A method of extending the sensor planning abilities of the MVP (machine vision planning) system to plan viewpoints for monitoring a pre-planned robot task is described. The dynamic sensor planning system presented analyzes geometric models of the environment and of the planned motions of the robot, as well as optical models of the vision sensor. Using a combination of swept volumes and a temporal interval search technique, it computes a series of viewpoints, each of which provides a valid viewpoint for a different interval of the task. By mounting a camera on another manipulator, the viewpoints can be executed at appropriate times during the task so that there is always a robust view suitable for monitoring the task. Experimental results monitoring a simulated robot operation are presented, and directions for future research are discussed

Sensor planning in an active robotic work cell

In this paper, we discuss techniques for extending the sensor planning capabilities of the machine vision planning system to include motion in a well-known environment. In a typical work cell, vision sensors are needed to monitor a task and provide feedback to motion control programs or to assess task completion or failure. In planning sensor locations and parameters for such a work-cell, all motion in the environment must be taken into account in order to avoid occlusions of desired features by moving objects and, in the case where the features to be monitored are being manipulated by the robot, to insure that the features are always within the camera's view. Several different sensor locations (or a single, movable sensor) may be required in order to view the features of interest during the course of the task. The goal is to minimize the number of sensors (or to minimize the motion of the single sensor) while guaranteeing a robust view at all times during the task, where a robust view is one which is unobstructed, in focus, and sufficiently magnified. In the past, sensor planning techniques have primarily focused on static environments. We present techniques which we have been exploring to include knowledge of motion in the sensor planning problem. Possible directions for future research are also presented