Due to large scale implementation of visual detection and tracking as a mean of sensor and navigation tool, target detection and tracking using image manipulation for autonomous robotic system becomes an interesting object of study for many researchers. In addition, there have been attempts to develop a system that can detect and track a moving target by using an image or video processing in a real time condition. Despite that, visual object tracking can be a subject of noise because of image manipulation. The noise can create uncertainty on state and observation model that can lead to control instability, especially that in remote operation. Therefore, an effective filter that can tackle or reduce this noise is needed in developing a visual object tracking system. In this work, a 2-degree of freedom (2-DOF) visual object tracking system was developed with an information filter. The system consists of an image capture unit, an image processing unit, a wireless communication unit, and a manipulator. Then to observe the filter effectiveness on real time visual object tracking in remote operation, performances of this visual object tracking system with and without the filter were tested based on video simulation and real time tracking. In the live streaming test, the information filter can reduce the error of the measurement about 30% than that without it

Prasetiyo, Radius Bhayu

English

Cylinder: Jurnal Ilmiah Teknik Mesin

26  Control Design and Implementation of Autonomous 2-DOF Wireless Visual Object Tracking System Radius Bhayu Prasetiyo University of Science and Technology, South Korea / Korea Institute of Industrial Technology E-mail: jq_rbp@yahoo.com  ABSTRAK Karena skala implementasi deteksi visual yang besar sebagai alat sensor dan navigasi, pelacakan target menggunakan manipulasi gambar untuk sistem robot otonom menjadi sebuah objek studi yang menarik bagi banyak peneliti. Hal ini pun memunculkan berbagai upaya untuk mengembangkan sistem yang dapat mendeteksi dan melacak target bergerak dengan menggunakan pemrosesan gambar atau video dalam kondisi real time. Meskipun begitu, pelacakan objek visual dapat menjadi subjek dari kesalahan karena manipulasi gambar. Kesalahan ini dapat menimbulkan ketidakpastian pada kontrol sistem yang dapat menyebabkan ketidakstabilan, terutama bagi operasi jarak jauh. Oleh karena itu, filter yang efektif yang dapat mengatasi atau mengurangi kesalahan ini sangatlah diperlukan dalam mengembangkan sistem pelacakan objek visual. Dalam karya ini, sebuah sistem pelacakan objek visual dalam 2 derajat kebebasan (2-DOF) dikembangkan dengan information filter atau filter informasi. Sistem ini terdiri dari sebuah unit pengambilan gambar, unit pengolahan gambar, komunikasi nirkabel, dan manipulator. Kemudian untuk mengamati efektivitas filter dalam kondisi real time dan jarak jauh, prestasi sistem pelacakan visual ini, baik dengan maupun tanpa filter tersebut, diuji berdasarkan simulasi video dan tes secara real time. Berdasarkan pengujian secara real time, filter informasi dapat mengurangi kesalahan pengukuran / deteksi sekitar 30% dibandingkan dengan deteksi tanpa menggunakan filter. Kata Kunci: Deteksi visual, pelacakan objek nirkabel, parameter Denavit-Hartenberg,  filter informasi, dua derajat kebebasan.  ABSTRACT Due to large scale implementation of visual detection and tracking as a mean of sensor and navigation tool, target detection and tracking using image manipulation for autonomous robotic system becomes an interesting object of study for many researchers. In addition, there have been attempts to develop a system that can detect and track a moving target by using an image or video processing in a real time condition. Despite that, visual object tracking can be a subject of noise because of image manipulation. The noise can create uncertainty on state and observation model that can lead to control instability, especially that in remote operation. Therefore, an effective filter that can tackle or reduce this noise is needed in developing a visual object tracking system. In this work, a 2-degree of freedom (2-DOF) visual object tracking system was developed with an information filter. The system consists of an image capture unit, an image processing unit, a wireless communication unit, and a manipulator. Then to observe the filter effectiveness on real time visual object tracking in remote operation, performances of this visual object tracking system with and without the filter were tested based on video simulation and real time tracking. In the live streaming test, the information filter can reduce the error of the measurement about 30% than that without it. Keywords: Visual detection, wireless object tracking, Denavit-Hartenberg parameter, information  filter, two-degree of freedom.   27   1. INTRODUCTION Recently, the study of visual object tracking has been an interesting object of study for many researchers. There have been attempts to develop a system that can detect and track a moving target by using image or video processing algorithm [1]–[3]. Additionally, this visual detection and tracking system can also be implemented as a mean of sensor and navigation tool for mobile robots, and/or UAV system [4]-[8]. The reason is because of this system can measure the position of the visual tracker relatively to the target object. With this relative position, the system can conduct a localization process and located its position in space. Besides that, visual object tracking can be a subject of noise because of image manipulation. The noise can create uncertainty on the state and observation model that can lead to control instability. Therefore, a filter that can tackle or reduce this measurement noise is needed in developing a visual object tracking system. In this work, an information filter was implemented to a wireless visual object tracking system to reduce noises and uncertainty of measurements from target position tracking process. Components of the visual object tracking system consist of an image capture unit, an image processing unit, and a manipulator. The image capture unit is a smartphone that can transfer images wirelessly to the image processing unit (computer) via HTTP streaming. For its manipulator, the system was equipped with two servo motors that can be controlled wirelessly from a computer via 2.4 GHz wireless communication system. The image capture unit is used to provide images for image processing unit which calculate visual object tracking procedures. In this work, the images were captured by an android smartphone, SKY  IM-A780L, with dimensions: about 66 mm length, 127 mm height, and 9 mm depth. This smartphone is powered by lithium Polymer (Li-Po) battery 1650 mAh which can provide about 3 hours of live HTTP streaming. By using its primary camera (back camera), an 8 mega pixel image can be captured by this smartphone. The smartphone can also be configured as a Wi-Fi router for the wireless streaming (as shown in Figure 1).  Figure 1. Wireless visual object tracking system diagram The image processing unit is a computer with Wi-Fi connection and 2.4 GHz wireless communication system. The Wi-Fi connection is used to receive wireless HTTP video streaming from the image capture unit. Then images were extracted and analyzed from this video to calculate the object position and movement. Based on this calculation, commands for manipulator unit were produced and transferred to manipulator by a transmitter. This transmitter is a remote control with TI CC2511 System on Chip (SoC) that can be connected to the computer via USB communication devices class (USB CDC) and provides 2.4 GHz wireless communication system. From the transmitter, the input commands are transferred wirelessly to a receiver in the manipulator. The wireless receiver is a TI CC2511 compatible 2.4 GHz RF transceiver SoC, TI CC2510. This SoC can produce Pulse Width Modulation (PWM) that can be used to control two servo-motors, HITEC HS-325HB, which acts as actuators in the system manipulator. Consist of 2 Cell Li-Po battery of 450 mAh, the SoC can receive the input commands via 2.4 GHz wireless communication system and control the actuators based on that input commands. http streaming, wifiImage capture deviceActuatorsImage processing2.4 GHz wireless communicationManipulatorTransmitterComputerReceiver28  Figure 2 shows the components of the wireless visual object tracking system.  Figure 2. Wireless visual object tracking system. (a) Manipulator (isometric). (b) Manipulator (side). (c) Transmitter and image processing unit.  2. CONTROL DESIGN AND IMPLEMEN-TATION Because the location of the camera on the smartphone is not in a same coordinate system with that of the servo motors, the camera and two servo motors will have different object position measurement values. Therefore, to track and follow the object movement from camera images, the object position must be transformed to each actuator’s coordinate system. With this coordinate transformation, the input commands for each actuator can be calculated based on each servo motor coordinate system. To provide such coordinate transformation, Denavit – Hartenberg parameters [9] are used in this work to calculate the kinematics of the manipulator. Based on the parameters, the coordinate transformation matrix can be calculated from one coordinate frame to the next as: 1cos sin cos sin sin cossin cos cos cos sin sin0 sin cos0 0 0 1i i i i i i ii i i i i i iiii i irrTd                   (1) Where   is a rotation angle around     axis from      axis to    axis,   is the rotation angle around    axis from      axis to    axis,    is a distant measured from      axis to    along   , and    is the distant measured from      axis to    along     .   Figure 3. Manipulator kinematics coordinate system As shown in Figure 3, there are four coordinate frames defined for the manipulator, e.g. frame 0, frame 1, frame 2, and frame 3. Frame 0 is the base coordinate system frame for the manipulator.     rotation is aligned with the rotation hinge of servo motor 1 and can be controlled by input command for servo motor 1.    is aligned with the hinge of servo motor 2 and can be controlled by input command for servo motor 2. Frame 3 is aligned with the image coordinate frame captured by the camera. These coordinate frames described how the manipulator can be controlled based on the rotation of    and    on each kinematics joint. The parameters of each coordinate frame are listed in Table 1.   Table 1. Kinematics parameters      (mm)    (deg)    (mm)    (deg) 1 14 (-90) 63 0 - 180 2 58 (-90) (-14) (-180) – (-90) 3 0 0 43 90   0x 0y0z1y1x1z2z2y2x3z3y3x1d1r2r2d3d2129  By aligning object position in the image with the center of the camera, object movement can be followed by moving the two servo motors. Assuming the center of the camera is the end effector of manipulator kinematics,    in Figure 4, and the object as the target,   , then by using an inverse kinematics method, the input commands for each servo motors in the manipulator can be calculated so that the end effector and the target position are always aligned together in camera coordinate system. One of the inverse kinematics methods that is relatively simple and can be used for a real time application is cyclic coordinate descent (CCD) which was studied by Wang [10].  Figure 4. Linkage diagram In CCD method, an iterative heuristic search for each joint angle of a manipulator kinematics is performed to make the end effector reach a desired location in space. In this tracking system, this would result in the closest distance of the camera center and the object position with given joint angles are the input commands for the servo motors. As shown in Figure 4, by using CCD method, the rotation angle of each joint can be calculated as:                                    (2)                              (3) Where    e is angle displacement for    so that       line can be aligned with       and   is the displacement vector for   that determine displacement direction. By iterative calculation of     from     to    , the input commands for two servo motors in the manipulator can be generated so that the tracked object position can be aligned with the center of the camera. 3. FILTER IMPLEMENTATION Measuring position of an object from images can be a subject of noises from image manipulation process such as scaling, resizing, color distortion, and other measurement noises from the camera. These noises give uncertainty in controlling the manipulator to keep following the object tracked from video streaming. In this system, information filter is used to reduce the noise from object position measurement so that manipulator can keep the center of the camera aligned with the object position. As shown in Figure 5, tracked object position measured in image plane (      and     ) was filtered by information filter before CCD calculation determines the input commands for manipulator.  Figure 5. Visual Object Tracking Control Block Diagram The information filter or inverse covariance filter is a well-known filter that has the same function as Kalman Filter [11]. With a relatively light initialization, the information filter can estimate unknown variables based on a series of measurements containing noise and other inaccuracies. Therefore, this filter can produce optimal parameters estimation of state and observation model from measurements with Gaussian noise, such as visual object tracking system.  The filter procedure consists of initialization, propagation, and update. Initialization of the information filter is started when the object is detected, and while the object position is being tracked, the filter is propagated and updated. In initialization, variables of information filter called information matrix and idiP 1iPePtPPD SaturationManipulatorProcessObject Detectionand TrackingInformationFilterCCDΣ                            30  information vector are equal to zero. This has an advantage for its lightweight computational loads when the system needs to regenerate detection, especially when the object disappears during tracking. Therefore, for this wireless visual object tracking system, the information filter can be used for a real time control. 4. TEST AND RESULTS To test the effectiveness of the information filter, the wireless visual object tracking system is used to detect object movement in two video simulations and in a real time environment. The first video is a simulation of an object moving in vertical direction. The second video shows an object that moves in horizontal direction.  Figure 6. Interface of the wireless visual object tracking system: (a) vertical simulation, (b) horizontal simulation Figure 6 shows computer interface of image processing unit for the wireless visual object tracking system. As shown in Figure 6(a), the interface is loading and playing the video of the object moving in vertical direction. During the play, the object movement is detected by image processing procedure. During the movement, the object position is tracked and recorded with two conditions, e.g. without information filter and with the filter. These conditions were also applied to the second video simulation.  Figure 7. Vertical object simulation track  Figure 8. RMS Error of vertical object simulation Figure 7 shows the result of object tracking on a vertically moving object. Root mean square (RMS) error of object tracking measurement without and with information filter (shown in Figure 8) is calculated based on the differences of measurement in x axis direction. Based on Figure 8, the information filter can actually reduce about 20% error of object tracking measurement. objectobjectdirectiondirection(b)(a)372 373 374 375 3760100200300400500600700X axis (pixel)Y axis (pixel)  objecttrackinginformation filter280 300 320 340 360 3800.350.40.450.50.550.60.650.70.75Simulation runError rms  trackinginformation filter31   Figure 9. Horizontal object simulation track  Figure 10. RMS Error of horizontal object simulation Figure 9 and Figure 10 show the test results on the horizontal moving object. As shown in Figure 10, the error of the original tracking measurement keeps increasing during the simulation run. This result shows the instability of the original object tracking measurement without information filter. On the other hand, the error of the tracking measurement with information filter is converged on 0.4 which shows the stability of the system. On real time application, the wireless pan tilt camera object is tested to follow a moving object while transferring a live video via HTTP streaming. As shown in Figure 11, the moving object is a face of a person that is moving randomly during live streaming. The live video has the image size of 144 x 176 pixels (portrait) that are streamed at an average of 24 frames per second (fps). In this test, the object is a face of a man that moves arbitrarily. The error is calculated based on the object displacement along the center of the camera in image coordinate frame.   Figure 11. The interface of the wireless visual object tracking system on live streaming  Figure 12. RMS Error of live streaming In the live streaming test, the information filter can reduce the error of the measurement about 30% than that without it as shown in Figure 12. With all results, information filter can effectively reduce the measurement noise of the wireless visual object tracking system.  0 100 200 300 400 500 600414416418420422424426X axis (pixel)Y axis (pixel)  objecttrackinginformation filter280 290 300 310 3200.20.40.60.811.21.41.6Simulation runError rms  trackinginformation filter1000 2000 3000 4000 5000 6000123456789Simulation runError rms  trackinginformation filter32  5. CONCLUSION In this work, control design for a 2-DOF wireless visual object tracking system was developed and implemented with information filter. The system can autonomously track a moving object in real time with wireless transmission of input commands and object images. Based on the test, information filter can reduce about 30% error on object tracking measurement during live streaming which proves the effectiveness of the information filter.  REFERENCES [1] S. K. Weng, C. M. Kuo, S. K. Tu, “Video object tracking using adaptive Kalman filter,” Journal of Visual Communication and Image Representation, vol. 17, issue 6, pp. 1196–1197, Dec. 2006. [2] B. Karasulu, “Review and evaluation of well-known methods for moving object detection and tracking in videos,” Journal of Aeronautics and Space Technologies, vol. 4, issue 4, pp. 11–22, Mar. 2010. [3]  J.C. Nascimento, J.S. Marques, “Performance evaluation of object detection algorithms for video surveillance," IEEE Transactions on Multimedia 8,” vol. 4, pp. 761–774, Aug. 2006. [4] J. Palacin, I. Valgañon, R. Pernia, “The optical mouse for indoor mobile robot odometry measurement, Sensors and Actuators,” Sensor and Actuators A: Physical, vol. 126, Issue 1, pp. 141-147, Jan. 2006. [5] G. Dobie, R. Summan, C. MacLeod, S. G. Pierce, “Visual odometry and image mosaicing for NDE,” NDT & E International, vol. 57, pp. 17-25, Jul. 2013. [6] M.V. Srinivasan, J.S. Chahl, K. Weber, S. Venkatesh, M.G. Nagle, S.W. Zhang, “Robot navigation inspired by principles of insect vision,” Robotics and Autonomous Systems, vol. 26, pp. 203-216, Feb. 1999. [7] G. L. Mariottini, S. Scheggi, F. Morbidi, D. Prattichizzo, “An accurate and robust visual-compass algorithm for robot-mounted omnidirectional cameras,” Robotics and Autonomous Systems, vol. 60, pp. 1179-1190, Sep. 2012. [8] H.M. Becerra, G. L. Nicolás, C. Sagues, “Omnidirectional visual control of mobile robots based on the 1D trifocal tensor,” Robotics and Autonomous Systems, vol. 58, pp 796-808, Jun. 2010. [9] R.S. Hartenberg and J. Denavit, Jacques, Kinematic synthesis of linkages, McGraw-Hill series in mechanical engineering, New York: McGraw-Hill, 1965. [10] Wang and Chen, “A Combined Optimization Method for Solving the Inverse Kinematics Problem of Mechanical Manipulators,” IEEE Tr. On Robotics and Automation, vol. 7, pp. 489 – 498 , August 1991. [11] S. Challa, M. R. Morelande, D. Musicki, and R. J. Evans, Fundamentals of Object Tracking, Cambridge University Press, New York, 2011.       

Control Design and Implementation of Autonomous 2-DOF Wireless Visual Object Tracking System

JURNAL PERKOTAAN

Damianus Journal of Medicine

http://ojs.atmajaya.ac.id/index.php/cylinder/article/download/567/440

Control Design and Implementation of Autonomous 2-DOF Wireless Visual Object Tracking System

Abstract

Similar works

Full text

Available Versions

Cylinder: Jurnal Ilmiah Teknik Mesin

JURNAL PERKOTAAN

Damianus Journal of Medicine