A novel method for surface tracking for ship hull inspection is presented. The proposed method is based on the USM-ROV visual surface tracking

Arshad, Mohd Rizal

Zainal Abidin, Zulkifli

English

Repository@USM

Proceedings of International Conference on Man-Machine Systems 2006 September 15-16 2006, Langkawi, Malaysia  Visual Servoing with application to ROV for Ship Hull Inspection  Zulkifli Zainal Abidin and Mohd Rizal Arshad. School of Electronics and Electrical, Universiti Sains Malaysia, Seri Ampangan, 14300 Nibong Tebal, Seberang Perai Selatan, Pulau Pinang, Malaysia E-mail: zzulkifli@iiu.edu.my and dr.rizal@gmail.com  Abstract —A novel method for surface tracking for ship hull inspection is presented. The proposed method is based on the USM-ROV visual surface tracking. The method integrates laser tracking and motor control. First the region of interest (which typically laser intensity) is track with adaptive region estimation. Then, the measured pixels position will determine the orientation of the motors. The proposed system has been implemented for real time supervisory control for ROV operator.  I. INTRODUCTION Remotely operated vehicle ROV is a machine operated by a host. Historically the ROV’s could be called a remotely controlled camera and tooling platform. ROV’s can be effectively used for underwater applications such as, drill, construction support and pipeline. Among the benefits of using these vehicles are; human safety, reduction in mission cost, and improved accuracy for repetitive and routine tasks. As the area of underwater robotics matures, ROV’s are also important in ship hull inspection process [1, 2, and 3]. The main difficulty of ship hull inspection is the complex shape structure, such as the bowl (Figure 1), especially at the front structure and thruster engine area [4]. New sensors or techniques to inspect difficult access areas are needed. This research proposes the design and development of a visual surface tracking system for ship hull inspection using an ROV. Initially, the system will detect the ship hull structure’s surface, localizing them, and then adjusting the camera position. The camera will move vertically and also horizontally, facing the targeted object. Imaging capabilities and real time feedback are important factors for this application.  This paper proposes a visual surface tracking system for ship hull inspection by ROVs that is implemented by initially detecting the target objects, localizing them, and then approaching them by taking video images while closely tracing their shape. Proper sensing and navigation methods must be developed to realize this behavior. Assuming that the visibility is adequate for visual investigation of underwater structures, a laser ranging system that consists of Laser pointers and a color CCD camera are introduced for sensing. For navigation, a method based on the relative position to the target objects is introduced.   Fig. 1.   Example of a Ship Hull II. TECHNICAL DIFFICULTIES The most important process in underwater ship hull inspection is visual monitoring. The ship owners are required to provide a copy of the visual tape and the written report by the diving company to the authority inspector/ Marine Guard [5]. The ROV operator and the expertise will make decision base on what they observed and data measured. But the visual data is not 100% accurate especially on the extreme curve surface. ROV can not maintain its static position because of the underwater condition. Inconsistency in ROV movement and the difficulty of ROV operator to guide and control the camera at the same time resulted in the visual inspection and collected NDT measurement error.  Proceedings of International Conference on Man-Machine Systems 2006 September 15-16 2006, Langkawi, Malaysia  The objectives for this research are as follow: i. To develop supervisory control system for ROV operator. ii. To build the real time image processing and controller that deals with the physical control of the system. iii. To make the system respond within 0.5 – 1 millisecond for the whole process to respond. III. PROPOSED SYSTEM DESIGN DESCRIPTIONS A. Laser ranging system The vision navigation system, consisting of a color CCD camera and laser pointing devices is introduced to overcome these difficulties. As shown in Fig. 2, three units of laser pointing devices will be use. Each distance is calculated by using triangulation from pixel position of the camera image through image processing. Thus the angle (Ө) between the ship hull’s surface and the vehicle orientation can be determined. In order to eliminate errors caused by distortion of camera lens and the acrylic dome, calibration has to be carried out in preliminary experiment in a test tank [6].The system can also be used as a means of measuring the size of the objects.  Although the underwater image-based sensor tends to encounter problem in lighting and optical backscatter, it provides pinpoint and precise distance data when the target is visible [7]. B. Estimation of distance and direction to object  The lateral geometric relationship of the video camera and an observation object (ship hull) is illustrated in Fig. 2 and Fig. 3. D is the distance from the camera to the wall and d is the distance between the image field and the laser pointer.     Fig. 2.  Geometric relationship in lateral plane.  The coordinates on the image field are represented by the pixel number and the entire image field is 352x240 pixels. L1, L2 and L3 are the reference point. It represents the total highest threshold pixel count on the scene. LA is the average value of L1 and L2. Set point for the highest threshold pixel is any value greater than 240.  By balancing each reference point, the ship hull surface is always parallel with the camera.   Fig. 3.  Geometric relationship in horizontal plane.   Fig. 4.  Position of laser in image.   Fig. 5.  Geometric relationship in horizontal plane (L3 misaligned).  The actual image in Fig. 6 shows the position of laser pointer. Perpendicular position occurs at, set point Ө = current Ө (vertical) and La = Lb (horizontal).  Camera Laser 3 Laser A D d y x d L3 LA Camera Laser 3 Laser 1 & 2 D d y x d L3 LA Camera Laser 1 Laser 2 D d z x d L1 L2 352 240 x [pixel] y [pixel] L1 L2 L3 Square ROI Proceedings of International Conference on Man-Machine Systems 2006 September 15-16 2006, Langkawi, Malaysia  Driver  4 bits input Pan Tilt Parallel Port Camera USB Port Laser Device  Reference point Center ( ) ( )22 1313 yLyLxLxLLb −+−= 1)  ( ) ( )22 2323 yLyLxLxLLa −+−= (2)  ( ) ( )22 1212 yLyLxLxLLc −+−=(3)  −−−= −abLbLaLc2cos2221θ (4)  Fig. 6.  Position of laser in image.      All laser pointers surrounded by square box of ROI. It will resize accordingly to the laser positions and sizes. By doing this, a total scanning time for each frame can be reduce thus the response time for the system can be maximize. C. Parallel Navigation The operator has an option either manually or let to respond automatically. This supervisory system then searches and detects the object so as to localize it and absorb the accumulated positioning errors. Once the system found the object, it changes its navigation basis from                        Fig. 10.  Structure of computer vision system. absolute position to relative position, from the target object and traces its surface keeping constant perpendicular view with respect to the ship hull.   Fig. 8 demonstrate the camera positions while facing the ship hull structure while, Fig. 9 shows the camera position with visual surface tracking. We have conducted an experiment to demonstrate our visual surface tracking system.  Structure of computer vision system shown in Fig. 10. A curve surface object has been used for analysis. The camera feed the images via USB connection. By calculating the geometrical relationship of pointing device, the proportional surface angle is determined. Both DC motors are connected to the driver via a parallel port.  Fig. 8.  Camera position without visual surface tracking system.   Fig. 8.  Camera position with visual surface tracking system. The collected data also can be use to keep constant distance of the ROV to the ship hull. It can be done by integrating the system with the ROV maneuver system.                   IV. ADAPTIVE REGION OF INTEREST ESTIMATION  L1 L2 Ө  L3 La Lb Lc Proceedings of International Conference on Man-Machine Systems 2006 September 15-16 2006, Langkawi, Malaysia  In this research, we would like propose an adaptive Region of Interest (ROI). ROI estimation, used extensively for video coding and enhancement, has been studied by a number of research groups. ROI based coding is also a key feature of the MPEG-4 [8] and the JPEG-2000 [9]. Since most of the ROI based coding algorithms are proposed for entertainment video and video conferencing applications, faces and moving objects are generally considered as ROI.  Hui Cheng [10] in his paper proposed adaptive ROI estimation with a frame-level. For each video frame, the operator’s ROI is modeled as a rectangular region centered in the middle of the camera Field of View (FOV). The size and the direction of an ROI are determined using the velocity of the camera FOV. In our case, we would like to propose a mathematical algorithm condition to perform real-time image processing and auto alignment. By utilizing Open CV and Visual C++, the total number of highest intensity level will be used as the reference point.  A. Dynamic size ROI algorithm The ROI algorithm for a fixed-size ROI can be described precisely as follows:  1. 3 different ROI generated for the 1st frame for each laser pointer (Fig. 10). 2. Scan L1 (top ROI), searching for any threshold level >= 240. 3. Get (top, right, left and bottom) position if threshold value >=240. 4. Repeat step 2 & 3 for L1 and L2 ROI. 5. Replace ROI area with new values (in step 3) and expand 5 units for each one (Fig. 11). 6. Evaluate total L1, L2 and L3, and send the data to controller for parallel bits output. 7. Next frame to process.   Fig. 10. Position of ROI (1st frame).   Fig. 11. Position of ROI (n + 1 frame).  The ROI can remain of fixed size, or can dynamically adjust its size to fit the properties of the video it is processing. In our case, the laser pointers were fixed on the camera. The targeted image does not move rapidly, it’s expanding or shrinking. We predict that the next ROI equal to current ROI + 5 pixel units. Our main objective is to make the system respond within 0.5 – 1 millisecond for the whole process to respond. V. PRELIMINARY EXPERIMENTAL RESULT We have conducted an experiment to demonstrate our visual surface tracking system.  Control architecture of the tracking system shown in Fig. 12. A curve surface object has been used for analysis. The camera feed the images via USB connection. By calculating the total numbers of highest intensity level of pointing device, the proportional surface angle is determined. Both DC motors are connected to the driver via a parallel port. The program below shows a pixel calculation for the top laser pointer.  //Top laser point for (row = nrow_L3; row < frow_L3; row++) { for (col = ncol_L3; col < fcol_L3; col++) { if(((uchar*)(grey->imageData+grey->widthStep*row))[col] >= 240)  { top = top + 1; if (top == 1) nrow_L3 = row - 5; if (row > dumbrow_L3 ) frow_L3 = row + 5; dumbrow_L3 = row;ncol_L3 = col-5; if (col > dumbcol_L3 ) fcol_L3 = col + 5; dumbcol_L3 = col;  }}}  352 240 x [pixel] y [pixel] L1 L2 L3 Square ROI 352 240 x [pixel] y [pixel] L1 L2 L3 Square ROI Proceedings of International Conference on Man-Machine Systems 2006 September 15-16 2006, Langkawi, Malaysia                      Fig. 11.  Control architecture of the tracking system.  In this experiment, adaptive ROI estimation algorithm was applied. Fig. 13 shows the surface tracking result. 10 data were recorded in 1 second.  The fluctuated points indicate that the system was trying to stabilize its surface tracking position. The center surface position occurs at pixel different = 0. Without ROI estimation, 240 x352 pixels square need to be scan. But in this case, each ROI takes approximately (15x15 to 20x20) pixels square or only (0.799% to 1.420%) area of the whole frame. VI. FURTHER WORKS In this paper, Visual Surface Tracking Using an ROV is proposed for Ship Hull Inspection. Three known ROI are used for measuring the pixels. The actual underwater experiment will be implemented for the next stage. The main                Fig. 13.  Numbers of pixel different versus time for horizontal surface tracking.  consideration is the fastest time taken for each frame to scan the pixel matrices and the distance accuracy. This supervisory control is used to help the ROV’s operator by minimizing his task of controlling the camera position while maneuvering the ROV. By analyzing the way an operator controls the camera system, this research hopefully will contribute a significant impact for underwater ship hull inspection process.  Fig. 14 shows the proposed ROV block diagram with visual surface tracking system for the further works and the actual implementation   Fig. 14.  ROV block diagram with visual surface tracking system.   Horizontal Tracking-2-1.75-1.5-1.25-1-0.75-0.5-0.2500.250.50.7511.251.51.7520 50 100 150 200 250 300 350Time (seconds)Ratio Diff (units)Video Intel Open CV / C++ Adaptive ROI Estimation Geometrical/ Optical Flow Analysis Serial Output Signal Pan-Tilt Position Frame by frame Camera Analyze Data Analyze Data Camera & Laser device Proceedings of International Conference on Man-Machine Systems 2006 September 15-16 2006, Langkawi, Malaysia  REFERENCE  [1] Lynn, D.C.; Bohlander, G.S., “Performing ship hull inspections using a remotely operated vehicle”  OCEANS '99 MTS/IEEE. Riding the Crest into the 21st Century Volume 2,  13-16 Sept. 1999 Page(s):555 - 562 vol.2.  [2] J. Cadiou, S . Coudray, P. LRon and M. Perrier, "Control architecture of a deep scientific ROV: VICTOR 6000", in Proceedings of the IEEE Oceans '98 Conference, Nice, France, 1998  [3] J. Amat, A. Monferrer, J. Batlle, X.Cufiv, “GARBI: a low-cost underwater vehicle”, in Microprocessors and Microsystems, Volume 23, Issue 2, Pages 61-67, 3 September 1999.  [4] David L. Olson, Brajendra Mishra, “Assessment of the Emerging Technologies for the Detection and Measurement of Corrosion Wastage of Coated Marine Structures” in Proceedings of OMAE’03, The 22nd International Conference on Offshore Mechanics & Arctic Engineering Cancun, Mexico, June 8-13, 2003.  [5] (Department of Transportation United States Coast Guard) - NAVIGATION AND VESSEL INSPECTION CIRCULAR NO. 1-89, www.uscg.mil/hq/g-m/nvic/1_89/n1-89.pdf  [6] Yu, S., & Ura, T. (2001) “Development of new distance measurement system for autonomous underwater vehicles”,  Proceedings of the 19th conference of RSJ, CD-ROM, 1N34 (in Japanese).  [7] Hayato Kondo and Tamaki Ura, “Navigation of an AUV for investigation of underwater structures” Control Engineering Practice, Volume 12, Issue 12, December 2004, Pages 1551-1559.  [8] Brady, N., “MPEG-4 standardized methods for the compression of arbitrarily shaped video objects” Circuits and Systems for Video Technology, IEEE Transactions on Volume 9,  Issue 8,  Dec. 1999 Page(s):1170 – 1189.  [9] A.P. Bradley and F.W.M. Stentiford, “ JPEG 2000 and Region of Interest Coding” , DICTA2002, Digital Image Computing Techniques and Applications, Melbourne, Australia, Jan. 2002.  [10] Hui Cheng; Wus, J., “Adaptive Region of Interest Estimation for Aerial Surveillance Video” Image Processing, 2005. ICIP 2005. IEEE International Conference on Volume 3, 11-14 Sept. 2005 Page(s):860 – 863.  

Visual Servoing With Application To ROV For Ship Hull Inspection.

Abstract

Similar works

Full text

Available Versions

Repository@USM