This paper describes a robot to collect golf balls in a driving range, operating as autonomous and/or remotely operated. It uses a set of sensors which gives the robot capacities of surrounding environment perception and digital image processing to search for the balls in places with higher concentrations of balls. This system avoids stopping the players for ball collection permitting a higher use rate of the field

Oliveira, André Joaquim Barbosa de

Pacheco, Luís Fernando Costa

Ribeiro, A. Fernando

Universidade do Minho: RepositoriUM

               MOBILE ROBOT FOR AUTONOMOUS GOLF BALLS PICKING   Luís Fernando Costa Pacheco, André Joaquim Barbosa de Oliveira, António Fernando Macedo Ribeiro   Dep. Electrónica Industrial, University of Minho, Guimarães     Abstract: This paper describes a robot to collect golf balls in a driving range, operating as autonomous and/or remotely operated. It uses a set of sensors which gives the robot capacities of surrounding environment perception and digital image processing to search for the balls in places with higher concentrations of balls. This system avoids stopping the players for ball collection permitting a higher use rate of the field. Copyright.  Keywords: golf, sensorial fusion, digital image processing, communication.      1.INTRODUCTION  In a golf game, the first stroke is so important that there are special places to practice (called driving range). This space is of extreme importance in a golf course for players to learn and improve their skills. But this type of training has some drawbacks like the huge number of golf balls in stock, the stopping time to pick up the balls, amongst others.  Nowadays, the balls pick up and its delivery to the balls dispensing machine is carried out with human intervention, by using some machines with trolleys but always driven by humans. In order to improve this task, the need for dedicated and specialized vehicles is becoming a must, not only to speed up the task but also to reduce the maintenance costs of the whole system.  The traditional systems do not allow the players to use the field while balls picking up or maintenance operations are being carried out, for safety reasons. The extreme violence of one hit in a human is unbearable.  In order to sort out this problem, an idea came up which consisted of developing an autonomous mobile platform which carries out the operation of golf ball pick up. This project consisted of building up a working prototype, develop the whole mechanics in house, to design the hardware boards necessary, to plan and choose the group of sensors required, and to develop software for controlling the robot, to grab and analyse the image of the camera, and the communication software. The prototype was in tests in a real driving range for a few days which allowed the team to come across some of the problems and enabled them to sorted out those problems.    2.PROTOTYPE DESCRIPTION  The prototype built to carry out the golf ball pick up task consists of two physical distinct parts:  • an autonomous robot developed from scratch, but based on the experience acquired on the wheelchair and on the robot footballers, see (Ribeiro, et al., 2004; Ribeiro, et al., 2007), which were also built at the Grupo de Automação e Robótica at University of Minho in Guimarães  • a trolley which is pulled by the autonomous robot, which contains a set of cylinders in which the balls get stuck on it passes over them, and a basket where the balls fall after half a turn of the cylinders.  A diagram is shown in Figure 1 where it can be seen both parts of the golf balls pick up system: the autonomous robot on the right side and the trolley on the left side.   Figure 1 – Autonomous Robot and the trolley  In order to meet all the necessary requirements, a prototype was developed with the following characteristics; it should have a system to detect obstacles, should inform about the robot surrounding environment and its positioning on the golf field. To perceive the surrounding environment several sensors are used being the most important ones the temperature sensor, humidity sensor, inclinometer, electronic campus, a Global Positioning System receiver, a vision system, amongst others.  The prototype also possesses a colour analogue camera linked to a digital colour image processing system. This system grabs the surrounding image and analyses this image to locate the largest concentration of golf balls.  The driving system is carried out by two 200W DC motors which are fed by two 24V lead batteries. A simple electric differential controlled by software is used to manoeuvre the robot.  Besides the whole electronic system, it was necessary to use a trolley whose main task is to pick up the balls and store them in a container so that afterwards the balls can be automatically released next to the balls dispensing machine, when the basket it full.  This trolley also works as the third support in order to equilibrate the whole platform. The device on the trolley which actually picks-up the ball was bought off the shelf. In order for the robot to plan and perform correctly his task, he needs some data acquisition about the surrounding environment as well as information about his actual status. This information is received through the many sensors mounted on the robot platform.  Figure 2 shows a diagram with the whole electronic system. Each device on the diagram is numbered and they all are described in this section. In Figure 2, the main component is the controller board which consists of a FoxBoard. It is a device which comes with embedded Linux operating system, with a CPU ETRAX 100LX (100 MIPS) with RISC architecture developed by AXIS, see (acme systems, 2007), with reduced size and low consumption rates, specially designed to allow an easy integration in many variable applications.    Figure 2 - Robot electronic devices diagram  It contains software specially developed for decision making, which monitors the status of all robot sensors using an i2c bus to communicate between all the electronic devices. It establishes the communication with the joystick or any other remote control device, with the image processing unit and performs the motors speed control using a PI control system, getting the speed feedback from standard optical encoders.  The device (1) consists of a remote control with a joystick to manoeuvre the robot and with a small screen to display relevant information read from some sensors. Inside this remote control (2), a XBee module is used to perform the communication between the joystick and the control board. The communication is carried out between the two modules, one on the FoxBoard and another on the joystick, and this device was developed by the company SAR - Soluções de Automação e Robótica, Guimarães, Portugal. The XBee, is a wireless system that uses the ZigBee protocol, see (Digi, 2008).  The device (3), consists of an accelerometer which detects sudden variations in speed. An accelerometer can be used to detect many things like acceleration, inclination, rotation, vibration, collision and gravity. The one used in this robot is “Memsic 2125 Dual Axis Accelerometer” from “Parallax”.  This accelerometer is made up by a gas chamber, with a device which increases its temperature in the centre forming a hot air bubble, with four temperature sensors in the edges. Since the hot air moves upwards, when the accelerometer is placed in a plan parallel to the floor, the hot air will reside on the top area of the chamber and all temperature sensors will read the same value. Should this accelerometer be tilted, the hot air will tend to move towards one of the sides. This way, it is possible to read the acceleration. Figure 3 shows the working principle of this device.  This device also converts the signals read by the temperature sensors in two signals with a square shaped wave, whose duty-cycle varies with the acceleration felt. It is 50% at 0g and varies proportionally with acceleration, see (parallax, 2004a).     Figure 3 - Accelerometer working principle (adapted from  (parallax, 2004b) )  Number (4) still in Figure 2 represents the encoders which are coupled to each motor and that are used to read the speed of the motor. These are optical encoders with three channels, HEDS-5540, see (Agilent Technologies, 1999).  On (5) are represented the DC motors used of 24V and 7,5A, normally used in wheelchairs.  The motor driving boards are represented on (6) and these have the reference MD03. An H bridge is implemented which supports around about 50V/20A. These boards use the i2c protocol, see (Robot Electronics, 2005).  In order to detect and avoid obstacles an ultra sound sensor is used and is pictured in (7). These are frequently used in mobile robots to measure distances and to detect obstacles. The signal read is based on acoustic signals – The sonar transmits a sound and reads the reflected signal, should it exist, calculating the time between the transmission and the reception. This way, when an obstacle is detected and knowing the time and speed of sound waves propagation, the distance can be determined. The ultra-sound used is SRF235 and also uses the i2c interface, see (Robot Electronics, 2007a).  In order for the robot to know his heading direction (especially when it is stopped) an electronic campus is used and represented on (8). It consists on a passive sensor which uses the magnetic field of the earth to orientate the robot in relation to a specified referential, normally the north. The main objective is to produce a number to represent the actual robot direction. This sensor is very sensitive and easily mistakable by external interferences like large metallic structures, requiring though a previous validation when reading its position. The device used, CMPS03, contains a magnetic field sensor (Philips KMZ51), which is able to detect the earth magnetic field and once again the bus it uses is i2c and controlled by the FoxBoard, see (Robot Electronics, 2007b).  The device (9), consists of a humidity sensor. Humidity is normally not easy to measure. Some devices to read humidity demand electronic circuits to make the analogue-digital conversion. They need operational amplifiers, oscillators, compensations, temperature adjustments to calculate the dew-point. Also, a calibration is necessary (knowing a humidity reference value), the reading shown be carried out in useful time and it should be reliable concerning the weather conditions.  Normally, the environment always has some humidity. The number of molecules of water in the air, vary considerably, which means, the air can be dry as in the desert, or humid as in the tropics. There exists na upper limit of the amount of humidity which the air can have at a certain temperature. If, for some reason, the humidity value is higher, condensation occurs. This limit is called dew-point, the temperature for which a certain amount of humidity in the air starts condensation.  The sensor used STH11 contains capacitive polymers, able to calculate the relative humidity and temperature sensors coupled to a 14 bits ADC which feeds information to the i2c interface, see (Parallax, 2004c).  The device (10), consists of a GPS (Global Positioning System) receiver. The American Department of Defence started this project, whose main purpose was to create a system to estimate the position of a certain coordinate on earth, based on satellites. It works based on the signal delay, in other words, the distance that a signal takes to travel knowing its speed and time. It allows previewing the longitude and latitude, elevation, atmospheric condition and the actual time on any place on the surface of the earth. There are two usage types; one for civil usage, having a certain degree of error and another for military usage. The GPS used is a 18PC from Garmin and uses a serial interface, see (Garmin, 2006).  One Linksys Wireless router is used and represented on (11), which permit to communicate between a remote computer and the robot. The image processing unit is represented on (12), and basically is a low consumption personal computer and on (13) it is pictured the analogue camera used for image acquisition in colours.   3.IMAGE PROCESSING  The image processing is one of the areas of automation and robotics that, in addition to increasing the speed of computerized processing systems, further development has suffered.  One can consider three types of image processing:  • Processing based on the modification of visual information characteristics, using optical elements – lenses and filters. This is a common processing application in photography, and the level of disclosure and the level of abstraction.  • Modification of the electrical signal that represents the image, using analogue electronics.  • Handling characteristics of the digital image. Through the digital handling of the information element characteristics – pixel, is acquiring greater adaptability and flexibility of automated systems.  Depending on the type of application, the three types of processing can do together, for perception of a unique object.  The processing of digital image is the most widely used. This is a result of various intermediate steps.    Figure 4 - Main Stages of Digital Image Processing  Figure 4 shows a clear outline of the various stages for processing digital image – the whole process starts with image acquisition, which normally corresponds to the acquisition of light reflections captured by the camera sensor (CCD). Following, the pre-processing techniques applied to decrease the noise, adjust contrast and brightness, calibration of cameras, and so on.   The next stage is image segmentation. This stage is defined as the separation of the constituent parts of an image in various objects of interest. Here, the automatic segmentation is the one which seems to be the most effective but also more difficult to achieve, due to the need for automatic algorithms which have to adapt to changes in light and brightness. A good segmentation means lesser need to apply complex detection algorithms.  The segmentation main target is normally limited to simple images, mostly binary images, which highlight relevant information of the objects of interest such as contours, shapes or specific colours. These objects pass then by the stages of representation and description, where important parameters from the object are collected such as subject-area, perimeter, shape, etc.  The last step, recognition and perception may or may not be implicit in the previous stage, according to the problem complexity. Through the recognition they differentiate themselves in the segmented image, objects with distinct characteristics between them and pre established at an earlier stage in the processing, which is the study of the objects to detect. For example, to detect circles in an image, one should know before hand what is the specific size of the circles and if they should be differentiated according to the predominant colour that is implicit in them - these data are implicit in a knowledge base inherent in an early stage findings of tests and practices, which can be isolated from the whole processing.  In perception a set of information from the recognition is used and priorities are established and / or decisions are taken. Sometimes, interpretation is confused with recognition, when only one object alone is to be detected. However, doing analogy with the previous example, for a given sequence of circles with known sizes and colours it can be inferred a different interpretation. It will not be enough only to recognize a circle, but also to position it globally throughout the image or sequence of images.   A. Methodology for golf balls detection  The pre-processing used is based on filters for noise reduction: Gaussian filter and blurring filter. Through the use of these two filters there has been a dramatically reduction on the amount of noise on the acquired images.  Demand for golf balls in the picture was made based on two principles:  1. The outline of a golf ball, in the picture, has geometric relationship C, which can be represented by:   πππ .4.).2(222===rrAPC  (1)  Being A = area, P = Perimeter and R = circle radius  2. Any ball has a predominance of colour close to pure white – it is assumed that only white golf balls are used (but this could change accordingly).   Applying these two principles a processing algorithm was developed, based on the following steps:  Capture image Filtering image Find contours For all contours  If C=4π +/- tolerance1   Find circles with const C   For each circle detected    If central pixel colour = (white +/- tolerance2)    Increment counter of balls  All this processing software was developed in C language based on Linux Operating Systems, and using the Intel Corporation – OpenCV Libraries available.   4.REMOTE DATA TRANSMISSION  In order to remotely visualize the robot behaviour and to link all the processing units – continuous processing/control unit, image processing unit and remote access unit, it was necessary to define a data/image transmission/reception protocol. Therefore, a Master-Slave communication protocol was developed as pictured in Figure 5. This protocol has as main server the continuous processing unit – FoxBoard. The image processing unit – PCLocal works as intermediary between the FoxBoard and the remote access unit – PCRemoto. If the PCRemoto requests a data stream from the FoxBoard, it is the PCLocal who manages that request, receives the answer and replies to the PCRemoto.   Figure 5 - Communication protocol between processing units  The transmission protocol was defined as follows:  • The FoxBoard must be the first system to turn-on and will stay in the “listen” status, waiting for data update requests, which contains the necessary data for monitoring the whole system, carried out by the PCLocal  • The PCLocal Works as server for the PCRemoto and as client for the FoxBoard – whenever the PCRemoto sends a data or image request o PCLocal sends the respective data. Should the request be of image type, the PCLocal starts the remote sending of the image that the robot camera is actually grabbing, stopping the transmission only when it receives a new image request  Both request streams and data streams have a specific header with the string GOLF, which is used to validate the stream. Another flag of the stream is the type of information that it contains (data or image).   5.CONCLUSION  An autonomous robot to pick up golf ball in a driving range was designed, developed, built and tested in a real field, with hall the outside constraints. It proved to work and is being used.  The complex and vast sensory process was extremely simplified with the use of i2c protocol based devices. Through the only one type of bus, it was possible to communicate all information percept about the surrounding robot environment.   Figure 6 – Robot in the real environment   The implementation of the whole system based in a continuous processing unit, proved to be feasible and simple due to the fact of using a reliable and tools full Linux based operating system.   REFERENCES  AcmeSystems, (2007). http://www.acmesystems.it/?id=4  Agilent Technologies, (1999). http://www.datasheetarchive.com/pdf/1581517.pdf Digi, (2008). http://www.maxstream.net/wireless/zigbee.php  Garmin, (2006). http://www.garmin.pt/index.asp?Tipo=Detalhe&ID=010-00321-54  Parallax, (2004a). http://www.parallax.com/dl/docs/prod/acc/memsickit.pdf Parallax, (2004b). http://www.parallax.com/dl/docs/prod/compshop/SICMemsicTut.pdf  Parallax, (2004c). http://www.parallax.com/detail.asp?product_id=28018 Ribeiro, F., Moutinho, I., Silva, P., Fraga, C. and Pereira, N., (2004). "Vision, Kinematics and Game strategy in Multi-Robot Systems like MSL RoboCup", ROBOCUP'2004 - Team Description Paper, RoboCup'2004 - Robot Soccer World Cup VIII - LNAI 3276, Nardi et all, Springer, published on CD-ROM. Ribeiro, F., Moutinho, I., Silva, P., Braga, P. and Pereira, N., (2007). "Mobile Robot Construction for Edutainment Application", Revista Robótica, n. 69, pag. 12-16, ISSN: 0874-9019. Robot Electronics, (2005). http://www.robot-electronics.co.uk/htm/md03tech.htm Robot Electronics, (2007a). http://www.robot-electronics.co.uk/htm/srf235tech.htm Robot Electronics, (2007b). http://www.robot-electronics.co.uk/htm/cmps3doc.shtml  

Mobile robot for autonomous golf balls picking

http://repositorium.sdum.uminho.pt/bitstream/1822/10247/1/Paper%20Golfe%20Norberto%20Pires.pdf

Mobile robot for autonomous golf balls picking

Abstract

Similar works

Full text

Available Versions

Universidade do Minho: RepositoriUM