Robot World Cup Initiative (RoboCup) is a worldwide competition proposed to advance research in robotics and artiﬁcial intelligence. It has a league called RoboCup soccer devoted for soccer robots. Robotic soccer is a challenge because robots are mobile, fully autonomous, multi-agents, and they play on a dynamic environment. Moreover, robots must recognize the game entities, which is a crucial task during a game. A camera is usually used as an input system to recognize ball, opponents, soccer ﬁeld, and so on. These elements may be recognized applying some tools of computational intelligence, for example an artiﬁcial neural network. This paper describes the application of an artiﬁcial neural network on middle size robotic football league, where a multilayer perceptron neural network is trained with the backpropagation algorithm, to classify elements on the image. The results show that an artiﬁcial neural network successfully classiﬁed the entities. They were recognized even when similar color entities were present on the image.info:eu-repo/semantics/publishedVersio

Lopes, Gil

Ribeiro, A. Fernando

Ribeiro, Paulo Rogério de Almeida

Journal of Software Engineering and Applications

Paulo Rogério de Almeida Ribeiro

Gil Lopes

Fernando Ribeiro

Crossref

Neural Network in Computer Vision for RoboCup Middle Size League

Universidade do Minho: RepositoriUM

Journal of Software Engineering and Applications, 2016, *,**Neural Network in Computer Vision for RoboCupMiddle Size LeaguePaulo Roge´rio de Almeida Ribeiro1, Gil Lopes1, Fernando Ribeiro11Department of Industrial Electronics - Robotics Group - Algoritmi Centre - University of Minho, Campusde Azure´m, 4800-058, Guimara˜es, PortugalEmail: paulorogeriocp@gmail.com, gil@dei.uminho.pt, fernando@dei.uminho.ptReceived **** 2016Copyright c© 2016 by author(s) and Scientific Research Publishing Inc.This work is licensed under the Creative Commons Attribution International License (CC BY).http://creativecommons.org/licenses/by/4.0/AbstractRobot World Cup Initiative (RoboCup) is a worldwide competition proposed to advance re-search in robotics and artificial intelligence. It has a league called RoboCup soccer devotedfor soccer robots. Robotic soccer is a challenge because robots are mobile, fully autonomous,multi-agents, and they play on a dynamic environment.Moreover, robots must recognize the game entities, which is a crucial task during a game.A camera is usually used as an input system to recognize ball, opponents, soccer field, and soon. These elements may be recognized applying some tools of computational intelligence, forexample an artificial neural network.This paper describes the application of an artificial neural network on middle size roboticfootball league, where a multilayer perceptron neural network is trained with the backpropa-gation algorithm, to classify elements on the image. The results show that an artificial neuralnetwork successfully classified the entities. They were recognized even when similar color en-tities were present on the image.KeywordsRoboCup; Computer Vision; Artificial Neural Network1. IntroductionRobotic soccer is a challenge since its robots are mobile, fully autonomous, multi-agents, and they play ona dynamic environment. Its main competition, Robot World Cup Initiative (RoboCup) [1], consists of ascientific challenge that has been created to develop research in robotics and artificial intelligence through astandard worldwide problem, e.g. [2].How to cite this paper: Ribeiro, P. R. A et al, (2016) Neural Network in Computer Vision for RoboCupMiddle Size League. ***********,*,***-***1Ribeiro, P. R. A et alMovement control and planning, and vision system are relevant research areas for this problem. Forexample, a precise perception of the soccer field, ball and other entities of the game are important taskssince several decisions rely on these information. An usual approach to recognize these elements is throughan algorithm applied on images grabbed by a camera. Some tools of computational intelligence have beenapplied to solve this recognition problem [3] considering their high classification rate. However, some of thesealgorithms have a high processing time to recognize the entities and this time during a game is importantas the classification rate. An ideal approach must find a compromise between the shortest processing timeand the highest rate of correct classification.Thus, this work proposes to apply an artificial neural network (ANN) to recognize entities - such as soccerfield, lines, ball, players - for RoboCup middle size league. An ANN was chosen because: it can approximateany function [4]; it works with noisy data; minimum knowledge about the problem is required [5], as wellas a minimum knowledge of the input statical distribution [6] is sufficient. As a proof of concept, this workclassifies only three entities, namely: soccer field, ball and “other elements”.This paper is organized as follows: On Section 2, the Robocup competition is described; Section 3reports the methodology applied for this work; Section 4 shows the obtained results; and Section 5 presentsthe conclusion and directions for further works .2. RobocupAs stated by the RoboCup federation on its website, the main objective of RoboCup challenge is “By themiddle of the 21st century, a team of fully autonomous humanoid robot soccer players shall win a soccergame, complying with the official rules of FIFA, against the winner of the most recent World Cup”.RoboCup intends to promote research on Artificial Intelligence and robotics in general, with a worldknown rules challenge like football, and therefore, teams from all around the world are building mobile andautonomous robotic platforms able to play football, and refereed by a human. In order to make it possible,the challenges have been split into several different leagues, to tackle different problems separately. TheRoboCup Soccer is divided into Simulation League, Small Size Robot League, Middle Size Robot League,Humanoid League, and Standard Platform League. Each one of these leagues tackle different aspects of thechallenge. While simulation concentrates more on the strategy of the game itself, the small size league teamsdo research on cooperation and fast moving, the Middle Size League sorts out most the physical problemswith hardware/kicking devices, etc. However, the middle size league is the one with robots playing mostsimilar to humans, regarding speed, weight and size, strategy, ball pass, kick strength, field characteristics,and so on.Nowadays, the RoboCup is not only football but also other challenges namely RoboCupRescue andRoboCup@Home. The Rescue league involves a physical space called arena that simulates a destroyedenvironment (like an earthquake) and the robots have to find out victims through sound, temperature ormovement. The @home league consists of a physical space similar to a real home and each robot has toperform domestic tasks like taking the bin outside, turning on a TV, following one person, picking up objectsfrom a table of from inside a fridge, etc. This league is becoming more and more attractive to teams due toits usefulness and because it consists of only one robot per team.The RoboCup Middle Size Robot League football (or soccer in USA) challenge involves many problemsto be sorted by researchers and the most important ones are described here: 1) it is a dynamical environmentwhere each robot is a different entity and never on the same place; 2) since cameras grab images at 60 framesper second and some teams move at speeds of up to four meters per second a real time system is crucial;3) the information available by each robot is incomplete, i.e. the information rapidly becomes out datedand each robot cannot see the whole field or perceive all information from all opponents and team mates,therefore some assumption has to be carried out; 4) The sensor reading is non symbolic and since each robotteam has a non standard shape it is difficult to grab information about opponents; 5) the control is notcentralized, i.e., each robot has his own computer/processor and takes his own decisions, and therefore alot of communication needs to be used, which is sometimes hard to achieve because there are hundreds ofcommunication devices on the pavilion.Robots that participate on the middle size league must have their own data acquisition system. The2Ribeiro, P. R. A et alvideo camera is one of the most important sensors on this league, and can be used for the localization andobstacle avoidance tasks. However, these tasks can be seriously affected due to the vision system that isdisturbed during light conditions changes. Algorithms for image processing and computer vision must berobust against several conditions such as fast moving robots, light conditions variations, vibrations, imageocclusion and so on. Figure 1 shows a game for the RoboCup middle size league where some entities of thegame can be seen, such as soccer field, robots and ball.Figure 1: Soccer robots - RoboCup middle size league.3. MethodologyThe image is obtained by an omnidirectional vision system. This system uses a parabolic/conic mirror facingdownwards and a color camera facing the mirror to obtain a 360 ◦ view, as demonstrated on Figure 2. AnANN is used to classify some entities of the RoboCup middle size league. As a proof of concept, an ANN istrained with the backpropagation algorithm to recognize soccer field, ball and “other elements” present onthe captured image.ANN’s input is a window with the three color channels, i.e. red, green and blue channel of the image.Each output neuron represents an entity, and its output value depends on the current entity that is presenton the image. Thus, there are 3 neurons on the output layer: the first neuron for the soccer field, the secondone is for the ball, and the third one is for “other elements”. In order to recognize one element the outputvalue should be 1 for its output neuron whereas the other output neurons should be −1, i.e. a soccer field asan input sample should lead to ANN’s output such as [1 −1 −1] whereas a ball as an input sample produces[−1 1 −1] as the output of the ANN.3Ribeiro, P. R. A et alFigure 2: Omnidirectional vision system: parabolic/conic mirror facing downwards and a color camerafacing the mirror.4. ResultsThe application was implemented in C++ and QtCreator [7] using the library OpenCV [8], and was runningunder the operating system Ubuntu. The computer was a conventional notebook with a 2.2 GHz Core 2 Duoprocessor. Figure 3 shows the system developed to recognize the entities during data acquisition to train theANN. For each input sample (numbered on Figure 3) a corresponding label for it is given - i.e. soccer field,ball, or others - on the left side of Figure 3.Figure 3: System developed to recognize the entities during sample acquisitionSeveral combinations of ANN architectures were tested, such as number of neurons in the hidden layers,values of leaning rate and momentum term, as well as the number of samples to train and epochs. Thebest obtained parameters for this case are shown on Table 1. The input layer has 75 neurons once the usedwindow had 25 pixels and the number of input neurons is equal to the window size multiplied by 3, whichis the number of color channels. The activation function was a sigmoid function.4Ribeiro, P. R. A et alParameters ValueNumber of input neurons 75Number of hidden neurons - first layer 60Number of hidden neurons - second layer 40Number of output neurons 3Learning rate 0.00004Momentum term 0.07Desired Error 0.0001Epochs 99999Table 1: Parameters used for the classification problem.The developed system can classify an image sample or an entire image. Figure 4 depicts the process toclassify an image sample, where on the right side an input sample (blue square on the image grabbed by thecamera) is shown and on the left side it displays the ANN’s output (on a table) for this input sample. Theaverage time to recognize a sample (size 5 by 5 pixels) is 0.007 seconds whereas for a full image its requiredtime is approximately 0.1 seconds.Figure 4: System’s output for an input sample (blue square) to recognize soccer fieldAcceptable results were obtained with the system as shown on Figure 4. Thus, new elements were placedon the soccer field to check system’s reliability, namely large paper cylinders and road cones. Paper cylinderswere used since these are used in real games as team markers. Road cones were used due to its color similaritywith the ball.The results with the paper cylinders are accurate, as shown on Figure 5, i.e. it shows no confusionbetween the ball and paper cylinders. On the other hand, the system was not so conclusive to distinguishbetween the ball and road cone. This was mainly due to the road cone’s color, which is very similar to the5Ribeiro, P. R. A et alball color. An input sample containing the road cone may generate an ANN’s output as [−1 1 −1] ratherthan [−1 −1 1] as expected, once it has to be classified as “other elements”. A tolerance value, which is thedifference between the desired output and the current ANN’s output, was used to solve this misclassification.Several experiments demonstrated an ideal tolerance value of 0.3. Figure 6 shows a tolerance value of 0.5 inorder to show a misclassification between ball and road cone.Figure 5: Ball reliably classified (blue square) after an insertion of new elementsFigure 6: Example of misclassification between ball and road cone.Two short movies were recorded to show the software working on a real environment, the first videoshows the classification of some parts of the image1 whereas the second one shows the classification for thewhole image2.1https://www.youtube.com/watch?v=6AKcsQxz4PQ2https://www.youtube.com/watch?v=XcK5C7oQZco6Ribeiro, P. R. A et al5. ConclusionANN have been widely used to recognize objects in several different applications. This work shows anapplication of ANN for RoboCup middle size league. The results show that the use of ANN for this taskis successful. The entities were completely recognized, even when similar color entities like paper cylindersand road cones are present on the image.Considering that a soccer robot acquires between 25 and 30 frames per second, the time to classify a fullimage is considered high, therefore, an alternative is to use an ANN only in some parts of the image andonly after image segmentation. For further work it is recommended to use a K-means to segment the image,in order to reduce classification time.AcknowledgmentsThe authors wish to thank all members of the Laborato´rio de Automac¸a˜o e Robo´tica, at University of Minho,Guimara˜es. Also special thanks to the ALGORITMI Research Centre for the opportunity to develop thisresearch.References[1] Robocup: (2016) http://www.robocup.org/ [Accessed in: 13 Mar. 2016].[2] Budden, D., Wang, P., Obst, O., and Prokopenko, M. Sept 2015 Robotics Automation Magazine, IEEE22(3), 140–146.[3] Kashanipour, A., Kashanipour, A. R., Milani, N. S., Akhlaghi, P., and Boukani, K. K. (2007) In RoboCup: pp. 548–555.[4] Haykin, S. (1999) Neural Networks: A Comprehensive Foundation, Prentice Hall, 2 edition.[5] Egmont-Petersen, M., deRidder, D., and Handels, H. (2002) Image processing with neural networks - areview. Pattern Recognition 35(10), 2279–2301.[6] Amoroso, C., Chella, A., Morreale, V., and Storniolo, P. (2000) In RoboCup-99: Robot Soccer WorldCup III London, UK: Springer-Verlag. pp. 136–147.[7] QtCreator (2016) http://www.qt.io/ [Accessed in: 10 Jan. 2016].[8] OpenCV (2016) http://opencv.org/ [Accessed in: 12 Feb. 2016].7

Neural network in computer vision for RoboCup middle size league

http://repositorium.sdum.uminho.pt/bitstream/1822/53789/1/Paper.pdf

Neural network in computer vision for RoboCup middle size league

Abstract

Similar works

Full text

Available Versions

Crossref

Universidade do Minho: RepositoriUM