Empirical control system development for intelligent mobile robot based on the elements of the reinforcement machine learning and axiomatic design theory

Ovaj rad predstavlja istraživanje autora u domenu koncepcijskog projektovanja upravljačkog sistema koji može da uči na osnovu sopstvenog iskustva. Sposobnost adaptivnog ponašanja pri izvršavanju postavljenog zadatka u realnim, nepredvidivim uslovima, jedan je od ključnih zadataka svakog inteligentnog robotskog sistema. U funkciji rešavanja ovog problema, predlaže se pristup baziran na učenju, i to kombinovanjem empirijske upravljačke strategije, mašinskog učenja ojačavanjem i aksiomatske teorije projektovanja. Predloženi koncept koristi najbolje osobine pomenutih teorijskih pristupa u cilju ostvarivanja optimalne odluke mobilnog robota za trenutno stanje sistema. Empirijska upravljačka teorija se, u ovom radu, a priori koristi u utvrđivanju idejnog rešenja za rešavanje problema navigacije mobilnog robota. Učenje ojačavanjem realizuje mehanizme koji memorišu i ažuriraju odgovore okruženja, a u kombinaciji sa empirijskom upravljačkom teorijom određuje najbolju moguću odluku u skladu sa trenutnim okolnostima. Aksiomatska teorija projektovanja se koristi pri definisanju upravljačkog problema, kao i pri uspostavljanju koncepcijskog rešenja za dati zadatak, sa aspekta primene pomenutih pristupa. Deo predloženog algoritma empirijskog upravljanja realizovan je pomoću LEGO Mindstorms NXT mobilnog robota, tretirajući problem navigacije u nepoznatom okruženju. Ostvareni eksperimentalni rezultati nagoveštavaju dobru perspektivu za realizaciju efikasnog upravljanja baziranog na iskustvu, čiji dalji razvoj može da dovede do ostvarenja autonomnog ponašanja mobilnog robota pri izbegavanju prepreka u tehnološkom okruženju, što je i očekivani naučni cilj.This paper presents the authors' efforts to conceptual design of control system that can learn from its own experience. The ability of adaptive behaviour regarding the given task in real, unpredictable conditions is one of the main demands for every intelligent robotic system. To solve this problem, the authors suggest a learning approach that combines empirical control strategy, reinforcement learning and axiomatic design theory. The proposed concept uses best features of mentioned theoretical approaches to produce optimal action in the current state of the mobile robot. In this paper empirical control theory imparts the basis of conceptual solution for the navigation problem of mobile robot. Reinforcement learning enables the mechanisms that memorize and update environment responses, and combining with the empirical control theory determines best possible action according to the present circumstances. Axiomatic design theory accurately defines the problem and possible solution for the given task in terms of the elements defined by two previously mentioned approaches. Part of the proposed algorithm was implemented on the LEGO Mindstorms NXT mobile robot for the navigation task in an unknown manufacturing environment. Experimental results have shown good perspective for development of efficient and adaptable control system, which could lead to autonomous mobile robot behaviour

Empirical control system development for intelligent mobile robot based on the elements of the reinforcement machine learning and axiomatic design theory

Ovaj rad predstavlja istraživanje autora u domenu koncepcijskog projektovanja upravljačkog sistema koji može da uči na osnovu sopstvenog iskustva. Sposobnost adaptivnog ponašanja pri izvršavanju postavljenog zadatka u realnim, nepredvidivim uslovima, jedan je od ključnih zadataka svakog inteligentnog robotskog sistema. U funkciji rešavanja ovog problema, predlaže se pristup baziran na učenju, i to kombinovanjem empirijske upravljačke strategije, mašinskog učenja ojačavanjem i aksiomatske teorije projektovanja. Predloženi koncept koristi najbolje osobine pomenutih teorijskih pristupa u cilju ostvarivanja optimalne odluke mobilnog robota za trenutno stanje sistema. Empirijska upravljačka teorija se, u ovom radu, a priori koristi u utvrđivanju idejnog rešenja za rešavanje problema navigacije mobilnog robota. Učenje ojačavanjem realizuje mehanizme koji memorišu i ažuriraju odgovore okruženja, a u kombinaciji sa empirijskom upravljačkom teorijom određuje najbolju moguću odluku u skladu sa trenutnim okolnostima. Aksiomatska teorija projektovanja se koristi pri definisanju upravljačkog problema, kao i pri uspostavljanju koncepcijskog rešenja za dati zadatak, sa aspekta primene pomenutih pristupa. Deo predloženog algoritma empirijskog upravljanja realizovan je pomoću LEGO Mindstorms NXT mobilnog robota, tretirajući problem navigacije u nepoznatom okruženju. Ostvareni eksperimentalni rezultati nagoveštavaju dobru perspektivu za realizaciju efikasnog upravljanja baziranog na iskustvu, čiji dalji razvoj može da dovede do ostvarenja autonomnog ponašanja mobilnog robota pri izbegavanju prepreka u tehnološkom okruženju, što je i očekivani naučni cilj.This paper presents the authors' efforts to conceptual design of control system that can learn from its own experience. The ability of adaptive behaviour regarding the given task in real, unpredictable conditions is one of the main demands for every intelligent robotic system. To solve this problem, the authors suggest a learning approach that combines empirical control strategy, reinforcement learning and axiomatic design theory. The proposed concept uses best features of mentioned theoretical approaches to produce optimal action in the current state of the mobile robot. In this paper empirical control theory imparts the basis of conceptual solution for the navigation problem of mobile robot. Reinforcement learning enables the mechanisms that memorize and update environment responses, and combining with the empirical control theory determines best possible action according to the present circumstances. Axiomatic design theory accurately defines the problem and possible solution for the given task in terms of the elements defined by two previously mentioned approaches. Part of the proposed algorithm was implemented on the LEGO Mindstorms NXT mobile robot for the navigation task in an unknown manufacturing environment. Experimental results have shown good perspective for development of efficient and adaptable control system, which could lead to autonomous mobile robot behaviour

Bioaccumulation of metallic trace elements and antioxidant enzyme activities in Apfelbeckia insculpta (L. Koch, 1867) (Diplopoda: Callipodida) from the cave Hadži-Prodanova Pećina (Serbia)

The concentration of 10 metallic trace elements or MTE (Cu, Fe, Zn, Mn, As, Hg, Pb, Cd, Ni, and Cr) was measured in specimens of the troglophilic millipede Apfelbeckia insculpta (L. Koch, 1867) and sediment of the cave Hadži-Prodanova Pećina in western Serbia. Some MTE, like Fe and Mn, displayed much higher concentrations compared to other elements, both in the sediment and in the body of A. insculpta. On the other hand, estimation of the bioaccumulation factor (BAF) in both males and females of A. insculpta showed values greater than 1 for xenobiotic elements compared to those that are essential. In addition to chemical analyses, we examined the activities of antioxidant enzymes (SOD, CAT, GPX, and GR) and the phase II biotransformation enzyme GST, as well as the content of –SH groups, in the body of A. insculpta. Activities of two (GR and GST) out of the five tested enzymes showed significant differences between the sexes. These results represent the first comprehensive report of antioxidant enzymes in myriapods. The noted differences in the investigated MTE and enzyme activities between the sexes of A. insculpta most likely reflect different metabolic activities and responses to environmental conditions in males and females.International Journal of Speleology (2017), 46(1): 99-10

On three new high-altitude endemic leiodids (Coleoptera: Leiodidae) from the Balkan Peninsula

Three new leiodid beetle species, viz. Petkovskiella henrikenghoffi sp. n. (Republic of Macedonia, Mt. Dautica), Babuniella jovanhadzii sp. n. (Republic of Macedonia, Mt. Karadžica) and Magdelainella milojebrajkovici sp. n. (Serbia, Mt. Javor), are described and diagnostified. Both adult genitalia and other taxonomically important characters are illustrated. Babuniella Z. Karaman is given a full generic status. All new species studied are clearly distinct from their closest congeners. These forms are of the Tertiary or even pre-Tertiary origin and age, and represent both relicts and endemics inhabiting central and southern areas of the Balkan Peninsula

Empirical Control for Intelligent Robotic Systems – State-of-the-Art

Емпиријско управљање представља нов приступ у концепцијском пројектовању управљачких система мобилних робота и робота вертикалне зглобне конфигурације. У односу на конвенционалне приступе, емпиријски системи имају способност машинског учења на основу прикупљених информација из технолошког окружења, перманентно унапређујући своје понашање сходно постављеном задатку. У раду је дат детаљан преглед истраживања у овој области са посебним освртом на развој и имплеметацију емпиријских управљачких система на бази машинског Q-учења ојачавањем и soft computing техника вештачке интелигенције. Извршена је анализа актуелних праваца истраживања са становишта карактеристичних проблема управљања роботских система (проблем навигације, избегавања препрека, праћења зида технолошког окружења, и/или визуелног навођења). Сваки од презентованих истраживачких резултата је укратко описан, са јасно наглашеном предношћу примене теорије емпиријског управљања у процесу концепцијског пројектовања управљачких система.Empirical control presents a new approach in the domain of the conceptual design of the control systems for mobile robots and robot manipulators. Compared to the conventional design methods, empirical control systems have the ability to learn based on the information obtained from the environment, continuously improving robot’s behaviour. This paper presents a review on current research results, with emphasis on control systems based on the Q-learning algorithm and soft computing techniques. Comparative analysis has been conducted in terms of common robot-based and vision-based tasks. Described algorithms and experimental evaluations in real world clearly points out the advantages of implementation of the empirical theory in the conceptual design process of the control systems

Empirical Control System Development for Intelligent Mobile Robot Based on the Elements of the Reinforcement Machine Learning and Axiomatic Design Theory

This paper presents the authors’ efforts to conceptual design of control system that can learn from its own experience. The ability of adaptive behaviour regarding the given task in real, unpredictable conditions is one of the main demands for every intelligent robotic system. To solve this problem, the authors suggest a learning approach that combines empirical control strategy, reinforcement learning and axiomatic design theory. The proposed concept uses best features of mentioned theoretical approaches to produce optimal action in the current state of the mobile robot. In this paper empirical control theory imparts the basis of conceptual solution for the navigation problem of mobile robot. Reinforcement learning enables the mechanisms that memorize and update environment responses, and combining with the empirical control theory determines best possible action according to the present circumstances. Axiomatic design theory accurately defines the problem and possible solution for the given task in terms of the elements defined by two previously mentioned approaches. Part of the proposed algorithm was implemented on the LEGO Mindstorms NXT mobile robot for the navigation task in an unknown manufacturing environment. Experimental results have shown good perspective for development of efficient and adaptable control system, which could lead to autonomous mobile robot behaviour

Empirical Control for Intelligent Robotic Systems – State-of-the-Art

Емпиријско управљање представља нов приступ у концепцијском пројектовању управљачких система мобилних робота и робота вертикалне зглобне конфигурације. У односу на конвенционалне приступе, емпиријски системи имају способност машинског учења на основу прикупљених информација из технолошког окружења, перманентно унапређујући своје понашање сходно постављеном задатку. У раду је дат детаљан преглед истраживања у овој области са посебним освртом на развој и имплеметацију емпиријских управљачких система на бази машинског Q-учења ојачавањем и soft computing техника вештачке интелигенције. Извршена је анализа актуелних праваца истраживања са становишта карактеристичних проблема управљања роботских система (проблем навигације, избегавања препрека, праћења зида технолошког окружења, и/или визуелног навођења). Сваки од презентованих истраживачких резултата је укратко описан, са јасно наглашеном предношћу примене теорије емпиријског управљања у процесу концепцијског пројектовања управљачких система.Empirical control presents a new approach in the domain of the conceptual design of the control systems for mobile robots and robot manipulators. Compared to the conventional design methods, empirical control systems have the ability to learn based on the information obtained from the environment, continuously improving robot’s behaviour. This paper presents a review on current research results, with emphasis on control systems based on the Q-learning algorithm and soft computing techniques. Comparative analysis has been conducted in terms of common robot-based and vision-based tasks. Described algorithms and experimental evaluations in real world clearly points out the advantages of implementation of the empirical theory in the conceptual design process of the control systems

Empirical Control System Development for Intelligent Mobile Robot Based on the Elements of the Reinforcement Machine Learning and Axiomatic Design Theory

This paper presents the authors’ efforts to conceptual design of control system that can learn from its own experience. The ability of adaptive behaviour regarding the given task in real, unpredictable conditions is one of the main demands for every intelligent robotic system. To solve this problem, the authors suggest a learning approach that combines empirical control strategy, reinforcement learning and axiomatic design theory. The proposed concept uses best features of mentioned theoretical approaches to produce optimal action in the current state of the mobile robot. In this paper empirical control theory imparts the basis of conceptual solution for the navigation problem of mobile robot. Reinforcement learning enables the mechanisms that memorize and update environment responses, and combining with the empirical control theory determines best possible action according to the present circumstances. Axiomatic design theory accurately defines the problem and possible solution for the given task in terms of the elements defined by two previously mentioned approaches. Part of the proposed algorithm was implemented on the LEGO Mindstorms NXT mobile robot for the navigation task in an unknown manufacturing environment. Experimental results have shown good perspective for development of efficient and adaptable control system, which could lead to autonomous mobile robot behaviour

A first record of the antioxidant defense and selected trace elements in Salamandra salamandra larvae on Mt. Avala and Mt. Vršački Breg (Serbia)

We investigated the activities of superoxide dismutase (SOD), catalase (CAT), glutathioneperoxidase (GSH-Px), glutathione reductase (GR) and the phase II biotransformation enzyme glutathioneS-transferase (GST) in the whole body of fire salamander larvae (Salamandra salamandra) from twolocalities on Mt. Avala (AVS and ABP) and one locality on Mt. Vršački Breg (VSB), Serbia. We alsodetermined the total glutathione (GSH) and sulfhydryl group (SH) contents, as well as the concentrations ofmanganese (Mn), copper (Cu), zinc (Zn), selenium (Se), arsenic (As), cadmium (Cd), lead (Pb) and uranium(U). The obtained results show that animals from VSB had significantly lower weights and lengths thananimals from AVS and ABP. The activities of all investigated enzymes were significantly higher, while theSH content was significantly lower in animals from VSB compared to those from AVS and ABP. Nocorrelations between trace-element concentrations in water and animal tissue were observed. We concludedthat the obtained results were more likely a consequence of the combination of developmental differencesand the effects of different habitat conditions, environmental and anthropogenic influences than ofconcentrations of trace elements in the water alone

Q-Learning Framework as a Solution for an Obstacle Avoidance Problem in Unknown Environment

This paper presents machine learning approach as a solution for an obstacle avoidance problem. Q-learning, as one of the reinforcement learning algorithms, imparts the learning process based on trial and error and a corresponding reward into the behaviour of an intelligent agent - a mobile robot. The adaptable actions of a mobile robot in situations when that behaviour is necessary are the main advantage over conventional methods for designing a navigational path. The implemented algorithm characterizes simplicity and efficiency, and certainty in terms of reaching optimal behaviour after the certain number of learning episodes. Experimental results show proper exploration strategy with gradually improving mobile robot state to action mapping by adjusting Q-value function in a described manner. With more episodes conducted this adaptable control system could lead to a fully autonomous mobile robot, which is one of the main demands in modern intelligent manufacturing systems in which stochastic changes in the environment can results with failure in the entire production process