Behavioral Repertoire via Generative Adversarial Policy Networks

Learning algorithms are enabling robots to solve increasingly challenging real-world tasks. These approaches often rely on demonstrations and reproduce the behavior shown. Unexpected changes in the environment may require using different behaviors to achieve the same effect, for instance to reach and grasp an object in changing clutter. An emerging paradigm addressing this robustness issue is to learn a diverse set of successful behaviors for a given task, from which a robot can select the most suitable policy when faced with a new environment. In this paper, we explore a novel realization of this vision by learning a generative model over policies. Rather than learning a single policy, or a small fixed repertoire, our generative model for policies compactly encodes an unbounded number of policies and allows novel controller variants to be sampled. Leveraging our generative policy network, a robot can sample novel behaviors until it finds one that works for a new environment. We demonstrate this idea with an application of robust ball-throwing in the presence of obstacles. We show that this approach achieves a greater diversity of behaviors than an existing evolutionary approach, while maintaining good efficacy of sampled behaviors, allowing a Baxter robot to hit targets more often when ball throwing in the presence of obstacles.Comment: In Proceedings of 2019 Joint IEEE 9th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob), pages 320 - 32

Generative neural data synthesis for autonomous systems

A significant number of Machine Learning methods for automation currently rely on data-hungry training techniques. The lack of accessible training data often represents an insurmountable obstacle, especially in the fields of robotics and automation, where acquiring new data can be far from trivial. Additional data acquisition is not only often expensive and time-consuming, but occasionally is not even an option. Furthermore, the real world applications sometimes have commercial sensitivity issues associated with the distribution of the raw data. This doctoral thesis explores bypassing the aforementioned difficulties by synthesising new realistic and diverse datasets using the Generative Adversarial Network (GAN). The success of this approach is demonstrated empirically through solving a variety of case-specific data-hungry problems, via application of novel GAN-based techniques and architectures. Specifically, it starts with exploring the use of GANs for the realistic simulation of the extremely high-dimensional underwater acoustic imagery for the purpose of training both teleoperators and autonomous target recognition systems. We have developed a method capable of generating realistic sonar data of any chosen dimension by image-translation GANs with Markov principle. Following this, we apply GAN-based models to robot behavioural repertoire generation, that enables a robot manipulator to successfully overcome unforeseen impedances, such as unknown sets of obstacles and random broken joints scenarios. Finally, we consider dynamical system identification for articulated robot arms. We show how using diversity-driven GAN models to generate exploratory trajectories can allow dynamic parameters to be identified more efficiently and accurately than with conventional optimisation approaches. Together, these results show that GANs have the potential to benefit a variety of robotics learning problems where training data is currently a bottleneck

Full-Scale Continuous Synthetic Sonar Data Generation with Markov Conditional Generative Adversarial Networks

Deployment and operation of autonomous underwater vehicles is expensive and time-consuming. High-quality realistic sonar data simulation could be of benefit to multiple applications, including training of human operators for post-mission analysis, as well as tuning and validation of autonomous target recognition (ATR) systems for underwater vehicles. Producing realistic synthetic sonar imagery is a challenging problem as the model has to account for specific artefacts of real acoustic sensors, vehicle altitude, and a variety of environmental factors. We propose a novel method for generating realistic-looking sonar side-scans of full-length missions, called Markov Conditional pix2pix (MC-pix2pix). Quantitative assessment results confirm that the quality of the produced data is almost indistinguishable from real. Furthermore, we show that bootstrapping ATR systems with MC-pix2pix data can improve the performance. Synthetic data is generated 18 times faster than real acquisition speed, with full user control over the topography of the generated data.Comment: 6 pages, 6 figures. Accepted to ICRA2020. 2020 IEEE International Conference on Robotics and Automatio

Adversarial Generation of Informative Trajectories for Dynamics System Identification

Dynamic System Identification approaches usually heavily rely on the evolutionary and gradient-based optimisation techniques to produce optimal excitation trajectories for determining the physical parameters of robot platforms. Current optimisation techniques tend to generate single trajectories. This is expensive, and intractable for longer trajectories, thus limiting their efficacy for system identification. We propose to tackle this issue by using multiple shorter cyclic trajectories, which can be generated in parallel, and subsequently combined together to achieve the same effect as a longer trajectory. Crucially, we show how to scale this approach even further by increasing the generation speed and quality of the dataset through the use of generative adversarial network (GAN) based architectures to produce a large databases of valid and diverse excitation trajectories. To the best of our knowledge, this is the first robotics work to explore system identification with multiple cyclic trajectories and to develop GAN-based techniques for scaleably producing excitation trajectories that are diverse in both control parameter and inertial parameter spaces. We show that our approach dramatically accelerates trajectory optimisation, while simultaneously providing more accurate system identification than the conventional approach.Comment: Accepted for publication in IEEE iROS 202

Survey: Leakage and Privacy at Inference Time

Leakage of data from publicly available Machine Learning (ML) models is an area of growing significance as commercial and government applications of ML can draw on multiple sources of data, potentially including users' and clients' sensitive data. We provide a comprehensive survey of contemporary advances on several fronts, covering involuntary data leakage which is natural to ML models, potential malevolent leakage which is caused by privacy attacks, and currently available defence mechanisms. We focus on inference-time leakage, as the most likely scenario for publicly available models. We first discuss what leakage is in the context of different data, tasks, and model architectures. We then propose a taxonomy across involuntary and malevolent leakage, available defences, followed by the currently available assessment metrics and applications. We conclude with outstanding challenges and open questions, outlining some promising directions for future research

Composite honeycomb cores

This thesis demonstrates how a certain type of a sandwich core - honeycomb is manufactured and post-processed, in case if it was made from composite material itself. Any sandwich structure consists of three essential elements: two faces, core and joints. The core represents good ability to carry shear loads. That part of the sandwich is incredibly significant, since it assures that sandwich is not just light but also stiff enough. Honeycomb core is one of the most structurally efficient core constructions, especially in stiffness-critical applications, which is provided with minimal density and high out-of-plane compression and shear properties. As it follows from its name, honeycomb core consists of hexagon-shaped cells, which are completely identical to each other. That core structure is an anisotropic one, which means it is anisotropically loaded inside the sandwich. That follows that honeycomb core could be made of an anisotropic material performing a maximum strength to mass ratio. Traditional honeycomb constructing methods include corrugation and expansion manufacturing processes as well as expansion combined with dipping into phenolic resin. Corrugation was chosen as a basis manufacturing process used to create honeycomb cores in the experimental part of this work. The process consisted of the following steps: prepreg glass fibre and carbon fibre sheets were shaped with a use of aluminium mould, heat and pressure and then sliced and glued together in order to create a composite honeycomb core. Post-processing by milling did not result in shear overload and delaminating of skins or delaminating of layers from each other. As a result some composite honeycomb cores and composite honeycomb sandwich panels were made. Macroscopic density (approx. 0.4 g/cm3) and price (at least 160.04 euro/kg) of the made products were competitive with Ultracor samples (approx. 0.3 g/cm3 and 16000 euro/kg). However these made products would have more performance if bonding between layers inside the honeycomb core would be stronger with less adhesive used

Defining customer experience in four star hotels in Finland and in Russia

The main objective of this case study was to define customer experience in four star hotels in Finland and in Russia. The commissioners of the project are a Finnish hotel “Helka” and a Russian hotel “Helvetia”. Both organizations belong to the four star hotel category with a similar range of services. The hotels’ location allows easy access to all central places such as the central railway station, shopping malls, theaters and museums. From the commissioners’ perspective the main purpose of the case study is to identify the hotel services’ weaknesses and to develop improvement suggestions based on the research results. In order to obtain reliable results related to the drawbacks of the hotel service, it is essential to identify similarities and differences between hotel management systems. Fur-thermore, customer satisfaction research provides most objective and reliable information about the current situation in the hotels as well as customers’ opinions and attitudes toward the organizations. Working on the project started with gathering information and building a theory base. In order to obtain additional information about the hotels, the managers were contacted and appointments were arranged for carrying out an interview. The theoretical framework of the thesis consists of theory about services and service design, consumer behavior and customers’ buying preferences, customer service and its interaction with customer satisfaction, as well as a definition of customer loyalty. Furthermore, the theoretical section provides information about service design, because the concept of service design was applied throughout the thesis working process. During the first stage of SID, Understanding, was applied in the format of a stakeholder analysis, blueprint and interview methods. The second stage, Creating, included benchmarking, 5 Why and personas methods. One of these methods, personas, was used for completing customer journey maps, implemented during the third stage. In the final stage of service design, Delivering, a customer journey map was drawn up, as well as mystery visitor and customer satisfaction research were conducted. The customer satisfaction research was carried out based on the customers’ reviews in the web portals booking.com and tripadvisor.com. Customers’ accommodation experiences are very important for presenting a real situation in the hotels. Analyzing the feedback enabled an understanding of the current situation in the hotels and identifying positive issues of the provided services. The quality of the hotels’ services is almost on the same level, the average score for Helka was 8.2 and for Helvetia 9. Customers are satisfied with the customer service and with the employees’ “Yes, I can” approach. Differencies between the hotels´ management systems were identified. The Finnish democratic style of management is not suitable in Russian hotels, due to a different mentality and culture. Based on the customers’ negative reviews, conclusions were listed, which determined the areas for improvement and improvement suggestions. Areas for improvement were related to customer service, hotel state, additional services, customer loyalty level and increasing customer satisfaction.Asiakaskokemuksen määritteleminen neljän tähden hotelleissa Suomessa ja Venäjällä Opinnäytetyön päätavoitteena oli määritellä asiakkaiden kokemukset neljän tähden hotelleissa Suomessa ja Venäjällä. Projektin asiakkaita olivat Suomen hotelli ”Helka” ja Venäjän hotelli ”Helvetia”. Molemmat yritykset kuuluvat neljän tähden hotelliluokkaan ja tarjoavat samanlaisen palveluvalikoiman. Hotellien sijainnit mahdollistavat helpon pääsyn kaikkiin tärkeisiin paikkoihin, kuten päärautatieasemalle, ostoksille, teattereihin ja museoihin. Asiakkaiden näkökulmasta tutkimuksen päätarkoituksena oli tunnistaa hotellien palveluiden heikkoudet ja kehittää niitä tukimustuloksien perusteella. Saadakseen luotettavia tuloksia, jotka liittyvät hotellin palveluiden epäonnistumiseen, oli tärkeää tunnistaa erot ja yhtäläisyydet hotellien hallintatavoissa ja järjestelmässä. Sen lisäksi asiakastyytyväisyys tutkimus tarjoaa eniten objektiivista ja luotettavaa tietoa hotellin nykytilasta, kuten myös asiakkaiden mielipiteistä ja asenteista hotellia kohtaan. Hankkeen työskentely alkoi tiedon keräyksestä ja teoriapohjan rakentamisesta. Saadakseen lisätietoja hotelleista opiskelijat ottivat yhteyttä hotellien edustajiin ja järjestivät tapaamiset haastatteluja varten. Opinnäytetyön teoria koostui palvelusta ja palvelun suunnittelusta, kuluttajakäyttäytymisestä sekä asiakkaiden ostomieltymyksistä, asiakaspalvelusta ja sen vuorovaikutuksesta asiakastyytyväisyyteen sekä asiakasuskollisuuden määrittelystä. Lisäksi yksi tärkeimmistä teoriaosuuksista liittyi palvelunmuotoiluun, koska palvelumuotoilu- käsitettä on sovellettu koko opinnäytetyöprosessiin aikana. Palvelumuotoilu koostuu kolmesta eri vaiheesta: ensimmäisen vaiheen aikana ”Ymmärtäminen” on sovellettu sidosryhmäanalyysi, blueprint ja haastattelu-menetelmiä. Palvelumuotoilun toisessa vaiheessa ”Luominen” on käytetty benchmarking, 5-Miksi ja persoonat menetelmiä. Yhtä näistä menetelmistä, persoonat, käytettiin asiakkaan matkakarttoja laatiessa, mikä toteutui kolmannessa vaiheessa. Palvelumuotoilun loppuvaiheessa ”Tulokset” on sovellettu asiakkaan matkakarttoja, mysteerikävijä tutkimusta sekä asiakastyytyväisyystutkimusta. Asiakastyytyväisyystutkimus perustuu asiakkaiden hotelliarviointeihih web-sivustoilla www.booking.com ja www.tripadvisor.com. Asiakkaiden mielipiteet ja kokemukset majoituksesta ovat erittäin tärkeitä, jotta voidaan kuvata hotellin todellinen tilanne. Palautteiden analyysin avulla opiskelijat ymmärsivät hotellin nykytilan ja tunnistivat positiiviset kohdat hotellien palveluissa. Hotellien palvelunlaatu on lähes samalla tasolla, keskiarvopisteillä Helka 8,2 ja Helvetia 9. Asiakkaat ovat tyytyväisiä hotellien asiakaspalveluun ja työntekijöiden ”Kyllä voin” lähestymistapoihin. Opiskelijat tunnistivat erot hotellien hallintajärjestelmissä; Suomen demokraattinen johtamistyyli ei sovi Venäjän hotelleihin, koska mailla on eri mentaliteetti ja kulttuuri. Asiakkaiden kielteisten arviointien perusteella opiskelijat tekivät johtopäätöksiä, määrittelivät kehittämisalueita ja kehittelivät parannusehdotuksia. Parannuskohteet liittyivät asiakaspalveluun, hotellirakennuksen kunnon parantamiseen, lisäpalveluihin, asiakasuskollisuuden sekä asiakastyytyväisyyden kasvuun