Training an Actor-Critic Reinforcement Learning Controller for Arm Movement Using Human-Generated Rewards

Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Functional Electrical Stimulation (FES) employs neuroprostheses to apply electrical current to the nerves and muscles of individuals paralyzed by spinal cord injury (SCI) to restore voluntary movement. Neuroprosthesis controllers calculate stimulation patterns to produce desired actions. To date, no existing controller is able to efficiently adapt its control strategy to the wide range of possible physiological arm characteristics, reaching movements, and user preferences that vary over time. Reinforcement learning (RL) is a control strategy that can incorporate human reward signals as inputs to allow human users to shape controller behavior. In this study, ten neurologically intact human participants assigned subjective numerical rewards to train RL controllers, evaluating animations of goal-oriented reaching tasks performed using a planar musculoskeletal human arm simulation. The RL controller learning achieved using human trainers was compared with learning accomplished using human-like rewards generated by an algorithm; metrics included success at reaching the specified target; time required to reach the target; and target overshoot. Both sets of controllers learned efficiently and with minimal differences, significantly outperforming standard controllers. Reward positivity and consistency were found to be unrelated to learning success. These results suggest that human rewards can be used effectively to train RL-based FES controllers.NIH #TRN030167Veterans Administration Rehabilitation Research & Development predoctoral fellowshipArdiem Medical Arm Control Device grant #W81XWH072004

Social Learning and Adoption of New Behavior in a Virtual Agent Society

Social learning and adoption of new behavior govern the rise of a variety of behaviors: from actions as mundane as dance steps to those as dangerous as new ways to make IED detonators. However, agents in immersive virtual environments lack the ability to realistically simulate the spread of new behavior. To address this gap, a cognitive model was designed that represents the well-known socio-cognitive factors of attention, social influence, and motivation that influence learning and the adoption of a new behavior. To explore the effectiveness of this model, simulations modeled the spread of two competing memes in Hamariyah, an archetypal Iraqi village developed for cross-cultural training. Diffusion and clustering analyses were used to examine adoption patterns in these simulations. Agents produced well-defined clusters of early versus late adoption based on their social influences, personality, and contextual factors, such as employment status. These findings indicate that the spread of behavior can be simulated plausibly in a virtual agent society and has the potential to increase the realism of immersive virtual environments

Development of Craniofacial Bone Defect Reconstruction Implant Based on Fibre-Reinforced Composite with Photopolymerisable Resin Systems. Experimental studies in vitro and in vivo.

Reconstruction of defects in the craniomaxillofacial (CMF) area has mainly been based on bone grafts or metallic fixing plates and screws. Particularly in the case of large calvarial and/or craniofacial defects caused by trauma, tumours or congenital malformations, there is a need for reliable reconstruction biomaterials, because bone grafts or metallic fixing systems do not completely fulfill the criteria for the best possible reconstruction methods in these complicated cases. In this series of studies, the usability of fibre-reinforced composite (FRC) was studied as a biostable, nonmetallic alternative material for reconstructing artificially created bone defects in frontal and calvarial areas of rabbits. The experimental part of this work describes the different stages of the product development process from the first in vitro tests with resin-impregnated fibrereinforced composites to the in vivo animal studies, in which this FRC was tested as an implant material for reconstructing different size bone defects in rabbit frontal and calvarial areas. In the first in vitro study, the FRC was polymerised in contact with bone or blood in the laboratory. The polymerised FRC samples were then incubated in water, which was analysed for residual monomer content by using high performance liquid chromatography (HPLC). It was found that this in vitro polymerisation in contact with bone and blood did not markedly increase the residual monomer leaching from the FRC. In the second in vitro study, different adhesive systems were tested in fixing the implant to bone surface. This was done to find an alternative implant fixing system to screws and pins. On the basis of this study, it was found that the surface of the calvarial bone needed both mechanical and chemical treatments before the resinimpregnated FRC could be properly fixed onto it. In three animal studies performed with rabbit frontal bone defects and critical size calvarial bone defect models, biological responses to the FRC implants were evaluated. On the basis of theseevaluations, it can be concluded that the FRC, based on E-glass (electrical glass) fibres forming a porous fibre veil enables the ingrowth of connective tissues to the inner structures of the material, as well as the bone formation and mineralization inside the fibre veil. Bone formation could be enhanced by using bioactive glass granules fixed to the FRC implants. FRC-implanted bone defects healed partly; no total healing of defects was achieved. Biological responses during the follow-up time, at a maximum of 12 weeks, to resin-impregnated composite implant seemed to depend on the polymerization time of the resin matrix of the FRC. Both of the studied resin systems used in the FRC were photopolymerised and the heat-induced postpolymerisation was used additionally.Valokovetteiseen kuitukomposiittiin perustuvan implantin kehittäminen kallon ja kasvojen alueen luupuutosten korjaamiseen. Kokeellisia in vitro- ja in vivo –tutkimuksia Kallon ja kasvojen alueen luupuutosten korjaus on perustunut tähän mennessä pääasiassa luusiirteiden sekä metallisten fiksaatiolevyjen ja ruuvien käyttöön. Erityisesti syöpäkasvainten, synnynnäisten epämuodostumien tai tapaturmien aiheuttamien alaisten luupuutosten korjaamisessa kallon ja/tai kasvojen alueella tarvitaan luotettavia biomateriaaleja, koska luusiirteillä tai metalleihin perustuvilla menetelmillä ei pystytä vauriokohtaa parhaalla mahdollisella tavalla korjaamaan. Tässä väitöskirjassa tutkittiin kuituvahvisteisen komposiittimateriaalin (FRC) käyttökelpoisuutta biostabiilina, metalleja korvaavana vaihtoehtoisena materiaalina luupuutosten korjaamisessa käyttäen mallina kanin otsan ja kallon luihin tehtyjä defektejä. Väitöskirjan kokeellisessa osassa kuvataan implantin tuotekehityksen vaiheita alkaen ensimmäisistä in vitro -testeistä resiini-impregnoiduilla kuitukomposiiteilla, ja päätyen in vivo -eläinkokeisiin, joissa kuituvahvisteista komposittia testattiin implanttimateriaalina erikokoisten luupuutosten korjauksessa kanin otsan ja kallon alueella. Ensimmäisessä in vitro-tutkimuksessa FRC-materiaali valokovetettiin laboratorioolosuhteissa luu- ja verikontaktissa, ja kovetetut näytekappaleet inkuboitiin vedessä, jonka jäännösmonomeeripitoisuudet analysoitiin nestekromatografia (HPLC)- menetelmällä. Tutkimuksen mukaan luu- ja verikontaktissa polymerointi ei lisännyt merkittävästi jäännösmonomeerien vapautumista FRC-materiaalista. Toisessa in vitro tutkimuksessa testattiin erilaisia adheesiomenetelmiä implantin luuhun kiinnittämisessä. Tämän tutkimuksen tarkoituksena oli löytää vaihtoehtoja implantin kiinnittämiselle luuhun ilman ruuveja ja nastoja. Tutkimuksessa havaittiin, että kuitukomposiitin riittävä kiinnitys kallon luun pintaan adhesiiveja käyttäen vaati luun pinnan mekaanista ja kemiallista esikäsittelyä. Kolmessa kanin otsaluu- tai kriittisen koon kallodefektimalliin perustuvassa eläinkokeessa arvioitiin kuitukomposiittiimplantin aiheuttamia biologisia reaktioita. Näiden tutkimusten perusteella voitiin havaita, että sidekudokset kasvavat E-lasikuituun (E=electrical, sähkölasikuitu) perustuvan kuituhuopamateriaalin sisään. Myös uudisluun muodostuminen ja sen mineralisaatio on mahdollista huovan sisällä. Bioaktiiviset lasirakeet edesauttoivat luunmuodostusta kuitukomposiitin yhteydessä käytettynä. Täydellistä luudefektien paranemista ei eläinkokeissa saavutettu, vaan defektit paranivat vain osittain. Biologinen vaste resiini-impregnoidulle komposiitti-implantille maksimissaan 12 viikon implantoinnin jälkeen oli riippuvainen myös muovimatriisin polymerisaatioajasta. Molemmat kokeissa käytetyt resiiniseokset jälkilämpöpolymeroitiin valokovetuksen lisäksi.Siirretty Doriast

Agriculture in responsibility for our common world

The vocational training course program Agriculture in Responsibility for our common World organised within the frame of the Banat Green Deal Project GreenERDE (Education and Research in the context of the digital and ecological transformation of agriculture in the Banat Region and Baden-Württemberg - towards resource efficiency and resilience) and delivered between June 2021 and May 2022 targets the knowledge and experience transfer to the farmer community in the Banat Region, Romania and other parts of the world. Current and future challenges, such as the ecological conversion and digital transformation of agricultural production, but also social, economic and cultural aspects haven been addressed transcending prevailing patterns. The innovative and relevant knowledge originating from practice, experiments, research or development projects throughout Europe and other continents is presented in a training format for interested participants.Das im Rahmen des Banat Green Deal Projekts GreenERDE (Bildung und Forschung im Kontext der digitalen und ökologischen Transformation des Agrarbereichs im Banat und Baden-Württemberg - auf dem Weg zu Ressourceneffizienz und Resilienz) von Juni 2021 bis Mai 2022 durchgeführte Fortbildungsprogramm Landwirtschaft in Verantwortung für unsere gemeinsame Welt), zielt auf den Wissens- und Erfahrungstransfer unter Landwirten und anderen interessirten Personen in der Banat-Region, Rumänien und anderen Teilen der Welt ab. Aktuelle und zukünftige Herausforderungen, wie die ökologische Umstellung und die digitale Transformation der landwirtschaftlichen Produktion, aber auch soziale, wirtschaftliche und kulturelle Aspekte wurden jenseits vorherrschender Muster adressiert. Das innovative und relevante Wissen aus Praxis, Forschungs- oder Entwicklungsprojekten in ganz Europa und anderen Kontinenten wird in einem Trainingsformat für interessierte Teilnehmer präsentiert

New Approaches to Urban Planning - Insights from Participatory Communities

The new approaches to urban planning, such as participatory time and e-planning, comprise methods that allow us to analyse, develop, implement and monitor physical, functional and participatory structures at the neighbourhood level and beyond. They enable models of planning that may bring about an architecture of opportunities. This means the building of a supportive infrastructure of everyday life that encourages citizens to participate not only in formal decision-making, but actually in the co-design and co-production of their own local environment, on the basis of daily and future activities, at different scales

Liquid cinema and the re-creation of thought: towards a philosophy of filmind

Includes bibliographical references.This research is towards the advancement of filmosophy as a progressive new approach to how we think about, and through, film. This explorative research aims to introduce, contextualise, and expand upon the thoughts and writings of Daniel Frampton, as found in his 2006 manifesto: Filmosophy. In order to provide a suitable platform from which to introduce Frampton’s contemporary concepts (i.e. ‘filmind’ and ‘fluid film-thinking’), this paper first outlines and discusses the various ways in which philosophy and film are said to overlap, culminating in a critical discussion of ‘film-as-philosophy’ in terms of the implications it posits for providing innovative philosophical contributions through uniquely cinematic means (the ‘problem of paraphrase’). This literature review concludes by presenting and discussing filmosophy and its major tenets as both an appropriate extension of the current canon, and as a potentially productive new paradigm through which both film and philosophy can be critically considered and advanced