UnifiedGesture: A Unified Gesture Synthesis Model for Multiple Skeletons

The automatic co-speech gesture generation draws much attention in computer animation. Previous works designed network structures on individual datasets, which resulted in a lack of data volume and generalizability across different motion capture standards. In addition, it is a challenging task due to the weak correlation between speech and gestures. To address these problems, we present UnifiedGesture, a novel diffusion model-based speech-driven gesture synthesis approach, trained on multiple gesture datasets with different skeletons. Specifically, we first present a retargeting network to learn latent homeomorphic graphs for different motion capture standards, unifying the representations of various gestures while extending the dataset. We then capture the correlation between speech and gestures based on a diffusion model architecture using cross-local attention and self-attention to generate better speech-matched and realistic gestures. To further align speech and gesture and increase diversity, we incorporate reinforcement learning on the discrete gesture units with a learned reward function. Extensive experiments show that UnifiedGesture outperforms recent approaches on speech-driven gesture generation in terms of CCA, FGD, and human-likeness. All code, pre-trained models, databases, and demos are available to the public at https://github.com/YoungSeng/UnifiedGesture.Comment: 16 pages, 11 figures, ACM MM 202

Automatic and adaptable registration of live RGBD video streams sharing partially overlapping views

In this thesis, we introduce DeReEs-4v, an algorithm for unsupervised and automatic registration of two video frames captured depth-sensing cameras. DeReEs-4V receives two RGBD video streams from two depth-sensing cameras arbitrary located in an indoor space that share a minimum amount of 25% overlap between their captured scenes. The motivation of this research is to employ multiple depth-sensing cameras to enlarge the field of view and acquire a more complete and accurate 3D information of the environment. A typical way to combine multiple views from different cameras is through manual calibration. However, this process is time-consuming and may require some technical knowledge. Moreover, calibration has to be repeated when the location or position of the cameras change. In this research, we demonstrate how DeReEs-4V registration can be used to find the transformation of the view of one camera with respect to the other at interactive rates. Our algorithm automatically finds the 3D transformation to match the views from two cameras, requires no human interference, and is robust to camera movements while capturing. To validate this approach, a thorough examination of the system performance under different scenarios is presented. The system presented here supports any application that might benefit from the wider field-of-view provided by the combined scene from both cameras, including applications in 3D telepresence, gaming, people tracking, videoconferencing and computer vision

All-Women Initiatives in Art and Technology 1986-2020. Atenea: Mentoring and Networking Project in Steam

[ES] La presente tesis doctoral es una exploración sobre las iniciativas auto-organizadas de mujeres que han surgido en la intersección del arte y la tecnología a nivel internacional. Estas iniciativas comunitarias han ido apareciendo en lugares donde existe una escena de arte y nuevos medios, la cual es generalmente de dominación masculina y baja en diversidad. Este trabajo de investigación es un mapeo de este fenómeno tecno-social y artístico, tratando de identificar las primeras prácticas. A través de entrevistas, encuestas y métodos de investigación etnográfica, esta tesis aspira a comprender el contexto que ha dado lugar a semejantes iniciativas a lo largo de tres décadas y media, examinando las diferentes tipologías por los formatos, extensión, espacio y estructura que éstas han adoptado para afrontar la situación de las mujeres en estas áreas. Existe un vació de referencias sobre formas organizativas y de comisariado en las áreas de arte y tecnología. Además, la literatura general señala las subculturas digitales de los noventa, tales como el ciberfeminismo y el movimiento DIY, como los primeros y principales referentes para las alianzas de mujeres y el activismo translocal en la era digital. Sin embargo, las estrategias de colaboración y segregación en áreas de arte y tecnología no han sido analizadas desde los estudios feministas desde una perspectiva histórica en profundidad. Por lo tanto, esta tesis establece la relación entre dichas iniciativas con la historia de los grupos de mujeres artistas y espacios de arte alternativos, la historia de las mujeres en el arte de nuevos medios y la cultura de redes. Las nuevas tecnologías de la información y comunicación han permitido nuevas formas de comunicación y conectividad transfronterizas; sin embargo, este trabajo investiga en qué medida estas iniciativas dependen de las TIC para el establecimiento de su comunidad, comparado con los formatos analógicos. Con los resultados, esta tesis aspira a convertirse en una fuente de referencias que enriquezca la escasa bibliografía de la temática, proporcionando una lista de formas organizativas y de comisariado, así como las practicas más tempranas nunca publicadas, y en conjunto contribuir a la historiografía actual sobre mujeres. Por último, a la vez que se comenta la situación actual de las mujeres en las áreas de arte y tecnología, este trabajo presenta Atenea, un proyecto de mentorización y networking para mujeres en las artes y carreras STEM. Atenea está compuesto de un programa de actividades con diferentes actividades organizativas y de comisariado, así como una plataforma de networking. Adicionalmente, el proyecto tiene un programa enfocado en transmitir conocimientos STEM a niñas a través de prácticas artísticas con tecnología de la mano de mujeres artistas nacionales como mentoras.[CAT] La present tesi doctoral és una exploració sobre les iniciatives autoorganitzades de dones que han sorgit en la intersecció de l'art i la tecnologia a nivell internacional. Aquestes iniciatives comunitàries han anat apareixent en llocs on hi ha una escena d'art i nous mitjans, la qual és generalment de dominació masculina i baixa en diversitat. Aquest treball de recerca és un mapeig d'aquest fenomen tecno-social i artístic, tractant d'identificar les primeres pràctiques. A través d'entrevistes, enquestes i mètodes d'investigació etnogràfica, aquesta tesi aspira a comprendre el context que ha donat lloc a semblants iniciatives al llarg de tres dècades i mig, examinant les diferents tipologies pels formats, extensió, espai i estructura que aquestes han adoptat per afrontar la situació de les dones en aquestes àrees. Hi ha un buit de referències sobre formes organitzatives i de comissariat en les àrees d'art i tecnologia. A més, la literatura general assenyala les subcultures digitals dels noranta, com ara el ciberfeminisme i el moviment DIY, com els primers i principals referents per a les aliances de dones i l'activisme translocal en l'era digital. No obstant això, les estratègies de col·laboració i segregació en àrees d'art i tecnologia no han estat analitzades des dels estudis feministes des d'una perspectiva històrica en profunditat. Per tant, aquesta tesi estableix la relació entre aquestes iniciatives amb la història dels grups de dones artistes i espais d'art alternatius, la història de les dones en l'art de nous mitjans i la cultura de xarxes. Les noves tecnologies de la informació i comunicació han permès noves formes de comunicació i connectivitat transfrontereres; però, aquest treball investiga en quina mesura aquestes iniciatives depenen de les TIC per a l'establiment de la seva comunitat, comparat amb els formats analògics. Amb els resultats, aquesta tesi aspira a convertir-se en una font de referències que enriqueixi l'escassa bibliografia de la temàtica, proporcionant una llista de formes organitzatives i de comissariat, així com les pràctiques més primerenques mai publicades, i en conjunt contribuir a la historiografia actual sobre dones. Finalment, alhora que es comenta la situació actual de les dones en les àrees d'art i tecnologia, aquest treball presenta Atenea, un projecte de mentorització i networking per a dones en les arts i carreres STEM. Atenea està compost d'un programmea d'activitats amb diferents activitats organitzatives i de comissariat, així com una plataforma de networking. Addicionalment, el projecte té un programmea enfocat a transmetre coneixements STEM a nenes a través de pràctiques artístiques amb tecnologia de la mà de dones artistes nacionals com mentores.[EN] This dissertation explores a series of all-women self-organised initiatives that have emerged at the intersection of art and technology internationally. These grassroot initiatives have continued to appear in places where there is a new media arts scene, generally male-dominated and low in diversity. This research work is a mapping of this techno-social and artistic phenomenon, attempting to identify its earliest practices. Through interviews, surveys and ethnographic research, this thesis seeks to comprehend the context that has given rise to such initiatives over three decades and half, examining the different typologies by formats, extension, space and structure that they have adopted to address the situation of women in these fields. There is a void of references on all-women curatorial and organisational forms in art and technology. Moreover, general literature points at digital subcultures of the 1990s such as cyberfeminism and DIY movement as first referents for women's alliance and translocal activism in the digital age. However, collaboration and segregation strategies in new media arts have not been discussed in feminist scholarship from a deeper historical perspective. Therefore, this dissertation elaborates the linkage of such initiatives to early history of all-women art groups and alternative art spaces, history of women in new media arts and to network cultures. New information and communication technologies have enabled new forms of communication and connectivity cross-borders; however, this work investigates in which measure these initiatives are dependent upon ICT for the establishment of a community, compared to analog formats. With the results, this dissertation aims to become a source of references that enriches the scarce bibliography on the topic, providing a list of curatorial and organisational forms with the earliest and unpublished practices that altogether contribute to current women's historiography. Lastly, while discussing the current situation of women in art and technology fields, this works presents Atenea, a mentoring and networking platform for women in Arts and STEM careers. Atenea is comprised of a programme having different organizational, curatorial and networking activities. Additionally, the platform has a project focused on teaching STEM skills to girls through new media arts with female artists as mentors. Keywords: All-women, self-organised, initiatives, new media arts, digital arts, electronic arts, art, technology, activism, mapping, alternative art spaces, collaboration, segregation, ICT, digital age, virtual, community, network cultures, feminism, cyberfeminism, Atenea, STEM, STEAM, girls, organisational, curatorial, networking platform, mentoring,Robles Mateo, E. (2020). All-Women Initiatives in Art and Technology 1986-2020. Atenea: Mentoring and Networking Project in Steam [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/159248TESI

Approches environnement-centrées pour la simulation de systèmes multi-agents: Pour un déplacement de la complexité des agents vers l'environnement

This habilitation thesis synthesizes research works which are mainly related to the field of Multi-Agent Based Simulation (MABS). MABS is a general framework for modeling and experimenting with systems in which the dynamics emerges from local interactions among individuals (autonomous agents). Examples of use range from the study of natural systems (e.g. ant colonies, crowds or traffic jams) to the engineering of artificial ones (e.g., collective robotics, distributed artificial intelligence-based softwares). To this end, MABS modeling represents the behavior of individuals, their environment and interactions, so that global dynamics can be computed and studied from the bottom up. In this context, we have been investigating research on the theory and practice of MABS from two different perspectives : (1) the design of generic abstractions dedicated to the modeling of multi-agent dynamics (e.g., the IRM4S model) and (2) the engineering of MABS (MaDKit and TurtleKit platforms). Besides, we have been experimenting with MABS in different application domains such as image processing, video games, and collective robotics. Contrary to approaches that put the emphasis on the agent behaviors, all these works have been done by considering the environment of the agents as a first order abstraction. In this thesis, we first reflect upon the research we have conducted according to this perspective. Next, we show how we actually use this perspective to propose an original approach for using General-Purpose processing on Graphics Processing Units (GPGPU) within MABS, and then present the research perspectives related to our positioning.Les travaux de recherche synthétisés dans ce mémoire s’inscrivent principalement dans le domaine de la modélisation et de la simulation de systèmes multi-agents (SMA). La simulation multi-agents met en œuvre des modèles où les individus, leur environnement et leurs interactions sont directement représentés. Dans ces modèles, chaque individu –agent autonome– possède son propre comportement et produit ses actions en fonction d’une perception locale de son environnement. Ainsi, la simulation multi-agents est utilisée pour étudier des systèmes naturels comme les colonies de fourmis, les dynamiques de foules ou le trafic urbain, mais aussi pour concevoir des systèmes artificiels, par exemple dans le cadre de la robotique collective ou le développement de logiciels basés sur de l’intelligence artificielle distribuée. Dans ce cadre, nos recherches ont porté sur des problématiques liées à la modélisation de simulations multi-agents, avec la proposition de modèles formels et conceptuels (e.g. le modèle IRM4S) et d’outils logiciels génériques (plates-formes MaDKit et TurtleKit), et sur leur utilisation dans divers domaines tels que le jeu vidéo, le traitement numérique de l’image ou la robotique collective. Contrairement aux approches centrées sur la conception des comportements individuels, dans ces travaux l’environnement des agents est considéré comme une abstraction de premier ordre. Dans ce mémoire, nous dressons tout d’abord un bilan de nos recherches en argumentant l’intérêt d’une telle démarche pour les modèles multi-agents. Nous montrons ensuite comment celle-ci nous a récemment permis de proposer une approche originale dans le cadre de l’utilisation du calcul haute performance sur carte graphique (GPGPU) pour la simulation de SMA, avant de présenter les perspectives de recherche associées à notre positionnement

Wearables at work:preferences from an employee’s perspective

This exploratory study aims to obtain a first impression of the wishes and needs of employees on the use of wearables at work for health promotion. 76 employ-ees with a mean age of 40 years old (SD ±11.7) filled in a survey after trying out a wearable. Most employees see the potential of using wearable devices for workplace health promotion. However, according to employees, some negative aspects should be overcome before wearables can effectively contribute to health promotion. The most mentioned negative aspects were poor visualization and un-pleasantness of wearing. Specifically for the workplace, employees were con-cerned about the privacy of data collection

Agora : unified framework for crowd simulation research

Crowd simulation focuses on modeling the movements and behaviors of large groups of people. This area of study has become increasingly important because of its several applications in various fields such as urban planning, safety, and entertainment. In each of these domains, the presence of virtual agents exhibiting realistic behavior greatly enhances the quality of the simulations. However, the inherently multifaceted and intricate nature of human behavior presents a unique challenge, necessitating the effective combination of multiple behavior models. This thesis introduces a novel theoretical framework for modeling human behavior in crowd simulations, addressing the unresolved issue of combining a plethora of behavior models, often developed in isolation. The proposed framework decomposes human behavior into fundamental driving stimuli, which are then represented graphically through the heatmap paradigm. Subsequently, the agent behavior is influenced by the heatmaps, which guide them toward attractive areas and steer them away from repulsive locations based on the encoded stimuli. A key advantage of this approach lies in the ability to combine heatmaps using well-defined color operations, effectively integrating different aspects of human behavior. Furthermore, the heatmap paradigm facilitates objective comparison of simulation output with real-world data, employing image similarity metrics to evaluate model accuracy. To realize this framework, the thesis presents a modular software architecture designed to support various tasks involved in crowd simulation, emphasizing the separation of concerns for each task. This architecture comprises a collection of abstract modules, which are subsequently implemented using appropriate software components to realize the underlying features, resulting in the Agora framework. To assess the ability of Agora to support the various tasks involved in crowd simulation, two case studies are implemented and analyzed. The first case study simulates tourists visiting Þingvellir national park in Iceland, examining how their behavior is influenced by the visibility of the surrounding environment. The second case study employs Agora to model the thermal and density comfort levels of virtual pedestrians in an urban setting. The results demonstrate that Agora successfully supports the development, combination, and evaluation of crowd simulation models against real-world data. The authoring process, assisted by Agora, is significantly more streamlined compared to its native counterpart. The integration of multiple models is achieved by combining the heatmaps, resulting in plausible behavior, and the model assessment is made convenient through the evaluator within the framework. The thesis concludes by discussing the implications of these findings for the field of crowd simulation, highlighting the contributions and potential future directions of the Agora framework.Mannfjöldahermun fæst við gerð líkana af hreyfingu og hegðun stórra hópa af fólki. Mikilvægi þessa rannsóknasviðs hefur vaxið stöðugt vegna hagnýtingar á margvíslegum vetvangi, eins og til dæmis á vetvangi borgarskipulags, öryggis og afþreyingar. Þegar sýndarmenni hegða sér á sannfærandi hátt, leiðir það til betri hermunar fyrir þessi notkunarsvið. En mannleg hegðun er í eðli sínu margbrotin og flókin og því er það sérstök áskorun við smíði sýndarmenna að sameina, með áhrifaríkum hætti, mörg mismunandi hegðunarlíkön. Þessi ritgerð kynnir nýja fræðilega umgjörð líkanasmíði mannlegrar hegðunar fyrir mannfjöldahermun, sem tekur á þeim óleysta vanda að sameina fjölda hegðunarlíkana, sem oft eru þróuð með aðskildum hætti. Umgjörðin brýtur mannlega hegðun niður í grundvallar drifáreiti, sem eru sett fram grafískt útfrá hugmyndafræði hitakorta. Sýndarmennin hegða sér síðan undir áhrifum frá hitakortunum, sem vísa þeim í áttina að aðlaðandi svæðum og stýra þeim burt frá fráhrindandi svæðum, útfrá hinu umritaða áreiti. Lykilkostur þessarar nálgunar er sá eiginleiki að geta blandað saman hitakortum með vel skilgreindum litaaðgerðum, sem eru þá í raun samþætting mismunandi hliða mannlegrar hegðunar. Hitakortshugmyndafræðin auðveldar ennfremur hlutlægan samanburð hermunarúttaks og raungagna með notkun myndsamanburðarmælinga, til að meta nákvæmni líkana. Varðandi útfærslu, þá kynnir þessi ritgerð einingadrifna hugbúnaðarhögun sem er hönnuð til að styðja við ýmsa ferla mannfjöldahermunar, með áherslu á aðskilnað helstu viðfangsefna hvers ferlis. Þessi högun inniheldur safn huglægra eininga, sem síðan eru útfærðar með viðeigandi hugbúnaðarhlutum, sem raungera undirliggjandi eiginleika. Útkoman er sjálf Agora umbjörðin. Tvö sýnidæmi eru útfærð og greind til að meta getu Agoru til að styðja við ýmis mannfjöldahermunarverkefni. Fyrra dæmið hermir eftir ferðamönnum sem heimsækja Þingvallaþjóðgarð, og skoðar hvernig hegðun þeirra verður fyrir áhrifum sýnileika umhverfisins sem umleikur þá. Seinna dæmið nýtir Agoru til að smíða líkan af hitauppstreymis- og þéttleikaþægindum hjá sýndarvegfarendum í borgarumhverfi. Niðurstöðurnar sýna góðan árangur Agoru við að styðja þróun, samþættingu og mat mannfjöldahermunarlíkana gagnvart raungögnum. Þróunarferlið er verulega þjálla með Agoru en með hefðbundnum aðferðum. Samþætting margra líkana tókst með blöndun hitakorta, möguleg hegðun var framkölluð og mat á líkönunum varð þægilegra með umgjörðinni. Ritgerðinni lýkur með því að fjalla um áhrif þessara niðurstaðna á svið mannfjöldahegðunar, með áherslu á nýstálegt framlag þessarar rannsóknar og mögulega framtíðarþróun Agora umgjarðarinnar