CP-EB: Talking Face Generation with Controllable Pose and Eye Blinking Embedding

This paper proposes a talking face generation method named "CP-EB" that takes an audio signal as input and a person image as reference, to synthesize a photo-realistic people talking video with head poses controlled by a short video clip and proper eye blinking embedding. It's noted that not only the head pose but also eye blinking are both important aspects for deep fake detection. The implicit control of poses by video has already achieved by the state-of-art work. According to recent research, eye blinking has weak correlation with input audio which means eye blinks extraction from audio and generation are possible. Hence, we propose a GAN-based architecture to extract eye blink feature from input audio and reference video respectively and employ contrastive training between them, then embed it into the concatenated features of identity and poses to generate talking face images. Experimental results show that the proposed method can generate photo-realistic talking face with synchronous lips motions, natural head poses and blinking eyes.Comment: Accepted by the 21st IEEE International Symposium on Parallel and Distributed Processing with Applications (IEEE ISPA 2023

Simulation FX: Cinema and the R&D Complex

This study looks at the ongoing development of tools and practices used to animate nonlinear physical phenomena, such as the crash of ocean waves or the movement of human hair, in the visual effect and animation industries. These tools and practices are developed in a nexus between public funding, research universities, the film industry, and various other sectors, such as aerospace and meteorology. This study investigates how technological development became integrated with film production, and in turn how epistemic paradigms were shared between the film industry, scientific research institutions and other industries. At the heart of these animation tools and practices, and the networks of institutions that developed them, is a way of thinking that seeks to make use of unpredictable nonlinear complexity by shaping it toward specific applications. I observe this in the way animation and visual effect studios seek the realistic appearance of nonlinear natural movement through simulation, while also implementing technologies and practices to direct the look of these simulations. I also observe this in a variety of related examples, from the way the concept of research and development unites science and application, to the way management science promotes hands off approaches that preserve the unpredictable nature of creative work. My methods consist of charting the circulation of ideas, technologies, moving images and people through contact zones such as the computer science special interest group ACM SIGGRAPH, using archival research of trade communications, scholarly publications and conference proceedings, as well as interviews with industry workers

All-Women Initiatives in Art and Technology 1986-2020. Atenea: Mentoring and Networking Project in Steam

[ES] La presente tesis doctoral es una exploración sobre las iniciativas auto-organizadas de mujeres que han surgido en la intersección del arte y la tecnología a nivel internacional. Estas iniciativas comunitarias han ido apareciendo en lugares donde existe una escena de arte y nuevos medios, la cual es generalmente de dominación masculina y baja en diversidad. Este trabajo de investigación es un mapeo de este fenómeno tecno-social y artístico, tratando de identificar las primeras prácticas. A través de entrevistas, encuestas y métodos de investigación etnográfica, esta tesis aspira a comprender el contexto que ha dado lugar a semejantes iniciativas a lo largo de tres décadas y media, examinando las diferentes tipologías por los formatos, extensión, espacio y estructura que éstas han adoptado para afrontar la situación de las mujeres en estas áreas. Existe un vació de referencias sobre formas organizativas y de comisariado en las áreas de arte y tecnología. Además, la literatura general señala las subculturas digitales de los noventa, tales como el ciberfeminismo y el movimiento DIY, como los primeros y principales referentes para las alianzas de mujeres y el activismo translocal en la era digital. Sin embargo, las estrategias de colaboración y segregación en áreas de arte y tecnología no han sido analizadas desde los estudios feministas desde una perspectiva histórica en profundidad. Por lo tanto, esta tesis establece la relación entre dichas iniciativas con la historia de los grupos de mujeres artistas y espacios de arte alternativos, la historia de las mujeres en el arte de nuevos medios y la cultura de redes. Las nuevas tecnologías de la información y comunicación han permitido nuevas formas de comunicación y conectividad transfronterizas; sin embargo, este trabajo investiga en qué medida estas iniciativas dependen de las TIC para el establecimiento de su comunidad, comparado con los formatos analógicos. Con los resultados, esta tesis aspira a convertirse en una fuente de referencias que enriquezca la escasa bibliografía de la temática, proporcionando una lista de formas organizativas y de comisariado, así como las practicas más tempranas nunca publicadas, y en conjunto contribuir a la historiografía actual sobre mujeres. Por último, a la vez que se comenta la situación actual de las mujeres en las áreas de arte y tecnología, este trabajo presenta Atenea, un proyecto de mentorización y networking para mujeres en las artes y carreras STEM. Atenea está compuesto de un programa de actividades con diferentes actividades organizativas y de comisariado, así como una plataforma de networking. Adicionalmente, el proyecto tiene un programa enfocado en transmitir conocimientos STEM a niñas a través de prácticas artísticas con tecnología de la mano de mujeres artistas nacionales como mentoras.[CAT] La present tesi doctoral és una exploració sobre les iniciatives autoorganitzades de dones que han sorgit en la intersecció de l'art i la tecnologia a nivell internacional. Aquestes iniciatives comunitàries han anat apareixent en llocs on hi ha una escena d'art i nous mitjans, la qual és generalment de dominació masculina i baixa en diversitat. Aquest treball de recerca és un mapeig d'aquest fenomen tecno-social i artístic, tractant d'identificar les primeres pràctiques. A través d'entrevistes, enquestes i mètodes d'investigació etnogràfica, aquesta tesi aspira a comprendre el context que ha donat lloc a semblants iniciatives al llarg de tres dècades i mig, examinant les diferents tipologies pels formats, extensió, espai i estructura que aquestes han adoptat per afrontar la situació de les dones en aquestes àrees. Hi ha un buit de referències sobre formes organitzatives i de comissariat en les àrees d'art i tecnologia. A més, la literatura general assenyala les subcultures digitals dels noranta, com ara el ciberfeminisme i el moviment DIY, com els primers i principals referents per a les aliances de dones i l'activisme translocal en l'era digital. No obstant això, les estratègies de col·laboració i segregació en àrees d'art i tecnologia no han estat analitzades des dels estudis feministes des d'una perspectiva històrica en profunditat. Per tant, aquesta tesi estableix la relació entre aquestes iniciatives amb la història dels grups de dones artistes i espais d'art alternatius, la història de les dones en l'art de nous mitjans i la cultura de xarxes. Les noves tecnologies de la informació i comunicació han permès noves formes de comunicació i connectivitat transfrontereres; però, aquest treball investiga en quina mesura aquestes iniciatives depenen de les TIC per a l'establiment de la seva comunitat, comparat amb els formats analògics. Amb els resultats, aquesta tesi aspira a convertir-se en una font de referències que enriqueixi l'escassa bibliografia de la temàtica, proporcionant una llista de formes organitzatives i de comissariat, així com les pràctiques més primerenques mai publicades, i en conjunt contribuir a la historiografia actual sobre dones. Finalment, alhora que es comenta la situació actual de les dones en les àrees d'art i tecnologia, aquest treball presenta Atenea, un projecte de mentorització i networking per a dones en les arts i carreres STEM. Atenea està compost d'un programmea d'activitats amb diferents activitats organitzatives i de comissariat, així com una plataforma de networking. Addicionalment, el projecte té un programmea enfocat a transmetre coneixements STEM a nenes a través de pràctiques artístiques amb tecnologia de la mà de dones artistes nacionals com mentores.[EN] This dissertation explores a series of all-women self-organised initiatives that have emerged at the intersection of art and technology internationally. These grassroot initiatives have continued to appear in places where there is a new media arts scene, generally male-dominated and low in diversity. This research work is a mapping of this techno-social and artistic phenomenon, attempting to identify its earliest practices. Through interviews, surveys and ethnographic research, this thesis seeks to comprehend the context that has given rise to such initiatives over three decades and half, examining the different typologies by formats, extension, space and structure that they have adopted to address the situation of women in these fields. There is a void of references on all-women curatorial and organisational forms in art and technology. Moreover, general literature points at digital subcultures of the 1990s such as cyberfeminism and DIY movement as first referents for women's alliance and translocal activism in the digital age. However, collaboration and segregation strategies in new media arts have not been discussed in feminist scholarship from a deeper historical perspective. Therefore, this dissertation elaborates the linkage of such initiatives to early history of all-women art groups and alternative art spaces, history of women in new media arts and to network cultures. New information and communication technologies have enabled new forms of communication and connectivity cross-borders; however, this work investigates in which measure these initiatives are dependent upon ICT for the establishment of a community, compared to analog formats. With the results, this dissertation aims to become a source of references that enriches the scarce bibliography on the topic, providing a list of curatorial and organisational forms with the earliest and unpublished practices that altogether contribute to current women's historiography. Lastly, while discussing the current situation of women in art and technology fields, this works presents Atenea, a mentoring and networking platform for women in Arts and STEM careers. Atenea is comprised of a programme having different organizational, curatorial and networking activities. Additionally, the platform has a project focused on teaching STEM skills to girls through new media arts with female artists as mentors. Keywords: All-women, self-organised, initiatives, new media arts, digital arts, electronic arts, art, technology, activism, mapping, alternative art spaces, collaboration, segregation, ICT, digital age, virtual, community, network cultures, feminism, cyberfeminism, Atenea, STEM, STEAM, girls, organisational, curatorial, networking platform, mentoring,Robles Mateo, E. (2020). All-Women Initiatives in Art and Technology 1986-2020. Atenea: Mentoring and Networking Project in Steam [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/159248TESI

APRIL: Approximating Polygons as Raster Interval Lists

The spatial intersection join an important spatial query operation, due to its popularity and high complexity. The spatial join pipeline takes as input two collections of spatial objects (e.g., polygons). In the filter step, pairs of object MBRs that intersect are identified and passed to the refinement step for verification of the join predicate on the exact object geometries. The bottleneck of spatial join evaluation is in the refinement step. We introduce APRIL, a powerful intermediate step in the pipeline, which is based on raster interval approximations of object geometries. Our technique applies a sequence of interval joins on 'intervalized' object approximations to determine whether the objects intersect or not. Compared to previous work, APRIL approximations are simpler, occupy much less space, and achieve similar pruning effectiveness at a much higher speed. Besides intersection joins between polygons, APRIL can directly be applied and has high effectiveness for polygonal range queries, within joins, and polygon-linestring joins. By applying a lightweight compression technique, APRIL approximations may occupy even less space than object MBRs. Furthermore, APRIL can be customized to apply on partitioned data and on polygons of varying sizes, rasterized at different granularities. Our last contribution is a novel algorithm that computes the APRIL approximation of a polygon without having to rasterize it in full, which is orders of magnitude faster than the computation of other raster approximations. Experiments on real data demonstrate the effectiveness and efficiency of APRIL; compared to the state-of-the-art intermediate filter, APRIL occupies 2x-8x less space, is 3.5x-8.5x more time-efficient, and reduces the end-to-end join cost up to 3 times.Comment: 12 page

Systematic literature review of hand gestures used in human computer interaction interfaces

Gestures, widely accepted as a humans' natural mode of interaction with their surroundings, have been considered for use in human-computer based interfaces since the early 1980s. They have been explored and implemented, with a range of success and maturity levels, in a variety of fields, facilitated by a multitude of technologies. Underpinning gesture theory however focuses on gestures performed simultaneously with speech, and majority of gesture based interfaces are supported by other modes of interaction. This article reports the results of a systematic review undertaken to identify characteristics of touchless/in-air hand gestures used in interaction interfaces. 148 articles were reviewed reporting on gesture-based interaction interfaces, identified through searching engineering and science databases (Engineering Village, Pro Quest, Science Direct, Scopus and Web of Science). The goal of the review was to map the field of gesture-based interfaces, investigate the patterns in gesture use, and identify common combinations of gestures for different combinations of applications and technologies. From the review, the community seems disparate with little evidence of building upon prior work and a fundamental framework of gesture-based interaction is not evident. However, the findings can help inform future developments and provide valuable information about the benefits and drawbacks of different approaches. It was further found that the nature and appropriateness of gestures used was not a primary factor in gesture elicitation when designing gesture based systems, and that ease of technology implementation often took precedence