12,983 research outputs found

    Vision- and tactile-based continuous multimodal intention and attention recognition for safer physical human-robot interaction

    Full text link
    Employing skin-like tactile sensors on robots enhances both the safety and usability of collaborative robots by adding the capability to detect human contact. Unfortunately, simple binary tactile sensors alone cannot determine the context of the human contact -- whether it is a deliberate interaction or an unintended collision that requires safety manoeuvres. Many published methods classify discrete interactions using more advanced tactile sensors or by analysing joint torques. Instead, we propose to augment the intention recognition capabilities of simple binary tactile sensors by adding a robot-mounted camera for human posture analysis. Different interaction characteristics, including touch location, human pose, and gaze direction, are used to train a supervised machine learning algorithm to classify whether a touch is intentional or not with an F1-score of 86%. We demonstrate that multimodal intention recognition is significantly more accurate than monomodal analyses with the collaborative robot Baxter. Furthermore, our method can also continuously monitor interactions that fluidly change between intentional or unintentional by gauging the user's attention through gaze. If a user stops paying attention mid-task, the proposed intention and attention recognition algorithm can activate safety features to prevent unsafe interactions. We also employ a feature reduction technique that reduces the number of inputs to five to achieve a more generalized low-dimensional classifier. This simplification both reduces the amount of training data required and improves real-world classification accuracy. It also renders the method potentially agnostic to the robot and touch sensor architectures while achieving a high degree of task adaptability.Comment: 11 pages, 8 figures, preprint under revie

    The Metaverse: Survey, Trends, Novel Pipeline Ecosystem & Future Directions

    Full text link
    The Metaverse offers a second world beyond reality, where boundaries are non-existent, and possibilities are endless through engagement and immersive experiences using the virtual reality (VR) technology. Many disciplines can benefit from the advancement of the Metaverse when accurately developed, including the fields of technology, gaming, education, art, and culture. Nevertheless, developing the Metaverse environment to its full potential is an ambiguous task that needs proper guidance and directions. Existing surveys on the Metaverse focus only on a specific aspect and discipline of the Metaverse and lack a holistic view of the entire process. To this end, a more holistic, multi-disciplinary, in-depth, and academic and industry-oriented review is required to provide a thorough study of the Metaverse development pipeline. To address these issues, we present in this survey a novel multi-layered pipeline ecosystem composed of (1) the Metaverse computing, networking, communications and hardware infrastructure, (2) environment digitization, and (3) user interactions. For every layer, we discuss the components that detail the steps of its development. Also, for each of these components, we examine the impact of a set of enabling technologies and empowering domains (e.g., Artificial Intelligence, Security & Privacy, Blockchain, Business, Ethics, and Social) on its advancement. In addition, we explain the importance of these technologies to support decentralization, interoperability, user experiences, interactions, and monetization. Our presented study highlights the existing challenges for each component, followed by research directions and potential solutions. To the best of our knowledge, this survey is the most comprehensive and allows users, scholars, and entrepreneurs to get an in-depth understanding of the Metaverse ecosystem to find their opportunities and potentials for contribution

    Economia colaborativa

    Get PDF
    A importância de se proceder à análise dos principais desafios jurídicos que a economia colaborativa coloca – pelas implicações que as mudanças de paradigma dos modelos de negócios e dos sujeitos envolvidos suscitam − é indiscutível, correspondendo à necessidade de se fomentar a segurança jurídica destas práticas, potenciadoras de crescimento económico e bem-estar social. O Centro de Investigação em Justiça e Governação (JusGov) constituiu uma equipa multidisciplinar que, além de juristas, integra investigadores de outras áreas, como a economia e a gestão, dos vários grupos do JusGov – embora com especial participação dos investigadores que integram o grupo E-TEC (Estado, Empresa e Tecnologia) – e de outras prestigiadas instituições nacionais e internacionais, para desenvolver um projeto neste domínio, com o objetivo de identificar os problemas jurídicos que a economia colaborativa suscita e avaliar se já existem soluções para aqueles, refletindo igualmente sobre a conveniência de serem introduzidas alterações ou se será mesmo necessário criar nova regulamentação. O resultado desta investigação é apresentado nesta obra, com o que se pretende fomentar a continuação do debate sobre este tema.Esta obra é financiada por fundos nacionais através da FCT — Fundação para a Ciência e a Tecnologia, I.P., no âmbito do Financiamento UID/05749/202

    VIVE3D: Viewpoint-Independent Video Editing using 3D-Aware GANs

    Full text link
    We introduce VIVE3D, a novel approach that extends the capabilities of image-based 3D GANs to video editing and is able to represent the input video in an identity-preserving and temporally consistent way. We propose two new building blocks. First, we introduce a novel GAN inversion technique specifically tailored to 3D GANs by jointly embedding multiple frames and optimizing for the camera parameters. Second, besides traditional semantic face edits (e.g. for age and expression), we are the first to demonstrate edits that show novel views of the head enabled by the inherent properties of 3D GANs and our optical flow-guided compositing technique to combine the head with the background video. Our experiments demonstrate that VIVE3D generates high-fidelity face edits at consistent quality from a range of camera viewpoints which are composited with the original video in a temporally and spatially consistent manner.Comment: CVPR 2023. Project webpage and video available at http://afruehstueck.github.io/vive3

    Accurate and Interpretable Solution of the Inverse Rig for Realistic Blendshape Models with Quadratic Corrective Terms

    Full text link
    We propose a new model-based algorithm solving the inverse rig problem in facial animation retargeting, exhibiting higher accuracy of the fit and sparser, more interpretable weight vector compared to SOTA. The proposed method targets a specific subdomain of human face animation - highly-realistic blendshape models used in the production of movies and video games. In this paper, we formulate an optimization problem that takes into account all the requirements of targeted models. Our objective goes beyond a linear blendshape model and employs the quadratic corrective terms necessary for correctly fitting fine details of the mesh. We show that the solution to the proposed problem yields highly accurate mesh reconstruction even when general-purpose solvers, like SQP, are used. The results obtained using SQP are highly accurate in the mesh space but do not exhibit favorable qualities in terms of weight sparsity and smoothness, and for this reason, we further propose a novel algorithm relying on a MM technique. The algorithm is specifically suited for solving the proposed objective, yielding a high-accuracy mesh fit while respecting the constraints and producing a sparse and smooth set of weights easy to manipulate and interpret by artists. Our algorithm is benchmarked with SOTA approaches, and shows an overall superiority of the results, yielding a smooth animation reconstruction with a relative improvement up to 45 percent in root mean squared mesh error while keeping the cardinality comparable with benchmark methods. This paper gives a comprehensive set of evaluation metrics that cover different aspects of the solution, including mesh accuracy, sparsity of the weights, and smoothness of the animation curves, as well as the appearance of the produced animation, which human experts evaluated

    Hi4D: 4D Instance Segmentation of Close Human Interaction

    Full text link
    We propose Hi4D, a method and dataset for the automatic analysis of physically close human-human interaction under prolonged contact. Robustly disentangling several in-contact subjects is a challenging task due to occlusions and complex shapes. Hence, existing multi-view systems typically fuse 3D surfaces of close subjects into a single, connected mesh. To address this issue we leverage i) individually fitted neural implicit avatars; ii) an alternating optimization scheme that refines pose and surface through periods of close proximity; and iii) thus segment the fused raw scans into individual instances. From these instances we compile Hi4D dataset of 4D textured scans of 20 subject pairs, 100 sequences, and a total of more than 11K frames. Hi4D contains rich interaction-centric annotations in 2D and 3D alongside accurately registered parametric body models. We define varied human pose and shape estimation tasks on this dataset and provide results from state-of-the-art methods on these benchmarks.Comment: Project page: https://yifeiyin04.github.io/Hi4D

    Evaluating 3D human face reconstruction from a frontal 2D image, focusing on facial regions associated with foetal alcohol syndrome

    Get PDF
    Foetal alcohol syndrome (FAS) is a preventable condition caused by maternal alcohol consumption during pregnancy. The FAS facial phenotype is an important factor for diagnosis, alongside central nervous system impairments and growth abnormalities. Current methods for analysing the FAS facial phenotype rely on 3D facial image data, obtained from costly and complex surface scanning devices. An alternative is to use 2D images, which are easy to acquire with a digital camera or smart phone. However, 2D images lack the geometric accuracy required for accurate facial shape analysis. Our research offers a solution through the reconstruction of 3D human faces from single or multiple 2D images. We have developed a framework for evaluating 3D human face reconstruction from a single-input 2D image using a 3D face model for potential use in FAS assessment. We first built a generative morphable model of the face from a database of registered 3D face scans with diverse skin tones. Then we applied this model to reconstruct 3D face surfaces from single frontal images using a model-driven sampling algorithm. The accuracy of the predicted 3D face shapes was evaluated in terms of surface reconstruction error and the accuracy of FAS-relevant landmark locations and distances. Results show an average root mean square error of 2.62 mm. Our framework has the potential to estimate 3D landmark positions for parts of the face associated with the FAS facial phenotype. Future work aims to improve on the accuracy and adapt the approach for use in clinical settings. Significance: Our study presents a framework for constructing and evaluating a 3D face model from 2D face scans and evaluating the accuracy of 3D face shape predictions from single images. The results indicate low generalisation error and comparability to other studies. The reconstructions also provide insight into specific regions of the face relevant to FAS diagnosis. The proposed approach presents a potential cost-effective and easily accessible imaging tool for FAS screening, yet its clinical application needs further research

    Being Comes from Not-being: Open-vocabulary Text-to-Motion Generation with Wordless Training

    Full text link
    Text-to-motion generation is an emerging and challenging problem, which aims to synthesize motion with the same semantics as the input text. However, due to the lack of diverse labeled training data, most approaches either limit to specific types of text annotations or require online optimizations to cater to the texts during inference at the cost of efficiency and stability. In this paper, we investigate offline open-vocabulary text-to-motion generation in a zero-shot learning manner that neither requires paired training data nor extra online optimization to adapt for unseen texts. Inspired by the prompt learning in NLP, we pretrain a motion generator that learns to reconstruct the full motion from the masked motion. During inference, instead of changing the motion generator, our method reformulates the input text into a masked motion as the prompt for the motion generator to ``reconstruct'' the motion. In constructing the prompt, the unmasked poses of the prompt are synthesized by a text-to-pose generator. To supervise the optimization of the text-to-pose generator, we propose the first text-pose alignment model for measuring the alignment between texts and 3D poses. And to prevent the pose generator from overfitting to limited training texts, we further propose a novel wordless training mechanism that optimizes the text-to-pose generator without any training texts. The comprehensive experimental results show that our method obtains a significant improvement against the baseline methods. The code is available at https://github.com/junfanlin/oohmg

    L’Asie du Sud-Est 2023 : bilan, enjeux et perspectives

    Get PDF
    Chaque année, l’Institut de recherche sur l’Asie du Sud-Est contemporaine (IRASEC), basé à Bangkok, mobilise une vingtaine de chercheurs et d’experts pour mieux comprendre l’actualité régionale de ce carrefour économique, culturel et religieux, au cœur de l’Indo-Pacifique. Cette collection permet de suivre au fil des ans l’évolution des grands enjeux contemporains de cette région continentale et archipélagique de plus de 680 millions d’habitants, et d’en comprendre les dynamiques d’intégration régionale et de connectivités avec le reste du monde. L’Asie du Sud-Est 2023 propose une analyse synthétique et détaillée des principaux événements politiques et diplomatiques, ainsi que des évolutions économiques, sociales et environnementales de l’année 2022 dans chacun des onze pays de la région. Ce décryptage est complété pour chaque pays par un focus sur deux personnalités de l’année et une actualité marquante en image. L’ouvrage propose également cinq dossiers thématiques qui abordent des sujets traités à l’échelle régionale sud-est asiatique : les ressorts institutionnels de l’approche de santé intégrée One Health, le vieillissement de la population et sa prise en compte par les politiques publiques, les câbles sous-marins au cœur de la connectivité sud-est asiatique, l’aménagement du bassin du Mékong et ses multiples acteurs, et les enjeux politiques et linguistiques des langues transnationales. Des outils pratiques sont également disponibles : une fiche et une chronologie par pays et un cahier des principaux indicateurs démographiques, sociaux, économiques et environnementaux

    Message Journal, Issue 5: COVID-19 SPECIAL ISSUE Capturing visual insights, thoughts and reflections on 2020/21 and beyond...

    Get PDF
    If there is a theme running through the Message Covid-19 special issue, it is one of caring. Of our own and others’ resilience and wellbeing, of friendship and community, of students, practitioners and their futures, of social justice, equality and of doing the right thing. The veins of designing with care run through the edition, wide and deep. It captures, not designers as heroes, but those with humble views, exposing the need to understand a diversity of perspectives when trying to comprehend the complexity that Covid-19 continues to generate. As graphic designers, illustrators and visual communicators, contributors have created, documented, written, visualised, reflected, shared, connected and co-created, designed for good causes and re-defined what it is to be a student, an academic and a designer during the pandemic. This poignant period in time has driven us, through isolation, towards new rules of living, and new ways of working; to see and map the world in a different light. A light that is uncertain, disjointed, and constantly being redefined. This Message issue captures responses from the graphic communication design community in their raw state, to allow contributors to communicate their experiences through both their written and visual voice. Thus, the reader can discern as much from the words as the design and visualisations. Through this issue a substantial number of contributions have focused on personal reflection, isolation, fear, anxiety and wellbeing, as well as reaching out to community, making connections and collaborating. This was not surprising in a world in which connection with others has often been remote, and where ‘normal’ social structures of support and care have been broken down. We also gain insight into those who are using graphic communication design to inspire and capture new ways of teaching and learning, developing themselves as designers, educators, and activists, responding to social justice and to do good; gaining greater insight into society, government actions and conspiracy. Introduction: Victoria Squire - Coping with Covid: Community, connection and collaboration: James Alexander & Carole Evans, Meg Davies, Matthew Frame, Chae Ho Lee, Alma Hoffmann, Holly K. Kaufman-Hill, Joshua Korenblat, Warren Lehrer, Christine Lhowe, Sara Nesteruk, Cat Normoyle & Jessica Teague, Kyuha Shim. - Coping with Covid: Isolation, wellbeing and hope: Sadia Abdisalam, Tom Ayling, Jessica Barness, Megan Culliford, Stephanie Cunningham, Sofija Gvozdeva, Hedzlynn Kamaruzzaman, Merle Karp, Erica V. P. Lewis, Kelly Salchow Macarthur, Steven McCarthy, Shelly Mayers, Elizabeth Shefrin, Angelica Sibrian, David Smart, Ane Thon Knutsen, Isobel Thomas, Darryl Westley. - Coping with Covid: Pedagogy, teaching and learning: Bernard J Canniffe, Subir Dey, Aaron Ganci, Elizabeth Herrmann, John Kilburn, Paul Nini, Emily Osborne, Gianni Sinni & Irene Sgarro, Dave Wood, Helena Gregory, Colin Raeburn & Jackie Malcolm. - Coping with Covid: Social justice, activism and doing good: Class Action Collective, Xinyi Li, Matt Soar, Junie Tang, Lisa Winstanley. - Coping with Covid: Society, control and conspiracy: Diana Bîrhală, Maria Borțoi, Patti Capaldi, Tânia A. Cardoso, Peter Gibbons, Bianca Milea, Rebecca Tegtmeyer, Danne Wo
    • …
    corecore