4,017 research outputs found
Virtual environments promoting interaction
Virtual reality (VR) has been widely researched in the academic environment and is now breaking
into the industry. Regular companies do not have access to this technology as a collaboration tool
because these solutions usually require specific devices that are not at hand of the common user in
offices. There are other collaboration platforms based on video, speech and text, but VR allows
users to share the same 3D space. In this 3D space there can be added functionalities or information
that in a real-world environment would not be possible, something intrinsic to VR.
This dissertation has produced a 3D framework that promotes nonverbal communication. It
plays a fundamental role on human interaction and is mostly based on emotion. In the academia,
confusion is known to influence learning gains if it is properly managed. We designed a study to
evaluate how lexical, syntactic and n-gram features influence perceived confusion and found results (not statistically significant) that point that it is possible to build a machine learning model
that can predict the level of confusion based on these features. This model was used to manipulate
the script of a given presentation, and user feedback shows a trend that by manipulating these
features and theoretically lowering the level of confusion on text not only drops the reported confusion, as it also increases reported sense of presence. Another contribution of this dissertation
comes from the intrinsic features of a 3D environment where one can carry actions that in a real
world are not possible. We designed an automatic adaption lighting system that reacts to the perceived user’s engagement. This hypothesis was partially refused as the results go against what we
hypothesized but do not have statistical significance.
Three lines of research may stem from this dissertation. First, there can be more complex features to train the machine learning model such as syntax trees. Also, on an Intelligent Tutoring
System this could adjust the avatar’s speech in real-time if fed by a real-time confusion detector.
When going for a social scenario, the set of basic emotions is well-adjusted and can enrich them.
Facial emotion recognition can extend this effect to the avatar’s body to fuel this synchronization
and increase the sense of presence. Finally, we based this dissertation on the premise of using
ubiquitous devices, but with the rapid evolution of technology we should consider that new devices
will be present on offices. This opens new possibilities for other modalities.A Realidade Virtual (RV) tem sido alvo de investigação extensa na academia e tem vindo a entrar
na indústria. Empresas comuns não têm acesso a esta tecnologia como uma ferramenta de colaboração porque estas soluções necessitam de dispositivos específicos que não estão disponíveis para
o utilizador comum em escritório. Existem outras plataformas de colaboração baseadas em vídeo,
voz e texto, mas a RV permite partilhar o mesmo espaço 3D. Neste espaço podem existir funcionalidades ou informação adicionais que no mundo real não seria possível, algo intrínseco à RV.
Esta dissertação produziu uma framework 3D que promove a comunicação não-verbal que tem
um papel fundamental na interação humana e é principalmente baseada em emoção. Na academia
é sabido que a confusão influencia os ganhos na aprendizagem quando gerida adequadamente.
Desenhámos um estudo para avaliar como as características lexicais, sintáticas e n-gramas influenciam a confusão percecionada. Construímos e testámos um modelo de aprendizagem automática
que prevê o nível de confusão baseado nestas características, produzindo resultados não estatisticamente significativos que suportam esta hipótese. Este modelo foi usado para manipular o texto
de uma apresentação e o feedback dos utilizadores demonstra uma tendência na diminuição do
nível de confusão reportada no texto e aumento da sensação de presença. Outra contribuição vem
das características intrínsecas de um ambiente 3D onde se podem executar ações que no mundo
real não seriam possíveis. Desenhámos um sistema automático de iluminação adaptativa que reage
ao engagement percecionado do utilizador. Os resultados não suportam o que hipotetizámos mas
não têm significância estatística, pelo que esta hipótese foi parcialmente rejeitada.
Três linhas de investigação podem provir desta dissertação. Primeiro, criar características mais
complexas para treinar o modelo de aprendizagem, tais como árvores de sintaxe. Além disso, num
Intelligent Tutoring System este modelo poderá ajustar o discurso do avatar em tempo real, alimentado por um detetor de confusão. As emoções básicas ajustam-se a um cenário social e podem
enriquecê-lo. A emoção expressada facialmente pode estender este efeito ao corpo do avatar para
alimentar o sincronismo social e aumentar a sensação de presença. Finalmente, baseámo-nos em
dispositivos ubíquos, mas com a rápida evolução da tecnologia, podemos considerar que novos
dispositivos irão estar presentes em escritórios. Isto abre possibilidades para novas modalidades
Bringing tabletop technologies to kindergarten children
Taking computer technology away from the desktop and into a more physical, manipulative space, is known that provide many benefits and is generally considered to result in a system that is easier to learn and more natural to use. This paper describes a design solution that allows kindergarten children to take the benefits of the new pedagogical possibilities that tangible interaction and tabletop technologies offer for manipulative learning. After analysis of children's cognitive and psychomotor skills, we have designed and tuned a prototype game that is suitable for children aged 3 to 4 years old. Our prototype uniquely combines low cost tangible interaction and tabletop technology with tutored learning. The design has been based on the observation of children using the technology, letting them freely play with the application during three play sessions. These observational sessions informed the design decisions for the game whilst also confirming the children's enjoyment of the prototype
VToonify: Controllable High-Resolution Portrait Video Style Transfer
Generating high-quality artistic portrait videos is an important and
desirable task in computer graphics and vision. Although a series of successful
portrait image toonification models built upon the powerful StyleGAN have been
proposed, these image-oriented methods have obvious limitations when applied to
videos, such as the fixed frame size, the requirement of face alignment,
missing non-facial details and temporal inconsistency. In this work, we
investigate the challenging controllable high-resolution portrait video style
transfer by introducing a novel VToonify framework. Specifically, VToonify
leverages the mid- and high-resolution layers of StyleGAN to render
high-quality artistic portraits based on the multi-scale content features
extracted by an encoder to better preserve the frame details. The resulting
fully convolutional architecture accepts non-aligned faces in videos of
variable size as input, contributing to complete face regions with natural
motions in the output. Our framework is compatible with existing StyleGAN-based
image toonification models to extend them to video toonification, and inherits
appealing features of these models for flexible style control on color and
intensity. This work presents two instantiations of VToonify built upon Toonify
and DualStyleGAN for collection-based and exemplar-based portrait video style
transfer, respectively. Extensive experimental results demonstrate the
effectiveness of our proposed VToonify framework over existing methods in
generating high-quality and temporally-coherent artistic portrait videos with
flexible style controls.Comment: ACM Transactions on Graphics (SIGGRAPH Asia 2022). Code:
https://github.com/williamyang1991/VToonify Project page:
https://www.mmlab-ntu.com/project/vtoonify
3D Virtual Worlds and the Metaverse: Current Status and Future Possibilities
Moving from a set of independent virtual worlds to an integrated network of 3D virtual worlds or Metaverse rests on progress in four areas: immersive realism, ubiquity of access and identity, interoperability, and scalability. For each area, the current status and needed developments in order to achieve a functional Metaverse are described. Factors that support the formation of a viable Metaverse, such as institutional and popular interest and ongoing improvements in hardware performance, and factors that constrain the achievement of this goal, including limits in computational methods and unrealized collaboration among virtual world stakeholders and developers, are also considered
Development of the huggable social robot Probo: on the conceptual design and software architecture
This dissertation presents the development of a huggable social robot named Probo. Probo embodies a stuffed imaginary animal, providing a soft touch and a huggable appearance. Probo's purpose is to serve as a multidisciplinary research platform for human-robot interaction focused on children. In terms of a social robot, Probo is classified as a social interface supporting non-verbal communication. Probo's social skills are thereby limited to a reactive level. To close the gap with higher levels of interaction, an innovative system for shared control with a human operator is introduced. The software architecture de nes a modular structure to incorporate all systems into a single control center. This control center is accompanied with a 3D virtual model of Probo, simulating all motions of the robot and providing a visual feedback to the operator. Additionally, the model allows us to advance on user-testing and evaluation of newly designed systems. The robot reacts on basic input stimuli that it perceives during interaction. The input stimuli, that can be referred to as low-level perceptions, are derived from vision analysis, audio analysis, touch analysis and object identification. The stimuli will influence the attention and homeostatic system, used to de ne the robot's point of attention, current emotional state and corresponding facial expression. The recognition of these facial expressions has been evaluated in various user-studies. To evaluate the collaboration of the software components, a social interactive game for children, Probogotchi, has been developed. To facilitate interaction with children, Probo has an identity and corresponding history. Safety is ensured through Probo's soft embodiment and intrinsic safe actuation systems. To convey the illusion of life in a robotic creature, tools for the creation and management of motion sequences are put into the hands of the operator. All motions generated from operator triggered systems are combined with the motions originating from the autonomous reactive systems. The resulting motion is subsequently smoothened and transmitted to the actuation systems. With future applications to come, Probo is an ideal platform to create a friendly companion for hospitalised children
VALE-Emotions: Teaching mobile application for individuals with Autism Spectrum Disorders
In this paper, the development of an interactive mobile application to strengthen the learning of emotion recognition in children with Autism Spectrum Disorders (ASD) is presented. This App is part of an authoring Virtual Advanced Learning Environment (VALE) devoted to support teaching and learning activities. VALE-Emotions App is based on the six basic emotions studied by (Paul Ekman, 1992), but it is not only limited to the recognition of such emotions in their highest intensity levels. In fact, the app allows for determining the effective recognition of these emotions at different intensity levels. Such intensity levels are generated by an authoring Dynamic Facial Expressions (DFE) coding using virtual avatars. Each learning activity is carried out through training and tests applications, giving to the users the opportunity of freely developing, learning, and strengthen social skills in an entertaining way. The results of the experimentation of the VALE-Emotions on subjects with ASD are also reported. In general, the participants showed efficient response at the stimulus during the developed activities obtaining a high and fast recognition of certain emotions
- …