Spatio-temporal centroid based sign language facial expressions for animation synthesis in virtual environment

Orientador: Eduardo TodtTese (doutorado) - Universidade Federal do Paraná, Setor de Ciências Exatas, Programa de Pós-Graduação em Informática. Defesa : Curitiba, 20/02/2019Inclui referências: p.86-97Área de concentração: Ciência da ComputaçãoResumo: Formalmente reconhecida como segunda lingua oficial brasileira, a BSL, ou Libras, conta hoje com muitas aplicacoes computacionais que integram a comunidade surda nas atividades cotidianas, oferecendo interpretes virtuais representados por avatares 3D construidos utilizando modelos formais que parametrizam as caracteristicas especificas das linguas de sinais. Estas aplicacoes, contudo, ainda consideram expressoes faciais como recurso de segundo plano em uma lingua primariamente gestual, ignorando a importancia que expressoes faciais e emocoes imprimem no contexto da mensagem transmitida. Neste trabalho, a fim de definir um modelo facial parametrizado para uso em linguas de sinais, um sistema de sintese de expressoes faciais atraves de um avatar 3D e proposto e um prototipo implementado. Neste sentido, um modelo de landmarks faciais separado por regioes e definido assim como uma modelagem de expressoes base utilizando as bases faciais AKDEF e JAFEE como referencia. Com este sistema e possivel representar expressoes complexas utilizando interpolacao dos valores de intensidade na animacao geometrica, de forma simplificada utilizando controle por centroides e deslocamento de regioes independentes no modelo 3D. E proposto ainda uma aplicacao de modelo espaco-temporal para os landmarks faciais, com o objetivo de observar o comportamento e relacao dos centroides na sintese das expressoes base definindo quais pontos geometricos sao relevantes no processo de interpolacao e animacao das expressoes. Um sistema de exportacao dos dados faciais seguindo o formato hierarquico utilizado na maioria dos avatares 3D interpretes de linguas de sinais e desenvolvido, incentivando a integracao em modelos formais computacionais ja existentes na literatura, permitindo ainda a adaptacao e alteracao de valores e intensidades na representacao das emocoes. Assim, os modelos e conceitos apresentados propoe a integracao de um modeo facial para representacao de expressoes na sintese de sinais oferecendo uma proposta simplificada e otimizada para aplicacao dos recursos em avatares 3D. Palavras-chave: Avatar 3D, Dados Espaco-Temporal, Libras, Lingua de sinais, Expressoes Faciais.Abstract: Formally recognized as the second official Brazilian language, BSL, or Libras, today has many computational applications that integrate the deaf community into daily activities, offering virtual interpreters represented by 3D avatars built using formal models that parameterize the specific characteristics of sign languages. These applications, however, still consider facial expressions as a background feature in a primarily gestural language, ignoring the importance that facial expressions and emotions imprint on the context of the transmitted message. In this work, in order to define a parametrized facial model for use in sign languages, a system of synthesis of facial expressions through a 3D avatar is proposed and a prototype implemented. In this way, a model of facial landmarks separated by regions is defined as a modeling of base expressions using the AKDEF and JAFEE facial bases as a reference. With this system it is possible to represent complex expressions using interpolation of the intensity values in the geometric animation, in a simplified way using control by centroids and displacement of independent regions in the 3D model. A spatial-temporal model is proposed for the facial landmarks, with the objective of define the behavior and relation of the centroids in the synthesis of the basic expressions, pointing out which geometric landmark are relevant in the process of interpolation and animation of the expressions. A system for exporting facial data following the hierarchical format used in most avatars 3D sign language interpreters is developed, encouraging the integration in formal computer models already existent in the literature, also allowing the adaptation and change of values and intensities in the representation of the emotions. Thus, the models and concepts presented propose the integration of a facial model to represent expressions in the synthesis of signals offering a simplified and optimized proposal for the application of the resources in 3D avatars. Keywords: 3D Avatar, Spatio-Temporal Data, BSL, Sign Language, Facial Expression

Exploring the expansion of planner’s engagement capabilities via accessing the data from a building information model for public consultation

A statutory urban planning consultation is the only opportunity to address concerns of the public regarding the shared built environment. However, current methods for consulting the public on urban planning proposals are archaic. They are often uncoordinated and can potentially fail to provide an inclusive forum for all socio-demographics. Face-to-face consultations are noted as the best method to consult. However, they only provide a means for those with time and mobility to attend the discussions, workshops, and exhibitions regarding the built environment. Urban planning consultation processes are also limited by stakeholder relationships. It is not often that stakeholders acknowledge each other. This thesis focuses on approaches to stakeholder identification, management, and engagement. Stakeholders can be identified at any stage of project development and addressed as internal (who have a direct hand in changing a project) and external (who are impacted by changes.) Stakeholder management and engagement can vary in theory and practice, and this thesis will explore this. Internal stakeholders utilise their own communication methods and processes to cooperate and consider risk at each stage of the project. Building information modelling (BIM) is a collaborative tool which shares textual and image data among the building development project team. Planners implementing a consultation with the public do not use BIM, as it is software for internal stakeholders to share information for the project development in a common data environment (CDE). BIM is a technical methodology primarily used to communicate the strategy of the project (including usual details and specifications) but is limited in its further applications for encouraging communication beyond this initial scope. External stakeholders rely on planners to reach wider audiences, but without mediation from an expert to explain the technical language in layman’s terms, there remains a gap in knowledge making it difficult for the public to understand specific design decisions. This PhD study aims to understand how 3D Building Information Models (BIM) and associated data can be utilised to facilitate communication throughout each stage of the planning consultation and explores how BIM may be used to address the knowledge gap specifically through an online planning consultation platform. This is increasingly important as more digital tools are being introduced into the planning sphere and incorporating the views of the public and other stakeholders, should be central to planning consultations in the UK. This research presents mixed methods in order to identify the requirements for a digital planning portal that best presents BIM data for the use of public consultations. The visual and textual information of a BIM model is documented and validated with a case study and recommendations are made for using BIM data within a public consultation. Throughout the study, the concepts of what consultations are in currently in place are considered, in parallel to current policy and best practice. The author presents the impact digital tools and how these might be utilised within the current planning consultation process. The findings show that, with the incorporation of digital platforms in planning consultations, a greater focus on stakeholder perspectives and relationships can be established