A framework for cardio-pulmonary resuscitation (CPR) scene retrieval from medical simulation videos based on object and activity detection.

In this thesis, we propose a framework to detect and retrieve CPR activity scenes from medical simulation videos. Medical simulation is a modern training method for medical students, where an emergency patient condition is simulated on human-like mannequins and the students act upon. These simulation sessions are recorded by the physician, for later debriefing. With the increasing number of simulation videos, automatic detection and retrieval of specific scenes became necessary. The proposed framework for CPR scene retrieval, would eliminate the conventional approach of using shot detection and frame segmentation techniques. Firstly, our work explores the application of Histogram of Oriented Gradients in three dimensions (HOG3D) to retrieve the scenes containing CPR activity. Secondly, we investigate the use of Local Binary Patterns in Three Orthogonal Planes (LBPTOP), which is the three dimensional extension of the popular Local Binary Patterns. This technique is a robust feature that can detect specific activities from scenes containing multiple actors and activities. Thirdly, we propose an improvement to the above mentioned methods by a combination of HOG3D and LBP-TOP. We use decision level fusion techniques to combine the features. We prove experimentally that the proposed techniques and their combination out-perform the existing system for CPR scene retrieval. Finally, we devise a method to detect and retrieve the scenes containing the breathing bag activity, from the medical simulation videos. The proposed framework is tested and validated using eight medical simulation videos and the results are presented

Using a Bayesian Framework to Develop 3D Gestural Input Systems Based on Expertise and Exposure in Anesthesia

Interactions with a keyboard and mouse fall short of human capabilities and what is lacking in the technological revolution is a surge of new and natural ways of interacting with computers. In-air gestures are a promising input modality as they are expressive, easy to use, quick to use, and natural for users. It is known that gestural systems should be developed within a particular context as gesture choice is dependent on the context; however, there is little research investigating other individual factors which may influence gesture choice such as expertise and exposure. Anesthesia providers’ hands have been linked to bacterial transmission; therefore, this research investigates the context of gestural technology for anesthetic task. The objective of this research is to understand how expertise and exposure influence gestural behavior and to develop Bayesian statistical models that can accurately predict how users would choose intuitive gestures in anesthesia based on expertise and exposure. Expertise and exposure may influence gesture responses for individuals; however, there is limited to no work investigating how these factors influence intuitive gesture choice and how to use this information to predict intuitive gestures to be used in system design. If researchers can capture users’ gesture variability within a particular context based on expertise and exposure, then statistical models can be developed to predict how users may gesturally respond to a computer system and use those predictions to design a gestural system which anticipates a user’s response and thus affords intuitiveness to multiple user groups. This allows designers to more completely understand the end user and implement intuitive gesture systems that are based on expected natural responses. Ultimately, this dissertation seeks to investigate the human factors challenges associated with gestural system development within a specific context and to offer statistical approaches to understanding and predicting human behavior in a gestural system. Two experimental studies and two Bayesian analyses were completed in this dissertation. The first experimental study investigated the effect of expertise within the context of anesthesiology. The main finding of this study was that domain expertise is influential when developing 3D gestural systems as novices and experts differ in terms of intuitive gesture-function mappings as well as reaction times to generate an intuitive mapping. The second study investigated the effect of exposure for controlling a computer-based presentation and found that there is a learning effect of gestural control in that participants were significantly faster at generating intuitive mappings as they gained exposure with the system. The two Bayesian analyses were in the form of Bayesian multinomial logistic regression models where intuitive gesture choice was predicted based on the contextual task and either expertise or exposure. The Bayesian analyses generated posterior predictive probabilities for all combinations of task, expertise level, and exposure level and showed that gesture choice can be predicted to some degree. This work provides further insights into how 3D gestural input systems should be designed and how Bayesian statistics can be used to model human behavior

U2XECS : avaliação de usabilidade e experiência de usuário de sistemas conversacionais

Orientadora: Natasha Malveira Costa ValentimDissertação (mestrado) - Universidade Federal do Paraná, Setor de Ciências Exatas, Programa de Pós-Graduação em Informática. Defesa : Curitiba, 26/11/2020Inclui referências: p. 111-121Área de concentração: Ciência da ComputaçãoResumo: Devido ao aumento do uso das tecnologias nos ultimos anos, novas formas de interacao estao presentes no cotidiano da sociedade. A interacao baseada em voz, caracteristica dos Sistemas Conversacionais (SC), e um exemplo dessas novas formas de interagir. Amazon Alexa, Siri, Google Assistant, Amazon Frame, dispostivos Amazon Echo e Google Home sao exemplos de SCs que utilizam a voz do usuario para desempenhar tarefas. Os SCs tem despertado interesse tanto da industria como da academia, recebendo investimentos e fazendo parte de pesquisa em Interacao Humano-Computador (IHC) e Engenharia de Software (ES). Como qualquer outro sistema, e necessario que os SCs fornecam uma boa experiencia e que atendam as necessidades de seus usuarios. Nesse sentido, a avaliacao de Usabilidade e de Experiencia do Usuario (User eXperience - UX) e vista como etapa importante que contribui com a verificacao da qualidade dos SCs. Na avaliacao da Usabilidade, geralmente sao verificados atributos referentes as metas comportamentais do sistema, como a eficacia, eficiencia e satisfacao do usuario. Ja na avaliacao da UX, geralmente sao considerados os atributos ligados ao sentimento do usuarios, como emocao e motivacao. No entanto, atraves de dois Mapeamentos Sistematicos da Literatura (MSL), foi identificado que as tecnologias de avaliacao utilizadas para avaliar a Usabilidade e/ou UX dos SCs eram genericas, e poderiam avaliar qualquer tipo software. Alem disso, foram identificados alguns questionarios de avaliacao de Usabilidade ou UX de interfaces conversacionais. Entretanto, estas tecnologias consideram apenas um aspecto de qualidade, Usabilidade, ou UX. Os MSLs tambem identificaram que alguns pesquisadores utilizam questionarios que desenvolveram para seus proprios estudos, sem passar por um processo de avaliacao empirica. Sendo assim, o objetivo deste trabalho e fornecer uma tecnologia de avaliacao conjunta de Usabilidade e UX especifica para SCs, a U2XECS (Usability and User eXperience Evaluation of Conversational Systems). A U2XECS e uma tecnologia de avaliacao baseada em questionario que fornece afirmacoes de Usabilidade e UX especificas para avaliar SCs. O objetivo do U2XECS e orientar pesquisadores e desenvolvedores para identificar melhorias e percepcoes dos usuarios nestes sistemas. Alem dos MSLs e da proposicao da tecnologia, sao apresentados tambem tres estudos que foram realizados no processo de elaboracao e refinamento da tecnologia: um estudo exploratorio, um survey e um estudo de viabilidade. Os resultados evidenciaram pontos positivos da U2XECS relacionados a facilidade de uso, utilidade e intencoes de uso. Alem disso, foram identificadas oportunidades de melhoria, tais como afirmacoes ambiguas, mudanca na estrutura e no tamanho do questionario. Palavras-chave: Avaliacao de Usabilidade. Avaliacao de Experiencia de Usuario. Interacao Baseada em Voz. Sistemas Conversacionais.Abstract: Due to the increased use of technologies in recent years, new forms of interaction are present in society. The voice-based interaction, characteristic of Conversational Systems (CSs), is an example of these new interaction forms. Amazon Alexa, Siri, Google Assistant, Amazon Frame, Amazon Echo devices, and Google Home are examples of CSs that use the voice to perform tasks. The CSs have aroused interest from both industry and academia, receiving investments and being part of research in Human-Computer Interaction (HCI) and Software Engineering (SE). Like any other system, CSs must provide a good experience and meet the needs of their users. In this sense, the evaluation of Usability and User eXperience (UX) is seen as an essential step that contributes to verifying the quality of the CSs. In the Usability evaluation, attributes regarding the system's behavioral goals, such as effectiveness, efficiency, and user satisfaction, are usually verified. In the UX evaluation, attributes related to the user's feelings, such as emotion and motivation, are usually considered. However, through two Systematic Mapping Studies (SMS), it was identified that the evaluation technologies used to evaluate the Usability and/or UX of the CSs were generic and could evaluate any software. Besides, some Usability or UX evaluation questionnaires of conversational interfaces were identified. However, these technologies consider only one aspect of quality, Usability, or UX. SMSs also identified that some researchers use questionnaires that they developed for their studies without going through an empirical evaluation process. Therefore, this work aims to provide a CS-specific joint Usability and UX evaluation technology, the U2XECS (Usability and User eXperience Evaluation of Conversational Systems). U2XECS is a questionnaire-based evaluation technology that provides Usability and UX specific statements to evaluate CSs. The goal of U2XECS is to guide researchers and developers to identify improvements and user perceptions in these systems. Besides the SMSs and the technology proposition, three studies that were carried out in the process of elaboration and refinement of the technology are presented: an exploratory study, a survey, and a feasibility study. The results showed positive points of U2XECS related to ease of use, utility, and intentions of use. Besides, opportunities for improvement were identified, such as ambiguous statements, change in the structure and size of the questionnaire. Keywords: Usability Evaluation. User Experience Evaluation. Voice-Based Interaction. Conversational Systems