Addressing the data bottleneck in medical deep learning models using a human-in-the-loop machine learning approach

[Abstract]: Any machine learning (ML) model is highly dependent on the data it uses for learning, and this is even more important in the case of deep learning models. The problem is a data bottleneck, i.e. the difficulty in obtaining an adequate number of cases and quality data. Another issue is improving the learning process, which can be done by actively introducing experts into the learning loop, in what is known as human-in-the-loop (HITL) ML. We describe an ML model based on a neural network in which HITL techniques were used to resolve the data bottleneck problem for the treatment of pancreatic cancer. We first augmented the dataset using synthetic cases created by a generative adversarial network. We then launched an active learning (AL) process involving human experts as oracles to label both new cases and cases by the network found to be suspect. This AL process was carried out simultaneously with an interactive ML process in which feedback was obtained from humans in order to develop better synthetic cases for each iteration of training. We discuss the challenges involved in including humans in the learning process, especially in relation to human–computer interaction, which is acquiring great importance in building ML models and can condition the success of a HITL approach. This paper also discusses the methodological approach adopted to address these challenges.This work has been supported by the State Research Agency of the Spanish Government (Grant PID2019-107194GB-I00/AEI/10.13039/501100011033) and by the Xunta de Galicia (Grant ED431C 2022/44), supported in turn by the EU European Regional Development Fund. We wish to acknowledge support received from the Centro de Investigación de Galicia CITIC, funded by the Xunta de Galicia and the European Regional Development Fund (Galicia 2014–2020 Program; Grant ED431G 2019/01).Xunta de Galicia; ED431C 2022/44Xunta de Galicia; ED431G 2019/0

Development of usability and context-of-use taxonomies, integration with techniques for the study of usability and application to real-world intelligent systems

[Resumen] Un importante obstáculo para la implantación del Diseño Centrado en el Usuario en el mundo real es que no existe una definición del concepto de usabilidad que sea precisa y comúnnmente aceptada y aplicada en la práctica. La literatura tiende a definir la usabilidad en términos demasiado escuetos y ambiguos, y a describir su aplicación en términos informales. Idénticas críticas pueden hacerse al concepto del contexto de uso, al cual la usabilidad es siempre relativa. Como consecuencia de esto, las técnicas ad hoc predominan en las metodologías de usabilidad. Esta tesis propone taxonomías detalladas para los conceptos de usabilidad y contexto de uso. Las taxonomías están organizadas jerárquicamente y contienen descripciones precisas de sus atributos y subatributos. Para ilustrar la utilidad práctica de las taxonomías, se describe cómo fueron integradas en el ciclo de vida de desarrollo de dos productos reales en el campo de los Sistemas Inteligentes. Concretamente, un dispositivo de Adaptación Inteligente de la Velocidad y un generador automático de interfaces de usuario. En un punto específico de cada proyecto se realizó un estudio de usabilidad, usando las taxonomías para estructurar y guiar actividades de usabilidad como el análisis de los requisitos de usabilidad, la evaluación heurística y el análisis subjetivo.[Abstract] A major obstacle to the implantation of User-Centered Design in the real world is the fact that no precise de nition of the concept of usability exists that is widely accepted and applied in practice. Generally speaking, the literature tends to de ne usability in overly brief and ambiguous terms and to describe its application in informal terms. The same criticisms can be leveled at the concept of context of use, to which usability is always relative. As a consequence of these drawbacks, ad hoc techniques predominate in usability study methodologies. This thesis proposes detailed taxonomies for the concepts of usability and the context of use. The taxonomies are organized hierarchically and contain precise descriptions of their attributes and subattributes. In order to illustrate the practical usefulness of the taxonomies, this thesis describes and discusses how the taxonomies were integrated into the development life cycle of two real-world projects in the eld of Intelligent Systems, namely, an Intelligent Speed Adaptation device and an automatic generator of user interfaces. At a speci c point of each project, a usability study was conducted, in which the taxonomies were used to structure and guide well-known usability activities such as usability requirements analysis, heuristic evaluation, and subjective analysis[Resumo] Un importante obstáculo para a implantación do Deseño Centrado no Usuario no mundo real é que non existe unha definición do concepto de usabilidade que sexa precisa e comunmente aceptada e aplicada na práctica. A literatura tende a definir a usabilidade en termos demasiado concisos e ambiguos, e a describir a súa aplicación en termos informais. Idénticas críticas poden facerse ao concepto do contexto de uso, ao cal a usabilidade de sempre relativa. Como consecuencia disto, as técnicas ad hoc predominan nas metodoloxías de usabilidade. Esta tese propón taxonomías detalladas para os conceptos de usabilidade e contexto de uso. As taxonomías están organizadas xerárquicamente e conteñen descricións precisas dos seus atributos e subatributos. Para ilustrar a utilidade práctica das taxonomías, descríbese como foron integradas no ciclo de vida de desenvolvemento de dous produtos reais no campo dos Sistemas Intelixentes. Concretamente, un dispositivo de Adaptación Intelixente da Velocidade e un xerador automático de interfaces de usuario. Nun punto específico de cada proxecto realizouse un estudo de usabilidade, usando as taxonomías para estruturar e guiar actividades de usabilidade como a análise dos requisitos de usabilidade, a avaliación heurística e a análise subxectiva

Defining Functional Illiteracy to Empower Inclusive Technology Design

Limited literacy presents a significant challenge in HCI research, yet the field lacks consistent definitions and measurement criteria. Researchers often interchange terms such as 'functional illiterates,' 'low literates,' and 'semi-literates,' further complicating the field. This paper conducts a systematic literature review (SLR) of 33 HCI studies, revealing concerns about the absence of a definition in 41% of the studies and the lack of measurement technique in 74%. Based on the results from our SLR and relevant research beyond HCI, we propose the following work-in-progress definition. 'Functional illiterates are motivated adults with some familiarity with text but insufficient to fully comprehend meanings and low skills in the measured digital skill, with enough language proficiency in the study language if they are literate in their native language. This understanding, coupled with addressing the identified issues, will empower the HCI4D community to design more inclusive technology solutions for functionally illiterate users in developing countries

A comparison of immersive realities and interaction methods: cultural learning in virtual heritage

In recent years, Augmented Reality (AR), Virtual Reality (VR), Augmented Virtuality (AV), and Mixed Reality (MxR) have become popular immersive reality technologies for cultural knowledge dissemination in Virtual Heritage (VH). These technologies have been utilized for enriching museums with a personalized visiting experience and digital content tailored to the historical and cultural context of the museums and heritage sites. Various interaction methods, such as sensor-based, device-based, tangible, collaborative, multimodal, and hybrid interaction methods, have also been employed by these immersive reality technologies to enable interaction with the virtual environments. However, the utilization of these technologies and interaction methods isn’t often supported by a guideline that can assist Cultural Heritage Professionals (CHP) to predetermine their relevance to attain the intended objectives of the VH applications. In this regard, our paper attempts to compare the existing immersive reality technologies and interaction methods against their potential to enhance cultural learning in VH applications. To objectify the comparison, three factors have been borrowed from existing scholarly arguments in the Cultural Heritage (CH) domain. These factors are the technology’s or the interaction method’s potential and/or demonstrated capability to: (1) establish a contextual relationship between users, virtual content, and cultural context, (2) allow collaboration between users, and (3) enable engagement with the cultural context in the virtual environments and the virtual environment itself. Following the comparison, we have also proposed a specific integration of collaborative and multimodal interaction methods into a Mixed Reality (MxR) scenario that can be applied to VH applications that aim at enhancing cultural learning in situ