Auditory and Spatial Assessment in Inattentive Children Using Smart Devices and Gesture Interaction

We present an interactive game for assessing auditory and spatial memory. We compared the performance of children with and without inattention using the game and using a variety of classical tools for assessment of auditory and spatial memory. The children with inattention showed statistically worse performance in the game. There were no statistically significant differences between the two groups for the satisfaction and interaction outcomes. Therefore, our game could be a good tool for distinguishing performances of children with and without inattention

The design and development of motion detection edutainment maths for use with slow learners’ children

This research is aimed to examine game-based motion detection technology in helping slow learners’ children to improve and enhance their levels of attention and concentration while learning mathematics. The study also aims to explore that game-based motion detection engage slow learners’ children while learning mathematics. Additionally, the current study examined the role of game-based motion detection in improving the attention and concentration of slow learners’ children as compared to normal healthy students, in terms of learning mathematics and the educational outcomes of such classes Slow learners are considering a wide different range of students who are not performing well in their study. These group could be including ADHD, Autism, impulsive students, inattention and many more. In this research, I have designed and developed a motion-based mathematic game using Kinect Xbox to test and check the effectiveness and efficiency of slow learners’ students in compare with normal students. For the above purpose, the game has been designed based on several learning theories such as Mayer Principles of learning, Kolb’s Learning style and Piaget Theory for K5 and grade 6-8 years old. In experiment design, I have used System Usability Scale (SUS) to rate the features and PACES, Physical Activity Enjoyment Scale have been used for experimentation of the participants. For testing, both qualitative and quantitative model have been accomplished. Qualitative model was based on the feedback from expert teachers who are observing the students and quantitative model was based on demographic analysis, normality test, reliability analysis and validity test. The outcome illustrates the value of game-based instruction, in specific physical activities and their impact on children's mathematics. The current study findings highlighted the suitability, needfulness, attention, and enhanced learning through game based instructional design for slow learners. The study answers the advantages of using game based instructional design for the slow learner students

Embodied interaction with visualization and spatial navigation in time-sensitive scenarios

Paraphrasing the theory of embodied cognition, all aspects of our cognition are determined primarily by the contextual information and the means of physical interaction with data and information. In hybrid human-machine systems involving complex decision making, continuously maintaining a high level of attention while employing a deep understanding concerning the task performed as well as its context are essential. Utilizing embodied interaction to interact with machines has the potential to promote thinking and learning according to the theory of embodied cognition proposed by Lakoff. Additionally, the hybrid human-machine system utilizing natural and intuitive communication channels (e.g., gestures, speech, and body stances) should afford an array of cognitive benefits outstripping the more static forms of interaction (e.g., computer keyboard). This research proposes such a computational framework based on a Bayesian approach; this framework infers operator\u27s focus of attention based on the physical expressions of the operators. Specifically, this work aims to assess the effect of embodied interaction on attention during the solution of complex, time-sensitive, spatial navigational problems. Toward the goal of assessing the level of operator\u27s attention, we present a method linking the operator\u27s interaction utility, inference, and reasoning. The level of attention was inferred through networks coined Bayesian Attentional Networks (BANs). BANs are structures describing cause-effect relationships between operator\u27s attention, physical actions and decision-making. The proposed framework also generated a representative BAN, called the Consensus (Majority) Model (CMM); the CMM consists of an iteratively derived and agreed graph among candidate BANs obtained by experts and by the automatic learning process. Finally, the best combinations of interaction modalities and feedback were determined by the use of particular utility functions. This methodology was applied to a spatial navigational scenario; wherein, the operators interacted with dynamic images through a series of decision making processes. Real-world experiments were conducted to assess the framework\u27s ability to infer the operator\u27s levels of attention. Users were instructed to complete a series of spatial-navigational tasks using an assigned pairing of an interaction modality out of five categories (vision-based gesture, glove-based gesture, speech, feet, or body balance) and a feedback modality out of two (visual-based or auditory-based). Experimental results have confirmed that physical expressions are a determining factor in the quality of the solutions in a spatial navigational problem. Moreover, it was found that the combination of foot gestures with visual feedback resulted in the best task performance (p\u3c .001). Results have also shown that embodied interaction-based multimodal interface decreased execution errors that occurred in the cyber-physical scenarios (p \u3c .001). Therefore we conclude that appropriate use of interaction and feedback modalities allows the operators maintain their focus of attention, reduce errors, and enhance task performance in solving the decision making problems

The design and development of motion detection edutainment maths for use with slow learners’ children

This research is aimed to examine game-based motion detection technology in helping slow learners’ children to improve and enhance their levels of attention and concentration while learning mathematics. The study also aims to explore that game-based motion detection engage slow learners’ children while learning mathematics. Additionally, the current study examined the role of game-based motion detection in improving the attention and concentration of slow learners’ children as compared to normal healthy students, in terms of learning mathematics and the educational outcomes of such classes Slow learners are considering a wide different range of students who are not performing well in their study. These group could be including ADHD, Autism, impulsive students, inattention and many more. In this research, I have designed and developed a motion-based mathematic game using Kinect Xbox to test and check the effectiveness and efficiency of slow learners’ students in compare with normal students. For the above purpose, the game has been designed based on several learning theories such as Mayer Principles of learning, Kolb’s Learning style and Piaget Theory for K5 and grade 6-8 years old. In experiment design, I have used System Usability Scale (SUS) to rate the features and PACES, Physical Activity Enjoyment Scale have been used for experimentation of the participants. For testing, both qualitative and quantitative model have been accomplished. Qualitative model was based on the feedback from expert teachers who are observing the students and quantitative model was based on demographic analysis, normality test, reliability analysis and validity test. The outcome illustrates the value of game-based instruction, in specific physical activities and their impact on children's mathematics. The current study findings highlighted the suitability, needfulness, attention, and enhanced learning through game based instructional design for slow learners. The study answers the advantages of using game based instructional design for the slow learner students

Multimodal Data Analysis of Dyadic Interactions for an Automated Feedback System Supporting Parent Implementation of Pivotal Response Treatment

abstract: Parents fulfill a pivotal role in early childhood development of social and communication skills. In children with autism, the development of these skills can be delayed. Applied behavioral analysis (ABA) techniques have been created to aid in skill acquisition. Among these, pivotal response treatment (PRT) has been empirically shown to foster improvements. Research into PRT implementation has also shown that parents can be trained to be effective interventionists for their children. The current difficulty in PRT training is how to disseminate training to parents who need it, and how to support and motivate practitioners after training. Evaluation of the parents’ fidelity to implementation is often undertaken using video probes that depict the dyadic interaction occurring between the parent and the child during PRT sessions. These videos are time consuming for clinicians to process, and often result in only minimal feedback for the parents. Current trends in technology could be utilized to alleviate the manual cost of extracting data from the videos, affording greater opportunities for providing clinician created feedback as well as automated assessments. The naturalistic context of the video probes along with the dependence on ubiquitous recording devices creates a difficult scenario for classification tasks. The domain of the PRT video probes can be expected to have high levels of both aleatory and epistemic uncertainty. Addressing these challenges requires examination of the multimodal data along with implementation and evaluation of classification algorithms. This is explored through the use of a new dataset of PRT videos. The relationship between the parent and the clinician is important. The clinician can provide support and help build self-efficacy in addition to providing knowledge and modeling of treatment procedures. Facilitating this relationship along with automated feedback not only provides the opportunity to present expert feedback to the parent, but also allows the clinician to aid in personalizing the classification models. By utilizing a human-in-the-loop framework, clinicians can aid in addressing the uncertainty in the classification models by providing additional labeled samples. This will allow the system to improve classification and provides a person-centered approach to extracting multimodal data from PRT video probes.Dissertation/ThesisDoctoral Dissertation Computer Science 201

Safe and Sound: Proceedings of the 27th Annual International Conference on Auditory Display

Complete proceedings of the 27th International Conference on Auditory Display (ICAD2022), June 24-27. Online virtual conference

The Effect of Hearing Impairment on Word Processing of Infant- and Adult-Directed Speech

Objective. Little is known about how children with hearing loss (CHL) process words. The Emergent Coalition Model (ECM) of early word learning proposes that multiple cues (e.g., perceptual, social, linguistic) are used to facilitate word learning. Because hearing loss influences speech perception, different word learning patterns may emerge in CHL relative to children with normal hearing (CNH). One perceptual cue used by young children to access word learning is infant-directed-speech (IDS). Specifically, twenty-one month-olds can learn words in IDS but not in adult directed speech (ADS); however, by 27 months children can learn words in ADS. Currently, it is unknown how CHL process words in IDS and ADS. This study examined how CHL and CNH process familiar and novel words in IDS and ADS. A Looking-WhileListening paradigm was used. We predicted that: 1) CNH would show faster reaction time (RT) and higher accuracy than CHL, 2) word processing may show different patterns for familiar versus novel words, and 3) vocabulary size would be correlated with word processing skills. Methods. Eleven children with bilateral, sensorineural hearing loss (M=32.48 months) using hearing aids or cochlear implants, and 11 CNH, matched for age, gender, and SES participated. Each child was tested in IDS and ADS on different days. At each visit, children were trained to map two novel labels to objects, counterbalanced across visits. Following training, accuracy and RT were assessed for both novel and familiar words. Vocabulary size was assessed using the McArthur-Bates Communicative Development Inventory. Results. In the familiar word condition, for the CHL accuracy was significantly better in IDS than ADS, and RT was faster in IDS than ADS (but not significant). For CNH, accuracy was not different in IDS than in ADS, but RT was significantly faster in ADS than IDS. A significant speech type by group interaction was found (p \u3c.05), for both accuracy and RT. Follow-up tests showed that CNH have higher accuracy and faster RTs than CHL. Results for familiar words suggest that while IDS may lead to more efficient speech processing for CHL, CNH are more efficient at processing ADS. In the novel word condition, only 10 CHL completed the task. For CHL, accuracy was marginally better in IDS than ADS, but no significant difference was observed in RT. For CNH no differences were seen in accuracy or RT between IDS and ADS. Analysis of Variance for RT showed that CNH have significantly faster RTs than CHL for novel word processing. For CHL, vocabulary size was negatively correlated with RT to familiar words in IDS and ADS, suggesting that children with larger vocabularies processed familiar words faster than children with smaller vocabularies. For the CHN, vocabulary size was marginally correlated with accuracy and RT to novel words in ADS. Conclusions. This study demonstrates 1) the facilitative effects of IDS on word processing for young CHL, and 2) the relationship between word processing and expressive vocabulary in young children, suggesting that children with larger vocabularies are faster and more efficient at word processing tasks. The present findings suggest that CHL do not perform as well as their normal hearing peers on word processing tasks in ADS. These findings provide empirical evidence that childhood hearing loss affects processing of IDS and ADS differently than for CNH

Models and Analysis of Vocal Emissions for Biomedical Applications

The International Workshop on Models and Analysis of Vocal Emissions for Biomedical Applications (MAVEBA) came into being in 1999 from the particularly felt need of sharing know-how, objectives and results between areas that until then seemed quite distinct such as bioengineering, medicine and singing. MAVEBA deals with all aspects concerning the study of the human voice with applications ranging from the newborn to the adult and elderly. Over the years the initial issues have grown and spread also in other fields of research such as occupational voice disorders, neurology, rehabilitation, image and video analysis. MAVEBA takes place every two years in Firenze, Italy. This edition celebrates twenty-two years of uninterrupted and successful research in the field of voice analysis

An immersive virtual reality task with physical movement for the assessment of spatial memory

Tesis por compendioLa Realidad Virtual (RV) aún no se ha explotado en la evaluación de la memoria espacial. Los sistemas actuales de RV para la evaluación de la memoria espacial incluyen interacción mediante el uso de dispositivos tradicionales (por ejemplo, pantallas y teclados). Este tipo de interacción clásica con el sistema es ineficaz porque la sensación de presencia del usuario es muy pobre. Si el usuario no tiene la sensación de "estar ahí" en el entorno virtual, es posible que la memoria espacial no se evalúe correctamente. Por tanto, desarrollar un sistema que permita a los participantes estar activos en el entorno virtual, podría evaluar la memoria espacial de la misma forma que se puede evaluar en un entorno real. El objetivo principal de esta tesis fue diseñar, desarrollar y validar un sistema inmersivo de RV en el que el usuario pueda interactuar mediante movimiento físico. El entorno virtual se diseñó a partir de una ciudad con forma cuadrada. La tarea cognitiva, basada en dicho entorno, constaba de seis niveles. El objetivo de los niveles para la tarea principal era evaluar la memoria a corto plazo de los niños para la localización de objetos en el entorno virtual. El sistema de RV incorporó dos tipos de interacción. 1) Una condición física activa (caminar físicamente sobre una Wii Balance Board y cambiar la dirección girando un volante inalámbrico). 2) Una condición inactiva (estando de pie y usando un gamepad). Para la visualización de la tarea, se utilizó una pantalla grande y estéreo. Para la validación, se realizaron dos estudios (N=212). El rendimiento de nuestra tarea se comparó con métodos tradicionales. Se encontraron correlaciones entre nuestra tarea y los métodos tradicionales, lo que indica que nuestra tarea ha demostrado ser una herramienta válida para evaluar la memoria espacial a corto plazo en niños. Con respecto al tipo de interacción, los resultados mostraron que no hubo diferencias estadísticamente significativas con respecto a la puntuación obtenida en nuestra tarea y en función de la interacción utilizada. Éste es un buen resultado porque significa que la tarea es adecuada para la evaluación de la memoria espacial y que los dos tipos de interacción se pueden utilizar para este fin. Con respecto a las diferencias de género en la puntuación de la tarea y Corsi Block Tapping Test, los resultados indicaron que no hubo diferencias estadísticamente significativas para el género. Con respecto a facilidad de uso y satisfacción, se demostró que el uso de la condición inactiva no difirió significativamente de la condición física activa para las preguntas de usabilidad y satisfacción. Previamente al desarrollo del sistema de RV mencionado, desarrollamos un sistema de RV, con interfaces de usuario naturales (NUI) y una pantalla autoestereoscópica, para aprendizaje dental. El sistema incluye dos modos: fondo neutro y mundo real. Este sistema fue validado con 33 estudiantes. Con este primer desarrollo, se adquirió el conocimiento necesario para poder afrontar el segundo desarrollo, núcleo de la tesis. Las siguientes conclusiones generales se extrajeron de los 2 desarrollos y los 3 estudios: Aprendizaje - Los sistemas de RV con autoestereoscopía, con diferentes fondos y NUI, han demostrado ser herramientas eficaces para aprender la morfología dental - Con este tipo de sistemas, los niños pueden aprender y, al mismo tiempo, pueden divertirse - La estereoscopía y NUI son apropiadas para el desarrollo de juegos educativos y pueden ser explotadas en su desarrollo Memoria espacial - Los sistemas de RV, con estereoscopía y dos interfaces de usuario diferentes, han demostrado ser herramientas fiables y efectivas para evaluar la memoria espacial en niños - Con este tipo de sistemas, los niños pueden ser evaluados mientras se divierten - Nuestra tarea y tareas similares podrían usarse para evaluación y entrenamiento de la memoria en niños y adultosLa Realitat Virtual (RV) encara no s'ha explotat en l'avaluació de la memòria espacial. Els sistemes actuals de RV per a l'avaluació de la memòria espacial inclouen interacció mitjançant l'ús de dispositius tradicionals (per exemple, pantalles, ratolins o teclats). Aquest tipus d'interacció clàssica amb el sistema és ineficaç perquè la sensació de presència de l'usuari és molt pobre. Si l'usuari no té la sensació de "ser-hi" en l'entorn virtual, és possible que la memòria espacial no siga avaluada correctament. Per tant, desenvolupar un sistema que permeta als participants estar actius en l'entorn virtual, podria avaluar la memòria espacial de la mateixa manera que es pot avaluar en un entorn real. L'objectiu principal d'aquesta tesi va ser dissenyar, desenvolupar i validar un sistema immersiu de RV en el qual l'usuari puga interactuar mitjançant moviment físic. L'entorn virtual es va dissenyar a partir d'una ciutat amb forma quadrada. La tasca cognitiva constava de sis nivells. L'objectiu dels nivells per a la tasca principal era avaluar la memòria a curt termini dels xiquets per a la localització d'objectes en l'entorn virtual. El sistema de RV va incorporar dos tipus d'interacció. 1) Una condició física activa (caminar físicament sobre una Wii Balance Board i canviar la direcció girant un volant inalàmbric). 2) Una condició inactiva (estar dret i fent servir un gamepad). Per a la visualització de la tasca, es va utilitzar una pantalla gran i estèreo. Per a la validació, es van realitzar dos estudis (N=212). El rendiment de la nostra tasca es va comparar amb mètodes tradicionals. Es van trobar correlacions entre la nostra tasca i els mètodes tradicionals, el que indica que la nostra tasca ha demostrat ser una eina vàlida per avaluar la memòria espacial a curt termini en xiquets. Pel que fa al tipus d'interacció, els resultats van mostrar que no hi va haver diferències estadísticament significatives respecte a la puntuació obtinguda en la nostra tasca i en funció de la interacció utilitzada. Aquest és un bon resultat perquè significa que la tasca és adequada per a l'avaluació de la memòria espacial i que els dos tipus d'interacció es poden utilitzar per a aquest fi. Pel que fa a les diferències de gènere en la puntuació de la tasca i Corsi Block Tapping Test, els resultats van indicar que no hi va haver diferències estadísticament significatives per al gènere. Pel que fa a facilitat d'ús i satisfacció, els nostres estudis han demostrat que l'ús de la condició inactiva no va diferir significativament de la condició física activa per a les preguntes d'usabilitat i satisfacció. Prèviament al desenvolupament del sistema de RV esmentat, vam desenvolupar un sistema de RV, amb interfícies d'usuari naturals (NUI) i una pantalla autoestereoscòpica, per aprenentatge dental. El sistema inclou dos tipus de fons: fons neutre i món real. Aquest sistema va ser validat amb 33 estudiants. Amb aquest primer desenvolupament, es va adquirir el coneixement necessari per poder afrontar el segon desenvolupament, nucli de la tesi. Les següents conclusions generals es van extraure dels dos desenvolupaments i els tres estudis: Aprenentatge - Els sistemes de RV amb autoestereoscòpia, amb diferents fons i NUI, han demostrat ser eines eficaces per a aprendre la morfologia dental - Amb aquest tipus de sistemes, els xiquets poden aprendre i, al mateix temps, poden divertir-se - La estereoscòpia i NUI són apropiades per al desenvolupament de jocs educatius i poden ser explotades en el seu desenvolupament Memòria espacial - Els sistemes de RV, amb estereoscòpia i dues interfícies d'usuari diferents, han demostrat ser eines fiables i efectives per avaluar la memòria espacial en xiquets - Amb aquest tipus de sistemes, els xiquets poden ser avaluats mentre es diverteixen - La nostra tasca i tasques similars podrien usar-se per avaluació i entrenament de la memòria en xiquet i adultsVirtual Reality (VR) has not yet been exploited in the assessment of spatial memory. Current VR systems for the assessment of spatial memory include interaction by using traditional devices (e.g., computer screens, mouses or keyboards). This classical type of interaction with the system is ineffective because the user's sense of presence is very poor. If the user does not have the feeling of "being there" in the virtual environment, the spatial memory may not be assessed correctly. The user's physical movement can contribute to a high level of presence. Therefore, by developing a system that allows the participants to become active in a virtual environment, spatial memory can be assessed as the same way it could be evaluated in a real environment. The main objective of this thesis was to design, develop and validate an immersive VR system in which the user could interact by physical movements. The virtual environment was designed based on a city square. The cognitive task, based on this environment, comprised six levels. The goal of the levels for the main task was to assess children short-term memory for object location in the virtual environment. The VR system incorporated two types of interaction. 1) A physical active condition (physically walking on a Wii Balance Board and changing the direction by turning a wireless steering wheel). 2) An inactive condition (stand up and use a gamepad). For the visualization of the task, a large stereo screen was used. For the validation, two studies were carried out. The performance of our task was compared with traditional methods (the Corsi Block Tapping Test). We carried out two studies involving 212 children. Correlations were found between our task and traditional methods, indicating that our task has proven to be a valid tool for assessing spatial short-term memory in children. With regard to the interaction type, the results showed that there were no statistically significant differences regarding the score obtained in our task based on the interaction used (inactive condition vs. physical active condition). Although unexpected, this is a good result because it means that the task is well suited for the assessment of spatial memory and that the two interaction types can be used for this purpose. With regard to gender differences in the task score and the Corsi Block Tapping Test, the results indicated that there were no statistically significant differences for gender. With regard to usability and satisfaction, our studies have shown that the use of the inactive condition did not differ significantly from the physical active condition for the usability and satisfaction questions. Previously to the development of the mentioned VR system, we developed a VR system, with Natural User Interfaces (NUI) and an autostereoscopic screen, for dental learning. The system included two modes: neutral and real world background. This system was validated with 33 dentistry students. With this first development, the required knowledge for facing the second development, core of the thesis, was acquired. The following general conclusions were extracted from the two developments and the three studies: Learning - Autostereoscopic VR systems, with different background modes and NUI, have proven to be effective tools for learning teeth morphology - With this type of systems, children can learn and at the same time, they can have a good time - Stereoscopy and NUI are appropriated for developing educational games and they can be exploited in their development Spatial memory - VR systems, with stereoscopy and two different user interfaces (inactive and physical active conditions), have proven to be reliable and effective tools to assess spatial memory in children - With this type of systems, the children can be assessed meanwhile are having a good time - Our task and similar tasks could be used for assessment and training of spatial memory in children and adultsRodríguez Andrés, D. (2018). An immersive virtual reality task with physical movement for the assessment of spatial memory [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/114823TESISCompendi