Sensing with Earables: A Systematic Literature Review and Taxonomy of Phenomena

Earables have emerged as a unique platform for ubiquitous computing by augmenting ear-worn devices with state-of-the-art sensing. This new platform has spurred a wealth of new research exploring what can be detected on a wearable, small form factor. As a sensing platform, the ears are less susceptible to motion artifacts and are located in close proximity to a number of important anatomical structures including the brain, blood vessels, and facial muscles which reveal a wealth of information. They can be easily reached by the hands and the ear canal itself is affected by mouth, face, and head movements. We have conducted a systematic literature review of 271 earable publications from the ACM and IEEE libraries. These were synthesized into an open-ended taxonomy of 47 different phenomena that can be sensed in, on, or around the ear. Through analysis, we identify 13 fundamental phenomena from which all other phenomena can be derived, and discuss the different sensors and sensing principles used to detect them. We comprehensively review the phenomena in four main areas of (i) physiological monitoring and health, (ii) movement and activity, (iii) interaction, and (iv) authentication and identification. This breadth highlights the potential that earables have to offer as a ubiquitous, general-purpose platform

Sensing with Earables: A Systematic Literature Review and Taxonomy of Phenomena

Earables have emerged as a unique platform for ubiquitous computing by augmenting ear-worn devices with state-of-the-art sensing. This new platform has spurred a wealth of new research exploring what can be detected on a wearable, small form factor. As a sensing platform, the ears are less susceptible to motion artifacts and are located in close proximity to a number of important anatomical structures including the brain, blood vessels, and facial muscles which reveal a wealth of information. They can be easily reached by the hands and the ear canal itself is affected by mouth, face, and head movements. We have conducted a systematic literature review of 271 earable publications from the ACM and IEEE libraries. These were synthesized into an open-ended taxonomy of 47 different phenomena that can be sensed in, on, or around the ear. Through analysis, we identify 13 fundamental phenomena from which all other phenomena can be derived, and discuss the different sensors and sensing principles used to detect them. We comprehensively review the phenomena in four main areas of (i) physiological monitoring and health, (ii) movement and activity, (iii) interaction, and (iv) authentication and identification. This breadth highlights the potential that earables have to offer as a ubiquitous, general-purpose platform

Entwicklung einer berührungslosen EEG-Mütze mittels kapazitiver Elektroden

Non-contact capacitive electrodes for bioelectric diagnostics provide an interesting alternative to classical galvanically coupled electrodes. Such a low cost diagnostic system can be applied without preparation time and in mobile wireless environments. For even higher user comfort textile capacitive electrodes are preferable. In this work, a comprehensive model for the electronic noise properties and frequency dependent responses of PCB-based, as well as textile non-contact capacitive electrodes, is presented. A thorough study of the influence of the electrical components on the resulting noise properties of these electrodes, is provided by independently measuring the corresponding noise spectra. The most important low frequency noise source of capacitive electrode is the necessary high input bias resistance. By comparing the noise measurements with the theoretical noise model of the electrode, it is concluded that the surface of the electrode contributes to an additional 1/f-power noise. It is also found that the highest possible coupling capacitance is most favorable for low noise behavior. Therefore, we implemented electrodes with electrically conducting fabric surfaces. With these electrodes, it is possible to enlarge the surface of the electrode while simultaneously maintaining a small distance between the body and the electrode over the whole surface area, thus maximizing the capacitance. We also show that the use of textile capacitive electrodes, reduces the noise considerably. Furthermore, this thesis describes the construction of a capacitive non-contact textile electroencephalography measuring hat (cEEG hat) with seven measuring channels. This hat benefits from the low noise characteristics of the integrated developed textile capacitive electrodes. The measured noise spectrum of this cEEG hat shows low noise characteristics at low frequencies. This fulfills many requirements for measuring brain signals. The implemented cEEG hat is comfortable to wear during very long measurements and even during sleep periods. In contrast to common methods, the cEEG hat provides a possibility of measuring EEG signal during sleep outside laboratories and in the comfort of home. EEG sleep measurements shown in this work, are recorded inside a normal apartment. The possibility of brain computer interface application is also shown by measuring steady state visually evoked potentials (SSVEP) at different frequencies.Berührungslose, kapazitive Elektroden für bioelekrische Untersuchungen stellen eine interessante Alternative zu klassischen galvanisch gekoppelten Elektroden dar. Ein solches preisgünstiges Diagnosesystem kann ohne lange Vorbereitungszeit und in mobilen Umgebungen eingesetzt werden. Für gesteigerten Tragekomfort sind textile Elektroden von Vorteil. In dieser Arbeit wird eine umfassende Beschreibung der elektronischen Rauscheigenschaften und des frequenzabhängigen Verhaltens von sowohl platinenbasierten, als auch textilen kapazitiven Elektroden vorgestellt. Die Einflüsse aller elektronischen Komponenten auf die resultierenden Rauscheigenschaften werden durch Messungen der entsprechenden Rauschspektren untersucht. Die wichtigste niederfrequente Rauschquelle kapazitiver Elektroden stellt der notwendige und zugleich hohe Bias-Eingangswiderstand dar. Durch Vergleich der gemessenen Rauschspektren mit dem theoretischen Modell wird die Oberfläche der Elektroden als eine zusätzliche 1/f-Rauschquelle identifiziert. Dabei ist die größtmögliche Kopplungskapazität vorteilhaft für ein niedriges Rauschen. Deshalb setzen wir im Folgenden Elektroden aus elektrisch leitfähigen Textilien ein. Mit diesen Elektroden ist es möglich, die Oberfläche der Elektrode unter gleichzeitiger Beibehaltung eines kleinen Abstandes zum Körper zu vergrößern. Dies maximiert wiederum die Kapazität. Wir zeigen zudem, dass die Verwendung textiler kapazitiver Elektroden die Rauscheigenschaften deutlich verbessert. Desweiteren wird in dieser Arbeit die Konstruktion eines kapazitiven, berührungslosen EEG-Helmes (cEEG-Mütze) mit sieben Kanälen beschrieben. Dieser Helm profitiert von den guten Rauscheigenschaften der zuvor entwickelten und hier integrierten textilen Elektroden. Die gemessenen Rauschspektren zeigen ein niedriges Rauschen im unteren Frequenzbereich. Dies erfüllt viele Voraussetzungen für die Messung von Gehirnsignalen. Die erstellte cEEG-Mütze lässt sich während langer Messzeiten und Schlafperioden angenehm tragen. Im Gegensatz zu herkömmlichen Methoden ermöglicht sie Messungen außerhalb von Laboratorien und im gewohnten Umfeld. Alle in dieser Arbeit gezeigten Schlafmessungen wurden in einer normalen Wohnung aufgezeichnet. Außerdem wird die Einsatzmöglichkeit für sogenannte ”Gehirn-Computer-Schnittstellen” anhand der Messung von ”steady state visually evoked potentials” (SSVEP) Signalen bei verschiedenen Frequenzen demonstriert

Knowledge Base for MENTAL AI, in Data Science Context

Globally, 1 in 7 people has some kind of mental or substance use disorder that affects their thinking, feelings, and behaviour in everyday life. Mental well-being is vital for physical health. No Health Without Mental Health! People with mental health disorders can carry on with normal life if they get the proper treatment and support. Mental disorders are complex to diagnose due to similar and common symptoms for numerous types of mental illnesses, with a minute difference among them. In the era of big, the challenge stays to make sense of the huge amount of health research and care data. Computational methods hold significant potential to enable superior patient stratification approaches to the established clinical practice, which in turn are a pre-requirement for the development of effective personalized medicine approaches. Personalized psychiatry also plays a vital role in predicting mental disorders and improving diagnosis and optimized treatment. The use of intelligent systems is expected to grow in the medical field, and it will continue to pose abundant opportunities for solutions that can help save patients’ lives. As it does for many industries, Artificial Intelligence (AI) systems can support mental health specialists in their jobs. Machine learning algorithms can be applied to find different patterns in the most diverse sets of data. This work aims to examine and compare different machine learning classification methodologies to predict different mental disorders and, from that, extract knowledge that can help mental health professionals in their tasks. Our algorithms were trained using a total dataset of 3353 patients from different hospital units. These data are divided into three subsets of data, mainly by the characteristics that the pathologies present. We evaluate the performance of the algorithms using different metrics. Among the metrics applied, we chose the F1 score to compare and analyze the algorithms, as it is the most suitable for the data we have since they found themselves imbalances. In the first evaluation, we trained our models, using all the patient’s symptoms and diagnoses. In the second evaluation, we trained our models, using only the symptoms that were somehow related to each other and that influenced the other pathologies.Milhões de pessoas em todo o mundo são afetadas por transtornos mentais que influenciam o seu pensamento, sentimento ou comportamento. A saúde mental é um pré-requisito essencial para a saúde física e geral. Pessoas com transtornos mentais geralmente precisam de tratamento e apoio adequados para levar uma vida normal. A saúde mental é uma condição de bem-estar em que um indivíduo reconhece as suas habilidades, pode lidar com as tensões quotidianas da vida, trabalhar de forma produtiva e pode contribuir para a sua comunidade. A saúde mental afeta a vida das pessoas com transtorno mental, as suas profissões e a produtividade da comunidade. Boa saúde mental e resiliência são essenciais para a nossa saúde biológica, conexões humanas, educação, trabalho e alcançar o nosso potencial. A pandemia do covid-19 impactou significativamente a saúde mental das pessoas, em particular grupos como saúde e outros trabalhadores da linha de frente, estudantes, pessoas que moram sozinhas e pessoas com condições de saúde mental pré-existentes. Além disso, os serviços para transtornos mentais, neurológicos e por uso de substâncias foram significativamente interrompidos. Os transtornos mentais são classificados como de diagnóstico complexo devido à semelhança dos sintomas. Consultas regulares de saúde de pessoas com transtornos mentais graves podem impedir a morte prematura. A dificuldade dos especialistas em diagnosticar é geralmente causada pela semelhança dos sintomas nos transtornos mentais, como por exemplo, transtorno de bordeline e bipolar. Os algoritmos de aprendizado de máquina podem ser aplicados para encontrar diferentes padrões nos mais diversos conjuntos de dados. Este trabalho, visa examinar e comparar diferentes metodologias de classificação de aprendizado de máquina para prever difentes transtornos mentais e disso, extrair conhecimento que possam auxiliar os profissionais da area de saude mental, nas suas tarefas. Os nossos algoritmos, foram treinados utilizando um conjunto total de dados de 3353 pacientes, provenientes de diferentes unidades hospitalares. Esses dados, estão repartidos em três subconjuntos de dados, principalmente, pelas características que as patologias apresentam. Avaliamos o desempenho dos algoritmos usando diferentes métricas. Dentre as métricas aplicadas, escolhemos o F1 score para comparar e analisar os algoritmos, pois é o mais adequado para os dados que possuímos. Visto que eles se encontravam desequilíbrios. Na primeira avaliação, treinamos os nossos modelos, utilizando todos os sintomas e diagnósticos dos pacientes. Na segunda avaliação, treinamos os nossos modelos, utilizando apenas os sintomas que apresentavam alguma relação entre si e que influenciavam nas outras patologias

IEEE Transactions On Biomedical Circuits And Systems : Vol. 9, No. 6, December 2015

1. A. Wearable EEG-HEG-HRV Multimodal System with Simultaneous Monitoring of tES for Mental Health Management / A. Burdett, D. Ham, R. Genov 2. Miniaturizing Ultrasonic System for Portable Health Care and Fitness / H.-Y Tang, et al. 3. A Circanidian and Cardiac Intraocular Pressure Sensor for Smart Implantable Lens / A. Donida, et al. 4. A Smart CMOS Assay SoC for Rapid Blood Screening Test of Risk Prediction / P-H. Kuo, et al. 5. A Multi-Modality CMOS Sensor Array for Cell-Based Assay and Drug Screening /T. Chi, et al. Etc

Hybridizing 3-dimensional multiple object tracking with neurofeedback to enhance preparation, performance, and learning

Le vaste domaine de l’amélioration cognitive traverse les applications comportementales, biochimiques et physiques. Aussi nombreuses sont les techniques que les limites de ces premières : des études de pauvre méthodologie, des pratiques éthiquement ambiguës, de faibles effets positifs, des effets secondaires significatifs, des couts financiers importants, un investissement de temps significatif, une accessibilité inégale, et encore un manque de transfert. L’objectif de cette thèse est de proposer une méthode novatrice d’intégration de l’une de ces techniques, le neurofeedback, directement dans un paradigme d’apprentissage afin d’améliorer la performance cognitive et l’apprentissage. Cette thèse propose les modalités, les fondements empiriques et des données à l’appui de ce paradigme efficace d’apprentissage ‘bouclé’. En manipulant la difficulté dans une tâche en fonction de l’activité cérébrale en temps réel, il est démontré que dans un paradigme d’apprentissage traditionnel (3-dimentional multiple object tracking), la vitesse et le degré d’apprentissage peuvent être améliorés de manière significative lorsque comparés au paradigme traditionnel ou encore à un groupe de contrôle actif. La performance améliorée demeure observée même avec un retrait du signal de rétroaction, ce qui suggère que les effets de l’entrainement amélioré sont consolidés et ne dépendent pas d’une rétroaction continue. Ensuite, cette thèse révèle comment de tels effets se produisent, en examinant les corrélés neuronaux des états de préparation et de performance à travers les conditions d’état de base et pendant la tâche, de plus qu’en fonction du résultat (réussite/échec) et de la difficulté (basse/moyenne/haute vitesse). La préparation, la performance et la charge cognitive sont mesurées via des liens robustement établis dans un contexte d’activité cérébrale fonctionnelle mesurée par l’électroencéphalographie quantitative. Il est démontré que l’ajout d’une assistance- à-la-tâche apportée par la fréquence alpha dominante est non seulement appropriée aux conditions de ce paradigme, mais influence la charge cognitive afin de favoriser un maintien du sujet dans sa zone de développement proximale, ce qui facilite l’apprentissage et améliore la performance. Ce type de paradigme d’apprentissage peut contribuer à surmonter, au minimum, un des limites fondamentales du neurofeedback et des autres techniques d’amélioration cognitive : le manque de transfert, en utilisant une méthode pouvant être intégrée directement dans le contexte dans lequel l’amélioration de la performance est souhaitée.The domain of cognitive enhancement is vast, spanning behavioral, biochemical and physical applications. The techniques are as numerous as are the limitations: poorly conducted studies, ethically ambiguous practices, limited positive effects, significant side-effects, high financial costs, significant time investment, unequal accessibility, and lack of transfer. The purpose of this thesis is to propose a novel way of integrating one of these techniques, neurofeedback, directly into a learning context in order to enhance cognitive performance and learning. This thesis provides the framework, empirical foundations, and supporting evidence for a highly efficient ‘closed-loop’ learning paradigm. By manipulating task difficulty based on a measure of cognitive load within a classic learning scenario (3-dimentional multiple object tracking) using real-time brain activity, results demonstrate that over 10 sessions, speed and degree of learning can be substantially improved compared with a classic learning system or an active sham-control group. Superior performance persists even once the feedback signal is removed, which suggests that the effects of enhanced training are consolidated and do not rely on continued feedback. Next, this thesis examines how these effects occur, exploring the neural correlates of the states of preparedness and performance across baseline and task conditions, further examining correlates related to trial results (correct/incorrect) and task difficulty (slow/medium/fast speeds). Cognitive preparedness, performance and load are measured using well-established relationships between real-time quantified brain activity as measured by quantitative electroencephalography. It is shown that the addition of neurofeedback-based task assistance based on peak alpha frequency is appropriate to task conditions and manages to influence cognitive load, keeping the subject in the zone of proximal development more often, facilitating learning and improving performance. This type of learning paradigm could contribute to overcoming at least one of the fundamental limitations of neurofeedback and other cognitive enhancement techniques : a lack of observable transfer effects, by utilizing a method that can be directly integrated into the context in which improved performance is sought

Muscle activation patterns in shoulder impingement patients

Introduction: Shoulder impingement is one of the most common presentations of shoulder joint problems 1. It appears to be caused by a reduction in the sub-acromial space as the humerus abducts between 60o -120o – the 'painful arc'. Structures between the humeral head and the acromion are thus pinched causing pain and further pathology 2. Shoulder muscle activity can influence this joint space but it is unclear whether this is a cause or effect in impingement patients. This study aimed to observe muscle activation patterns in normal and impingement shoulder patients and determine if there were any significant differences. Method: 19 adult subjects were asked to perform shoulder abduction in their symptomatic arm and non-symptomatic. 10 of these subjects (age 47.9 ± 11.2) were screened for shoulder impingement, and 9 subjects (age 38.9 ± 14.3) had no history of shoulder pathology. Surface EMG was used to collect data for 6 shoulder muscles (Upper, middle and lower trapezius, serratus anterior, infraspinatus, middle deltoids) which was then filtered and fully rectified. Subjects performed 3 smooth unilateral abduction movements at a cadence of 16 beats of a metronome set at 60bpm, and the mean of their results was recorded. T-tests were used to indicate any statistical significance in the data sets. Significance was set at P<0.05. Results: There was a significant difference in muscle activation with serratus anterior in particular showing a very low level of activation throughout the range when compared to normal shoulder activation patterns (<30%). Middle deltoid recruitment was significantly reduced between 60-90o in the impingement group (30:58%).Trends were noted in other muscles with upper trapezius and infraspinatus activating more rapidly and erratically (63:25%; 60:27% respectively), and lower trapezius with less recruitment (13:30%) in the patient group, although these did not quite reach significance. Conclusion: There appears to be some interesting alterations in muscle recruitment patterns in impingement shoulder patients when compared against their own unaffected shoulders and the control group. In particular changes in scapula control (serratus anterior and trapezius) and lateral rotation (infraspinatus), which have direct influence on the sub-acromial space, should be noted. It is still not clear whether these alterations are causative or reactionary, but this finding gives a clear indication to the importance of addressing muscle reeducation as part of a rehabilitation programme in shoulder impingement patients