Heterogeneous recognition of bioacoustic signals for human-machine interfaces

Human-machine interfaces (HMI) provide a communication pathway between man and machine. Not only do they augment existing pathways, they can substitute or even bypass these pathways where functional motor loss prevents the use of standard interfaces. This is especially important for individuals who rely on assistive technology in their everyday life. By utilising bioacoustic activity, it can lead to an assistive HMI concept which is unobtrusive, minimally disruptive and cosmetically appealing to the user. However, due to the complexity of the signals it remains relatively underexplored in the HMI field. This thesis investigates extracting and decoding volition from bioacoustic activity with the aim of generating real-time commands. The developed framework is a systemisation of various processing blocks enabling the mapping of continuous signals into M discrete classes. Class independent extraction efficiently detects and segments the continuous signals while class-specific extraction exemplifies each pattern set using a novel template creation process stable to permutations of the data set. These templates are utilised by a generalised single channel discrimination model, whereby each signal is template aligned prior to classification. The real-time decoding subsystem uses a multichannel heterogeneous ensemble architecture which fuses the output from a diverse set of these individual discrimination models. This enhances the classification performance by elevating both the sensitivity and specificity, with the increased specificity due to a natural rejection capacity based on a non-parametric majority vote. Such a strategy is useful when analysing signals which have diverse characteristics, false positives are prevalent and have strong consequences, and when there is limited training data available. The framework has been developed with generality in mind with wide applicability to a broad spectrum of biosignals. The processing system has been demonstrated on real-time decoding of tongue-movement ear pressure signals using both single and dual channel setups. This has included in-depth evaluation of these methods in both offline and online scenarios. During online evaluation, a stimulus based test methodology was devised, while representative interference was used to contaminate the decoding process in a relevant and real fashion. The results of this research provide a strong case for the utility of such techniques in real world applications of human-machine communication using impulsive bioacoustic signals and biosignals in general

Makine öğrenmesi algoritmaları kullanılarak glossokinetik potansiyel tabanlı dil - makine arayüzü tasarımı

06.03.2018 tarihli ve 30352 sayılı Resmi Gazetede yayımlanan “Yükseköğretim Kanunu İle Bazı Kanun Ve Kanun Hükmünde Kararnamelerde Değişiklik Yapılması Hakkında Kanun” ile 18.06.2018 tarihli “Lisansüstü Tezlerin Elektronik Ortamda Toplanması, Düzenlenmesi ve Erişime Açılmasına İlişkin Yönerge” gereğince tam metin erişime açılmıştır.Yardımcı teknolojiler, ağır engelli bireylerin diğer aygıtlara veya bireylere niyetlerini iletmelerini sağlayabilir. Bu teknolojiler, bireylerin sürekli yardım alma ihtiyacını kolaylaştırarak, aile üyelerinin yükünü ve sağlık maliyetlerini azaltacaktır. Omurilik yaralanmalarında veya amiyotrofik lateral sklerozda, engelli insanlar dış dünyayla sınırlı derecede iletişim kurabilirler. Bu tez çalışmasında, makine öğrenmesi algoritmalarını uygulayarak 1-boyutlu hareketler içeren yardımcı teknolojileri kontrol etmek için glossokinetik potansiyel (GKP) tabanlı dil-makine arayüzü geliştirilmiştir. GKP sinyalleri, dil hareket bilgilerini içeren elektrik sinyalleridir. Tez çalışmasında GKP sinyalleri, deneysel düzenler içinde dil ucunun yanak duvarlarıyla teması sırasında kafa derisine yerleştirilen elektrotlarla ölçülmüştür. İnsan vücudunun en esnek organlarından biri olan dil, yardımcı teknolojiler alanında çalışan araştırmacılar tarafından ileri motor kontrol görevlerine aday olarak kabul edilmiştir. Dil, omurilik yaralanmaları ve çoğu sinir-kas bozukluklarında bile genellikle ağır hasarlardan kaçabilir ve beyine hipoglosal kraniel sinir yoluyla bağlanır. Bu nedenle, yüksek düzeyde omurilik yaralanması olan felçli kişiler bile, dil kontrol yeteneklerini korurlar. Bununla beraber dil, çok fazla çaba gerektirmeden ağız boşluğu içinde hızlı ve doğru bir şekilde hareket edebilir. Dahası, bu yetenekli organ, ağız boşluğu içinde olmasından dolayı engelli bireyler için mahremiyet sağlayabilir. Dil-makine arayüzlerini kullanan araştırma çalışmalarının çoğu, ağız boşluğu içinde ve baş çevresinde rahatsızlık veren, hijyenik olmayan ekipmanlara sahiptir. Ancak, bu tez çalışması, engelli insanlara yardımcı cihazları doğal, rahatsızlık vermeyen, hızlı ve güvenilir bir şekilde kontrol etmeye hizmet edebilir. Çalışmada, geleneksel makine öğrenmesi algoritmaları ve konvolüsyonel yapay sinir ağı kullanarak sırasıyla %99 ve %100'e ulaşan sınıflandırma doğrulukları elde edilmiş ve yöntemlerin karşılaştırmalı analizi yapılmıştır. Zaman alanı ve frekans alanı özellik çıkarma metotlarının yanı sıra ayrık dalgacık dönüşümü, temel bileşen analizi ve bağımsız bileşen analizi sinyal işleme teknikleri de kullanılmıştır. Ayrıca, glossokinetik potansiyel tabanlı dil-makine arayüzü, elektroensefalografi (EEG) sinyallerinden kaynaklanan önemli yetersizlikleri içeren geleneksel EEG tabanlı beyin-bilgisayar arayüzleri için alternatif veya yardımcı kontrol ve iletişim kanalı olabileceği beklenmektedir.Assistive technologies (ATs) can enable severely disabled individuals to communicate their intentions to other devices or individuals. These technologies will ease the burden on family members and health costs by facilitating the need for continuous help for individuals. In spinal cord injuries (SCIs) or amyotrophic lateral sclerosis (ALS), diasabled people can communicate with the external world to a limited degree. In this thesis study, we have developed glossokinetic potential (GKP) based tongue-machine interface (TMI) to control assistive technologies for 1-D movements via implementing machine learning algorithms. GKP signals are electrical signals that consist of information on tongue movements. In the thesis study, GKP signals were measured by electrodes placed on the scalp during contact of the tongue tip and buccal walls in the experimental setups. Tongue, one of the most flexible organs of the human body, has been accepted as a candidate for advanced motor control tasks by researchers in the field of assistive technologies. The tongue is connected to the brain via the hypoglossal cranial nerve and can generally escape severe damages in SCIs and most neuromuscular disorders. Hence, high-level SCIs still maintain intact tongue control capabilities. Then the tongue is able move quickly and accurately without so much effort. Moreover, this gifted organ may provide privacy for paralytics because in the oral cavity. Most of the research using TMIs have obtrusive, unhygienic pieces of equipment in the oral cavity and around the headset. However, this dissertation may serve disabled people to control assistive technologies in natural, unobtrusive, speedy and reliable manner. In the study, traditional machine learning algorithms and convolutional neural network were used and classification accuracies of %99 and %100 were achieved respectively. And then comparative analysis of the algorithms was performed. In addition to time domain and frequency domain feature extraction methods, discrete wavelet transform, principal component analysis and independent component analysis signal processing techniques were also used. Moreover, it is expected that GKP-based TMI could be alternative or partner control and communication channel for traditional electroencephalography (EEG)-based brain-computer interfaces (BCIs) which involve significant inadequacies arisen from the EEG signals

Models and Analysis of Vocal Emissions for Biomedical Applications

The International Workshop on Models and Analysis of Vocal Emissions for Biomedical Applications (MAVEBA) came into being in 1999 from the particularly felt need of sharing know-how, objectives and results between areas that until then seemed quite distinct such as bioengineering, medicine and singing. MAVEBA deals with all aspects concerning the study of the human voice with applications ranging from the neonate to the adult and elderly. Over the years the initial issues have grown and spread also in other aspects of research such as occupational voice disorders, neurology, rehabilitation, image and video analysis. MAVEBA takes place every two years always in Firenze, Italy. This edition celebrates twenty years of uninterrupted and succesfully research in the field of voice analysis

2022 - The Third Annual Fall Symposium of Student Scholars

The full program book from the Fall 2022 Symposium of Student Scholars, held on November 17, 2022. Includes abstracts from the presentations and posters.https://digitalcommons.kennesaw.edu/sssprograms/1026/thumbnail.jp

The role of phonology in visual word recognition: evidence from Chinese

Posters - Letter/Word Processing V: abstract no. 5024The hypothesis of bidirectional coupling of orthography and phonology predicts that phonology plays a role in visual word recognition, as observed in the effects of feedforward and feedback spelling to sound consistency on lexical decision. However, because orthography and phonology are closely related in alphabetic languages (homophones in alphabetic languages are usually orthographically similar), it is difficult to exclude an influence of orthography on phonological effects in visual word recognition. Chinese languages contain many written homophones that are orthographically dissimilar, allowing a test of the claim that phonological effects can be independent of orthographic similarity. We report a study of visual word recognition in Chinese based on a mega-analysis of lexical decision performance with 500 characters. The results from multiple regression analyses, after controlling for orthographic frequency, stroke number, and radical frequency, showed main effects of feedforward and feedback consistency, as well as interactions between these variables and phonological frequency and number of homophones. Implications of these results for resonance models of visual word recognition are discussed.postprin

Interactive effects of orthography and semantics in Chinese picture naming

Posters - Language Production/Writing: abstract no. 4035Picture-naming performance in English and Dutch is enhanced by presentation of a word that is similar in form to the picture name. However, it is unclear whether facilitation has an orthographic or a phonological locus. We investigated the loci of the facilitation effect in Cantonese Chinese speakers by manipulating—at three SOAs (2100, 0, and 1100 msec)—semantic, orthographic, and phonological similarity. We identified an effect of orthographic facilitation that was independent of and larger than phonological facilitation across all SOAs. Semantic interference was also found at SOAs of 2100 and 0 msec. Critically, an interaction of semantics and orthography was observed at an SOA of 1100 msec. This interaction suggests that independent effects of orthographic facilitation on picture naming are located either at the level of semantic processing or at the lemma level and are not due to the activation of picture name segments at the level of phonological retrieval.postprin

Electrostimulation Contingencies and Attention, Electrocortical Activity and Neurofeedback

There is a growing body of evidence for diverse ways of modulating neuronal processing to improve cognitive performance. These include brain-based feedback, self-regulation techniques such as EEG-neurofeedback, and stimulation strategies, alone or in combination. The thesis goal was to determine whether a combined strategy would have advantages for normal cognitive function; specifically operant control of EEG activity in combination with transcutaneous electro-acustimulation. In experiment one the association between transcutaneous electroacustimulation (EA) and improved perceptual sensitivity was demonstrated with a visual GO/NOGO attention task (Chen et al, 2011). Furthermore reduced commission errors were related to an electrocortical motor inhibition component during and after alternating high and low frequency EA, whereas habituation in the control group with sham stimulation was related to different independent components. Experiment two applied frequency-domain ICA to detect changes in EEG power spectra from the eyes-closed to the eyes-open state (Chen et al, 2012). A multiple step approach was provided for analysing the spatiotemporal dynamics of default mode and resting state networks of cerebral EEG sources, preferable to conventional scalp EEG data analysis. Five regions were defined, compatible with fMRI studies. In experiment three the EA approach of Exp I was combined with sensorimotor rhythm (SMR) neurofeedback. SMR training improved perceptual sensitivity, an effect not found in a noncontingent feedback group. However, non-significant benefits resulted from EA. With ICA spectral power analysis changes in frontal beta power were associated with contingent SMR training. Possible long-term effects on an attention network in the resting EEG were also found after SMR training, compared with mock SMR training. In conclusion, this thesis has supplied novel evidence for significant cognitive and electrocortical effects of neurofeedback training and transcutaneous electro-acustimulation in healthy humans. Possible implications of these findings and suggestions for future research are considered

Visual Impairment and Blindness

Blindness and vision impairment affect at least 2.2 billion people worldwide with most individuals having a preventable vision impairment. The majority of people with vision impairment are older than 50 years, however, vision loss can affect people of all ages. Reduced eyesight can have major and long-lasting effects on all aspects of life, including daily personal activities, interacting with the community, school and work opportunities, and the ability to access public services. This book provides an overview of the effects of blindness and visual impairment in the context of the most common causes of blindness in older adults as well as children, including retinal disorders, cataracts, glaucoma, and macular or corneal degeneration