Estimation of the Shoulder Joint Angle using Brainwaves

This paper presents the angle of the shoulder joint as basic research for developing a machine interface using EEG. The raw EEG voltage signals and power density spectrum of the voltage value were used as the learning feature. Hebbian learning was used on a multilayer perceptron network for pattern classification for the estimation of joint angles   0o, 90o and 180o of the shoulder joint. Experimental results showed that it was possible to correctly classify up to 63.3% of motion using voltage values of the raw EEG signal with the neural network. Further, with selected electrodes and power density spectrum features, accuracy rose to 93.3% with more stable motion estimation

Adaptive extreme edge computing for wearable devices

Wearable devices are a fast-growing technology with impact on personal healthcare for both society and economy. Due to the widespread of sensors in pervasive and distributed networks, power consumption, processing speed, and system adaptation are vital in future smart wearable devices. The visioning and forecasting of how to bring computation to the edge in smart sensors have already begun, with an aspiration to provide adaptive extreme edge computing. Here, we provide a holistic view of hardware and theoretical solutions towards smart wearable devices that can provide guidance to research in this pervasive computing era. We propose various solutions for biologically plausible models for continual learning in neuromorphic computing technologies for wearable sensors. To envision this concept, we provide a systematic outline in which prospective low power and low latency scenarios of wearable sensors in neuromorphic platforms are expected. We successively describe vital potential landscapes of neuromorphic processors exploiting complementary metal-oxide semiconductors (CMOS) and emerging memory technologies (e.g. memristive devices). Furthermore, we evaluate the requirements for edge computing within wearable devices in terms of footprint, power consumption, latency, and data size. We additionally investigate the challenges beyond neuromorphic computing hardware, algorithms and devices that could impede enhancement of adaptive edge computing in smart wearable devices

Natural Hand Gestures Recognition System for Intelligent HCI: A Survey

Abstract: Gesture recognition is to recognizing meaningful expressions of motion by a human, involving the hands, arms, face, head, and/or body. Hand Gestures have greater importance in designing an intelligent and efficient human-computer interface. The applications of gesture recognition are manifold, ranging from sign language through medical rehabilitation to virtual reality. In this paper a survey on various recent gesture recognition approaches is provided with particular emphasis on hand gestures. A review of static hand posture methods are explained with different tools and algorithms applied on gesture recognition system, including connectionist models, hidden Markov model, and fuzzy clustering. Challenges and future research directions are also highlighted

Seeing things

This paper is concerned with the problem of attaching meaningful symbols to aspects of the visible environment in machine and biological vision. It begins with a review of some of the arguments commonly used to support either the 'symbolic' or the 'behaviourist' approach to vision. Having explored these avenues without arriving at a satisfactory conclusion, we then present a novel argument, which starts from the question : given a functional description of a vision system, when could it be said to support a symbolic interpretation? We argue that to attach symbols to a system, its behaviour must exhibit certain well defined regularities in its response to its visual input and these are best described in terms of invariance and equivariance to transformations which act in the world and induce corresponding changes of the vision system state. This approach is illustrated with a brief exploration of the problem of identifying and acquiring visual representations having these symmetry properties, which also highlights the advantages of using an 'active' model of vision

Yapay sinir ağlarının uyarlanabilir donanımsal yapılarda gerçeklenmesi

06.03.2018 tarihli ve 30352 sayılı Resmi Gazetede yayımlanan “Yükseköğretim Kanunu İle Bazı Kanun Ve Kanun Hükmünde Kararnamelerde Değişiklik Yapılması Hakkında Kanun” ile 18.06.2018 tarihli “Lisansüstü Tezlerin Elektronik Ortamda Toplanması, Düzenlenmesi ve Erişime Açılmasına İlişkin Yönerge” gereğince tam metin erişime açılmıştır.Yapay Sinir Ağları (YSA'lar), biyolojik sinir sistemine dayalı elektronik modellerdir. YSA'lar girişlerden gelen verileri işleyen birbirine bağlı yapay nöronlardan oluşmaktadır. Bu mimariler, yazılım ya da donanım olarak gerçekleştirilebilirler. YSA'nın yazılım uygulamasının avantajı, tasarımcının YSA bileşenlerinin iç işleyişini bilmesine gerek olmamasıdır. Bununla birlikte, gerçek zamanlı uygulamalarda, yazılım tabanlı YSA'lar donanım tabanlı YSA'lardan daha yavaştır. YSA hesaplamaları paralel olarak gerçekleştirilmektedir ve paralel işlem için özel donanım aygıtları gereklidir. Birçok alandan araştırmacılar optimizasyon, sınıflandırma, kontrol, görüntü işleme vb. problemlerin çözümü için YSA donanım uygulamaları gerçekleştirmişlerdir. Bu uygulamalar, YSA'ların paralel doğasından yararlanmak için farklı türde cihazlar üzerinde gerçekleştirilmiştir. YSA'nın FPGA uygulamaları, yeniden yapılandırılabilir yapısı ve paralel mimarisi nedeniyle son yirmi yılda büyük ilgi uyandırmıştır. Bu tez çalışmasında, Quartus II şematik tasarım kullanılarak eğitilebilir çok katmanlı sinir ağı (MLNN) yapısının donanım uygulaması FPGA üzerinde tamamen kombinasyonel mantık olarak gerçekleştirilmiştir. Yapay sinir ağını eğitmek için eğim düşüm metodunu kullanan geri yayılım algoritması uygulanmıştır. Nümerik tanımlama için IEEE tek-hassasiyetli kayan-noktalı format kullanılmıştır. Bu çalışma aynı zamanda IEEE tek-hassasiyetli kayan-noktalı format ile tam uyumlu hızlı bir kayan noktalı toplayıcı, bir paralel çarpıcı ve bir sigmoid aktivasyon fonksiyonu bloğunu sunmaktadır. İşlemleri paralel olarak gerçekleştiren toplayıcı, paralel çarpıcı ve aktivasyon fonksiyonu bloğu tamamen kombinasyonel mantık olarak tasarlanmıştır. Bu yeni tasarımda, gecikmeyi azaltmak için kaydırma işlemlerinde kaydırmalı yazmaçlar yerine üç-durumlu tampon serileri kullanılmıştır. Üç-durumlu tampon serileri kullanıldığından kaydırma işlemi için saat darbesi gerekli değildir ve böylece sonuç tek bir çevrimde üretilir. Sadece kapı gecikmesi maliyetli önerilen tasarım, YSA'nın donanım uygulamaları için uygundur.Artificial Neural Networks (ANNs) are electronic models based on biological nervous system. ANNs are made up of interconnected artificial neurons which can process values from inputs. These architectures can be implemented either in software or in hardware. The advantage of the software implementation of ANN is that the designer does not need to know the inner workings of ANN components. However, in real time applications, software based ANNs are slower than hardware based ANNs. ANN computations are carried out in parallel and special hardware devices are required for parallel processing. Researchers from many disciplines have been performing ANN hardware implementations to solve a variety of problems in optimization, classification, control, image processing etc. These applications have been performed on different types of devices to take advantage of the parallel nature inherent to ANNs. FPGA implementations of ANN have aroused great interest during the last two decades due to its reconfigurable structure and parallel architecture. In this thesis, hardware implementation of trainable Multi Layer Neural Network (MLNN) structure on FPGA (Field Programmable Gate Array) is realized as entirely combinational logic by using Quartus II schematic design. The back propagation algortihm, which uses gradient descent metod is implemented in order to train the neural network. IEEE single-precision floating-point format is used for numerical description. This study also presents the hardware designs of a fast floating point adder, a parallel multiplier and a sigmoid activation function block that are fully compliant with the IEEE single-precision floating-point format. The adder, parallel multiplier and the activation function block are designed as entirely combinational logic that perform operations in parallel. In this novel design, tri state buffer series are used for shifting operations instead of shift registers for reducing latency. Because the use of tri-state buffer series, clock pulse is not required for shifting and thus the result is generated in only a single clock-cycle. The proposed design is suitable for hardware implementation of ANN at the cost of gate delays only

Yapay sinir ağlarının uyarlanabilir donanımsal yapılarda gerçeklenmesi

06.03.2018 tarihli ve 30352 sayılı Resmi Gazetede yayımlanan “Yükseköğretim Kanunu İle Bazı Kanun Ve Kanun Hükmünde Kararnamelerde Değişiklik Yapılması Hakkında Kanun” ile 18.06.2018 tarihli “Lisansüstü Tezlerin Elektronik Ortamda Toplanması, Düzenlenmesi ve Erişime Açılmasına İlişkin Yönerge” gereğince tam metin erişime açılmıştır.Yapay Sinir Ağları (YSA'lar), biyolojik sinir sistemine dayalı elektronik modellerdir. YSA'lar girişlerden gelen verileri işleyen birbirine bağlı yapay nöronlardan oluşmaktadır. Bu mimariler, yazılım ya da donanım olarak gerçekleştirilebilirler. YSA'nın yazılım uygulamasının avantajı, tasarımcının YSA bileşenlerinin iç işleyişini bilmesine gerek olmamasıdır. Bununla birlikte, gerçek zamanlı uygulamalarda, yazılım tabanlı YSA'lar donanım tabanlı YSA'lardan daha yavaştır. YSA hesaplamaları paralel olarak gerçekleştirilmektedir ve paralel işlem için özel donanım aygıtları gereklidir. Birçok alandan araştırmacılar optimizasyon, sınıflandırma, kontrol, görüntü işleme vb. problemlerin çözümü için YSA donanım uygulamaları gerçekleştirmişlerdir. Bu uygulamalar, YSA'ların paralel doğasından yararlanmak için farklı türde cihazlar üzerinde gerçekleştirilmiştir. YSA'nın FPGA uygulamaları, yeniden yapılandırılabilir yapısı ve paralel mimarisi nedeniyle son yirmi yılda büyük ilgi uyandırmıştır. Bu tez çalışmasında, Quartus II şematik tasarım kullanılarak eğitilebilir çok katmanlı sinir ağı (MLNN) yapısının donanım uygulaması FPGA üzerinde tamamen kombinasyonel mantık olarak gerçekleştirilmiştir. Yapay sinir ağını eğitmek için eğim düşüm metodunu kullanan geri yayılım algoritması uygulanmıştır. Nümerik tanımlama için IEEE tek-hassasiyetli kayan-noktalı format kullanılmıştır. Bu çalışma aynı zamanda IEEE tek-hassasiyetli kayan-noktalı format ile tam uyumlu hızlı bir kayan noktalı toplayıcı, bir paralel çarpıcı ve bir sigmoid aktivasyon fonksiyonu bloğunu sunmaktadır. İşlemleri paralel olarak gerçekleştiren toplayıcı, paralel çarpıcı ve aktivasyon fonksiyonu bloğu tamamen kombinasyonel mantık olarak tasarlanmıştır. Bu yeni tasarımda, gecikmeyi azaltmak için kaydırma işlemlerinde kaydırmalı yazmaçlar yerine üç-durumlu tampon serileri kullanılmıştır. Üç-durumlu tampon serileri kullanıldığından kaydırma işlemi için saat darbesi gerekli değildir ve böylece sonuç tek bir çevrimde üretilir. Sadece kapı gecikmesi maliyetli önerilen tasarım, YSA'nın donanım uygulamaları için uygundur.Artificial Neural Networks (ANNs) are electronic models based on biological nervous system. ANNs are made up of interconnected artificial neurons which can process values from inputs. These architectures can be implemented either in software or in hardware. The advantage of the software implementation of ANN is that the designer does not need to know the inner workings of ANN components. However, in real time applications, software based ANNs are slower than hardware based ANNs. ANN computations are carried out in parallel and special hardware devices are required for parallel processing. Researchers from many disciplines have been performing ANN hardware implementations to solve a variety of problems in optimization, classification, control, image processing etc. These applications have been performed on different types of devices to take advantage of the parallel nature inherent to ANNs. FPGA implementations of ANN have aroused great interest during the last two decades due to its reconfigurable structure and parallel architecture. In this thesis, hardware implementation of trainable Multi Layer Neural Network (MLNN) structure on FPGA (Field Programmable Gate Array) is realized as entirely combinational logic by using Quartus II schematic design. The back propagation algortihm, which uses gradient descent metod is implemented in order to train the neural network. IEEE single-precision floating-point format is used for numerical description. This study also presents the hardware designs of a fast floating point adder, a parallel multiplier and a sigmoid activation function block that are fully compliant with the IEEE single-precision floating-point format. The adder, parallel multiplier and the activation function block are designed as entirely combinational logic that perform operations in parallel. In this novel design, tri state buffer series are used for shifting operations instead of shift registers for reducing latency. Because the use of tri-state buffer series, clock pulse is not required for shifting and thus the result is generated in only a single clock-cycle. The proposed design is suitable for hardware implementation of ANN at the cost of gate delays only

Enhancing brain/neural-machine interfaces for upper limb motor restoration in chronic stroke and cervical spinal cord injury

Operation of assistive exoskeletons based on voluntary control of sensorimotor rhythms (SMR, 8-12 Hz) enables intuitive control of finger or arm movements in severe paralysis after chronic stroke or cervical spinal cord injury (SCI). To improve reliability of such systems outside the laboratory, in particular when brain activity is recorded non-invasively with scalp electroencephalography (EEG), a hybrid EEG/electrooculography (EOG) brain/neural-machine interface (B/NMI) was recently introduced. Besides providing assistance, recent studies indicate that repeated use of such systems can trigger neural recovery. However, important prerequisites have to achieved before broader use in clinical settings or everyday life environments is feasible. Current B/NMI systems predominantly restore hand function, but do not allow simultaneous control of more proximal joints for whole-arm motor coordination as required for most stroke survivors suffering from paralysis in the entire upper limb. Besides paralysis, cognitive impairments including post-stroke fatigue due to the brain lesion reduce the capacity to maintain effortful B/NMI control over a longer period of time. This impedes the applicability in daily life assistance and might even limits the efficacy of neurorehabilitation training. In contrast to stroke survivors, tetraplegics due to cervical SCI lack motor function in both hands. Given that most activities of daily living (ADL) involve bimanual manipulation, e.g., to open the lid of a bottle, bilateral exoskeleton control is required but was not shown yet in tetraplegics. To further enhance B/NMI systems, we first investigated whether B/NMI whole-arm exoskeleton control in hemiplegia after chronic stroke is feasible and safe. In contrast to simple grasping, control of more complex tasks involving the entire upper limb was not feasible with established B/NMIs because high- dimensionality of such multiple joint systems exceeds the bandwidth of these interfaces. Thus, we blended B/NMI control with vision-guidance to receive a semiautonomous whole-arm exoskeleton control. Such setup allowed to divide ADL tasks into a sequence of EEG/EOG-triggered sub-tasks reducing complexity for the user. While, for instance, a drinking task was resolved into EOG-induced reaching, lifting and placing back the cup, grasping and releasing movements were based on intuitive SMR control. Feasibility of such shared vision-guided B/NMI control was assumed when executions were initialized within 3 s (fluent control) and a minimum of 75 % of subtasks were executed within that time (reliable control). We showed feasibility in healthy subjects as well as stroke survivors without report of any side effects documenting safe use. Similarly, feasibility and safety of bilateral B/NMI control after cervical SCI was evaluated. To enable bilateral B/NMI control, established EEG-based grasping and EOG-based releasing or stop commands were complemented with a novel EOG command allowing to switch laterality by performing prolonged horizontal eye movements (>1 s) to the left or to the right. Study results with healthy subjects and tetraplegics document fluent initialization of grasping motions below 3 s as well as safe use as unintended grasping could be stopped before a full motion was conducted. Superiority of novel bilateral control was documented by a higher accuracy of up to 22 % in tetraplegics compared to a bilateral control without prolonged EOG command. Lastly, as reliable B/NMI control is cognitively demanding, e.g., by imagining or attempting the desired movements, we investigated whether heart rate variability (HRV) can be used as biomarker to predict declining control performance, which is often reported in stroke survivors due to their cognitive impairments. Referring to the close brain-heart connection, we showed in healthy subjects that a decline in HRV is specific as well as predictive to a decline in B/NMI control performance within a single training session. The predictive link was revealed by a Granger-causality analysis. In conclusion, we could demonstrate important enhancements in B/NMI control paradigms including complex whole-arm exoskeleton control as well as individual performance monitoring within a training session based on HRV. Both achievements contribute to broaden the use as a standard therapy in stroke neurorehabilitation. Especially the predictive characteristic of HRV paves the way for adaptive B/NMI control paradigms to account for individual differences among impaired stroke survivors. Moreover, we also showed feasibility and safety of a novel implementation for bilateral B/NMI control, which is necessary for reliable operation of two hand-exoskeletons for bimanual ADLs after SCI

Neuromorphic Computing between Reality and Future Needs

Neuromorphic computing is a one of computer engineering methods that to model their elements as the human brain and nervous system. Many sciences as biology, mathematics, electronic engineering, computer science and physics have been integrated to construct artificial neural systems. In this chapter, the basics of Neuromorphic computing together with existing systems having the materials, devices, and circuits. The last part includes algorithms and applications in some fields

EEG and ECoG features for Brain Computer Interface in Stroke Rehabilitation

The ability of non-invasive Brain-Computer Interface (BCI) to control an exoskeleton was used for motor rehabilitation in stroke patients or as an assistive device for the paralyzed. However, there is still a need to create a more reliable BCI that could be used to control several degrees of Freedom (DoFs) that could improve rehabilitation results. Decoding different movements from the same limb, high accuracy and reliability are some of the main difficulties when using conventional EEG-based BCIs and the challenges we tackled in this thesis. In this PhD thesis, we investigated that the classification of several functional hand reaching movements from the same limb using EEG is possible with acceptable accuracy. Moreover, we investigated how the recalibration could affect the classification results. For this reason, we tested the recalibration in each multi-class decoding for within session, recalibrated between-sessions, and between sessions. It was shown the great influence of recalibrating the generated classifier with data from the current session to improve stability and reliability of the decoding. Moreover, we used a multiclass extension of the Filter Bank Common Spatial Patterns (FBCSP) to improve the decoding accuracy based on features and compared it to our previous study using CSP. Sensorimotor-rhythm-based BCI systems have been used within the same frequency ranges as a way to influence brain plasticity or controlling external devices. However, neural oscillations have shown to synchronize activity according to motor and cognitive functions. For this reason, the existence of cross-frequency interactions produces oscillations with different frequencies in neural networks. In this PhD, we investigated for the first time the existence of cross-frequency coupling during rest and movement using ECoG in chronic stroke patients. We found that there is an exaggerated phase-amplitude coupling between the phase of alpha frequency and the amplitude of gamma frequency, which can be used as feature or target for neurofeedback interventions using BCIs. This coupling has been also reported in another neurological disorder affecting motor function (Parkinson and dystonia) but, to date, it has not been investigated in stroke patients. This finding might change the future design of assistive or therapeuthic BCI systems for motor restoration in stroke patients