Generalization of Fitness Exercise Recognition from Doppler Measurements by Domain-adaption and Few-Shot Learning

In previous works, a mobile application was developed using an unmodified commercial off-the-shelf smartphone to recognize whole-body exercises. The working principle was based on the ultrasound Doppler sensing with the device built-in hardware. Applying such a lab-environment trained model on realistic application variations causes a significant drop in performance, and thus decimate its applicability. The reason of the reduced performance can be manifold. It could be induced by the user, environment, and device variations in realistic scenarios. Such scenarios are often more complex and diverse, which can be challenging to anticipate in the initial training data. To study and overcome this issue, this paper presents a database with controlled and uncontrolled subsets of fitness exercises. We propose two concepts to utilize small adaption data to successfully improve model generalization in an uncontrolled environment, increasing the recognition accuracy by two to six folds compared to the baseline for different users.Comment: accepted at International Conference on Pattern Recognition (ICPR) workshop 202

Performing Realistic Workout Activity Recognition on Consumer Smartphones

Smartphones have become an essential part of our lives. Especially its computing power and its current specifications make a modern smartphone a powerful device for human activity recognition tasks. Equipped with various integrated sensors, a modern smartphone can be leveraged for lots of smart applications. We already investigated the possibility of using an unmodified commercial smartphone to recognize eight strength-based exercises. App-based workouts have become popular in the last few years. The advantage of using a mobile device is that you can practice anywhere at anytime. In our previous work, we proved the possibility of turning a commercial smartphone into an active sonar device to leverage the echo reflected from exercising movement close to the device. By conducting a test study with 14 participants, we showed the first results for cross person evaluation and the generalization ability of our inference models on disjoint participants. In this work, we extended another model to further improve the model generalizability and provided a thorough comparison of our proposed system to other existing state-of-the-art approaches. Finally, a concept of counting the repetitions is also provided in this study as a parallel task to classification

ExerTrack - Towards Smart Surfaces to Track Exercises

The concept of the quantified self has gained popularity in recent years with the hype of miniaturized gadgets to monitor vital fitness levels. Smartwatches or smartphone apps and other fitness trackers are overwhelming the market. Most aerobic exercises such as walking, running, or cycling can be accurately recognized using wearable devices. However whole-body exercises such as push-ups, bridges, and sit-ups are performed on the ground and thus cannot be precisely recognized by wearing only one accelerometer. Thus, a floor-based approach is preferred for recognizing whole-body activities. Computer vision techniques on image data also report high recognition accuracy; however, the presence of a camera tends to raise privacy issues in public areas. Therefore, we focus on combining the advantages of ubiquitous proximity-sensing with non-optical sensors to preserve privacy in public areas and maintain low computation cost with a sparse sensor implementation. Our solution is the ExerTrack, an off-the-shelf sports mat equipped with eight sparsely distributed capacitive proximity sensors to recognize eight whole-body fitness exercises with a user-independent recognition accuracy of 93.5 % and a user-dependent recognition accuracy of 95.1 % based on a test study with 9 participants each performing 2 full sessions. We adopt a template-based approach to count repetitions and reach a user-independent counting accuracy of 93.6 %. The final model can run on a Raspberry Pi 3 in real time. This work includes data-processing of our proposed system and model selection to improve the recognition accuracy and data augmentation technique to regularize the network

Fitness Activity Recognition on Smartphones Using Doppler Measurements

Quantified Self has seen an increased interest in recent years, with devices including smartwatches, smartphones, or other wearables that allow you to monitor your fitness level. This is often combined with mobile apps that use gamification aspects to motivate the user to perform fitness activities, or increase the amount of sports exercise. Thus far, most applications rely on accelerometers or gyroscopes that are integrated into the devices. They have to be worn on the body to track activities. In this work, we investigated the use of a speaker and a microphone that are integrated into a smartphone to track exercises performed close to it. We combined active sonar and Doppler signal analysis in the ultrasound spectrum that is not perceivable by humans. We wanted to measure the body weight exercises bicycles, toe touches, and squats, as these consist of challenging radial movements towards the measuring device. We have tested several classification methods, ranging from support vector machines to convolutional neural networks. We achieved an accuracy of 88% for bicycles, 97% for toe-touches and 91% for squats on our test set

Monolayer hydrophilic MoS2 with strong charge trapping for atomically thin neuromorphic vision systems

Effective control of electrical and optoelectronic properties of two-dimensional layered materials, one of the key requirements for applications in advanced optoelectronics with multiple functions, has been hindered by the difficulty of elemental doping, which is commonly utilized in Si technology. In this study, we proposed a new method to synthesize hydrophilic MoS2 monolayers through covalently introducing hydroxyl groups during their growth process. These hydroxyl groups exhibit a strong capability of charge trapping, and thus the hydrophilic MoS2 monolayers achieve excellent electrical, optical, and memory properties. Optical memory transistors, made from a single component of monolayer hydrophilic MoS2, exhibit not only excellent light-dependent and time-dependent photoelectric performance, but also good photo-responsive memory characteristics with over multi-bit storage and more than 104 switching ratios. Atomically thin neuromorphic vision systems (with a concept of proof of 10 × 10 neuromorphic visual image) are manufactured from arrays of hydrophilic MoS2 optical memory transistors, showing high quality image sensing and memory functions with a high color resolution. These results proved our new concepts to realize image memorization and simplify the pixel matrix preparation process, which is a significant step toward the development of future artificial visual systems

The Interplay Between Hydrogen Sulfide and Phytohormone Signaling Pathways Under Challenging Environments

Hydrogen sulfide (H2S) serves as an important gaseous signaling molecule that is involved in intra- and intercellular signal transduction in plant–environment interactions. In plants, H2S is formed in sulfate/cysteine reduction pathways. The activation of endogenous H2S and its exogenous application has been found to be highly effective in ameliorating a wide variety of stress conditions in plants. The H2S interferes with the cellular redox regulatory network and prevents the degradation of proteins from oxidative stress via post-translational modifications (PTMs). H2S-mediated persulfidation allows the rapid response of proteins in signaling networks to environmental stimuli. In addition, regulatory crosstalk of H2S with other gaseous signals and plant growth regulators enable the activation of multiple signaling cascades that drive cellular adaptation. In this review, we summarize and discuss the current understanding of the molecular mechanisms of H2S-induced cellular adjustments and the interactions between H2S and various signaling pathways in plants, emphasizing the recent progress in our understanding of the effects of H2S on the PTMs of proteins. We also discuss future directions that would advance our understanding of H2S interactions to ultimately mitigate the impacts of environmental stresses in the plants

Engineering zinc oxide hybrid selenium nanoparticles for synergetic anti-tuberculosis treatment by combining Mycobacterium tuberculosis killings and host cell immunological inhibition

IntroductionAs a deadly disease induced by Mycobacterium tuberculosis (Mtb), tuberculosis remains one of the top killers among infectious diseases. The low intracellular Mtb killing efficiency of current antibiotics introduced the long duration anti-TB therapy in clinic with strong side effects and increased drug-resistant mutants. Therefore, the exploration of novel anti-TB agents with potent anti-TB efficiency becomes one of the most urgent issues for TB therapies. MethodsHere, we firstly introduced a novel method for the preparation of zinc oxide-selenium nanoparticles (ZnO-Se NPs) by the hybridization of zinc oxide and selenium to combine the anti-TB activities of zinc oxide nanoparticles and selenium nanoparticles. We characterized the ZnO-Se NPs by dynamic laser light scattering and transmission electron microscopy, and then tested the inhibition effects of ZnO-Se NPs on extracellular Mtb by colony-forming units (CFU) counting, bacterial ATP analysis, bacterial membrane potential analysis and scanning electron microscopy imaging. We also analyzed the effects of ZnO-Se NPs on the ROS production, mitochondrial membrane potential, apoptosis, autophagy, polarization and PI3K/Akt/mTOR signaling pathway of Mtb infected THP-1 macrophages. At last, we also tested the effects of ZnO-Se NPs on intracellular Mtb in THP-1 cells by colony-forming units (CFU) counting. ResultsThe obtained spherical core-shell ZnO-Se NPs with average diameters of 90 nm showed strong killing effects against extracellular Mtb, including BCG and the virulent H37Rv, by disrupting the ATP production, increasing the intracellular ROS level and destroying the membrane structures. More importantly, ZnO-Se NPs could also inhibit intracellular Mtb growth by promoting M1 polarization to increase the production of antiseptic nitric oxide and also promote apoptosis and autophagy of Mtb infected macrophages by increasing the intracellular ROS, disrupting mitochondria membrane potential and inhibiting PI3K/Akt/mTOR signaling pathway. DiscussionThese ZnO-Se NPs with synergetic anti-TB efficiency by combining the Mtb killing effects and host cell immunological inhibition effects were expected to serve as novel anti-TB agents for the development of more effective anti-TB strategy

Sensor Applications for Human Activity Recognition in Smart Environments

Human activity recognition (HAR) is the automated recognition of individual or group activities from sensor inputs. It deals with a wide range of application areas, such as for health care, assisting technologies, quantified-self and safety applications. HAR is the key to build human-centred applications and enables users to seamlessly and naturally interact with each other or with a smart environment. A smart environment is an instrumented room or space equipped with sensors and actuators to perceive the physical state or human activities within this space. The diversity of sensors makes it difficult to use the appropriate sensor to build specific applications. This work aims at presenting sensor-driven applications for human activity recognition in smart environments by using novel sensing categories beyond the existing sensor technologies commonly applied to these tasks. The intention is to improve the interaction for various sub-fields of human activities. Each application addresses the difficulties following the typical process pipeline for designing a smart environment application. At first, I survey most prominent research works with focus on sensor-driven categorization in the research domain of HAR to identify possible research gaps to position my work. I identify two use-cases: quantified-self and smart home applications. Quantified-self aims at self-tracking and self-knowledge through numbers. Common sensor technology for daily tracking of various aerobic endurance training activities, such as walking, running or cycling are based on acceleration data with wearable. However, more stationary exercises, such as strength-based training or stretching are also important for a healthy life-style, as they improve body coordination and balance. These exercises are not well tracked by wearing only a single wearable sensor, as these activities rely on coordinated movement of the entire body. I leverage two sensing categories to design two portable mobile applications for remote sensing of these more stationary exercises of physical workout. Sensor-driven applications for smart home domain aim at building systems to make the life of the occupants safer and more convenient. In this thesis, I target at stationary applications to be integrated into the environment to allow a more natural interaction between the occupant and the smart environment. I propose two possible solutions to achieve this task. The first system is a surface acoustic based system which provides a sparse sensor setup to detect a basic set of activities of daily living including the investigation of minimalist sensor arrangement. The second application is a tag-free indoor positioning system. Indoor localization aims at providing location information to build intelligent services for smart homes. Accurate indoor position offers the basic context for high-level reasoning system to achieve more complex contexts. The floor-based localization system using electrostatic sensors is scalable to different room geometries due to its layout and modular composition. Finally, privacy with non-visual input is the main aspect for applications proposed in this thesis. In addition, this thesis addresses the issue of adaptivity from prototypes towards real-world applications. I identify the issues of data sparsity in the training data and data diversity in the real-world data. In order to solve the issue of data sparsity, I demonstrate the data augmentation strategy to be applied on time series to increase the amount of training data by generating synthetic data. Towards mitigating the inherent difference of the development dataset and the real-world scenarios, I further investigate several approaches including metric-based learning and fine-tuning. I explore these methods to finetune the trained model on limited amount of individual data with and without retrain the pre-trained inference model. Finally some examples are stated as how to deploy the offline model to online processing device with limited hardware resources

Biometric Recognition in 3D Medical Images: A Survey

This work explores the potential applications of biometric recognition in 3D medical imaging data. We investigate various 3D imaging techniques commonly used in the medical domain, including 3D ultrasound imaging, magnetic resonance imaging (MRI), computer tomography (CT) scans, and 3D near-infrared (NIR) imaging. For each technique, we provide an overview of its working principle and discuss the advantages of integrating biometrics into 3D medical imaging data. Major advantage of using biometrics in this context is motivated by the research that using biometrics could not only increase data security but, more importantly, decrease the mix-up errors in patient’s medical data and thus improving patient safety and patient care. Our analysis uncovers certain weaknesses in current algorithms and limitations in existing research. Possible reasons include insufficient data availability, the under-utilization of deep-learning-based approaches to enhance accuracy and performance, and the absence of standardized benchmarking databases to support research. Our survey frames existing works and lead to practical recommendations and motivates efforts to improve the current state of research. Beyond exploring the utilization of biometrics in 3D medical imaging data, our study touches on further potential interactions between them, such as extracting health information from biometric captures. This work is thus the first work to survey and present an overview on works proposing the use of medical images for biometric recognition