Ubiquitous Integration and Temporal Synchronisation (UbilTS) framework : a solution for building complex multimodal data capture and interactive systems

Contemporary Data Capture and Interactive Systems (DCIS) systems are tied in with various technical complexities such as multimodal data types, diverse hardware and software components, time synchronisation issues and distributed deployment configurations. Building these systems is inherently difficult and requires addressing of these complexities before the intended and purposeful functionalities can be attained. The technical issues are often common and similar among diverse applications. This thesis presents the Ubiquitous Integration and Temporal Synchronisation (UbiITS) framework, a generic solution to address the technical complexities in building DCISs. The proposed solution is an abstract software framework that can be extended and customised to any application requirements. UbiITS includes all fundamental software components, techniques, system level layer abstractions and reference architecture as a collection to enable the systematic construction of complex DCISs. This work details four case studies to showcase the versatility and extensibility of UbiITS framework’s functionalities and demonstrate how it was employed to successfully solve a range of technical requirements. In each case UbiITS operated as the core element of each application. Additionally, these case studies are novel systems by themselves in each of their domains. Longstanding technical issues such as flexibly integrating and interoperating multimodal tools, precise time synchronisation, etc., were resolved in each application by employing UbiITS. The framework enabled establishing a functional system infrastructure in these cases, essentially opening up new lines of research in each discipline where these research approaches would not have been possible without the infrastructure provided by the framework. The thesis further presents a sample implementation of the framework on a device firmware exhibiting its capability to be directly implemented on a hardware platform. Summary metrics are also produced to establish the complexity, reusability, extendibility, implementation and maintainability characteristics of the framework.Engineering and Physical Sciences Research Council (EPSRC) grants - EP/F02553X/1, 114433 and 11394

Towards a Cyber-Physical Gaming System for Training in the Construction and Engineering Industry

Introducing serious gaming systems (SGS) has the potential to enhance trainee experience and performance across the construction industry and its supply chain, such as mechanical engineering services. SGS as an ‘enabler’ in architectural engineering has received limited research in its role to assess and enhance the performance of its workforce. In a personnel high-risk environment, improving training standards to eliminate or reduce health and safety risks, in addition to providing an understanding of workers’ ergonomics, ensures sustainability of both the project and its workforce. This paper presents an activity tracking and feedback system that captures the physical activity of a construction worker climbing a ladder. Climbing is captured with a 3D motion capture system and processed in real-time to identify potential areas of underperformance. A simple and representative scoring method was established as a reporting method (game statistics) for giving feedback about the correctness of the activity. It can nonetheless be tuned to characterise and adjust to various complexity levels in-line with the required training standards. Furthermore, the motion data and feedback information are fed into a virtual gaming environment enabling the real-time visualisation of the trainee’s motion and experiential learning of the performance through visual and audio feedback. The gaming concepts are employed here with multiple purposes, particularly for accelerating and facilitating the learning process of the trainee. In addition to the 3D motion capturing system, this paper outlines and tests a proposed serious cyber-physical gaming system that incorporates wearable technologies that has the potential to support both construction training and practice.</jats:p

Towards a Cyber-Physical Gaming System for Training in the Construction and Engineering Industry

Antidepressants are among the most commonly detected human pharmaceuticals in the aquatic environment. Since their mode of action is by modulating the neurotransmitters serotonin, dopamine, and norepinephrine, aquatic invertebrates who possess transporters and receptors sensitive to activation by these pharmaceuticals are potentially affected by them. We review the various types of antidepressants, their occurrence and concentrations in aquatic environments, and the actions of neurohormones modulated by antidepressants in molluscs and crustaceans. Recent studies on the effects of antidepressants on these two important groups show that molluscan reproductive and locomotory systems are affected by antidepressants at environmentally relevant concentrations. In particular, antidepressants affect spawning and larval release in bivalves and disrupt locomotion and reduce fecundity in snails. In crustaceans, antidepressants affect freshwater amphipod activity patterns, marine amphipod photo- and geotactic behavior, crayfish aggression, and daphnid reproduction and development. We note with interest the occurrence of non-monotonic dose responses curves in many studies on effects of antidepressants on aquatic animals, often with effects at low concentrations, but not at higher concentrations, and we suggest future experiments consider testing a broader range of concentrations. Furthermore, we consider invertebrate immune responses, genomic and transcriptomic sequencing of invertebrate genes, and the ever-present and overwhelming question of how contaminant mixtures could affect the action of neurohormones as topics for future study. In addressing the question, if antidepressants affect aquatic invertebrates at concentrations currently found in the environment, there is strong evidence to suggest the answer is yes. Furthermore, the examples highlighted in this review provide compelling evidence that the effects could be quite multifaceted across a variety of biological systems

ADVANCES IN BALANCE AND BIOFEEDBACK MEASUREMENT: THE CASE FOR HEALTH-BASED, POSTURAL SERIOUS GAMES

Health games are increasingly seen as a means to address issues from therapy and rehabilitation. Yet, as a transformative technology, rarely have such games been explored or exploited to assist research into pathologies. Serious games for research (SGR) to uncover pathologies would allow clinicians to develop new differential diagnostics while providing a positive experience for the subject. This paper is not about game design; nevertheless it presents an outlook to considerations that could be taken forward when developing health-based SGRs for pathomechanics, etiopathogenesis and biofeedback. This work relates to preliminary studies on balance challenges manifested in pathologies of the central nervous system. As technology advancements seek to augment human sensory contact between virtual and real worlds this may impact on how virtual environments are used and designed in future. As a consequence heightened sensory (or lack of thereof) may result in falls, for example users with vestibular disorder – because postural stability is a key aspect of motor ability that allows individuals to sustain and maintain the desired physical position of their body Here, our investigation is specific to functional correspondence of the incidental properties in human body sway between healthy subjects and subjects with dyslexia. Our early results suggest postural sway between healthy subjects and those with mild disorders can be distinguished

Brain–computer interface game applications for combined neurofeedback and biofeedback treatment for children on the autism spectrum

Individuals with Autism Spectrum Disorder (ASD) show deficits in social and communicative skills, including imitation, empathy, and shared attention, as well as restricted interests and repetitive patterns of behaviors. Evidence for and against the idea that dysfunctions in the mirror neuron system are involved in imitation and could be one underlying cause for ASD is discussed in this review. Neurofeedback interventions have reduced symptoms in children with ASD by self-regulation of brain rhythms. However, cortical deficiencies are not the only cause of these symptoms. Peripheral physiological activity, such as the heart rate, is closely linked to neurophysiological signals and associated with social engagement. Therefore, a combined approach targeting the interplay between brain, body and behavior could be more effective. Brain-computer interface applications for combined neurofeedback and biofeedback treatment for children with ASD are currently nonexistent. To facilitate their use, we have designed an innovative game that includes social interactions and provides neural- and body-based feedback that corresponds directly to the underlying significance of the trained signals as well as to the behavior that is reinforced

Knowledge Capture in CMM Inspection Planning: Barriers and Challenges

Coordinate Measuring Machines (CMM) have been widely used as a means of evaluating product quality and controlling quality manufacturing processes. Many techniques have been developed to facilitate the generation of CMM measurement plans. However, there are major gaps in the understanding of planning such strategies. This significant lack of explicitly available knowledge on how experts prepare plans and carry out measurements slows down the planning process, leading to the repetitive reinvention of new plans while preventing the automation or even semi-automation of the process. The objectives of this paper are twofold: (i) to provide a review of the existing inspection planning systems and discuss the barriers and challenges, especially from the aspect of knowledge capture and formalization; and (ii) to propose and demonstrate a novel digital engineering mixed reality paradigm which has the potential to facilitate the rapid capture of implicit inspection knowledge and explicitly represent this in a formalized way. An outline and the results of the development of an early stage prototype - which will form the foundation of a more complex system to address the aforementioned technological challenges identified in the literature survey - will be given

A Real-Time Olive Fruit Detection for Harvesting Robot Based on Yolo Algorithms

Deep neural network models have become powerful tools of machine learning and artificial intelligence. They can approximate functions and dynamics by learning from examples. This paper reviews the state-of-art of deep learning-based object detection frameworks that are used for fruit detection in general and for olive fruit in particular. A dataset of olive fruit on the tree is built to train and evaluate deep models. The ultimate goal of this work is the capability of on-edge real-time olive fruit detection on the tree from digital videos. Recent work in deep neural networks has led to the development of a state-of-the-art object detector termed You Only Look Once version five (YOLOv5). This paper builds a dataset of 1.2 K source images of olive fruit on the tree and evaluates the latest object detection algorithms focusing on variants of YOLOv5 and YOLOR. The results of the YOLOv5 models show that the YOLOv5 new network models are able to extract rich olive features from images and detect the olive fruit with a high precision of higher than 0.75 mAP_0.5. YOLOv5s performs better for real-time olive fruit detection on the tree over other YOLOv5 variants and YOLOR