Inferring Complex Activities for Context-aware Systems within Smart Environments

The rising ageing population worldwide and the prevalence of age-related conditions such as physical fragility, mental impairments and chronic diseases have significantly impacted the quality of life and caused a shortage of health and care services. Over-stretched healthcare providers are leading to a paradigm shift in public healthcare provisioning. Thus, Ambient Assisted Living (AAL) using Smart Homes (SH) technologies has been rigorously investigated to help address the aforementioned problems. Human Activity Recognition (HAR) is a critical component in AAL systems which enables applications such as just-in-time assistance, behaviour analysis, anomalies detection and emergency notifications. This thesis is aimed at investigating challenges faced in accurately recognising Activities of Daily Living (ADLs) performed by single or multiple inhabitants within smart environments. Specifically, this thesis explores five complementary research challenges in HAR. The first study contributes to knowledge by developing a semantic-enabled data segmentation approach with user-preferences. The second study takes the segmented set of sensor data to investigate and recognise human ADLs at multi-granular action level; coarse- and fine-grained action level. At the coarse-grained actions level, semantic relationships between the sensor, object and ADLs are deduced, whereas, at fine-grained action level, object usage at the satisfactory threshold with the evidence fused from multimodal sensor data is leveraged to verify the intended actions. Moreover, due to imprecise/vague interpretations of multimodal sensors and data fusion challenges, fuzzy set theory and fuzzy web ontology language (fuzzy-OWL) are leveraged. The third study focuses on incorporating uncertainties caused in HAR due to factors such as technological failure, object malfunction, and human errors. Hence, existing studies uncertainty theories and approaches are analysed and based on the findings, probabilistic ontology (PR-OWL) based HAR approach is proposed. The fourth study extends the first three studies to distinguish activities conducted by more than one inhabitant in a shared smart environment with the use of discriminative sensor-based techniques and time-series pattern analysis. The final study investigates in a suitable system architecture with a real-time smart environment tailored to AAL system and proposes microservices architecture with sensor-based off-the-shelf and bespoke sensing methods. The initial semantic-enabled data segmentation study was evaluated with 100% and 97.8% accuracy to segment sensor events under single and mixed activities scenarios. However, the average classification time taken to segment each sensor events have suffered from 3971ms and 62183ms for single and mixed activities scenarios, respectively. The second study to detect fine-grained-level user actions was evaluated with 30 and 153 fuzzy rules to detect two fine-grained movements with a pre-collected dataset from the real-time smart environment. The result of the second study indicate good average accuracy of 83.33% and 100% but with the high average duration of 24648ms and 105318ms, and posing further challenges for the scalability of fusion rule creations. The third study was evaluated by incorporating PR-OWL ontology with ADL ontologies and Semantic-Sensor-Network (SSN) ontology to define four types of uncertainties presented in the kitchen-based activity. The fourth study illustrated a case study to extended single-user AR to multi-user AR by combining RFID tags and fingerprint sensors discriminative sensors to identify and associate user actions with the aid of time-series analysis. The last study responds to the computations and performance requirements for the four studies by analysing and proposing microservices-based system architecture for AAL system. A future research investigation towards adopting fog/edge computing paradigms from cloud computing is discussed for higher availability, reduced network traffic/energy, cost, and creating a decentralised system. As a result of the five studies, this thesis develops a knowledge-driven framework to estimate and recognise multi-user activities at fine-grained level user actions. This framework integrates three complementary ontologies to conceptualise factual, fuzzy and uncertainties in the environment/ADLs, time-series analysis and discriminative sensing environment. Moreover, a distributed software architecture, multimodal sensor-based hardware prototypes, and other supportive utility tools such as simulator and synthetic ADL data generator for the experimentation were developed to support the evaluation of the proposed approaches. The distributed system is platform-independent and currently supported by an Android mobile application and web-browser based client interfaces for retrieving information such as live sensor events and HAR results

Recognition Situations Using Extended Dempster-Shafer Theory

Weiser’s [111] vision of pervasive computing describes a world where technology seamlessly integrates into the environment, automatically responding to peoples’ needs. Underpinning this vision is the ability of systems to automatically track the situation of a person. The task of situation recognition is critical and complex: noisy and unreliable sensor data, dynamic situations, unpredictable human behaviour and changes in the environment all contribute to the complexity. No single recognition technique is suitable in all environments. Factors such as availability of training data, ability to deal with uncertain information and transparency to the user will determine which technique to use in any particular environment. In this thesis, we propose the use of Dempster-Shafer theory as a theoretically sound basis for situation recognition - an approach that can reason with uncertainty, but which does not rely on training data. We use existing operations from Dempster-Shafer theory and create new operations to establish an evidence decision network. The network is used to generate and assess situation beliefs based on processed sensor data for an environment. We also define two specific extensions to Dempster-Shafer theory to enhance the knowledge that can be used for reasoning: 1) temporal knowledge about situation time patterns 2) quality of evidence sources (sensors) into the reasoning process. To validate the feasibility of our approach, this thesis creates evidence decision networks for two real-world data sets: a smart home data set and an officebased data set. We analyse situation recognition accuracy for each of the data sets, using the evidence decision networks with temporal/quality extensions. We also compare the evidence decision networks against two learning techniques: Naïve Bayes and J48 Decision Tree

Application of Audible Signals in Tool Condition Monitoring using Machine Learning Techniques

Machining is always accompanied by many difficulties like tool wear, tool breakage, improper machining conditions, non-uniform workpiece properties and some other irregularities, which are some of major barriers to highly-automated operations. Effective tool condition monitoring (TCM) system provides a best solution to monitor those irregular machining processes and suggest operators to take appropriate actions. Even though a wide variety of monitoring techniques have been developed for the online detection of tool condition, it remains an unsolved problem to look for a reliable, simple and cheap solution. This research work mainly focuses on developing a real-time tool condition monitoring model to detect the tool condition, part quality in machining process by using machine learning techniques through sound monitoring. The present study shows the development of a process model capable of on-line process monitoring utilizing machine learning techniques to analyze the sound signals collected during machining and train the proposed system to predict the cutting phenomenon during machining. A decision-making system based on the machine learning technique involving Support Vector Machine approach is developed. The developed system is trained with pre-processed data and tested, and the system showed a significant prediction accuracy in different applications which proves to be an effective model in applying to machining process as an on-line process monitoring system. In addition, this system also proves to be effective, cheap, compact and sensory position invariant. The successful development of the proposed TCM system can provide a practical tool to reduce downtime for tool changes and minimize the amount of scrap in metal cutting industry

Deep Space Network information system architecture study

The purpose of this article is to describe an architecture for the Deep Space Network (DSN) information system in the years 2000-2010 and to provide guidelines for its evolution during the 1990s. The study scope is defined to be from the front-end areas at the antennas to the end users (spacecraft teams, principal investigators, archival storage systems, and non-NASA partners). The architectural vision provides guidance for major DSN implementation efforts during the next decade. A strong motivation for the study is an expected dramatic improvement in information-systems technologies, such as the following: computer processing, automation technology (including knowledge-based systems), networking and data transport, software and hardware engineering, and human-interface technology. The proposed Ground Information System has the following major features: unified architecture from the front-end area to the end user; open-systems standards to achieve interoperability; DSN production of level 0 data; delivery of level 0 data from the Deep Space Communications Complex, if desired; dedicated telemetry processors for each receiver; security against unauthorized access and errors; and highly automated monitor and control

Design methodology for smart actuator services for machine tool and machining control and monitoring

This paper presents a methodology to design the services of smart actuators for machine tools. The smart actuators aim at replacing the traditional drives (spindles and feed-drives) and enable to add data processing abilities to implement monitoring and control tasks. Their data processing abilities are also exploited in order to create a new decision level at the machine level. The aim of this decision level is to react to disturbances that the monitoring tasks detect. The cooperation between the computational objects (the smart spindle, the smart feed-drives and the CNC unit) enables to carry out functions for accommodating or adapting to the disturbances. This leads to the extension of the notion of smart actuator with the notion of agent. In order to implement the services of the smart drives, a general design is presented describing the services as well as the behavior of the smart drive according to the object oriented approach. Requirements about the CNC unit are detailed. Eventually, an implementation of the smart drive services that involves a virtual lathe and a virtual turning operation is described. This description is part of the design methodology. Experimental results obtained thanks to the virtual machine are then presented

A Novel Approach for Crop Selection and Water Management using Mamdani’s Fuzzy Inference & IOT

In the modern world, technology is always evolving to replace more human labour with artificial intelligence. Moreover, farmers are under constant pressure to irrigate their farms at regular intervals without even a rudimentary grasp of the rainfall pattern and soil humidity, since it is extremely difficult to cultivate any agricultural food in regions with irregular rainfall patterns and high mean temperatures. This paper proposes a crop predictor and smart irrigation system using Mamdani’s fuzzy inference and IoT. The system aims to optimize water usage and crop yield by considering various factors such as soil moisture, temperature, humidity, rainfall, crop type and season. The system consists of three modules: a crop predictor module that uses fuzzy logic to suggest the best crop for a given location and season, an IOT module that collects and transmits the environmental data from sensors to a cloud server, and a smart irrigation module that uses fuzzy logic to control the water flow to the crops based on the data and the crop predictor module. The system is implemented and tested on a NodeMCU and MATLAB platform and shows promising results in terms of water conservation and crop productivity

Trends in the control of hexapod robots: a survey

The static stability of hexapods motivates their design for tasks in which stable locomotion is required, such as navigation across complex environments. This task is of high interest due to the possibility of replacing human beings in exploration, surveillance and rescue missions. For this application, the control system must adapt the actuation of the limbs according to their surroundings to ensure that the hexapod does not tumble during locomotion. The most traditional approach considers their limbs as robotic manipulators and relies on mechanical models to actuate them. However, the increasing interest in model-free models for the control of these systems has led to the design of novel solutions. Through a systematic literature review, this paper intends to overview the trends in this field of research and determine in which stage the design of autonomous and adaptable controllers for hexapods is.The first author received funding through a doctoral scholarship from the Portuguese Foundation for Science and Technology (FCT) (Grant No. SFRH/BD/145818/2019), with funds from the Portuguese Ministry of Science, Technology and Higher Education and the European Social Fund through the Programa Operacional Regional Norte. This work has been supported by the FCT national funds, under the national support to R&D units grant, through the reference project UIDB/04436/2020 and UIDP/04436/2020

Qualitative Distances and Qualitative Description of Images for Indoor Scene Description and Recognition in Robotics

The automatic extraction of knowledge from the world by a robotic system as human beings interpret their environment through their senses is still an unsolved task in Artificial Intelligence. A robotic agent is in contact with the world through its sensors and other electronic components which obtain and process mainly numerical information. Sonar, infrared and laser sensors obtain distance information. Webcams obtain digital images that are represented internally as matrices of red, blue and green (RGB) colour coordinate values. All this numerical values obtained from the environment need a later interpretation in order to provide the knowledge required by the robotic agent in order to carry out a task. Similarly, light wavelengths with specific amplitude are captured by cone cells of human eyes obtaining also stimulus without meaning. However, the information that human beings can describe and remember from what they see is expressed using words, that is qualitatively. The research work done in this thesis tries to narrow the gap between the acquisition of low level information by robot sensors and the need of obtaining high level or qualitative information for enhancing human-machine communication and for applying logical reasoning processes based on concepts. Moreover, qualitative concepts can be added a meaning by relating them to others. They can be used for reasoning applying qualitative models that have been developed in the last twenty years for describing and interpreting metrical and mathematical concepts such as orientation, distance, velocity, acceleration, and so on. And they can be also understood by human-users both written and read aloud. The first contribution presented is the definition of a method for obtaining fuzzy distance patterns (which include qualitative distances such as near , far , very far and so on) from the data obtained by any kind of distance sensors incorporated in a mobile robot and the definition of a factor to measure the dissimilarity between those fuzzy patterns. Both have been applied to the integration of the distances obtained by the sonar and laser distance sensors incorporated in a Pioneer 2 dx mobile robot and, as a result, special obstacles have been detected as glass window , mirror , and so on. Moreover, the fuzzy distance patterns provided have been also defuzzified in order to obtain a smooth robot speed and used to classify orientation reference systems into open (it defines an open space to be explored) or closed . The second contribution presented is the definition of a model for qualitative image description (QID) based on qualitative models of shape, colour, topology and orientation. This model can qualitatively describe any kind of digital image and is independent of the image segmentation method used. The QID model have been tested in two scenarios in robotics: (i) the description of digital images captured by the camera of a Pioneer 2 dx mobile robot and (ii) the description of digital images of tile mosaics taken by an industrial camera located on a platform used by a robot arm to assemble tile mosaics. In order to provide a formal and explicit meaning to the qualitative description of the images generated, a Description Logic (DL) based ontology has been designed and presented as the third contribution. Our approach can automatically process any random image and obtain a set of DL-axioms that describe it visually and spatially. And objects included in the images are classified according to the ontology schema using a DL reasoner. Tests have been carried out using digital images captured by a webcam incorporated in a Pioneer 2 dx mobile robot. The images taken correspond to the corridors of a building at University Jaume I and objects with them have been classified into walls , floor , office doors and fire extinguishers under different illumination conditions and from different observer viewpoints. The final contribution is the definition of a similarity measure between qualitative descriptions of shape, colour, topology and orientation. And the integration of those measures into the definition of a general similarity measure between two qualitative descriptions of images. These similarity measures have been applied to: (i) extract objects with similar shapes from the MPEG7 CE Shape-1 library; (ii) assemble tile mosaics by qualitative shape and colour similarity matching; (iii) compare images of tile compositions; and (iv) compare images of natural landmarks in a mobile robot world for their recognition