A Big Bang–Big Crunch Type-2 Fuzzy Logic System for Machine-Vision-Based Event Detection and Summarization in Real-World Ambient-Assisted Living

The area of ambient-assisted living (AAL) focuses on developing new technologies, which can improve the quality of life and care provided to elderly and disabled people. In this paper, we propose a novel system based on 3-D RGB-D vision sensors and interval type-2 fuzzy-logic-based systems (IT2FLSs) employing the big bang-big crunch algorithm for the real-time automatic detection and summarization of important events and human behaviors from the large-scale data. We will present several real-world experiments, which were conducted for AAL-related behaviors with various users. It will be shown that the proposed BB-BC IT2FLSs outperform the type-1 fuzzy logic system counterparts as well as other conventional nonfuzzy methods, and the performance improves when the number of subjects increases

A Big Bang Big Crunch Type-2 Fuzzy Logic System for Machine Vision-Based Event Detection and Summarization in Real-world Ambient Assisted Living

The recent years have witnessed the prevalence and abundance of vision sensors in various applications such as security surveillance, healthcare and Ambient Assisted Living (AAL) among others. This is so as to realize intelligent environments which are capable of detecting users’ actions and gestures so that the needed services can be provided automatically and instantly to maximize user comfort and safety as well as to minimize energy. However, it is very challenging to automatically detect important events and human behaviour from vision sensors and summarize them in real time. This is due to the massive data sizes related to video analysis applications and the high level of uncertainties associated with the real world unstructured environments occupied by various users. Machine vision based systems can help detect and summarize important information which cannot be detected by any other sensor; for example, how much water a candidate drank and whether or not they had something to eat. However, conventional non-fuzzy based methods are not robust enough to recognize the various complex types of behaviour in AAL applications. Fuzzy logic system (FLS) is an established field of research to robustly handle uncertainties in complicated real-world problems. In this thesis, we will present a general recognition and classification framework based on fuzzy logic systems which allows for behaviour recognition and event summarisation using 2D/3D video sensors in AAL applications. I started by investigating the use of 2D CCTV camera based system where I proposed and developed novel IT2FLS-based methods for silhouette extraction and 2D behaviour recognition which outperform the traditional on the publicly available Weizmann human action dataset. I will also present a novel system based on 3D RGB-D vision sensors and Interval Type-2 Fuzzy Logic based Systems (IT2FLSs) ) generated by the Big Bang Big Crunch (BB-BC) algorithm for the real time automatic detection and summarization of important events and human behaviour. I will present several real world experiments which were conducted for AAL related behaviour with various users. It will be shown that the proposed BB-BC IT2FLSs outperforms its Type-1 FLSs (T1FLSs) counterpart as well as other conventional non-fuzzy methods, and that performance improvement rises when the number of subjects increases. It will be shown that by utilizing the recognized output activity together with relevant event descriptions (such as video data, timestamp, location and user identification) detailed events are efficiently summarized and stored in our back-end SQL event database, which provides services including event searching, activity retrieval and high-definition video playback to the front-end user interfaces

A Fuzzy Logic-Based System for Soccer Video Scenes Classification

Massive global video surveillance worldwide captures data but lacks detailed activity information to flag events of interest, while the human burden of monitoring video footage is untenable. Artificial intelligence (AI) can be applied to raw video footage to identify and extract required information and summarize it in linguistic formats. Video summarization automation usually involves text-based data such as subtitles, segmenting text and semantics, with little attention to video summarization in the processing of video footage only. Classification problems in recorded videos are often very complex and uncertain due to the dynamic nature of the video sequence and light conditions, background, camera angle, occlusions, indistinguishable scene features, etc. Video scene classification forms the basis of linguistic video summarization, an open research problem with major commercial importance. Soccer video scenes present added challenges due to specific objects and events with similar features (e.g. “people” include audiences, coaches, and players), as well as being constituted from a series of quickly changing and dynamic frames with small inter-frame variations. There is an added difficulty associated with the need to have light weight video classification systems working in real time with massive data sizes. In this thesis, we introduce a novel system based on Interval Type-2 Fuzzy Logic Classification Systems (IT2FLCS) whose parameters are optimized by the Big Bang–Big Crunch (BB-BC) algorithm, which allows for the automatic scenes classification using optimized rules in broadcasted soccer matches video. The type-2 fuzzy logic systems would be unequivocal to present a highly interpretable and transparent model which is very suitable for the handling the encountered uncertainties in video footages and converting the accumulated data to linguistic formats which can be easily stored and analysed. Meanwhile the traditional black box techniques, such as support vector machines (SVMs) and neural networks, do not provide models which could be easily analysed and understood by human users. The BB-BC optimization is a heuristic, population-based evolutionary approach which is characterized by the ease of implementation, fast convergence and low computational cost. We employed the BB-BC to optimize our system parameters of fuzzy logic membership functions and fuzzy rules. Using the BB-BC we are able to balance the system transparency (through generating a small rule set) together with increasing the accuracy of scene classification. Thus, the proposed fuzzy-based system allows achieving relatively high classification accuracy with a small number of rules thus increasing the system interpretability and allowing its real-time processing. The type-2 Fuzzy Logic Classification System (T2FLCS) obtained 87.57% prediction accuracy in the scene classification of our testing group data which is better than the type-1 fuzzy classification system and neural networks counterparts. The BB-BC optimization algorithms decrease the size of rule bases both in T1FLCS and T2FLCS; the T2FLCS finally got 85.716% with reduce rules, outperforming the T1FLCS and neural network counterparts, especially in the “out-of-range data” which validates the T2FLCSs capability to handle the high level of faced uncertainties. We also presented a novel approach based on the scenes classification system combined with the dynamic time warping algorithm to implement the video events detection for real world processing. The proposed system could run on recorded or live video clips and output a label to describe the event in order to provide the high level summarization of the videos to the user

Enabling Field Force Operational Sustainability: A Big Bang-Big Crunch Type-2 Fuzzy Logic System for Goal-Driven Simulation

Business operational sustainability must allow creating economic value, building healthy ecosystems and developing strong communities. Hence, there is a need to develop solutions which can safeguard companies' business sustainability. Various solutions could have different costs and deliver different benefits. Therefore, there is a need to evaluate these solutions before being implemented. In reality, companies require achieving certain targets according to their plans and strategies. Goal-Driven Simulation (GDS) is an approach that allows evaluating solutions before implementing them in real-life while focusing on achieving desired targets. This paper presents a GDS based on interval type-2 Fuzzy Logic System (IT2FLS) optimized by the big bang-big crunch (BU-BC) algorithm with application to field force allocation within the telecommunications sector. The obtained results show the suitability of the proposed approach to model unexpected factors to protect the business sustainability in the telecommunications industry field force allocation domain

A type-2 fuzzy logic based goal-driven simulation for optimising field service delivery

This thesis develops an intelligent system capable of incorporating the conditions that drive operational activity while implementing the means to handle unexpected factors to protect business sustainability. This solution aims to optimise field service operations in the utility-based industry, especially within one of the world's leading communications services companies, namely BT (British Telecom), which operates in highly regulated and competitive markets. Notably, the telecommunication sector is an essential driver of economic activity. Consequently, intelligent solutions must incorporate the ability to explain their underlying algorithms that power their final decisions to humans. In this regard, this thesis studies the following research gaps: the lack of integrated solutions that go beyond isolated monolithic architectures, the lack of agile end-to-end frameworks for handling uncertainty while business targets are defined, current solutions that address target-oriented problems do not incorporate explainable methodologies; as a result, limited explainability features result in inapplicability for highly regulated industries, and most tools do not support scalability for real-world scenarios. Hence, the need for an integrated, intelligent solution to address these target-oriented simulation problems. This thesis aims to reduce the gaps mentioned above by exploiting fuzzy logic capabilities such as mimicking human thinking and handling uncertainty. Moreover, this thesis also finds support in the Explainable AI field, particularly in the strategies and characteristics to deploy more transparent intelligent solutions that humans can understand. Hence, these foundations support the thesis to unlock explainability, transparency and interpretability. This thesis develops a series of techniques with the following features: the formalisation of an end-to-end framework that dynamically learns form data, the implementation of a novel fuzzy membership correlation analysis approach to enhance performance, the development of a novel fuzzy logic-based method to evaluate the relevancy of inputs, the modelling of a robust optimisation method for operational sustainability in the telecommunications sector, the design of an agile modelling approach for scalability and consistency, the formalisation of a novel fuzzy-logic system for goal-driven simulation for achieving specific business targets before being implemented in real-life conditions, and a novel simulation environment that incorporates visual tools to enhance interpretability while moving from conventional simulation to a target-oriented model. The proposed tool was developed based on data from BT, reflecting their real-world operational conditions. The data was protected and anonymised in compliance with BT’s sharing of information regulations. The techniques presented in the development of this thesis yield significant improvements aligned to institutional targets. Precisely, as detailed in Section 9.5, the proposed system can model a reduction between 3.78% and 5.36% of footprint carbon emission due to travel times for jobs completion on customer premises for specific geographical areas. The proposed framework allows generating simulation scenarios 13 times faster than conventional approaches. As described in Section 9.6, these improvements contribute to increased productivity and customer satisfaction metrics regarding keeping appointment times, completing orders in the promised timeframe or fixing faults when agreed by an estimated 2.6%. The proposed tool allows to evaluate decisions before acting; as detailed in Section 9.7, this contributes to the ‘promoters’ minus ‘detractors’ across business units measure by an estimated 1%

An Explainable AI Approach to Process Data in Mixed Reality Environments for Field Service Operations

Digital Twins is a concept that describes how physical objects can be represented and connected to the virtual world, the main goal of a Digital Twin is to centralise all the available information of an object of interest in a single virtual model. The Digital Twin consist of three main components: the physical object, a virtual representation of that object (typically a 3D model), and a real-time connection between both objects so that any change can be communicated to the other part. The possibility of understanding, visualising, and interacting with physical objects through a virtual environment is, at a very high level, the main benefit of using Digital Twins. The adoption of this concept has grown a lot in the recent years in industries such as the manufacturing, construction, health, and energy. Utility companies in the telecommunication industry, water services, and gas services are still falling behind in the adoption of these new concepts. The potential benefit for these sectors is huge where some of these benefits are real-time remote monitoring, predictive maintenance, scenario and risk assessment, better collaboration between stakeholders (internal and external), and better documentation. Existing Mixed Reality, Virtual Reality and Augmented Reality technologies can help with the interaction and visualisation of the virtual twin. The different levels of reality in combination with the digital twins will help with different tasks, for example, Virtual Reality is useful for remote tasks were most of the interaction happens with the virtual twin and Augmented Reality will help bringing the virtual twin and all its information to onsite tasks to help field engineers. However, there are different challenges when trying to connect all the different components and some of these challenges did slow down the adoption of these technologies by the utility companies. The research work in this thesis will focus on two main challenges: the cost of creating these digital twins from existing sources of information and the lack of an explainable AI approach that can be used as an enabler for the interaction between human and Digital Twin in the mixed reality environment. To address the challenge of automating the creation of digital representations at a low cost, two interval type-2 Fuzzy Rule-based Systems are presented as the best explainable AI alternatives to the opaque AI models for processing images and extracting information of the objects of interest. One of them was used to extract information about trees in a satellite image and generate a 3D representation of the geographic area combined with terrain data. This will be used for remote scenario and risk assessment and prediction of the telecommunication equipment getting damaged by natural elements like trees. The proposed approach achieved an 86.90% of accuracy, 3.5% better than the type-1 but 3.0% worse than the opaque Multilayer Perceptron model. The second interval type-2 Fuzzy Rule-based System is an explainable AI model that incorporates the use of context information in its rule to process 2D floor plan images, identify elements of interest and create a 3D digital representation of the building floors. This will benefit the telecommunication company by automating, at a low cost, the process of creating a more detailed in-building map with the telecommunication assets and improve the collaboration with external stakeholders like contractors for maintenance tasks or construction companies for any works in the building. The proposed method achieved a 97.5% Intersection over Union metric value which was comparable to the 99.3% Intersection over Union of the opaque Convolutional Neural Network model, however our proposed solution is highly interpretable and augmentable by human experts which cannot be achieved via opaque box AI models. Additionally, another interval type-2 Fuzzy Rule-based System for hand gesture classification is also presented in this thesis. This rule-based system achieved a 96.4% accuracy, and it is an easily adjustable model that can be modified to include more hand gestures, the opaque model alternative, a K-Nearest Neighbour algorithm achieved a 98.9% accuracy, however, this model cannot be easily modified by a human expert and re-training is needed which results in a cost of time. This hand gesture recognition model, alongside another fuzzy rule-based system, will help to address the challenge of the interaction between human and digital twin objects in Mixed Reality environments

A Type-2 Fuzzy Based Explainable AI System for Predictive Maintenance within the Water Pumping Industry

Industrial maintenance has undergone a paradigm shift due to the emergence of artificial intelligence (AI), the Internet of Things (IoT), and cloud computing. Rather than accepting the drawbacks of reactive maintenance, leading firms worldwide are embracing "predict-and-prevent" maintenance. However, opaque box AI models are sophisticated and complex for the average user to comprehend and explain. This limits the AI employment in predictive maintenance, where it is vital to understand and evaluate the model before deployment. In addition, it's also important to comprehend the maintenance system's decisions. This paper presents a type-2 fuzzy-based Explainable AI (XAI) system for predictive maintenance within the water pumping industry. The proposed system is optimised via Big-Bang Big-Crunch (BB-BC), which maximises the model accuracy for predicting faults while maximising model interpretability. We evaluated the proposed system on water pumps using real-time data obtained by our hardware placed at real-world locations around the United Kingdom and compared our model with Type-1 Fuzzy Logic System (T1FLS), a Multi-Layer Perceptron (MLP) Neural Network, an effective deep learning method known as stacked autoencoders (SAEs) and an interpretable model like decision trees (DT). The proposed system predicted water pumping equipment failures with good accuracy (outperforming the T1FLS accuracy by 8.9% and DT by 529.2% while providing comparable results to SAEs and MLPs) and interpretability. The system predictions comprehend why a specific problem may occur, which leads to better and more informed customer visits to reduce equipment failure disturbances. It will be shown that 80.3% of water industry specialists strongly agree with the model's explanation, determining its acceptance

Fuzzy Logic in Surveillance Big Video Data Analysis: Comprehensive Review, Challenges, and Research Directions

CCTV cameras installed for continuous surveillance generate enormous amounts of data daily, forging the term “Big Video Data” (BVD). The active practice of BVD includes intelligent surveillance and activity recognition, among other challenging tasks. To efficiently address these tasks, the computer vision research community has provided monitoring systems, activity recognition methods, and many other computationally complex solutions for the purposeful usage of BVD. Unfortunately, the limited capabilities of these methods, higher computational complexity, and stringent installation requirements hinder their practical implementation in real-world scenarios, which still demand human operators sitting in front of cameras to monitor activities or make actionable decisions based on BVD. The usage of human-like logic, known as fuzzy logic, has been employed emerging for various data science applications such as control systems, image processing, decision making, routing, and advanced safety-critical systems. This is due to its ability to handle various sources of real world domain and data uncertainties, generating easily adaptable and explainable data-based models. Fuzzy logic can be effectively used for surveillance as a complementary for huge-sized artificial intelligence models and tiresome training procedures. In this paper, we draw researchers’ attention towards the usage of fuzzy logic for surveillance in the context of BVD. We carry out a comprehensive literature survey of methods for vision sensory data analytics that resort to fuzzy logic concepts. Our overview highlights the advantages, downsides, and challenges in existing video analysis methods based on fuzzy logic for surveillance applications. We enumerate and discuss the datasets used by these methods, and finally provide an outlook towards future research directions derived from our critical assessment of the efforts invested so far in this exciting field

An Explainable Artificial Intelligence Approach Based on Deep Type-2 Fuzzy Logic System

Artificial intelligence (AI) systems have benefitted from the easy availability of computing power and the rapid increase in the quantity and quality of data which has led to the widespread adoption of AI techniques across a wide variety of fields. However, the use of complex (or Black box) AI systems such as Deep Neural Networks, support vector machines, etc., could lead to a lack of transparency. This lack of transparency is not specific to deep learning or complex AI algorithms; other interpretable AI algorithms such as kernel machines, logistic regressions, decision trees, or rules-based algorithms can also become difficult to interpret for high dimensional inputs. The lack of transparency or explainability reduces the effectiveness of AI models in regulated applications (such as medical, financial, etc.), where it is essential to explain the model operation and how it arrived at a given prediction. The need for explainability in AI has led to a new line of research that focuses on developing Explainable AI techniques. There are three main avenues of research that are being explored to achieve explainability; first, Deep Explanations, which involves the modification of existing Deep learning models to add explainability. The methods proposed to do Deep explanations generally provide details about all the input features that affect the output, generally in a visual format as there might be a large number of features. This type of explanation is useful for tasks such as image recognition, but in other tasks, it might be hard to distinguish the most important features. Second, Model induction, which involves methods that are model agnostic, but these methods might not be suitable for use in regulated applications. The third method is to use existing interpretable models such as decision trees, fuzzy logic, etc., but the problem with them is that they can also become opaque for high dimensional data. Hence, this thesis presents a novel AI system by combining the predictive power of Deep Learning with the interpretability of Interval Type-2 Fuzzy Logic Systems. The advantages of such a system are, first, the ability to be trained via labelled and unlabelled data (i.e., mixing supervised and unsupervised learning). Second, having embedded feature selection abilities (i.e., can be trained by hundreds and thousands of inputs with no need for feature selection) while delivering explainable models with small rules bases composed of short rules to maximize the model’s interpretability. The proposed model was developed with data from British Telecom (BT). It achieved comparable performance to the deep models such as Stacked Autoencoder (SAE) and Convolution Neural Networks (CNN). In categorical datasets, the model outperformed the SAE by 2%, performed within 2-3% of the CNN and outperformed Multi-Layer Perceptron (MLP) and IT2FLS by 4%. In the regression datasets, the model performed slightly worse than the SAE, MLP and CNN models, but it outperformed the IT2FLS with a 15% lower error. The proposed model achieved excellent interpretability in a survey where it was rated within 2% of the highly interpretable IT2FLS. It was also rated 20% and 17% better than Deep learning XAI tools LIME and SHAP, respectively. The proposed model shows a small loss in performance for significantly higher interpretability, making it a suitable replacement for the other AI models in applications with many features where interpretability is paramount

Lightning search algorithm: a comprehensive survey

The lightning search algorithm (LSA) is a novel meta-heuristic optimization method, which is proposed in 2015 to solve constraint optimization problems. This paper presents a comprehensive survey of the applications, variants, and results of the so-called LSA. In LSA, the best-obtained solution is defined to improve the effectiveness of the fitness function through the optimization process by finding the minimum or maximum costs to solve a specific problem. Meta-heuristics have grown the focus of researches in the optimization domain, because of the foundation of decision-making and assessment in addressing various optimization problems. A review of LSA variants is displayed in this paper, such as the basic, binary, modification, hybridization, improved, and others. Moreover, the classes of the LSA’s applications include the benchmark functions, machine learning applications, network applications, engineering applications, and others. Finally, the results of the LSA is compared with other optimization algorithms published in the literature. Presenting a survey and reviewing the LSA applications is the chief aim of this survey paper