Metaheuristic design of feedforward neural networks: a review of two decades of research

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era

Artificial Intelligence and Machine Learning Approaches to Energy Demand-Side Response: A Systematic Review

Recent years have seen an increasing interest in Demand Response (DR) as a means to provide flexibility, and hence improve the reliability of energy systems in a cost-effective way. Yet, the high complexity of the tasks associated with DR, combined with their use of large-scale data and the frequent need for near real-time de-cisions, means that Artificial Intelligence (AI) and Machine Learning (ML) — a branch of AI — have recently emerged as key technologies for enabling demand-side response. AI methods can be used to tackle various challenges, ranging from selecting the optimal set of consumers to respond, learning their attributes and pref-erences, dynamic pricing, scheduling and control of devices, learning how to incentivise participants in the DR schemes and how to reward them in a fair and economically efficient way. This work provides an overview of AI methods utilised for DR applications, based on a systematic review of over 160 papers, 40 companies and commercial initiatives, and 21 large-scale projects. The papers are classified with regards to both the AI/ML algorithm(s) used and the application area in energy DR. Next, commercial initiatives are presented (including both start-ups and established companies) and large-scale innovation projects, where AI methods have been used for energy DR. The paper concludes with a discussion of advantages and potential limitations of reviewed AI techniques for different DR tasks, and outlines directions for future research in this fast-growing area

Advances in Meta-Heuristic Optimization Algorithms in Big Data Text Clustering

This paper presents a comprehensive survey of the meta-heuristic optimization algorithms on the text clustering applications and highlights its main procedures. These Artificial Intelligence (AI) algorithms are recognized as promising swarm intelligence methods due to their successful ability to solve machine learning problems, especially text clustering problems. This paper reviews all of the relevant literature on meta-heuristic-based text clustering applications, including many variants, such as basic, modified, hybridized, and multi-objective methods. As well, the main procedures of text clustering and critical discussions are given. Hence, this review reports its advantages and disadvantages and recommends potential future research paths. The main keywords that have been considered in this paper are text, clustering, meta-heuristic, optimization, and algorithm

Clustering in Recommendation Systems Using Swarm Intelligence

Ένα σύστημα συστάσεων είναι μία εφαρμογή που εκμεταλλεύεται πληροφορίες για να βοηθήσει τους χρήστες στη λήψη αποφάσεων προτείνοντας αντικείμενα που μπορεί να τους αρέσουν. Ένα σύστημα συστάσεων που βασίζεται στην τεχνική του συνεργατικού φιλτραρίσματος (collaborative filtering) δημιουργεί συστάσεις στους χρήστες με βάση τις προτιμήσεις παρόμοιων χρηστών. Ωστόσο, αυτός ο τύπος συστήματος συστάσεων δεν είναι τόσο αποτελεσματικός όταν τα δεδομένα αυξάνονται σε μεγάλο βαθμό (scalability) ή όταν δεν υπάρχει αρκετή πληροφορία (sparsity), καθώς δεν ομαδοποιούνται σωστά οι παρόμοιοι χρήστες. Αυτή η διπλωματική εργασία προτείνει τρείς υβριδικούς αλγορίθμους που ο καθένας συνδυάζει τον αλγόριθμο k-means με έναν αλγόριθμο ευφυΐας σμήνους για να βελτιώσει την ομαδοποίηση των χρηστών, και κατ’ επέκταση την ποιότητα των συστάσεων. Οι αλγόριθμοι ευφυΐας σμήνους που χρησιμοποιούνται είναι o αλγόριθμος τεχνητής κοινωνίας μελισσών (artificial bee colony), ο αλγόριθμος βελτιστοποίησης αναζήτησης κούκων (cuckoo search optimization) και ο αλγόριθμος βελτιστοποίησης γκρίζων λύκων (grey-wolf optimization). Οι προτεινόμενες μέθοδοι αξιολογήθηκαν χρησιμοποιώντας ένα σύνολο δεδομένων του MovieLens. Η αξιολόγηση δείχνει πως τα προτεινόμενα συστήματα συστάσεων αποδίδουν καλύτερα σε σύγκριση με τις ήδη υπάρχουσες τεχνικές όσον αφορά τις μετρικές του μέσου απόλυτου σφάλματος (mean absolute error - MAE), της ακρίβειας (precision), του αθροίσματος των τετραγωνικών σφαλμάτων (sum of squared errors - SSE) και της ανάκλησης (recall). Επιπλέον, τα αποτελέσματα της αξιολόγησης δείχνουν πως ο υβριδικός αλγόριθμος που χρησιμοποιεί την μέθοδο της τεχνητής κοινωνίας μελισσών αποδίδει ελαφρώς καλύτερα από τους άλλους δύο προτεινόμενους αλγορίθμους.A recommender system (RS) is an application that exploits information to help users in decision making by suggesting items they might like. A collaborative recommender system generates recommendations to users based on their similar neighbor’s preferences. However, this type of recommender system faces the data sparsity and scalability problems making the neighborhood selection a challenging task. This thesis proposes three hybrid collaborative recommender systems that each one combines the k-means algorithm with a different bio-inspired technique to enhance the clustering task, and therefore to improve the recommendation quality. The used bio-inspired techniques are artificial bee colony (ABC), cuckoo search optimization (CSO), and grey-wolf optimizer (GWO). The proposed approaches were evaluated over a MovieLens dataset. The evaluation shows that the proposed recommender systems perform better compared to already existing techniques in terms of mean absolute error (MAE), precision, sum of squared errors (SSE), and recall. Moreover, the experimental results indicate that the hybrid recommender system that uses the ABC method performs slightly better than the other two proposed hybrid algorithms

Improved Multi-Verse Optimizer In Text Document Clustering For Topic Extraction

This study aims to propose a suitable TE approach, which provides a better overview of the text documents. To achieve this aim: First, A new feature selection method for TDC, that is, binary multi-verse optimizer algorithm (BMVO) is proposed to eliminate irrelevantly, redundant features and obtain a new subset of more informative features. Second, three multi-verse optimizer algorithm (MVOs), namely, basic MVO, modified MVO, hybrid MVO is proposed to solve the TDC problem; these algorithms are incremental improvements of the preceding versions. Third, a novel ensemble method for an automatic TE from a collection of text document is proposed to extract the topics from the clustered document

Projection-Based Clustering through Self-Organization and Swarm Intelligence

It covers aspects of unsupervised machine learning used for knowledge discovery in data science and introduces a data-driven approach to cluster analysis, the Databionic swarm (DBS). DBS consists of the 3D landscape visualization and clustering of data. The 3D landscape enables 3D printing of high-dimensional data structures. The clustering and number of clusters or an absence of cluster structure are verified by the 3D landscape at a glance. DBS is the first swarm-based technique that shows emergent properties while exploiting concepts of swarm intelligence, self-organization and the Nash equilibrium concept from game theory. It results in the elimination of a global objective function and the setting of parameters. By downloading the R package DBS can be applied to data drawn from diverse research fields and used even by non-professionals in the field of data mining

IoT in smart communities, technologies and applications.

Internet of Things is a system that integrates different devices and technologies, removing the necessity of human intervention. This enables the capacity of having smart (or smarter) cities around the world. By hosting different technologies and allowing interactions between them, the internet of things has spearheaded the development of smart city systems for sustainable living, increased comfort and productivity for citizens. The Internet of Things (IoT) for Smart Cities has many different domains and draws upon various underlying systems for its operation, in this work, we provide a holistic coverage of the Internet of Things in Smart Cities by discussing the fundamental components that make up the IoT Smart City landscape, the technologies that enable these domains to exist, the most prevalent practices and techniques which are used in these domains as well as the challenges that deployment of IoT systems for smart cities encounter and which need to be addressed for ubiquitous use of smart city applications. It also presents a coverage of optimization methods and applications from a smart city perspective enabled by the Internet of Things. Towards this end, a mapping is provided for the most encountered applications of computational optimization within IoT smart cities for five popular optimization methods, ant colony optimization, genetic algorithm, particle swarm optimization, artificial bee colony optimization and differential evolution. For each application identified, the algorithms used, objectives considered, the nature of the formulation and constraints taken in to account have been specified and discussed. Lastly, the data setup used by each covered work is also mentioned and directions for future work have been identified. Within the smart health domain of IoT smart cities, human activity recognition has been a key study topic in the development of cyber physical systems and assisted living applications. In particular, inertial sensor based systems have become increasingly popular because they do not restrict users’ movement and are also relatively simple to implement compared to other approaches. Fall detection is one of the most important tasks in human activity recognition. With an increasingly aging world population and an inclination by the elderly to live alone, the need to incorporate dependable fall detection schemes in smart devices such as phones, watches has gained momentum. Therefore, differentiating between falls and activities of daily living (ADLs) has been the focus of researchers in recent years with very good results. However, one aspect within fall detection that has not been investigated much is direction and severity aware fall detection. Since a fall detection system aims to detect falls in people and notify medical personnel, it could be of added value to health professionals tending to a patient suffering from a fall to know the nature of the accident. In this regard, as a case study for smart health, four different experiments have been conducted for the task of fall detection with direction and severity consideration on two publicly available datasets. These four experiments not only tackle the problem on an increasingly complicated level (the first one considers a fall only scenario and the other two a combined activity of daily living and fall scenario) but also present methodologies which outperform the state of the art techniques as discussed. Lastly, future recommendations have also been provided for researchers

A Survey on Evolutionary Computation for Computer Vision and Image Analysis: Past, Present, and Future Trends

Computer vision (CV) is a big and important field in artificial intelligence covering a wide range of applications. Image analysis is a major task in CV aiming to extract, analyse and understand the visual content of images. However, imagerelated tasks are very challenging due to many factors, e.g., high variations across images, high dimensionality, domain expertise requirement, and image distortions. Evolutionary computation (EC) approaches have been widely used for image analysis with significant achievement. However, there is no comprehensive survey of existing EC approaches to image analysis. To fill this gap, this paper provides a comprehensive survey covering all essential EC approaches to important image analysis tasks including edge detection, image segmentation, image feature analysis, image classification, object detection, and others. This survey aims to provide a better understanding of evolutionary computer vision (ECV) by discussing the contributions of different approaches and exploring how and why EC is used for CV and image analysis. The applications, challenges, issues, and trends associated to this research field are also discussed and summarised to provide further guidelines and opportunities for future research

Projection-Based Clustering through Self-Organization and Swarm Intelligence: Combining Cluster Analysis with the Visualization of High-Dimensional Data

Cluster Analysis; Dimensionality Reduction; Swarm Intelligence; Visualization; Unsupervised Machine Learning; Data Science; Knowledge Discovery; 3D Printing; Self-Organization; Emergence; Game Theory; Advanced Analytics; High-Dimensional Data; Multivariate Data; Analysis of Structured Dat

Applied Metaheuristic Computing

For decades, Applied Metaheuristic Computing (AMC) has been a prevailing optimization technique for tackling perplexing engineering and business problems, such as scheduling, routing, ordering, bin packing, assignment, facility layout planning, among others. This is partly because the classic exact methods are constrained with prior assumptions, and partly due to the heuristics being problem-dependent and lacking generalization. AMC, on the contrary, guides the course of low-level heuristics to search beyond the local optimality, which impairs the capability of traditional computation methods. This topic series has collected quality papers proposing cutting-edge methodology and innovative applications which drive the advances of AMC