Unconstrained Road Sign Recognition

There are many types of road signs, each of which carries a different meaning and function: some signs regulate traffic, others indicate the state of the road or guide and warn drivers and pedestrians. Existent image-based road sign recognition systems work well under ideal conditions, but experience problems when the lighting conditions are poor or the signs are partially occluded. The aim of this research is to propose techniques to recognize road signs in a real outdoor environment, especially to deal with poor lighting and partially occluded road signs. To achieve this, hybrid segmentation and classification algorithms are proposed. In the first part of the thesis, we propose a hybrid dynamic threshold colour segmentation algorithm based on histogram analysis. A dynamic threshold is very important in road sign segmentation, since road sign colours may change throughout the day due to environmental conditions. In the second part, we propose a geometrical shape symmetry detection and reconstruction algorithm to detect and reconstruct the shape of the sign when it is partially occluded. This algorithm is robust to scale changes and rotations. The last part of this thesis deals with feature extraction and classification. We propose a hybrid feature vector based on histograms of oriented gradients, local binary patterns, and the scale-invariant feature transform. This vector is fed into a classifier that combines a Support Vector Machine (SVM) using a Random Forest and a hybrid SVM k-Nearest Neighbours (kNN) classifier. The overall method proposed in this thesis shows a high accuracy rate of 99.4% in ideal conditions, 98.6% in noisy and fading conditions, 98.4% in poor lighting conditions, and 92.5% for partially occluded road signs on the GRAMUAH traffic signs dataset

WEATHER LORE VALIDATION TOOL USING FUZZY COGNITIVE MAPS BASED ON COMPUTER VISION

Published ThesisThe creation of scientific weather forecasts is troubled by many technological challenges (Stern & Easterling, 1999) while their utilization is generally dismal. Consequently, the majority of small-scale farmers in Africa continue to consult some forms of weather lore to reach various cropping decisions (Baliscan, 2001). Weather lore is a body of informal folklore (Enock, 2013), associated with the prediction of the weather, and based on indigenous knowledge and human observation of the environment. As such, it tends to be more holistic, and more localized to the farmers’ context. However, weather lore has limitations; for instance, it has an inability to offer forecasts beyond a season. Different types of weather lore exist, utilizing almost all available human senses (feel, smell, sight and hearing). Out of all the types of weather lore in existence, it is the visual or observed weather lore that is mostly used by indigenous societies, to come up with weather predictions. On the other hand, meteorologists continue to treat this knowledge as superstition, partly because there is no means to scientifically evaluate and validate it. The visualization and characterization of visual sky objects (such as moon, clouds, stars, and rainbows) in forecasting weather are significant subjects of research. To realize the integration of visual weather lore in modern weather forecasting systems, there is a need to represent and scientifically substantiate this form of knowledge. This research was aimed at developing a method for verifying visual weather lore that is used by traditional communities to predict weather conditions. To realize this verification, fuzzy cognitive mapping was used to model and represent causal relationships between selected visual weather lore concepts and weather conditions. The traditional knowledge used to produce these maps was attained through case studies of two communities (in Kenya and South Africa).These case studies were aimed at understanding the weather lore domain as well as the causal effects between metrological and visual weather lore. In this study, common astronomical weather lore factors related to cloud physics were identified as: bright stars, dispersed clouds, dry weather, dull stars, feathery clouds, gathering clouds, grey clouds, high clouds, layered clouds, low clouds, stars, medium clouds, and rounded clouds. Relationships between the concepts were also identified and formally represented using fuzzy cognitive maps. On implementing the verification tool, machine vision was used to recognize sky objects captured using a sky camera, while pattern recognition was employed in benchmarking and scoring the objects. A wireless weather station was used to capture real-time weather parameters. The visualization tool was then designed and realized in a form of software artefact, which integrated both computer vision and fuzzy cognitive mapping for experimenting visual weather lore, and verification using various statistical forecast skills and metrics. The tool consists of four main sub-components: (1) Machine vision that recognizes sky objects using support vector machine classifiers using shape-based feature descriptors; (2) Pattern recognition–to benchmark and score objects using pixel orientations, Euclidean distance, canny and grey-level concurrence matrix; (3) Fuzzy cognitive mapping that was used to represent knowledge (i.e. active hebbian learning algorithm was used to learn until convergence); and (4) A statistical computing component was used for verifications and forecast skills including brier score and contingency tables for deterministic forecasts. Rigorous evaluation of the verification tool was carried out using independent (not used in the training and testing phases) real-time images from Bloemfontein, South Africa, and Voi-Kenya. The real-time images were captured using a sky camera with GPS location services. The results of the implementation were tested for the selected weather conditions (for example, rain, heat, cold, and dry conditions), and found to be acceptable (the verified prediction accuracies were over 80%). The recommendation in this study is to apply the implemented method for processing tasks, towards verifying all other types of visual weather lore. In addition, the use of the method developed also requires the implementation of modules for processing and verifying other types of weather lore, such as sounds, and symbols of nature. Since time immemorial, from Australia to Asia, Africa to Latin America, local communities have continued to rely on weather lore observations to predict seasonal weather as well as its effects on their livelihoods (Alcock, 2014). This is mainly based on many years of personal experiences in observing weather conditions. However, when it comes to predictions for longer lead-times (i.e. over a season), weather lore is uncertain (Hornidge & Antweiler, 2012). This uncertainty has partly contributed to the current status where meteorologists and other scientists continue to treat weather lore as superstition (United-Nations, 2004), and not capable of predicting weather. One of the problems in testing the confidence in weather lore in predicting weather is due to wide varieties of weather lore that are found in the details of indigenous sayings, which are tightly coupled to locality and pattern variations(Oviedo et al., 2008). This traditional knowledge is entrenched within the day-to-day socio-economic activities of the communities using it and is not globally available for comparison and validation (Huntington, Callaghan, Fox, & Krupnik, 2004). Further, this knowledge is based on local experience that lacks benchmarking techniques; so that harmonizing and integrating it within the science-based weather forecasting systems is a daunting task (Hornidge & Antweiler, 2012). It is partly for this reason that the question of validation of weather lore has not yet been substantially investigated. Sufficient expanded processes of gathering weather observations, combined with comparison and validation, can produce some useful information. Since forecasting weather accurately is a challenge even with the latest supercomputers (BBC News Magazine, 2013), validated weather lore can be useful if it is incorporated into modern weather prediction systems. Validation of traditional knowledge is a necessary step in the management of building integrated knowledge-based systems. Traditional knowledge incorporated into knowledge-based systems has to be verified for enhancing systems’ reliability. Weather lore knowledge exists in different forms as identified by traditional communities; hence it needs to be tied together for comparison and validation. The development of a weather lore validation tool that can integrate a framework for acquiring weather data and methods of representing the weather lore in verifiable forms can be a significant step in the validation of weather lore against actual weather records using conventional weather-observing instruments. The success of validating weather lore could stimulate the opportunity for integrating acceptable weather lore with modern systems of weather prediction to improve actionable information for decision making that relies on seasonal weather prediction. In this study a hybrid method is developed that includes computer vision and fuzzy cognitive mapping techniques for verifying visual weather lore. The verification tool was designed with forecasting based on mimicking visual perception, and fuzzy thinking based on the cognitive knowledge of humans. The method provides meaning to humanly perceivable sky objects so that computers can understand, interpret, and approximate visual weather outcomes. Questionnaires were administered in two case study locations (KwaZulu-Natal province in South Africa, and Taita-Taveta County in Kenya), between the months of March and July 2015. The two case studies were conducted by interviewing respondents on how visual astronomical and meteorological weather concepts cause weather outcomes. The two case studies were used to identify causal effects of visual astronomical and meteorological objects to weather conditions. This was followed by finding variations and comparisons, between the visual weather lore knowledge in the two case studies. The results from the two case studies were aggregated in terms of seasonal knowledge. The causal links between visual weather concepts were investigated using these two case studies; results were compared and aggregated to build up common knowledge. The joint averages of the majority of responses from the case studies were determined for each set of interacting concepts. The modelling of the weather lore verification tool consists of input, processing components and output. The input data to the system are sky image scenes and actual weather observations from wireless weather sensors. The image recognition component performs three sub-tasks, including: detection of objects (concepts) from image scenes, extraction of detected objects, and approximation of the presence of the concepts by comparing extracted objects to ideal objects. The prediction process involves the use of approximated concepts generated in the recognition component to simulate scenarios using the knowledge represented in the fuzzy cognitive maps. The verification component evaluates the variation between the predictions and actual weather observations to determine prediction errors and accuracy. To evaluate the tool, daily system simulations were run to predict and record probabilities of weather outcomes (i.e. rain, heat index/hotness, dry, cold index). Weather observations were captured periodically using a wireless weather station. This process was repeated several times until there was sufficient data to use for the verification process. To match the range of the predicted weather outcomes, the actual weather observations (measurement) were transformed and normalized to a range [0, 1].In the verification process, comparisons were made between the actual observations and weather outcome prediction values by computing residuals (error values) from the observations. The error values and the squared error were used to compute the Mean Squared Error (MSE), and the Root Mean Squared Error (RMSE), for each predicted weather outcome. Finally, the validity of the visual weather lore verification model was assessed using data from a different geographical location. Actual data in the form of daily sky scenes and weather parameters were acquired from Voi, Kenya, from December 2015 to January 2016.The results on the use of hybrid techniques for verification of weather lore is expected to provide an incentive in integrating indigenous knowledge on weather with modern numerical weather prediction systems for accurate and downscaled weather forecasts

Pattern Recognition

Pattern recognition is a very wide research field. It involves factors as diverse as sensors, feature extraction, pattern classification, decision fusion, applications and others. The signals processed are commonly one, two or three dimensional, the processing is done in real- time or takes hours and days, some systems look for one narrow object class, others search huge databases for entries with at least a small amount of similarity. No single person can claim expertise across the whole field, which develops rapidly, updates its paradigms and comprehends several philosophical approaches. This book reflects this diversity by presenting a selection of recent developments within the area of pattern recognition and related fields. It covers theoretical advances in classification and feature extraction as well as application-oriented works. Authors of these 25 works present and advocate recent achievements of their research related to the field of pattern recognition

Investigation of iris recognition in the visible spectrum

mong the biometric systems that have been developed so far, iris recognition systems have emerged as being one of the most reliable. In iris recognition, most of the research was conducted on operation under near infrared illumination. For unconstrained scenarios of iris recognition systems, the iris images are captured under visible light spectrum and therefore incorporate various types of imperfections. In this thesis the merits of fusing information from various sources for improving the state of the art accuracies of colour iris recognition systems is evaluated. An investigation of how fundamentally different fusion strategies can increase the degree of choice available in achieving certain performance criteria is conducted. Initially, simple fusion mechanisms are employed to increase the accuracy of an iris recognition system and then more complex fusion architectures are elaborated to further enhance the biometric system’s accuracy. In particular, the design process of the iris recognition system with reduced constraints is carried out using three different fusion approaches: multi-algorithmic, texture and colour fusion and multiple classifier systems. In the first approach, one novel iris feature extraction methodology is proposed and a multi-algorithmic iris recognition system using score fusion, composed of 3 individual systems, is benchmarked. In the texture and colour fusion approach, the advantages of fusing information from the iris texture with data extracted from the eye colour are illustrated. Finally, the multiple classifier systems approach investigates how the robustness and practicability of an iris recognition system operating on visible spectrum images can be enhanced by training individual classifiers on different iris features. Besides the various fusion techniques explored, an iris segmentation algorithm is proposed and a methodology for finding which colour channels from a colour space reveal the most discriminant information from the iris texture is introduced. The contributions presented in this thesis indicate that iris recognition systems that operate on visible spectrum images can be designed to operate with an accuracy required by a particular application scenario. Also, the iris recognition systems developed in the present study are suitable for mobile and embedded implementations

Computational intelligence techniques for maritime and coastal remote sensing

The aim of this thesis is to investigate the potential of computational intelligence techniques for some applications in the analysis of remotely sensed multi-spectral images. In particular, two problems are addressed. The first one is the classification of oil spills at sea, while the second one is the estimation of sea bottom depth. In both cases, the exploitation of optical satellite data allows to develop operational tools for easily accessing and monitoring large marine areas, in an efficient and cost effective way. Regarding the oil spill problem, today public opinion is certainly aware of the huge impact that oil tanker accidents and oil rig leaks have on marine and coastal environment. However, it is less known that most of the oil released in our seas cannot be ascribed to accidental spills, but rather to illegal ballast waters discharge, and to pollutant dumping at sea, during routine operations of oil tankers. For this reason, any effort for improving oil spill detection systems is of great importance. So far, Synthetic Aperture Radar (SAR) data have been preferred to multi-spectral data for oil spill detection applications, because of their all-weather and all-day capabilities, while optical images necessitate of clear sky conditions and day-light. On the other hand, many features make an optical approach desirable, such as lower cost and higher revisit time. Moreover, unlike SAR data, optical data are not affected by sea state, and are able to reduce false alarm rate, since they do not suffer from the main false alarm source in SAR data, that is represented by the presence of calm sea regions. In this thesis the problem of oil spill classification is tackled by applying different machine learning techniques to a significant dataset of regions of interest, collected in multi-spectral satellite images, acquired by MODIS sensor. These regions are then classified in one of two possible classes, that are oil spills and look-alikes, where look-alikes include any phenomena other than oil spills (e.g. algal blooms...). Results show that efficient and reliable oil spill classification systems based on optical data are feasible, and could offer a valuable support to the existing satellite-based monitoring systems. The estimation of sea bottom depth from high resolution multi-spectral satellite images is the second major topic of this thesis. The motivations for dealing with this problem arise from the necessity of limiting expensive and time consuming measurement campaigns. Since satellite data allow to quickly analyse large areas, a solution for this issue is to employ intelligent techniques, which, by exploiting a small set of depth measurements, are able to extend bathymetry estimate to a much larger area, covered by a multi-spectral satellite image. Such techniques, once that the training phase has been completed, allow to achieve very accurate results, and, thanks to their generalization capabilities, provide reliable bathymetric maps which cover wide areas. A crucial element is represented by the training dataset, which is built by coupling a number of depth measurements, located in a limited part of the image, with corresponding radiances, acquired by the satellite sensor. A successful estimate essentially depends on how the training dataset resembles the rest of the scene. On the other hand, the result is not affected by model uncertainties and systematic errors, as results from model-based analytic approaches are. In this thesis a neuro-fuzzy technique is applied to two case studies, more precisely, two high resolution multi-spectral images related to the same area, but acquired in different years and in different meteorological conditions. Different situations of in-situ depths availability are considered in the study, and the effect of limited in-situ data availability on performance is evaluated. The effect of both meteorological conditions and training set size reduction on the overall performance is also taken into account. Results outperform previous studies on bathymetry estimation techniques, and allow to give indications on the optimal paths which can be adopted when planning data collection at sea

Virtual Reality Games for Motor Rehabilitation

This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any product’s acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion

Assembly Line

An assembly line is a manufacturing process in which parts are added to a product in a sequential manner using optimally planned logistics to create a finished product in the fastest possible way. It is a flow-oriented production system where the productive units performing the operations, referred to as stations, are aligned in a serial manner. The present edited book is a collection of 12 chapters written by experts and well-known professionals of the field. The volume is organized in three parts according to the last research works in assembly line subject. The first part of the book is devoted to the assembly line balancing problem. It includes chapters dealing with different problems of ALBP. In the second part of the book some optimization problems in assembly line structure are considered. In many situations there are several contradictory goals that have to be satisfied simultaneously. The third part of the book deals with testing problems in assembly line. This section gives an overview on new trends, techniques and methodologies for testing the quality of a product at the end of the assembling line

Feature extraction and selection algorithm based on self adaptive ant colony system for sky image classification

Sky image classification is crucial in meteorology to forecast weather and climatic conditions. The fine-grained cloud detection and recognition (FGCDR) algorithm is use to extract colour, inside texture and neighbour texture features from multiview of superpixels sky images. However, the FGCDR produced a substantial amount of redundant and insignificant features. The ant colony optimisation (ACO) algorithm have been used to select feature subset. However, the ACO suffers from premature convergence which leads to poor feature subset. Therefore, an improved feature extraction and selection for sky image classification (FESSIC) algorithm is proposed. This algorithm consists of (i) Gaussian smoothness standard deviation method that formulates informative features within sky images; (ii) nearest-threshold based technique that converts feature map into a weighted directed graph to represent relationship between features; and (iii) an ant colony system with self-adaptive parameter technique for local pheromone update. The performance of FESSIC was evaluated against ten benchmark image classification algorithms and six classifiers on four ground-based sky image datasets. The Friedman test result is presented for the performance rank of six benchmark feature selection algorithms and FESSIC algorithm. The Man-Whitney U test is then performed to statistically evaluate the significance difference of the second rank and FESSIC algorithms. The experimental results for the proposed algorithm are superior to the benchmark image classification algorithms in terms of similarity value on Kiel, SWIMCAT and MGCD datasets. FESSIC outperforms other algorithms for average classification accuracy for the KSVM, MLP, RF and DT classifiers. The Friedman test has shown that the FESSIC has the first rank for all classifiers. Furthermore, the result of Man-Whitney U test indicates that FESSIC is significantly better than the second rank benchmark algorithm for all classifiers. In conclusion, the FESSIC can be utilised for image classification in various applications such as disaster management, medical diagnosis, industrial inspection, sports management, and content-based image retrieval

Robust Modular Feature-Based Terrain-Aided Visual Navigation and Mapping

The visual feature-based Terrain-Aided Navigation (TAN) system presented in this thesis addresses the problem of constraining inertial drift introduced into the location estimate of Unmanned Aerial Vehicles (UAVs) in GPS-denied environment. The presented TAN system utilises salient visual features representing semantic or human-interpretable objects (roads, forest and water boundaries) from onboard aerial imagery and associates them to a database of reference features created a-priori, through application of the same feature detection algorithms to satellite imagery. Correlation of the detected features with the reference features via a series of the robust data association steps allows a localisation solution to be achieved with a finite absolute bound precision defined by the certainty of the reference dataset. The feature-based Visual Navigation System (VNS) presented in this thesis was originally developed for a navigation application using simulated multi-year satellite image datasets. The extension of the system application into the mapping domain, in turn, has been based on the real (not simulated) flight data and imagery. In the mapping study the full potential of the system, being a versatile tool for enhancing the accuracy of the information derived from the aerial imagery has been demonstrated. Not only have the visual features, such as road networks, shorelines and water bodies, been used to obtain a position ’fix’, they have also been used in reverse for accurate mapping of vehicles detected on the roads into an inertial space with improved precision. Combined correction of the geo-coding errors and improved aircraft localisation formed a robust solution to the defense mapping application. A system of the proposed design will provide a complete independent navigation solution to an autonomous UAV and additionally give it object tracking capability