15 research outputs found
Pedestrian detection in far-infrared daytime images using a hierarchical codebook of SURF
One of the main challenges in intelligent vehicles concerns pedestrian detection for driving assistance. Recent experiments have showed that state-of-the-art descriptors provide better performances on the far-infrared (FIR) spectrum than on the visible one, even in daytime conditions, for pedestrian classification. In this paper, we propose a pedestrian detector with on-board FIR camera. Our main contribution is the exploitation of the specific characteristics of FIR images to design a fast, scale-invariant and robust pedestrian detector. Our system consists of three modules, each based on speeded-up robust feature (SURF) matching. The first module allows generating regions-of-interest (ROI), since in FIR images of the pedestrian shapes may vary in large scales, but heads appear usually as light regions. ROI are detected with a high recall rate with the hierarchical codebook of SURF features located in head regions. The second module consists of pedestrian full-body classification by using SVM. This module allows one to enhance the precision with low computational cost. In the third module, we combine the mean shift algorithm with inter-frame scale-invariant SURF feature tracking to enhance the robustness of our system. The experimental evaluation shows that our system outperforms, in the FIR domain, the state-of-the-art Haar-like Adaboost-cascade, histogram of oriented gradients (HOG)/linear SVM (linSVM) and MultiFtrpedestrian detectors, trained on the FIR images
Pedestrian Detection at Day/Night Time with Visible and FIR Cameras : A Comparison
Altres ajuts: DGT (SPIP2014-01352)Despite all the significant advances in pedestrian detection brought by computer vision for driving assistance, it is still a challenging problem. One reason is the extremely varying lighting conditions under which such a detector should operate, namely day and nighttime. Recent research has shown that the combination of visible and non-visible imaging modalities may increase detection accuracy, where the infrared spectrum plays a critical role. The goal of this paper is to assess the accuracy gain of different pedestrian models (holistic, part-based, patch-based) when training with images in the far infrared spectrum. Specifically, we want to compare detection accuracy on test images recorded at day and nighttime if trained (and tested) using (a) plain color images; (b) just infrared images; and (c) both of them. In order to obtain results for the last item, we propose an early fusion approach to combine features from both modalities. We base the evaluation on a new dataset that we have built for this purpose as well as on the publicly available KAIST multispectral dataset
Foot Detection Method for Footwear Augmented Reality Applications
Liitreaalsus on populaarsust koguv platvorm rõivaste ning aksessuaaride kasutamise visualiseerimiseks. Ideaalis võimaldab see kasutajatel proovida erinevaid riideid, jalatseid ja aksessuaare, kasutades ainult üht kaamerat ning sobivat rakendust, mis võimaldab kuvada erinevaid valikuid.\n\rJalatsite liitreaalsuses on palju erinevaid lahendusi, et pakkuda kasutajatele liitreaalsuse kogemust. Need lahendused kasutavad erinevaid meetodeid, nagu fikseeritud kaamera, muutumatu taust ja markerid jalgadel tuvastuse hõlbustamiseks. Nende meetodite hulgas pole ükski kindlalt parem, lihtsam või kiirem. Lisaks puudub tihtipeale avalikkusel ligipääs arendatud rakendustele.\n\rKäesolev magistritöö proovis leida universaalset lahendust, mis sobiks kasutamiseks kõigi tulevaste jalatsite liitreaalsuse rakendustega.Augmented reality is gaining popularity as a technique for visualizing apparel usage. Ide-ally it allows users virtually to try out different clothes, shoes, and accessories, with only a camera and suitable application which encompasses different apparel choices.\n\rFocusing on augmented reality for footwear, there is a multitude of different solutions on how to offer the reality augmentation experience to the end users. These solutions employ different methods to deliver the end result, such as using fixed camera and constant back-ground or requiring markers on feet for detection. Among the variety of techniques used to approach the footwear reality augmentation, there is no single best, simplest, or fastest solution. The solutions’ sources aren’t usually even publicly available. \n\rThis thesis tries to come up with a solution for the footwear reality augmentation problem, which can be used as a base for any proceeding footwear augmented reality projects. This intentionally universal approach will be created by researching possible combinations of potential methods that can ensure a solutions regarding footwear reality augmentation. \n\rIn general, the idea behind this thesis work is to conduct a literature review about different techniques and come up with the best and robust algorithm or combination of methods that can be used for footwear augmented reality.\n\rA researched, documented, implemented and publicized solution would allow any upcom-ing footwear augmented reality related project to start working from an established base, therefore reducing time waste on already solved issues and possibly improving the quality of the end result.\n\rThe solution presented in this thesis is developed with focus on augmented reality applica-tions. The method is neither specific to any platform nor does it have heavy location re-quirements. The result is a foot detection algorithm, capable of working on commonly available hardware, which is beneficial for augmented reality application
Pedestrian and cyclist detection and intent estimation for autonomous vehicles: A survey
© 2019 by the authors. As autonomous vehicles become more common on the roads, their advancement draws on safety concerns for vulnerable road users, such as pedestrians and cyclists. This paper presents a review of recent developments in pedestrian and cyclist detection and intent estimation to increase the safety of autonomous vehicles, for both the driver and other road users. Understanding the intentions of the pedestrian/cyclist enables the self-driving vehicle to take actions to avoid incidents. To make this possible, development of methods/techniques, such as deep learning (DL), for the autonomous vehicle will be explored. For example, the development of pedestrian detection has been significantly advanced using DL approaches, such as; Fast Region-Convolutional Neural Network (R-CNN), Faster R-CNN and Single Shot Detector (SSD). Although DL has been around for several decades, the hardware to realise the techniques have only recently become viable. Using these DL methods for pedestrian and cyclist detection and applying it for the tracking, motion modelling and pose estimation can allow for a successful and accurate method of intent estimation for the vulnerable road users. Although there has been a growth in research surrounding the study of pedestrian detection using vision-based approaches, further attention should include focus on cyclist detection. To further improve safety for these vulnerable road users (VRUs), approaches such as sensor fusion and intent estimation should be investigated
A Review of Sensor Technologies for Perception in Automated Driving
After more than 20 years of research, ADAS are
common in modern vehicles available in the market. Automated
Driving systems, still in research phase and limited in their
capabilities, are starting early commercial tests in public roads.
These systems rely on the information provided by on-board
sensors, which allow to describe the state of the vehicle, its
environment and other actors. Selection and arrangement of
sensors represent a key factor in the design of the system. This
survey reviews existing, novel and upcoming sensor technologies,
applied to common perception tasks for ADAS and Automated
Driving. They are put in context making a historical review of
the most relevant demonstrations on Automated Driving, focused
on their sensing setup. Finally, the article presents a snapshot of
the future challenges for sensing technologies and perception,
finishing with an overview of the commercial initiatives and
manufacturers alliances that will show future market trends in
sensors technologies for Automated Vehicles.This work has been partly supported by ECSEL Project ENABLE-
S3 (with grant agreement number 692455-2), by the
Spanish Government through CICYT projects (TRA2015-
63708-R and TRA2016-78886-C3-1-R)
Improving the utilization of training samples in visual recognition
Recognition is a fundamental computer vision problem, in which training samples are used to learn models, that then assign labels to test samples. The utilization of training samples is of vital importance to visual recognition, which can be addressed by increasing the capability of the description methods and the model learning methods. Two visual recognition tasks namely object detection and action recognition and are considered in this thesis.
Active learning utilizes selected subsets of the training dataset as training samples. Active learning methods select the most informative training samples in each iteration, and therefore require fewer training samples to attain comparable performance to passive learning methods. In this thesis, an active learning method for object detection that exploits the distribution of training samples is presented. Experiments show that the proposed method outperforms a passive learning method and a simple margin active learning method.
Weakly supervised learning facilitates learning on training samples with weak labels. In this thesis, a weakly supervised object detection method is proposed to utilize training samples with probabilistic labels. Base detectors are used to create object proposals from training samples with weak labels. Then the object proposals are assigned estimated probabilistic labels. A Generalized Hough Transform based object detector is extended to utilize the object proposals with probabilistic labels as training samples. The proposed method is shown to outperform both a comparison method that assigns strong labels to object proposals, and a weakly supervised deformable part-based models method. The proposed method also attains comparable performance to supervised learning methods.
Increasing the capability of the description method can improve the utilization of training samples. In this thesis, temporal pyramid histograms are proposed to address the problem of missing temporal information in the classical bag of features description method used in action recognition. Experiments show that the proposed description method outperforms the classical bag of features method in action recognition
Real-time motion planning methods for autonomous on-road driving: state-of-the-art and future research directions
Currently autonomous or self-driving vehicles are at the heart of academia and industry research because of its multi-faceted advantages that includes improved safety, reduced congestion, lower emissions and greater mobility. Software is the key driving factor underpinning autonomy within which planning algorithms that are responsible for mission-critical decision making hold a significant position. While transporting passengers or goods from a given origin to a given destination, motion planning methods incorporate searching for a path to follow, avoiding obstacles and generating the best trajectory that ensures safety, comfort and efficiency. A range of different planning approaches have been proposed in the literature. The purpose of this paper is to review existing approaches and then compare and contrast different methods employed for the motion planning of autonomous on-road driving that consists of (1) finding a path, (2) searching for the safest manoeuvre and (3) determining the most feasible trajectory. Methods developed by researchers in each of these three levels exhibit varying levels of complexity and performance accuracy. This paper presents a critical evaluation of each of these methods, in terms of their advantages/disadvantages, inherent limitations, feasibility, optimality, handling of obstacles and testing operational environments. Based on a critical review of existing methods, research challenges to address current limitations are identified and future research directions are suggested so as to enhance the performance of planning algorithms at all three levels. Some promising areas of future focus have been identified as the use of vehicular communications (V2V and V2I) and the incorporation of transport engineering aspects in order to improve the look-ahead horizon of current sensing technologies that are essential for planning with the aim of reducing the total cost of driverless vehicles. This critical review on planning techniques presented in this paper, along with the associated discussions on their constraints and limitations, seek to assist researchers in accelerating development in the emerging field of autonomous vehicle research
WEATHER LORE VALIDATION TOOL USING FUZZY COGNITIVE MAPS BASED ON COMPUTER VISION
Published ThesisThe creation of scientific weather forecasts is troubled by many technological challenges (Stern
& Easterling, 1999) while their utilization is generally dismal. Consequently, the majority of
small-scale farmers in Africa continue to consult some forms of weather lore to reach various
cropping decisions (Baliscan, 2001). Weather lore is a body of informal folklore (Enock, 2013),
associated with the prediction of the weather, and based on indigenous knowledge and human
observation of the environment. As such, it tends to be more holistic, and more localized to the
farmers’ context. However, weather lore has limitations; for instance, it has an inability to offer
forecasts beyond a season. Different types of weather lore exist, utilizing almost all available
human senses (feel, smell, sight and hearing). Out of all the types of weather lore in existence, it
is the visual or observed weather lore that is mostly used by indigenous societies, to come up
with weather predictions.
On the other hand, meteorologists continue to treat this knowledge as superstition, partly because
there is no means to scientifically evaluate and validate it. The visualization and characterization
of visual sky objects (such as moon, clouds, stars, and rainbows) in forecasting weather are
significant subjects of research. To realize the integration of visual weather lore in modern
weather forecasting systems, there is a need to represent and scientifically substantiate this form
of knowledge. This research was aimed at developing a method for verifying visual weather lore that is used by
traditional communities to predict weather conditions. To realize this verification, fuzzy
cognitive mapping was used to model and represent causal relationships between selected visual
weather lore concepts and weather conditions. The traditional knowledge used to produce these
maps was attained through case studies of two communities (in Kenya and South Africa).These
case studies were aimed at understanding the weather lore domain as well as the causal effects
between metrological and visual weather lore. In this study, common astronomical weather lore
factors related to cloud physics were identified as: bright stars, dispersed clouds, dry weather,
dull stars, feathery clouds, gathering clouds, grey clouds, high clouds, layered clouds, low
clouds, stars, medium clouds, and rounded clouds. Relationships between the concepts were also
identified and formally represented using fuzzy cognitive maps. On implementing the verification tool, machine vision was used to recognize sky objects
captured using a sky camera, while pattern recognition was employed in benchmarking and
scoring the objects. A wireless weather station was used to capture real-time weather parameters.
The visualization tool was then designed and realized in a form of software artefact, which
integrated both computer vision and fuzzy cognitive mapping for experimenting visual weather
lore, and verification using various statistical forecast skills and metrics. The tool consists of four
main sub-components: (1) Machine vision that recognizes sky objects using support vector
machine classifiers using shape-based feature descriptors; (2) Pattern recognition–to benchmark
and score objects using pixel orientations, Euclidean distance, canny and grey-level concurrence
matrix; (3) Fuzzy cognitive mapping that was used to represent knowledge (i.e. active hebbian
learning algorithm was used to learn until convergence); and (4) A statistical computing
component was used for verifications and forecast skills including brier score and contingency
tables for deterministic forecasts.
Rigorous evaluation of the verification tool was carried out using independent (not used in the
training and testing phases) real-time images from Bloemfontein, South Africa, and Voi-Kenya.
The real-time images were captured using a sky camera with GPS location services. The results
of the implementation were tested for the selected weather conditions (for example, rain, heat, cold, and dry conditions), and found to be acceptable (the verified prediction accuracies were
over 80%). The recommendation in this study is to apply the implemented method for processing
tasks, towards verifying all other types of visual weather lore. In addition, the use of the method
developed also requires the implementation of modules for processing and verifying other types
of weather lore, such as sounds, and symbols of nature. Since time immemorial, from Australia to Asia, Africa to Latin America, local communities have
continued to rely on weather lore observations to predict seasonal weather as well as its effects
on their livelihoods (Alcock, 2014). This is mainly based on many years of personal experiences
in observing weather conditions. However, when it comes to predictions for longer lead-times
(i.e. over a season), weather lore is uncertain (Hornidge & Antweiler, 2012). This uncertainty has
partly contributed to the current status where meteorologists and other scientists continue to treat
weather lore as superstition (United-Nations, 2004), and not capable of predicting weather.
One of the problems in testing the confidence in weather lore in predicting weather is due to
wide varieties of weather lore that are found in the details of indigenous sayings, which are
tightly coupled to locality and pattern variations(Oviedo et al., 2008). This traditional knowledge
is entrenched within the day-to-day socio-economic activities of the communities using it and is
not globally available for comparison and validation (Huntington, Callaghan, Fox, & Krupnik,
2004). Further, this knowledge is based on local experience that lacks benchmarking techniques;
so that harmonizing and integrating it within the science-based weather forecasting systems is a
daunting task (Hornidge & Antweiler, 2012). It is partly for this reason that the question of
validation of weather lore has not yet been substantially investigated. Sufficient expanded
processes of gathering weather observations, combined with comparison and validation, can produce some useful information. Since forecasting weather accurately is a challenge even with
the latest supercomputers (BBC News Magazine, 2013), validated weather lore can be useful if it
is incorporated into modern weather prediction systems.
Validation of traditional knowledge is a necessary step in the management of building integrated
knowledge-based systems. Traditional knowledge incorporated into knowledge-based systems
has to be verified for enhancing systems’ reliability. Weather lore knowledge exists in different
forms as identified by traditional communities; hence it needs to be tied together for comparison
and validation. The development of a weather lore validation tool that can integrate a framework
for acquiring weather data and methods of representing the weather lore in verifiable forms can
be a significant step in the validation of weather lore against actual weather records using
conventional weather-observing instruments. The success of validating weather lore could
stimulate the opportunity for integrating acceptable weather lore with modern systems of weather prediction to improve actionable information for decision making that relies on seasonal weather
prediction.
In this study a hybrid method is developed that includes computer vision and fuzzy cognitive
mapping techniques for verifying visual weather lore. The verification tool was designed with
forecasting based on mimicking visual perception, and fuzzy thinking based on the cognitive
knowledge of humans. The method provides meaning to humanly perceivable sky objects so that
computers can understand, interpret, and approximate visual weather outcomes.
Questionnaires were administered in two case study locations (KwaZulu-Natal province in South
Africa, and Taita-Taveta County in Kenya), between the months of March and July 2015. The
two case studies were conducted by interviewing respondents on how visual astronomical and
meteorological weather concepts cause weather outcomes. The two case studies were used to
identify causal effects of visual astronomical and meteorological objects to weather conditions.
This was followed by finding variations and comparisons, between the visual weather lore
knowledge in the two case studies. The results from the two case studies were aggregated in
terms of seasonal knowledge. The causal links between visual weather concepts were
investigated using these two case studies; results were compared and aggregated to build up
common knowledge. The joint averages of the majority of responses from the case studies were determined for each set of interacting concepts.
The modelling of the weather lore verification tool consists of input, processing components and
output. The input data to the system are sky image scenes and actual weather observations from
wireless weather sensors. The image recognition component performs three sub-tasks, including:
detection of objects (concepts) from image scenes, extraction of detected objects, and
approximation of the presence of the concepts by comparing extracted objects to ideal objects.
The prediction process involves the use of approximated concepts generated in the recognition
component to simulate scenarios using the knowledge represented in the fuzzy cognitive maps.
The verification component evaluates the variation between the predictions and actual weather
observations to determine prediction errors and accuracy.
To evaluate the tool, daily system simulations were run to predict and record probabilities of
weather outcomes (i.e. rain, heat index/hotness, dry, cold index). Weather observations were
captured periodically using a wireless weather station. This process was repeated several times until there was sufficient data to use for the verification process. To match the range of the
predicted weather outcomes, the actual weather observations (measurement) were transformed
and normalized to a range [0, 1].In the verification process, comparisons were made between the
actual observations and weather outcome prediction values by computing residuals (error values)
from the observations. The error values and the squared error were used to compute the Mean
Squared Error (MSE), and the Root Mean Squared Error (RMSE), for each predicted weather
outcome.
Finally, the validity of the visual weather lore verification model was assessed using data from a
different geographical location. Actual data in the form of daily sky scenes and weather
parameters were acquired from Voi, Kenya, from December 2015 to January 2016.The results on
the use of hybrid techniques for verification of weather lore is expected to provide an incentive
in integrating indigenous knowledge on weather with modern numerical weather prediction
systems for accurate and downscaled weather forecasts
Gaze-Based Human-Robot Interaction by the Brunswick Model
We present a new paradigm for human-robot interaction based on social signal processing, and in particular on the Brunswick model. Originally, the Brunswick model copes with face-to-face dyadic interaction, assuming that the interactants are communicating through a continuous exchange of non verbal social signals, in addition to the spoken messages. Social signals have to be interpreted, thanks to a proper recognition phase that considers visual and audio information. The Brunswick model allows to quantitatively evaluate the quality of the interaction using statistical tools which measure how effective is the recognition phase. In this paper we cast this theory when one of the interactants is a robot; in this case, the recognition phase performed by the robot and the human have to be revised w.r.t. the original model. The model is applied to Berrick, a recent open-source low-cost robotic head platform, where the gazing is the social signal to be considered