Radar-based Feature Design and Multiclass Classification for Road User Recognition

The classification of individual traffic participants is a complex task, especially for challenging scenarios with multiple road users or under bad weather conditions. Radar sensors provide an - with respect to well established camera systems - orthogonal way of measuring such scenes. In order to gain accurate classification results, 50 different features are extracted from the measurement data and tested on their performance. From these features a suitable subset is chosen and passed to random forest and long short-term memory (LSTM) classifiers to obtain class predictions for the radar input. Moreover, it is shown why data imbalance is an inherent problem in automotive radar classification when the dataset is not sufficiently large. To overcome this issue, classifier binarization is used among other techniques in order to better account for underrepresented classes. A new method to couple the resulting probabilities is proposed and compared to others with great success. Final results show substantial improvements when compared to ordinary multiclass classificationComment: 8 pages, 6 figure

Radar-based Road User Classification and Novelty Detection with Recurrent Neural Network Ensembles

Radar-based road user classification is an important yet still challenging task towards autonomous driving applications. The resolution of conventional automotive radar sensors results in a sparse data representation which is tough to recover by subsequent signal processing. In this article, classifier ensembles originating from a one-vs-one binarization paradigm are enriched by one-vs-all correction classifiers. They are utilized to efficiently classify individual traffic participants and also identify hidden object classes which have not been presented to the classifiers during training. For each classifier of the ensemble an individual feature set is determined from a total set of 98 features. Thereby, the overall classification performance can be improved when compared to previous methods and, additionally, novel classes can be identified much more accurately. Furthermore, the proposed structure allows to give new insights in the importance of features for the recognition of individual classes which is crucial for the development of new algorithms and sensor requirements.Comment: 8 pages, 9 figures, accepted paper for 2019 IEEE Intelligent Vehicles Symposium (IV), Paris, France, June 201

Deep Multi Temporal Scale Networks for Human Motion Analysis

The movement of human beings appears to respond to a complex motor system that contains signals at different hierarchical levels. For example, an action such as ``grasping a glass on a table'' represents a high-level action, but to perform this task, the body needs several motor inputs that include the activation of different joints of the body (shoulder, arm, hand, fingers, etc.). Each of these different joints/muscles have a different size, responsiveness, and precision with a complex non-linearly stratified temporal dimension where every muscle has its temporal scale. Parts such as the fingers responds much faster to brain input than more voluminous body parts such as the shoulder. The cooperation we have when we perform an action produces smooth, effective, and expressive movement in a complex multiple temporal scale cognitive task. Following this layered structure, the human body can be described as a kinematic tree, consisting of joints connected. Although it is nowadays well known that human movement and its perception are characterised by multiple temporal scales, very few works in the literature are focused on studying this particular property. In this thesis, we will focus on the analysis of human movement using data-driven techniques. In particular, we will focus on the non-verbal aspects of human movement, with an emphasis on full-body movements. The data-driven methods can interpret the information in the data by searching for rules, associations or patterns that can represent the relationships between input (e.g. the human action acquired with sensors) and output (e.g. the type of action performed). Furthermore, these models may represent a new research frontier as they can analyse large masses of data and focus on aspects that even an expert user might miss. The literature on data-driven models proposes two families of methods that can process time series and human movement. The first family, called shallow models, extract features from the time series that can help the learning algorithm find associations in the data. These features are identified and designed by domain experts who can identify the best ones for the problem faced. On the other hand, the second family avoids this phase of extraction by the human expert since the models themselves can identify the best set of features to optimise the learning of the model. In this thesis, we will provide a method that can apply the multi-temporal scales property of the human motion domain to deep learning models, the only data-driven models that can be extended to handle this property. We will ask ourselves two questions: what happens if we apply knowledge about how human movements are performed to deep learning models? Can this knowledge improve current automatic recognition standards? In order to prove the validity of our study, we collected data and tested our hypothesis in specially designed experiments. Results support both the proposal and the need for the use of deep multi-scale models as a tool to better understand human movement and its multiple time-scale nature

Automatic methods for crop classification by merging satellite radar (sentinel 1) and optical (sentinel 2) . data and artificial intelligence analysis

Land use and land cover maps can support our understanding of coupled human- environment systems and provide important information for environmental modelling and water resource management. Satellite data are a valuable source for land use and land cover mapping. However, cloud-free or weather independent data are necessary to map cloud-prone regions. Merging radar with optical images would increase the accuracy of the study. Agricultural land cover is characterized by strong variations within relatively short time intervals. These dynamics are challenging for land cover classifications on the one hand, but deliver crucial information that can be used to improve the machine learning classifier’s performance on the other hand. A parcel-based map of the main crop classes of the Netherlands was produced implementing a script on GEE and using Copernicus data. The machine-learning model used is a Random Forest Classifier. This was done by combining time series of radar and multispectral images from Sentinel 1 and Sentinel 2 satellites, respectively. The results show the potential of providing useful information delivered by entirely open source data and uses a cloud computing-based approach. The algorithm combines the two satellites data of one year in a multibands image to feed in the classifier. Standard deviation and several vegetation indexes were added in order to have more variables for each 15-day-median image composite. The process paid particular attention to time variability of mean values of each field. This will provide useful information both for understanding differences among crops and variability over the phenology of the plant. The accuracy assessment demonstrates that several crop types (i.e. corn, tulip) can be better classified with both radar and optical images while others (i.e. sugar beet, barley) have an increased accuracy with only radar. The overall accuracy of RFC with optical and radar is 76% while it is 74% if only radar is used

Deep Instance Segmentation with Automotive Radar Detection Points

Automotive radar provides reliable environmental perception in all-weather conditions with affordable cost, but it hardly supplies semantic and geometry information due to the sparsity of radar detection points. With the development of automotive radar technologies in recent years, instance segmentation becomes possible by using automotive radar. Its data contain contexts such as radar cross section and micro-Doppler effects, and sometimes can provide detection when the field of view is obscured. The outcome from instance segmentation could be potentially used as the input of trackers for tracking targets. The existing methods often utilize a clustering based classification framework, which fits the need of real-time processing but has limited performance due to minimum information provided by sparse radar detection points. In this paper, we propose an efficient method based on clustering of estimated semantic information to achieve instance segmentation for the sparse radar detection points. In addition, we show that the performance of the proposed approach can be further enhanced by incorporating the visual multi-layer perceptron. The effectiveness of the proposed method is verified by experimental results on the popular RadarScenes dataset, achieving 89.53% mCov and 86.97% mAP0.5, which is the best comparing to other approaches in the literature. More significantly, the proposed algorithm consumes memory around 1MB, and the inference time is less than 40ms. These two criteria ensure the practicality of the proposed method in real-world system