Multi-task deep learning with incomplete training samples for the image-based prediction of variables describing silk fabrics

This paper presents a method for the classification of images of silk fabrics with the aim to predict properties such as the placeand time of origin and the production technique. The proposed method was developed in the context of the EU project SILKNOW(http://silknow.eu/). In the context of classification, we address the problem of limited as well as not fully labelled data andinvestigate the connection between the distinct variables. A pre-trained Convolutional Neural Network (CNN) is used for thefeature extraction and a classification network realizing Multi-task learning (MTL) is trained based on these features. The trainingprocedure is adapted to enable the consideration of images that do not have a label for all tasks. Additionally, MTL with fullylabeled training data is investigated for the classification of silk fabrics. The impact of both MTL approaches is compared to singletask learning based on two different class structures. We achieve overall accuracies of 92-95% and average F1-scores of 88-90% inour best experiments. © 2019 Authors

An iterative inference procedure applying conditional random fields for simultaneous classification of land cover and land use

Land cover and land use exhibit strong contextual dependencies. We propose a novel approach for the simultaneous classification of land cover and land use, where semantic and spatial context is considered. The image sites for land cover and land use classification form a hierarchy consisting of two layers: a land cover layer and a land use layer. We apply Conditional Random Fields (CRF) at both layers. The layers differ with respect to the image entities corresponding to the nodes, the employed features and the classes to be distinguished. In the land cover layer, the nodes represent super-pixels; in the land use layer, the nodes correspond to objects from a geospatial database. Both CRFs model spatial dependencies between neighbouring image sites. The complex semantic relations between land cover and land use are integrated in the classification process by using contextual features. We propose a new iterative inference procedure for the simultaneous classification of land cover and land use, in which the two classification tasks mutually influence each other. This helps to improve the classification accuracy for certain classes. The main idea of this approach is that semantic context helps to refine the class predictions, which, in turn, leads to more expressive context information. Thus, potentially wrong decisions can be reversed at later stages. The approach is designed for input data based on aerial images. Experiments are carried out on a test site to evaluate the performance of the proposed method. We show the effectiveness of the iterative inference procedure and demonstrate that a smaller size of the super-pixels has a positive influence on the classification result

Vision-based indoor localization via a visual slam approach

With an increasing interest in indoor location based services, vision-based indoor localization techniques have attracted many attentions from both academia and industry. Inspired by the development of simultaneous localization and mapping technique (SLAM), we present a visual SLAM-based approach to achieve a 6 degrees of freedom (DoF) pose in indoor environment. Firstly, the indoor scene is explored by a keyframe-based global mapping technique, which generates a database from a sequence of images covering the entire scene. After the exploration, a feature vocabulary tree is trained for accelerating feature matching in the image retrieval phase, and the spatial structures obtained from the keyframes are stored. Instead of querying by a single image, a short sequence of images in the query site are used to extract both features and their relative poses, which is a local visual SLAM procedure. The relative poses of query images provide a pose graph-based geometric constraint which is used to assess the validity of image retrieval results. The final positioning result is obtained by selecting the pose of the first correct corresponding image. © Authors 2019

Contextual classification of point cloud data by exploiting individual 3d neigbourhoods

The fully automated analysis of 3D point clouds is of great importance in photogrammetry, remote sensing and computer vision. For reliably extracting objects such as buildings, road inventory or vegetation, many approaches rely on the results of a point cloud classification, where each 3D point is assigned a respective semantic class label. Such an assignment, in turn, typically involves statistical methods for feature extraction and machine learning. Whereas the different components in the processing workflow have extensively, but separately been investigated in recent years, the respective connection by sharing the results of crucial tasks across all components has not yet been addressed. This connection not only encapsulates the interrelated issues of neighborhood selection and feature extraction, but also the issue of how to involve spatial context in the classification step. In this paper, we present a novel and generic approach for 3D scene analysis which relies on (i) individually optimized 3D neighborhoods for (ii) the extraction of distinctive geometric features and (iii) the contextual classification of point cloud data. For a labeled benchmark dataset, we demonstrate the beneficial impact of involving contextual information in the classification process and that using individual 3D neighborhoods of optimal size significantly increases the quality of the results for both pointwise and contextual classification

Investigating 2d and 3d convolutions for multitemporal land cover classification using remote sensing images

With the availability of large amounts of satellite image time series (SITS), the identification of different materials of the Earth's surface is possible with a high temporal resolution. One of the basic tasks is the pixel-wise classification of land cover, i.e.The task of identifying the physical material of the Earth's surface in an image. Fully convolutional neural networks (FCN) are successfully used for this task. In this paper, we investigate different FCN variants, using different methods for the computation of spatial, spectral, and temporal features. We investigate the impact of 3D convolutions in the spatial-Temporal as well as in the spatial-spectral dimensions in comparison to 2D convolutions in the spatial dimensions only. Additionally, we introduce a new method to generate multitemporal input patches by using time intervals instead of fixed acquisition dates. We then choose the image that is closest in time to the middle of the corresponding time interval, which makes our approach more flexible with respect to the requirements for the acquisition of new data. Using these multi-Temporal input patches, generated from Sentinel-2 images, we improve the classification of land cover by 4% in the mean F1-score and 1.3% in the overall accuracy compared to a classification using mono-Temporal input patches. Furthermore, the usage of 3D convolutions instead of 2D convolutions improves the classification performance by a small amount of 0.4% in the mean F1-score and 1.2% in the overall accuracy

3D building change detection using high resolution stereo images and a GIS database

In this paper, a workflow is proposed to detect 3D building changes in urban and sub-urban areas using high-resolution stereoscopic satellite images of different epochs and a GIS database. Semi-global matching (SGM) is used to derive Digital Surface Models (DSM) and subsequently normalised digital surface models (nDSM, the difference of a DSM and a digital elevation model (DEM)), from the stereo pairs at each epoch. Large differences between the two DSMs are assumed to represent height changes. In order to reduce the effect of matching errors, heights in the nDSM of at least one epoch must also lie above a certain threshold in order to be considered as candidates for building change. A GIS database is used to check the existence of buildings at epoch 1. As a result of geometric discrepancies during data acquisition caused by different view directions and illumination conditions, the outlines of existing buildings do not necessarily match even in non-changed areas. Consequently, in the change map, there are streaking-shaped structures along the building outlines which do not correspond to actual changes. To eliminate these effects morphologic filtering is applied. The mask we use operates as a threshold on the shape and size of detected new blobs and effectively removes small objects such as cars, small trees and salt and pepper noise. The results of the proposed algorithm using IKONOS and GeoEye images demonstrate its performance for detecting 3D building changes and to extract building boundaries.DAA

Multi-source hierarchical conditional random field model for feature fusion of remote sensing images and LiDAR data

Feature fusion of remote sensing images and LiDAR points cloud data, which have strong complementarity, can effectively play the advantages of multi-class features to provide more reliable information support for the remote sensing applications, such as object classification and recognition. In this paper, we introduce a novel multi-source hierarchical conditional random field (MSHCRF) model to fuse features extracted from remote sensing images and LiDAR data for image classification. Firstly, typical features are selected to obtain the interest regions from multi-source data, then MSHCRF model is constructed to exploit up the features, category compatibility of images and the category consistency of multi-source data based on the regions, and the outputs of the model represents the optimal results of the image classification. Competitive results demonstrate the precision and robustness of the proposed method

PRECISE VEHICLE RECONSTRUCTION FOR AUTONOMOUS DRIVING APPLICATIONS

Interactive motion planing and collaborative positioning will play a key role in future autonomous driving applications. For this purpose, the precise reconstruction and pose estimation of other traffic participants, especially of other vehicles, is a fundamental task and will be tackled in this paper based on street level stereo images obtained from a moving vehicle. We learn a shape prior, consisting of vehicle geometry and appearance features, and we fit a vehicle model to initially detected vehicles. This is achieved by minimising an energy function, jointly incorporating 3D and 2D information to infer the model’s optimal and precise pose parameters. For evaluation we use the object detection and orientation benchmark of the KITTI dataset (Geiger et al., 2012). We can show a significant benefit of each of the individual energy terms of the overall objective function. We achieve good results with up to 94.8% correct and precise pose estimations with an average absolute error smaller than 3° for the orientation and 33 cm for position

Long-term physical activity modulates brain processing of somatosensory stimuli : Evidence from young male twins

Leisure-time physical activity is a key contributor to physical and mental health. Yet the role of physical activity in modulating cortical function is poorly known. We investigated whether precognitive sensory brain functions are associated with the level of physical activity. Physical activity history (3-yr-LTMET), physiological measures and somatosensory mismatch response (sMMR) in EEG were recorded in 32 young healthy twins. In all participants, 3-yr-LTMET correlated negatively with body fat%, r=0.77 and positively with VO2max, r=0.82. The fat% and VO2max differed between 15 physically active and 17 inactive participants. Trend toward larger sMMR was seen in inactive compared to active participants. This finding was significant in a pairwise comparison of 9 monozygotic twin pairs discordant for physical activity. Larger sMMR reflecting stronger synchronous neural activity may reveal diminished gating of precognitive somatosensory information in physically inactive healthy young men compared to the active ones possibly rendering them more vulnerable to somatosensory distractions from their surroundings. (C) 2016 Elsevier B.V. All rights reserved.Peer reviewe

Cross-sectional associations between the diversity of sport activities and the type of low back pain in adulthood

Leisure-time physical activity has a complex relationship with low back pain (LBP). Thus, we aimed to investigate whether the diversity of sport activities is associated with the type of LBP. In the FinnTwin16 study, 4246 (55% females) Finnish twins at mean age 34.1 years replied to a health behaviour survey in 2010-2012. Based on the participation in different sport activities, we created two measures of diversity: quantity (i.e. the number of sport activities: 1, 2, 3, 4 and >= 5) and quality (i.e. the type of sport activity: endurance, strength, body care, etc.). Based on the frequency, duration and type of LBP, we created three groups: no history of LBP lasting more than one day, radiating LBP and non-radiating LBP. The associations between the quantity and quality of sport activities and the type of LBP were investigated with logistic regression analyses. Participation in >= 5 sport activities associated with less radiating and non-radiating LBP in analyses pooled across sex (odds ratio 0.46, 95% CI 0.30-0.69 and 0.66, 0.44-0.99, respectively). However, the associations attenuated after adjusting for several confounders. Participation in endurance sports was associated with less radiating (0.58, 0.43-0.76) and non-radiating (0.60, 0.44-0.81) LBP, whereas strength sports and body care only with less radiating LBP (0.76, 0.58-1.00 and 0.26, 0.09-0.74, respectively) adjusted for all sport types. On a sport-specific level, running and cycling were associated with less radiating and non-radiating LBP. In adulthood, the diversity of sport activities, particularly participation in endurance sports, may be associated with less radiating and non-radiating LBP.Peer reviewe