664 research outputs found
Modeling, classifying and annotating weakly annotated images using bayesian network
International audienceIn this paper, we propose a probabilistic graphical model to represent weakly annotated images. We consider an image as weakly annotated if the number of keywords defined for it is less than the maximum number defined in the ground truth. This model is used to classify images and automatically extend existing annotations to new images by taking into account semantic relations between keywords. The proposed method has been evaluated in visual-textual classification and automatic annotation of images. The visualtextual classification is performed by using both visual and textual information. The experimental results, obtained from a database of more than 30000 images, show an improvement by 50.5% in terms of recognition rate against only visual information classification. Taking into account semantic relations between keywords improves the recognition rate by 10.5%. Moreover, the proposed model can be used to extend existing annotations to weakly annotated images, by computing distributions of missing keywords. Semantic relations improve the mean rate of good annotations by 6.9%. Finally, the proposed method is competitive with a state-of-art model
Adversarially Tuned Scene Generation
Generalization performance of trained computer vision systems that use
computer graphics (CG) generated data is not yet effective due to the concept
of 'domain-shift' between virtual and real data. Although simulated data
augmented with a few real world samples has been shown to mitigate domain shift
and improve transferability of trained models, guiding or bootstrapping the
virtual data generation with the distributions learnt from target real world
domain is desired, especially in the fields where annotating even few real
images is laborious (such as semantic labeling, and intrinsic images etc.). In
order to address this problem in an unsupervised manner, our work combines
recent advances in CG (which aims to generate stochastic scene layouts coupled
with large collections of 3D object models) and generative adversarial training
(which aims train generative models by measuring discrepancy between generated
and real data in terms of their separability in the space of a deep
discriminatively-trained classifier). Our method uses iterative estimation of
the posterior density of prior distributions for a generative graphical model.
This is done within a rejection sampling framework. Initially, we assume
uniform distributions as priors on the parameters of a scene described by a
generative graphical model. As iterations proceed the prior distributions get
updated to distributions that are closer to the (unknown) distributions of
target data. We demonstrate the utility of adversarially tuned scene generation
on two real-world benchmark datasets (CityScapes and CamVid) for traffic scene
semantic labeling with a deep convolutional net (DeepLab). We realized
performance improvements by 2.28 and 3.14 points (using the IoU metric) between
the DeepLab models trained on simulated sets prepared from the scene generation
models before and after tuning to CityScapes and CamVid respectively.Comment: 9 pages, accepted at CVPR 201
The intersection of video capsule endoscopy and artificial intelligence: addressing unique challenges using machine learning
Introduction: Technical burdens and time-intensive review processes limit the
practical utility of video capsule endoscopy (VCE). Artificial intelligence
(AI) is poised to address these limitations, but the intersection of AI and VCE
reveals challenges that must first be overcome. We identified five challenges
to address. Challenge #1: VCE data are stochastic and contains significant
artifact. Challenge #2: VCE interpretation is cost-intensive. Challenge #3: VCE
data are inherently imbalanced. Challenge #4: Existing VCE AIMLT are
computationally cumbersome. Challenge #5: Clinicians are hesitant to accept
AIMLT that cannot explain their process.
Methods: An anatomic landmark detection model was used to test the
application of convolutional neural networks (CNNs) to the task of classifying
VCE data. We also created a tool that assists in expert annotation of VCE data.
We then created more elaborate models using different approaches including a
multi-frame approach, a CNN based on graph representation, and a few-shot
approach based on meta-learning.
Results: When used on full-length VCE footage, CNNs accurately identified
anatomic landmarks (99.1%), with gradient weighted-class activation mapping
showing the parts of each frame that the CNN used to make its decision. The
graph CNN with weakly supervised learning (accuracy 89.9%, sensitivity of
91.1%), the few-shot model (accuracy 90.8%, precision 91.4%, sensitivity
90.9%), and the multi-frame model (accuracy 97.5%, precision 91.5%, sensitivity
94.8%) performed well. Discussion: Each of these five challenges is addressed,
in part, by one of our AI-based models. Our goal of producing high performance
using lightweight models that aim to improve clinician confidence was achieved
Semantic multimedia modelling & interpretation for annotation
The emergence of multimedia enabled devices, particularly the incorporation of cameras in mobile phones, and the accelerated revolutions in the low cost storage devices, boosts the multimedia data production rate drastically. Witnessing such an iniquitousness of digital images and videos, the research community has been projecting the issue of its significant utilization and management. Stored in monumental multimedia corpora, digital data need to be retrieved and organized in an intelligent way, leaning on the rich semantics involved. The utilization of these image and video collections demands proficient image and video annotation and retrieval techniques. Recently, the multimedia research community is progressively veering its emphasis to the personalization of these media. The main impediment in the image and video analysis is the semantic gap, which is the discrepancy among a user’s high-level interpretation of an image and the video and the low level computational interpretation of it. Content-based image and video annotation systems are remarkably susceptible to the semantic gap due to their reliance on low-level visual features for delineating semantically rich image and video contents. However, the fact is that the visual similarity is not semantic similarity, so there is a demand to break through this dilemma through an alternative way. The semantic gap can be narrowed by counting high-level and user-generated information in the annotation. High-level descriptions of images and or videos are more proficient of capturing the semantic meaning of multimedia content, but it is not always applicable to collect this information. It is commonly agreed that the problem of high level semantic annotation of multimedia is still far from being answered. This dissertation puts forward approaches for intelligent multimedia semantic extraction for high level annotation. This dissertation intends to bridge the gap between the visual features and semantics. It proposes a framework for annotation enhancement and refinement for the object/concept annotated images and videos datasets. The entire theme is to first purify the datasets from noisy keyword and then expand the concepts lexically and commonsensical to fill the vocabulary and lexical gap to achieve high level semantics for the corpus. This dissertation also explored a novel approach for high level semantic (HLS) propagation through the images corpora. The HLS propagation takes the advantages of the semantic intensity (SI), which is the concept dominancy factor in the image and annotation based semantic similarity of the images. As we are aware of the fact that the image is the combination of various concepts and among the list of concepts some of them are more dominant then the other, while semantic similarity of the images are based on the SI and concept semantic similarity among the pair of images. Moreover, the HLS exploits the clustering techniques to group similar images, where a single effort of the human experts to assign high level semantic to a randomly selected image and propagate to other images through clustering. The investigation has been made on the LabelMe image and LabelMe video dataset. Experiments exhibit that the proposed approaches perform a noticeable improvement towards bridging the semantic gap and reveal that our proposed system outperforms the traditional systems
ImageNet Large Scale Visual Recognition Challenge
The ImageNet Large Scale Visual Recognition Challenge is a benchmark in
object category classification and detection on hundreds of object categories
and millions of images. The challenge has been run annually from 2010 to
present, attracting participation from more than fifty institutions.
This paper describes the creation of this benchmark dataset and the advances
in object recognition that have been possible as a result. We discuss the
challenges of collecting large-scale ground truth annotation, highlight key
breakthroughs in categorical object recognition, provide a detailed analysis of
the current state of the field of large-scale image classification and object
detection, and compare the state-of-the-art computer vision accuracy with human
accuracy. We conclude with lessons learned in the five years of the challenge,
and propose future directions and improvements.Comment: 43 pages, 16 figures. v3 includes additional comparisons with PASCAL
VOC (per-category comparisons in Table 3, distribution of localization
difficulty in Fig 16), a list of queries used for obtaining object detection
images (Appendix C), and some additional reference
SoccerNet: A Scalable Dataset for Action Spotting in Soccer Videos
In this paper, we introduce SoccerNet, a benchmark for action spotting in
soccer videos. The dataset is composed of 500 complete soccer games from six
main European leagues, covering three seasons from 2014 to 2017 and a total
duration of 764 hours. A total of 6,637 temporal annotations are automatically
parsed from online match reports at a one minute resolution for three main
classes of events (Goal, Yellow/Red Card, and Substitution). As such, the
dataset is easily scalable. These annotations are manually refined to a one
second resolution by anchoring them at a single timestamp following
well-defined soccer rules. With an average of one event every 6.9 minutes, this
dataset focuses on the problem of localizing very sparse events within long
videos. We define the task of spotting as finding the anchors of soccer events
in a video. Making use of recent developments in the realm of generic action
recognition and detection in video, we provide strong baselines for detecting
soccer events. We show that our best model for classifying temporal segments of
length one minute reaches a mean Average Precision (mAP) of 67.8%. For the
spotting task, our baseline reaches an Average-mAP of 49.7% for tolerances
ranging from 5 to 60 seconds. Our dataset and models are available at
https://silviogiancola.github.io/SoccerNet.Comment: CVPR Workshop on Computer Vision in Sports 201
Weakly Supervised Localization using Deep Feature Maps
Object localization is an important computer vision problem with a variety of
applications. The lack of large scale object-level annotations and the relative
abundance of image-level labels makes a compelling case for weak supervision in
the object localization task. Deep Convolutional Neural Networks are a class of
state-of-the-art methods for the related problem of object recognition. In this
paper, we describe a novel object localization algorithm which uses
classification networks trained on only image labels. This weakly supervised
method leverages local spatial and semantic patterns captured in the
convolutional layers of classification networks. We propose an efficient beam
search based approach to detect and localize multiple objects in images. The
proposed method significantly outperforms the state-of-the-art in standard
object localization data-sets with a 8 point increase in mAP scores
TrackletMapper: Ground Surface Segmentation and Mapping from Traffic Participant Trajectories
Robustly classifying ground infrastructure such as roads and street crossings
is an essential task for mobile robots operating alongside pedestrians. While
many semantic segmentation datasets are available for autonomous vehicles,
models trained on such datasets exhibit a large domain gap when deployed on
robots operating in pedestrian spaces. Manually annotating images recorded from
pedestrian viewpoints is both expensive and time-consuming. To overcome this
challenge, we propose TrackletMapper, a framework for annotating ground surface
types such as sidewalks, roads, and street crossings from object tracklets
without requiring human-annotated data. To this end, we project the robot
ego-trajectory and the paths of other traffic participants into the ego-view
camera images, creating sparse semantic annotations for multiple types of
ground surfaces from which a ground segmentation model can be trained. We
further show that the model can be self-distilled for additional performance
benefits by aggregating a ground surface map and projecting it into the camera
images, creating a denser set of training annotations compared to the sparse
tracklet annotations. We qualitatively and quantitatively attest our findings
on a novel large-scale dataset for mobile robots operating in pedestrian areas.
Code and dataset will be made available at
http://trackletmapper.cs.uni-freiburg.de.Comment: 19 pages, 14 figures, CoRL 2022 v
A comprehensive survey on deep active learning and its applications in medical image analysis
Deep learning has achieved widespread success in medical image analysis,
leading to an increasing demand for large-scale expert-annotated medical image
datasets. Yet, the high cost of annotating medical images severely hampers the
development of deep learning in this field. To reduce annotation costs, active
learning aims to select the most informative samples for annotation and train
high-performance models with as few labeled samples as possible. In this
survey, we review the core methods of active learning, including the evaluation
of informativeness and sampling strategy. For the first time, we provide a
detailed summary of the integration of active learning with other
label-efficient techniques, such as semi-supervised, self-supervised learning,
and so on. Additionally, we also highlight active learning works that are
specifically tailored to medical image analysis. In the end, we offer our
perspectives on the future trends and challenges of active learning and its
applications in medical image analysis.Comment: Paper List on Github:
https://github.com/LightersWang/Awesome-Active-Learning-for-Medical-Image-Analysi
- …