Content-sensitive superpixel generation with boundary adjustment.

Superpixel segmentation has become a crucial tool in many image processing and computer vision applications. In this paper, a novel content-sensitive superpixel generation algorithm with boundary adjustment is proposed. First, the image local entropy was used to measure the amount of information in the image, and the amount of information was evenly distributed to each seed. It placed more seeds to achieve the lower under-segmentation in content-dense regions, and placed the fewer seeds to increase computational efficiency in content-sparse regions. Second, the Prim algorithm was adopted to generate uniform superpixels efficiently. Third, a boundary adjustment strategy with the adaptive distance further optimized the superpixels to improve the performance of the superpixel. Experimental results on the Berkeley Segmentation Database show that our method outperforms competing methods under evaluation metrics

Superpixel-based segmentation of muscle fibers in multi-channel microscopy

Background Confetti fluorescence and other multi-color genetic labelling strategies are useful for observing stem cell regeneration and for other problems of cell lineage tracing. One difficulty of such strategies is segmenting the cell boundaries, which is a very different problem from segmenting color images from the real world. This paper addresses the difficulties and presents a superpixel-based framework for segmentation of regenerated muscle fibers in mice. Results We propose to integrate an edge detector into a superpixel algorithm and customize the method for multi-channel images. The enhanced superpixel method outperforms the original and another advanced superpixel algorithm in terms of both boundary recall and under-segmentation error. Our framework was applied to cross-section and lateral section images of regenerated muscle fibers from confetti-fluorescent mice. Compared with “ground-truth” segmentations, our framework yielded median Dice similarity coefficients of 0.92 and higher. Conclusion Our segmentation framework is flexible and provides very good segmentations of multi-color muscle fibers. We anticipate our methods will be useful for segmenting a variety of tissues in confetti fluorecent mice and in mice with similar multi-color labels.National University of Singapore (Duke-NUS SRP Phase 2 Research Block Grant)Singapore. National Research Foundation (CREATE programme)Singapore-MIT Alliance for Research and Technology (SMART

Hybrid Superpixel Segmentation

Poster presentation: paper no. 27postprin

Human shape modelling for carried object detection and segmentation

La détection des objets transportés est un des prérequis pour développer des systèmes qui cherchent à comprendre les activités impliquant des personnes et des objets. Cette thèse présente de nouvelles méthodes pour détecter et segmenter les objets transportés dans des vidéos de surveillance. Les contributions sont divisées en trois principaux chapitres. Dans le premier chapitre, nous introduisons notre détecteur d’objets transportés, qui nous permet de détecter un type générique d’objets. Nous formulons la détection d’objets transportés comme un problème de classification de contours. Nous classifions le contour des objets mobiles en deux classes : objets transportés et personnes. Un masque de probabilités est généré pour le contour d’une personne basé sur un ensemble d’exemplaires (ECE) de personnes qui marchent ou se tiennent debout de différents points de vue. Les contours qui ne correspondent pas au masque de probabilités généré sont considérés comme des candidats pour être des objets transportés. Ensuite, une région est assignée à chaque objet transporté en utilisant la Coupe Biaisée Normalisée (BNC) avec une probabilité obtenue par une fonction pondérée de son chevauchement avec l’hypothèse du masque de contours de la personne et du premier plan segmenté. Finalement, les objets transportés sont détectés en appliquant une Suppression des Non-Maxima (NMS) qui élimine les scores trop bas pour les objets candidats. Le deuxième chapitre de contribution présente une approche pour détecter des objets transportés avec une méthode innovatrice pour extraire des caractéristiques des régions d’avant-plan basée sur leurs contours locaux et l’information des super-pixels. Initiallement, un objet bougeant dans une séquence vidéo est segmente en super-pixels sous plusieurs échelles. Ensuite, les régions ressemblant à des personnes dans l’avant-plan sont identifiées en utilisant un ensemble de caractéristiques extraites de super-pixels dans un codebook de formes locales. Ici, les régions ressemblant à des humains sont équivalentes au masque de probabilités de la première méthode (ECE). Notre deuxième détecteur d’objets transportés bénéficie du nouveau descripteur de caractéristiques pour produire une carte de probabilité plus précise. Les compléments des super-pixels correspondants aux régions ressemblant à des personnes dans l’avant-plan sont considérés comme une carte de probabilité des objets transportés. Finalement, chaque groupe de super-pixels voisins avec une haute probabilité d’objets transportés et qui ont un fort support de bordure sont fusionnés pour former un objet transporté. Finalement, dans le troisième chapitre, nous présentons une méthode pour détecter et segmenter les objets transportés. La méthode proposée adopte le nouveau descripteur basé sur les super-pixels pour iii identifier les régions ressemblant à des objets transportés en utilisant la modélisation de la forme humaine. En utilisant l’information spatio-temporelle des régions candidates, la consistance des objets transportés récurrents, vus dans le temps, est obtenue et sert à détecter les objets transportés. Enfin, les régions d’objets transportés sont raffinées en intégrant de l’information sur leur apparence et leur position à travers le temps avec une extension spatio-temporelle de GrabCut. Cette étape finale sert à segmenter avec précision les objets transportés dans les séquences vidéo. Nos méthodes sont complètement automatiques, et font des suppositions minimales sur les personnes, les objets transportés, et les les séquences vidéo. Nous évaluons les méthodes décrites en utilisant deux ensembles de données, PETS 2006 et i-Lids AVSS. Nous évaluons notre détecteur et nos méthodes de segmentation en les comparant avec l’état de l’art. L’évaluation expérimentale sur les deux ensembles de données démontre que notre détecteur d’objets transportés et nos méthodes de segmentation surpassent de façon significative les algorithmes compétiteurs.Detecting carried objects is one of the requirements for developing systems that reason about activities involving people and objects. This thesis presents novel methods to detect and segment carried objects in surveillance videos. The contributions are divided into three main chapters. In the first, we introduce our carried object detector which allows to detect a generic class of objects. We formulate carried object detection in terms of a contour classification problem. We classify moving object contours into two classes: carried object and person. A probability mask for person’s contours is generated based on an ensemble of contour exemplars (ECE) of walking/standing humans in different viewing directions. Contours that are not falling in the generated hypothesis mask are considered as candidates for carried object contours. Then, a region is assigned to each carried object candidate contour using Biased Normalized Cut (BNC) with a probability obtained by a weighted function of its overlap with the person’s contour hypothesis mask and segmented foreground. Finally, carried objects are detected by applying a Non-Maximum Suppression (NMS) method which eliminates the low score carried object candidates. The second contribution presents an approach to detect carried objects with an innovative method for extracting features from foreground regions based on their local contours and superpixel information. Initially, a moving object in a video frame is segmented into multi-scale superpixels. Then human-like regions in the foreground area are identified by matching a set of extracted features from superpixels against a codebook of local shapes. Here the definition of human like regions is equivalent to a person’s probability map in our first proposed method (ECE). Our second carried object detector benefits from the novel feature descriptor to produce a more accurate probability map. Complement of the matching probabilities of superpixels to human-like regions in the foreground are considered as a carried object probability map. At the end, each group of neighboring superpixels with a high carried object probability which has strong edge support is merged to form a carried object. Finally, in the third contribution we present a method to detect and segment carried objects. The proposed method adopts the new superpixel-based descriptor to identify carried object-like candidate regions using human shape modeling. Using spatio-temporal information of the candidate regions, consistency of recurring carried object candidates viewed over time is obtained and serves to detect carried objects. Last, the detected carried object regions are refined by integrating information of their appearances and their locations over time with a spatio-temporal extension of GrabCut. This final stage is used to accurately segment carried objects in frames. Our methods are fully automatic, and make minimal assumptions about a person, carried objects and videos. We evaluate the aforementioned methods using two available datasets PETS 2006 and i-Lids AVSS. We compare our detector and segmentation methods against a state-of-the-art detector. Experimental evaluation on the two datasets demonstrates that both our carried object detection and segmentation methods significantly outperform competing algorithms

슈퍼픽셀을 이용한 적응형 이미지 처리

학위논문(박사)--서울대학교 대학원 :자연과학대학 수리과학부,2020. 2. 강명주.Human-like Artificial Intelligence(AI) is also crucial issue in the field of image processing. In this trend, classical methods, processing images based on each pixel show some limitation because human being dont focus information of a single pixel. As recent studies shows, humans interpret images as the complicated combination of the number of meaningful `clusters'. Therefore in order to deal with various and complex images in the human-like way, we should process images based on these `clusters'. This paper will cover superpixels that can act as these `clusters' in images.We will introduce several superpixel generating algorithms and their advantages and disadvantages. And we will show the effectiveness of the superpixels in the image processing based on their contribution to the image evaluation field. Next, we propose a new approach to image analysis based on pivot colors of them. To find pivot colors, we propose a novel method called Superpixelwise Mean shift. This method combines the idea of mean shift procedure and the representative superpixels and is fast and robust. In the latter chapters, we will show its application to the image segmentation problem and the color mapping problem and result of them.인간과 비슷하게 동작하게 인공 지능의 개발은 이미지 프로세싱 분야에서도 중요한 이슈 중에 하나이다. 이러한 추세에서 픽셀을 기반으로 이미지를 다루는 고전적인 방법들은 여러 가지 한계를 보여주는데, 가장 큰 이유는 인간은 개별 픽셀이 가지는 정보에 큰 관심을 주지 않기 때문이다. 많은 연구가 보여주듯이 인간을 이미지를 의미를 가지는 수많은 덩어리 들의 복합적인 결합으로 보는 경향이 있으며 다양하고 복잡한 이미지를 다루기 위해서는 픽셀보다는 이러한 '덩어리'를 기반으로 이미지를 파악할 필요가 있다. 이 논문에서는 이미지에서 이러한 덩어리 역할을 할 수 있는 슈퍼픽셀을 다룬다. 앞부분에서는 먼저 다양한 슈퍼픽셀 생성 기법들과 그들의 장단점을 소개하고 이미지 평가 분야에서의 결과를 바탕으로 슈퍼픽셀이 이미지를 다루는데 얼마나 효과적인지를 보이겠다. 그 다음에 우리는 이미지의 기조 색상을 바탕으로 한 새로운 이미지 분석 방법을 제시하고 그 기조 색상을 구하기 위해서 Superpixelwise Mean Shift라는 새로운 방법론을 제시하였다. 이 방식은 Mean shift procedure와 슈퍼픽셀의 대표값을 결합시킨 방식으로 매우 빠르고 확고하다. 뒤의 장에서는 이 방법론을 이미지 분할 문제와 색 이동 문제에서 적용하고 그 결과를 보이겠다.1 Introduction 1 2 Preliminaries 4 2.1 Superpixel Generating Methods 5 2.1.1 Various Superpixel Generating Methods 5 2.1.2 Performance Comparison between the Superpixels and Choosing the Method 16 2.2 Image Quality Assessment System and Superpixels 17 2.2.1 Object of Image Evaluation System 17 2.2.2 Various Image Quality Assessment System 18 2.2.3 Applying Superpixels to IQA System 24 2.2.4 Performance Comparison of IQA System 28 3 Adaptive Image Segmentation Based on Superpixel 30 3.1 Superpixelwise Mean Shift 31 3.2 Two-Step Approach using S-Mean Shift 36 3.3 Gradient Transition and Eliminating Small Pieces 39 3.4 Merging On Balanced Gradient Transition 42 3.5 Experimental Result 45 3.5.1 Experiment of CSIQ Dataset 45 3.5.2 Experiment of Berkeley Dataset 50 3.5.3 Computational Time and Parameters 55 4 Color Mapping Based on Superpixel 57 4.1 Color Mapping Problem and Superpixels 58 4.2 Applying S-Mean Shift to Color Mapping 63 5 Conclusion 67 Abstract (in Korean) 73 Acknowledgement (in Korean) 74Docto