Probabilistic Models for Motion Segmentation in Image Sequences

Motion segmentation is the task of assigning a binary label to every pixel in an image sequence specifying whether it is a moving foreground object or stationary background. It is often an important task in many computer vision applications such as automatic surveillance and tracking systems. Depending on whether the camera is stationary or moving, different approaches are possible for segmentation. Motion segmentation when the camera is stationary is a well studied problem with many effective algorithms and systems in use today. In contrast, the problem of segmentation with a moving camera is much more complex. In this thesis, we make contributions to the problem of motion segmentation in both camera settings. First for the stationary camera case, we develop a probabilistic model that intuitively combines the various aspects of the problem in a system that is easy to interpret and extend. In most stationary camera systems, a distribution over feature values for the background at each pixel location is learned from previous frames in the sequence and used for classification in the current frame. These pixelwise models fail to account for the influence of neighboring pixels on each other. We propose a model that by spatially spreading the information in the pixelwise distributions better reflects the spatial influence between pixels. Further, we show that existing algorithms that use a constant variance value for the distributions at every pixel location in the image are inaccurate and present an alternate pixelwise adaptive variance method. These improvements result in a system that outperforms all existing algorithms on a standard benchmark. Compared to stationary camera videos, moving camera videos have fewer established solutions for motion segmentation. One of the contributions of this thesis is the development of a viable segmentation method that is effective on a wide range of videos and robust to complex background settings. In moving camera videos, motion segmentation is commonly performed using the image plane motion of pixels, or optical flow. However, objects that are at different depths from the camera can exhibit different optical flows, even if they share the same real-world motion. This can cause a depth-dependent segmentation of the scene. While such a segmentation is meaningful, it can be ineffective for the purpose of identifying independently moving objects. Our goal is to develop a segmentation algorithm that clusters pixels that have similar real-world motion. Our solution uses optical flow orientations instead of the complete vectors and exploits the well-known property that under translational camera motion, optical flow orientations are independent of object depth. We introduce a non-parametric probabilistic model that automatically estimates the number of observed independent motions and results in a labeling that is consistent with real-world motion in the scene. Most importantly, static objects are correctly identified as one segment even if they are at different depths. Finally, a rotation compensation algorithm is proposed that can be applied to real-world videos taken with hand-held cameras. We benchmark the system on over thirty videos from multiple data sets containing videos taken in challenging scenarios. Our system is particularly robust on complex background scenes containing objects at significantly different depths

View management for lifelong visual maps

The time complexity of making observations and loop closures in a graph-based visual SLAM system is a function of the number of views stored. Clever algorithms, such as approximate nearest neighbor search, can make this function sub-linear. Despite this, over time the number of views can still grow to a point at which the speed and/or accuracy of the system becomes unacceptable, especially in computation- and memory-constrained SLAM systems. However, not all views are created equal. Some views are rarely observed, because they have been created in an unusual lighting condition, or from low quality images, or in a location whose appearance has changed. These views can be removed to improve the overall performance of a SLAM system. In this paper, we propose a method for pruning views in a visual SLAM system to maintain its speed and accuracy for long term use.Comment: IEEE International Conference on Intelligent Robots and Systems (IROS), 201

Graph-CoVis: GNN-based Multi-view Panorama Global Pose Estimation

In this paper, we address the problem of wide-baseline camera pose estimation from a group of 360$^\circ$ panoramas under upright-camera assumption. Recent work has demonstrated the merit of deep-learning for end-to-end direct relative pose regression in 360$^\circ$ panorama pairs [11]. To exploit the benefits of multi-view logic in a learning-based framework, we introduce Graph-CoVis, which non-trivially extends CoVisPose [11] from relative two-view to global multi-view spherical camera pose estimation. Graph-CoVis is a novel Graph Neural Network based architecture that jointly learns the co-visible structure and global motion in an end-to-end and fully-supervised approach. Using the ZInD [4] dataset, which features real homes presenting wide-baselines, occlusion, and limited visual overlap, we show that our model performs competitively to state-of-the-art approaches

Genetic dissection of marker trait associations for grain micro-nutrients and thousand grain weight under heat and drought stress conditions in wheat

IntroductionWheat is grown and consumed worldwide, making it an important staple food crop for both its calorific and nutritional content. In places where wheat is used as a staple food, suboptimal micronutrient content levels, especially of grain iron (Fe) and zinc (Zn), can lead to malnutrition. Grain nutrient content is influenced by abiotic stresses, such as drought and heat stress. The best method for addressing micronutrient deficiencies is the biofortification of food crops. The prerequisites for marker-assisted varietal development are the identification of the genomic region responsible for high grain iron and zinc contents and an understanding of their genetics.MethodsA total of 193 diverse wheat genotypes were evaluated under drought and heat stress conditions across the years at the Indian Agricultural Research Institute (IARI), New Delhi, under timely sown irrigated (IR), restricted irrigated (RI) and late sown (LS) conditions. Grain iron content (GFeC) and grain zinc content (GZnC) were estimated from both the control and treatment groups. Genotyping of all the lines under study was carried out with the single nucleotide polymorphisms (SNPs) from Breeder’s 35K Axiom Array.Result and DiscussionThree subgroups were observed in the association panel based on both principal component analysis (PCA) and dendrogram analysis. A large whole-genome linkage disequilibrium (LD) block size of 3.49 Mb was observed. A genome-wide association study identified 16 unique stringent marker trait associations for GFeC, GZnC, and 1000-grain weight (TGW). In silico analysis demonstrated the presence of 28 potential candidate genes in the flanking region of 16 linked SNPs, such as synaptotagmin-like mitochondrial-lipid-binding domain, HAUS augmin-like complex, di-copper center-containing domain, protein kinase, chaperonin Cpn60, zinc finger, NUDIX hydrolase, etc. Expression levels of these genes in vegetative tissues and grain were also found. Utilization of identified markers in marker-assisted breeding may lead to the rapid development of biofortified wheat genotypes to combat malnutrition

Marker-assisted selection for transfer of QTLs to a promising line for drought tolerance in wheat (Triticum aestivum L.)

Wheat crop is subjected to various biotic and abiotic stresses, which affect crop productivity and yield. Among various abiotic stresses, drought stress is a major problem considering the current global climate change scenario. A high-yielding wheat variety, HD3086, has been released for commercial cultivation under timely sown irrigated conditions for the North Western Plain Zone (NWPZ) and North Eastern Plain Zone NEPZ of India. Presently, HD3086 is one of the highest breeder seed indented wheat varieties and has a stable yield over the years. However, under moisture deficit conditions, its potential yield cannot be achieved. The present study was undertaken to transfer drought-tolerant QTLs in the background of the variety HD3086 using marker-assisted backcross breeding. QTLs governing Biomass (BIO), Canopy Temperature (CT), Thousand Kernel Weight (TKW), Normalized Difference Vegetation Index (NDVI), and Yield (YLD) were transferred to improve performance under moisture deficit conditions. In BC1F1, BC2F1, and BC2F2 generations, the foreground selection was carried out to identify the plants with positive QTLs conferring drought tolerance and linked to traits NDVI, CT, TKW, and yield. The positive homozygous lines for targeted QTLs were advanced from BC2F2 to BC2F4via the pedigree-based phenotypic selection method. Background analysis was carried out in BC2F5 and obtained 78-91% recovery of the recurrent parent genome in the improved lines. Furthermore, the advanced lines were evaluated for 2 years under drought stress to assess improvement in MABB-derived lines. Increased GWPS, TKW, and NDVI and reduced CT was observed in improved lines. Seven improved lines were identified with significantly higher yields in comparison to HD3086 under stress conditions

Outcomes in grade 3B follicular lymphoma: an international study led by the Australasian Lymphoma Alliance

Grade (G) 3B follicular lymphoma (FL) is a rare FL subtype which exists on a histological continuum between ‘lowgrade’ (Grade 1, 2 and 3A FL) and diffuse large B-cell lymphoma (DLBCL) appearing to share features with each. Clinical characteristics and outcomes are poorly understood due to lack of adequate representation in prospective trials and large-scale analyses. We analyzed 157 G3BFL cases from 18 international centers, and two comparator groups; G3AFL (n=302) and DLBCL (n=548). Composite histology with DLBCL or low-grade FL occurred in approximately half of the G3BFL cases. With a median of 5 years follow-up, the overall survival and progression-free survival of G3BFL patients was better than that of DLBCL patients (P<0.001 and P<0.001, respectively); however, G3BFL patients were younger (P<0.001) with better performance status (P<0.001), less extranodal disease (P<0.001) and more frequently had normal lactate dehydrogenase (P<0.001) at baseline. The overall and progression-free survival of patients with G3BFL and G3AFL were similar (P=0.83 and P=0.80, respectively). After frontline immunochemotherapy, 24% of G3BFL relapsed; relapse rates were 63% in the DLBCL cohort and 19% in the low-grade FL cohort. Eight percent of relapses occurred beyond 5 years. In this G3BFL cohort, the revised International Prognostic Index successfully delineated risk groups, but the Follicular Lymphoma International Prognostic Index did not. We conclude that patients with immunochemotherapy-treated G3BFL have similar survival outcomes to those with G3AFL, yet a favorable baseline profile and distinctly superior prognosis compared to patients with DLBCL