Low Compute and Fully Parallel Computer Vision with HashMatch

Numerous computer vision problems such as stereo depth estimation, object-class segmentation and fore-ground/background segmentation can be formulated as per-pixel image labeling tasks. Given one or many images as input, the desired output of these methods is usually a spatially smooth assignment of labels. The large amount of such computer vision problems has lead to significant research efforts, with the state of art moving from CRF-based approaches to deep CNNs and more recently, hybrids of the two. Although these approaches have significantly advanced the state of the art, the vast majority has solely focused on improving quantitative results and are not designed for low-compute scenarios. In this paper, we present a new general framework for a variety of computer vision labeling tasks, called HashMatch. Our approach is designed to be both fully parallel, i.e. each pixel is independently processed, and low-compute, with a model complexity an order of magnitude less than existing CNN and CRF-based approaches. We evaluate HashMatch extensively on several problems such as disparity estimation, image retrieval, feature approximation and background subtraction, for which HashMatch achieves high computational efficiency while producing high quality results

From Multiview Image Curves to 3D Drawings

Reconstructing 3D scenes from multiple views has made impressive strides in recent years, chiefly by correlating isolated feature points, intensity patterns, or curvilinear structures. In the general setting - without controlled acquisition, abundant texture, curves and surfaces following specific models or limiting scene complexity - most methods produce unorganized point clouds, meshes, or voxel representations, with some exceptions producing unorganized clouds of 3D curve fragments. Ideally, many applications require structured representations of curves, surfaces and their spatial relationships. This paper presents a step in this direction by formulating an approach that combines 2D image curves into a collection of 3D curves, with topological connectivity between them represented as a 3D graph. This results in a 3D drawing, which is complementary to surface representations in the same sense as a 3D scaffold complements a tent taut over it. We evaluate our results against truth on synthetic and real datasets.Comment: Expanded ECCV 2016 version with tweaked figures and including an overview of the supplementary material available at multiview-3d-drawing.sourceforge.ne

An Analysis of Errors in Graph-Based Keypoint Matching and Proposed Solutions

International audienceAn error occurs in graph-based keypoint matching when key-points in two different images are matched by an algorithm but do not correspond to the same physical point. Most previous methods acquire keypoints in a black-box manner, and focus on developing better algorithms to match the provided points. However to study the complete performance of a matching system one has to study errors through the whole matching pipeline, from keypoint detection, candidate selection to graph optimisation. We show that in the full pipeline there are six different types of errors that cause mismatches. We then present a matching framework designed to reduce these errors. We achieve this by adapting keypoint detectors to better suit the needs of graph-based matching, and achieve better graph constraints by exploiting more information from their keypoints. Our framework is applicable in general images and can handle clutter and motion discontinuities. We also propose a method to identify many mismatches a posteriori based on Left-Right Consistency inspired by stereo matching due to the asymmetric way we detect keypoints and define the graph

A review of silhouette extraction algorithms for use within visual hull pipelines

© 2020, © 2020 Informa UK Limited, trading as Taylor & Francis Group. Markerless motion capture would permit the study of human biomechanics in environments where marker-based systems are impractical, e.g. outdoors or underwater. The visual hull tool may enable such data to be recorded, but it requires the accurate detection of the silhouette of the object in multiple camera views. This paper reviews the top-performing algorithms available to date for silhouette extraction, with the visual hull in mind as the downstream application; the rationale is that higher-quality silhouettes would lead to higher-quality visual hulls, and consequently better measurement of movement. This paper is the first attempt in the literature to compare silhouette extraction algorithms that belong to different fields of Computer Vision, namely background subtraction, semantic segmentation, and multi-view segmentation. It was found that several algorithms exist that would be substantial improvements over the silhouette extraction algorithms traditionally used in visual hull pipelines. In particular, FgSegNet v2 (a background subtraction algorithm), DeepLabv3+ JFT (a semantic segmentation algorithm), and Djelouah 2013 (a multi-view segmentation algorithm) are the most accurate and promising methods for the extraction of silhouettes from 2D images to date, and could seamlessly be integrated within a visual hull pipeline for studies of human movement or biomechanics

Quantifying Absolute Neutralization Titers against SARS-CoV-2 by a Standardized Virus Neutralization Assay Allows for CrossCohort Comparisons of COVID-19 Sera

The global coronavirus disease 2019 (COVID-19) pandemic has mobilized efforts to develop vaccines and antibody-based therapeutics, including convalescent-phase plasma therapy, that inhibit viral entry by inducing or transferring neutralizing antibodies (nAbs) against the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) spike glycoprotein (CoV2-S). However, rigorous efficacy testing requires extensive screening with live virus under onerous biosafety level 3 (BSL3) conditions, which limits high-throughput screening of patient and vaccine sera. Myriad BSL2-compatible surrogate virus neutralization assays (VNAs) have been developed to overcome this barrier. Yet, there is marked variability between VNAs and how their results are presented, making intergroup comparisons difficult. To address these limitations, we developed a standardized VNA using CoV2-S pseudotyped particles (CoV2pp) based on vesicular stomatitis virus bearing the Renilla luciferase gene in place of its G glyco-protein (VSVDG); this assay can be robustly produced at scale and generate accurate neutralizing titers within 18 h postinfection. Our standardized CoV2pp VNA showed a strong positive correlation with CoV2-S enzyme-linked immunosorbent assay (ELISA) results and live-virus neutralizations in confirmed convalescent-patient sera. Three independent groups subsequently validated our standardized CoV2pp VNA (n . 120). Our data (i) show that absolute 50% inhibitory concentration (absIC50), absIC80, and absIC90 values can be legitimately compared across diverse cohorts, (ii) highlight the substantial but consistent variability in neutralization potency across these cohorts, and (iii) support the use of the absIC80 as a more meaningful metric for assessing the neutralization potency of a vaccine or convalescent-phase sera. Lastly, we used our CoV2pp in a screen to identify ultrapermissive 293T clones that stably express ACE2 or ACE2 plus TMPRSS2. When these are used in combination with our CoV2pp, we can produce CoV2pp sufficient for 150,000 standardized VNAs/week. IMPORTANCE Vaccines and antibody-based therapeutics like convalescent-phase plasma therapy are premised upon inducing or transferring neutralizing antibodies that inhibit SARS-CoV-2 entry into cells. Virus neutralization assays (VNAs) for measuring neutralizing antibody titers (NATs) are an essential part of determining vaccine or therapeutic efficacy. However, such efficacy testing is limited by the inherent dangers of working with the live virus, which requires specialized high-level biocontainment facilities. We there-fore developed a standardized replication-defective pseudotyped particle system that mimics the entry of live SARS-CoV-2. This tool allows for the safe and efficient measurement of NATs, determination of other forms of entry inhibition, and thorough investigation of virus entry mechanisms. Four independent labs across the globe validated our standardized VNA using diverse cohorts. We argue that a standardized and scalable assay is necessary for meaningful comparisons of the myriad of vaccines and antibody-based therapeutics becoming available. Our data provide generalizable metrics for assessing their efficacy.Fil: Oguntuyo, Kasopefoluwa. Icahn School of Medicine at Mount Sinai; Estados UnidosFil: Stevens, Christian S.. Icahn School of Medicine at Mount Sinai; Estados UnidosFil: Hung, Chuan Tien. Icahn School of Medicine at Mount Sinai; Estados UnidosFil: Ikegame, Satoshi. Icahn School of Medicine at Mount Sinai; Estados UnidosFil: Acklin, Joshua A.. Icahn School of Medicine at Mount Sinai; Estados UnidosFil: Kowdle, Shreyas S.. Icahn School of Medicine at Mount Sinai; Estados UnidosFil: Carmichael, Jillian C.. Icahn School of Medicine at Mount Sinai; Estados UnidosFil: Chiu, Hsin Ping. Icahn School of Medicine at Mount Sinai; Estados UnidosFil: Azarm, Kristopher D.. Icahn School of Medicine at Mount Sinai; Estados UnidosFil: Haas, Griffin D.. Icahn School of Medicine at Mount Sinai; Estados UnidosFil: Amanat, Fatima. Icahn School of Medicine at Mount Sinai; Estados UnidosFil: Klingler, Jéromine. Icahn School of Medicine at Mount Sinai; Estados UnidosFil: Baine, Ian. Icahn School of Medicine at Mount Sinai; Estados UnidosFil: Arinsburg, Suzanne. Icahn School of Medicine at Mount Sinai; Estados UnidosFil: Bandres, Juan C.. Icahn School of Medicine at Mount Sinai; Estados UnidosFil: Siddiquey, Mohammed N. A.. Icahn School of Medicine at Mount Sinai; Estados UnidosFil: Schilke, Robert M.. Icahn School of Medicine at Mount Sinai; Estados UnidosFil: Woolard, Matthew D.. State University of Louisiana; Estados UnidosFil: Zhang, Hongbo. State University of Louisiana; Estados UnidosFil: Duty, Andrew J.. Icahn School of Medicine at Mount Sinai; Estados UnidosFil: Kraus, Thomas A.. Icahn School of Medicine at Mount Sinai; Estados UnidosFil: Moran, Thomas M.. Icahn School of Medicine at Mount Sinai; Estados UnidosFil: Tortorella, Domenico. Icahn School of Medicine at Mount Sinai; Estados UnidosFil: Lim, Jean K.. Icahn School of Medicine at Mount Sinai; Estados UnidosFil: Gamarnik, Andrea Vanesa. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Parque Centenario. Instituto de Investigaciones Bioquímicas de Buenos Aires. Fundación Instituto Leloir. Instituto de Investigaciones Bioquímicas de Buenos Aires; Argentina. Icahn School of Medicine at Mount Sinai; Estados UnidosFil: Hioe, Catarina E.. Icahn School of Medicine at Mount Sinai; Estados UnidosFil: Zolla Pazner, Susan. Icahn School of Medicine at Mount Sinai; Estados UnidosFil: Ivanov, Stanimir S.. State University of Louisiana; Estados UnidosFil: Kamil, Jeremy. State University of Louisiana; Estados UnidosFil: Krammer, Florian. Icahn School of Medicine at Mount Sinai; Estados UnidosFil: Lee, Benhur. Icahn School of Medicine at Mount Sinai; Estados UnidosFil: Ojeda, Diego Sebastian. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Houssay. Instituto de Investigaciones Biomédicas en Retrovirus y Sida. Universidad de Buenos Aires. Facultad de Medicina. Instituto de Investigaciones Biomédicas en Retrovirus y Sida; ArgentinaFil: González López Ledesma, María Mora. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Parque Centenario. Instituto de Investigaciones Bioquímicas de Buenos Aires. Fundación Instituto Leloir. Instituto de Investigaciones Bioquímicas de Buenos Aires; ArgentinaFil: Costa Navarro, Guadalupe Soledad. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Parque Centenario. Instituto de Investigaciones Bioquímicas de Buenos Aires. Fundación Instituto Leloir. Instituto de Investigaciones Bioquímicas de Buenos Aires; ArgentinaFil: Pallarés, H. M.. No especifíca;Fil: Sanchez, Lautaro Nicolas. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Parque Centenario. Instituto de Investigaciones Bioquímicas de Buenos Aires. Fundación Instituto Leloir. Instituto de Investigaciones Bioquímicas de Buenos Aires; ArgentinaFil: Perez, P.. No especifíca;Fil: Ostrowsk, M.. No especifíca;Fil: Villordo, S. M.. No especifíca;Fil: Alvarez, D. E.. No especifíca;Fil: Caramelo, J. J.. No especifíca;Fil: Carradori, J.. No especifíca;Fil: Yanovsky, M. J.. No especifíca

Learning to segment a video to clips based on scene and camera motion

10.1007/978-3-642-33712-3_20Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)7574 LNCSPART 3272-28

Revisiting depth layers from occlusions

10.1109/CVPR.2013.272Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition2091-2098PIVR

Video categorization using Object of Interest detection

10.1109/ICIP.2010.5652300Proceedings - International Conference on Image Processing, ICIP4569-457

Combining monocular geometric cues with traditional stereo cues for consumer camera stereo

10.1007/978-3-642-33868-7_11Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)7584 LNCSPART 2103-11