Bootstrapping Deep Neural Networks from Approximate Image Processing Pipelines

Complex image processing and computer vision systems often consist of a processing pipeline of functional modules. We intend to replace parts or all of a target pipeline with deep neural networks to achieve benefits such as increased accuracy or reduced computational requirement. To acquire a large amount of labeled data necessary to train the deep neural network, we propose a workflow that leverages the target pipeline to create a significantly larger labeled training set automatically, without prior domain knowledge of the target pipeline. We show experimentally that despite the noise introduced by automated labeling and only using a very small initially labeled data set, the trained deep neural networks can achieve similar or even better performance than the components they replace, while in some cases also reducing computational requirements.Comment: 6 pages, 5 figure

TextManiA: Enriching Visual Feature by Text-driven Manifold Augmentation

We propose TextManiA, a text-driven manifold augmentation method that semantically enriches visual feature spaces, regardless of class distribution. TextManiA augments visual data with intra-class semantic perturbation by exploiting easy-to-understand visually mimetic words, i.e., attributes. This work is built on an interesting hypothesis that general language models, e.g., BERT and GPT, encompass visual information to some extent, even without training on visual training data. Given the hypothesis, TextManiA transfers pre-trained text representation obtained from a well-established large language encoder to a target visual feature space being learned. Our extensive analysis hints that the language encoder indeed encompasses visual information at least useful to augment visual representation. Our experiments demonstrate that TextManiA is particularly powerful in scarce samples with class imbalance as well as even distribution. We also show compatibility with the label mix-based approaches in evenly distributed scarce data.Comment: Accepted at ICCV 2023. [Project Pages] https://textmania.github.io

Bootstrapping Deep Neural Networks from Approximate Image Processing Pipelines

Complex image processing and computer vision systems often consist of a processing pipeline of functional modules. We intend to replace parts or all of a target pipeline with deep neural networks to achieve benefits such as increased accuracy or reduced computational requirement. To acquire a large amount of labeled data necessary to train the deep neural network, we propose a workflow that leverages the target pipeline to create a significantly larger labeled training set automatically, without prior domain knowledge of the target pipeline. We show experimentally that despite the noise introduced by automated labeling and only using a very small initially labeled data set, the trained deep neural networks can achieve similar or even better performance than the components they replace, while in some cases also reducing computational requirements. Comment: 6 pages, 5 figur

Solving Square Jigsaw Puzzles with Loop Constraints

Abstract. We present a novel algorithm based on “loop constraints ” for assem-bling non-overlapping square-piece jigsaw puzzles where the rotation and the position of each piece are unknown. Our algorithm finds small loops of puzzle pieces which form consistent cycles. These small loops are in turn aggregated into higher order “loops of loops ” in a bottom-up fashion. In contrast to pre-vious puzzle solvers which avoid or ignore puzzle cycles, we specifically seek out and exploit these loops as a form of outlier rejection. Our algorithm signifi-cantly outperforms state-of-the-art algorithms in puzzle reconstruction accuracy. For the most challenging type of image puzzles with unknown piece rotation we reduce the reconstruction error by up to 70%. We determine an upper bound on reconstruction accuracy for various data sets and show that, in some cases, our algorithm nearly matches the upper bound

Enriching Visual Features via Text-driven Manifold Augmentation

1

Microfluidic multifunctional probe array dielectrophoretic force spectroscopy with wide loading rates

The simultaneous investigation of a large number of events with different types of intermolecular interactions, from nonequilibrium high-force pulling assays to quasi-equilibrium unbinding events in the same environment, can be very important for fully understanding intermolecular bond-rupture mechanisms. Here, we describe a novel dielectrophoretic force spectroscopy technique that utilizes microsized beads as multifunctional probes for parallel measurement of intermolecular forces with an extremely wide range of force rate (10 -4 to 10 4 pN/s) inside a microfluidic device. In our experiments, various forces, which broadly form the basis of all molecular interactions, were measured across a range of force loading rates by multifunctional probes of various diameters with a throughput of over 600 events per mm 2, simultaneously and in the same environment. Furthermore, the individual bond-rupture forces, the parameters for the characterization of entire energy landscapes, and the effective stiffness of the force spectroscopy were determined on the basis of the measured results. This method of determining intermolecular forces could be very useful for the precise and simultaneous examination of various molecular interactions, as it can be easily and cost-effectively implemented within a microfluidic device for a range of applications including immunoassays, molecular mechanics, chemical and biological screening, and mechanobiology. © 2012 American Chemical Society

Microfluidic Multifunctional Probe Array Dielectrophoretic Force Spectroscopy with Wide Loading Rates

The simultaneous investigation of a large number of events with different types of intermolecular interactions, from nonequilibrium high-force pulling assays to quasi-equilibrium unbinding events in the same environment, can be very important for fully understanding intermolecular bond-rupture mechanisms. Here, we describe a novel dielectrophoretic force spectroscopy technique that utilizes microsized beads as multifunctional probes for parallel measurement of intermolecular forces with an extremely wide range of force rate (10^–4 to 10⁴ pN/s) inside a microfluidic device. In our experiments, various forces, which broadly form the basis of all molecular interactions, were measured across a range of force loading rates by multifunctional probes of various diameters with a throughput of over 600 events per mm², simultaneously and in the same environment. Furthermore, the individual bond-rupture forces, the parameters for the characterization of entire energy landscapes, and the effective stiffness of the force spectroscopy were determined on the basis of the measured results. This method of determining intermolecular forces could be very useful for the precise and simultaneous examination of various molecular interactions, as it can be easily and cost-effectively implemented within a microfluidic device for a range of applications including immunoassays, molecular mechanics, chemical and biological screening, and mechanobiology