Image-Based Flexible Endoscope Steering

Manually steering the tip of a flexible endoscope to navigate through an endoluminal path relies on the physician’s dexterity and experience. In this paper we present the realization of a robotic flexible endoscope steering system that uses the endoscopic images to control the tip orientation towards the direction of the lumen. Two image-based control algorithms are investigated, one is based on the optical flow and the other is based on the image intensity. Both are evaluated using simulations in which the endoscope was steered through the lumen. The RMS distance to the lumen center was less than 25% of the lumen width. An experimental setup was built using a standard flexible endoscope, and the image-based control algorithms were used to actuate the wheels of the endoscope for tip steering. Experiments were conducted in an anatomical model to simulate gastroscopy. The image intensity- based algorithm was capable of steering the endoscope tip through an endoluminal path from the mouth to the duodenum accurately. Compared to manual control, the robotically steered endoscope performed 68% better in terms of keeping the lumen centered in the image

Image-based Localization using Hourglass Networks

In this paper, we propose an encoder-decoder convolutional neural network (CNN) architecture for estimating camera pose (orientation and location) from a single RGB-image. The architecture has a hourglass shape consisting of a chain of convolution and up-convolution layers followed by a regression part. The up-convolution layers are introduced to preserve the fine-grained information of the input image. Following the common practice, we train our model in end-to-end manner utilizing transfer learning from large scale classification data. The experiments demonstrate the performance of the approach on data exhibiting different lighting conditions, reflections, and motion blur. The results indicate a clear improvement over the previous state-of-the-art even when compared to methods that utilize sequence of test frames instead of a single frame.Comment: Camera-ready version for ICCVW 2017 (fixed glitches in abstract

Image-based Recommendations on Styles and Substitutes

Humans inevitably develop a sense of the relationships between objects, some of which are based on their appearance. Some pairs of objects might be seen as being alternatives to each other (such as two pairs of jeans), while others may be seen as being complementary (such as a pair of jeans and a matching shirt). This information guides many of the choices that people make, from buying clothes to their interactions with each other. We seek here to model this human sense of the relationships between objects based on their appearance. Our approach is not based on fine-grained modeling of user annotations but rather on capturing the largest dataset possible and developing a scalable method for uncovering human notions of the visual relationships within. We cast this as a network inference problem defined on graphs of related images, and provide a large-scale dataset for the training and evaluation of the same. The system we develop is capable of recommending which clothes and accessories will go well together (and which will not), amongst a host of other applications.Comment: 11 pages, 10 figures, SIGIR 201

Visual BFI: an Exploratory Study for Image-based Personality Test

This paper positions and explores the topic of image-based personality test. Instead of responding to text-based questions, the subjects will be provided a set of "choose-your-favorite-image" visual questions. With the image options of each question belonging to the same concept, the subjects' personality traits are estimated by observing their preferences of images under several unique concepts. The solution to design such an image-based personality test consists of concept-question identification and image-option selection. We have presented a preliminary framework to regularize these two steps in this exploratory study. A demo version of the designed image-based personality test is available at http://www.visualbfi.org/. Subjective as well as objective evaluations have demonstrated the feasibility of image-based personality test in limited questions

Image-based 3D Scene Reconstruction and Rescue Simulation Framework for Railway Accidents

Although the railway transport is regarded as a relatively safe transportation tool, many railway accidents have still happened worldwide. In this research, an image-based 3D scene reconstruction framework was proposed to help railway accident emergency rescues. Based on the improved constrained non-linear least square optimization, the framework can automatically model the accident scene with only one panorama in a short time. We embedded the self-developed global terrain module into the commercial visualization and physics engine, which makes the commercial engine can be used to render the static scene at anywhere and simulate the dynamic rescue process respectively. In addition, a Head Mounted Device (HMD) was integrated into this framework to allow users to verify their rescue plan and review previous railway accidents in an immersive environment

VITON: An Image-based Virtual Try-on Network

We present an image-based VIirtual Try-On Network (VITON) without using 3D information in any form, which seamlessly transfers a desired clothing item onto the corresponding region of a person using a coarse-to-fine strategy. Conditioned upon a new clothing-agnostic yet descriptive person representation, our framework first generates a coarse synthesized image with the target clothing item overlaid on that same person in the same pose. We further enhance the initial blurry clothing area with a refinement network. The network is trained to learn how much detail to utilize from the target clothing item, and where to apply to the person in order to synthesize a photo-realistic image in which the target item deforms naturally with clear visual patterns. Experiments on our newly collected Zalando dataset demonstrate its promise in the image-based virtual try-on task over state-of-the-art generative models

Experimental study of optimal energy weighting in energy-resolved CT using a CZT detector

Recent advances in energy-resolved CT can potentially improve contrast-to-noise ratio (CNR), which could subsequently reduce dose in conventional and dedicated breast CT. Two methods have been proposed for optimal energy weighting: weighting the energy-bin data prior to log normalization (projection-based weighting) and weighting the energy-bin data after log normalization (image-based weighting). Previous studies suggested that optimal projection-based and image-based energy weighting provide similar CNR improvements for energy-resolved CT compared to photon-counting or conventional energy-integrating CT. This study experimentally investigated the improvement in CNR of projection-based and image-based weighted images relative to photon-counting for six different energy-bin combinations using a bench top system with a CZT detector. The results showed CNR values ranged between 0.85 and 1.01 for the projection-based weighted images and between 0.91 and 1.43 for the image-based weighted images, relative to the CNR for the photon-counting image. The range of CNR values demonstrates the effects of energy-bin selection on CNR for a particular energy weighting scheme. The non-ideal spectral response of the CZT detector caused spectral tailing, which appears to generally reduce the CNR for the projection-based weighted images. Image-based weighting increased CNR in five of the six bin combinations despite the non-ideal spectral effects