Video from a Single Coded Exposure Photograph using a Learned Over-Complete Dictionary

Cameras face a fundamental tradeoff between the spatial and temporal resolution - digital still cameras can capture images with high spatial resolution, but most high-speed video cameras suffer from low spatial resolution. It is hard to overcome this tradeoff without incurring a significant increase in hardware costs. In this paper, we propose techniques for sampling, representing and reconstructing the space-time volume in order to overcome this tradeoff. Our approach has two important distinctions compared to previous works: (1) we achieve sparse representation of videos by learning an over-complete dictionary on video patches, and (2) we adhere to practical constraints on sampling scheme which is imposed by architectures of present image sensor devices. Consequently, our sampling scheme can be implemented on image sensors by making a straightforward modification to the control unit. To demonstrate the power of our approach, we have implemented a prototype imaging system with per-pixel coded exposure control using a liquid crystal on silicon (LCoS) device. Using both simulations and experiments on a wide range of scenes, we show that our method can effectively reconstruct a video from a single image maintaining high spatial resolution

Frameless Representation and Manipulation of Image Data

Most image sensors mimic film, integrating light during an exposure interval and then reading the latent image as a complete frame. In contrast, frameless image capture attempts to construct a continuous waveform for each sensel describing how the Ev (exposure value required at each pixel) changes over time. This allows great flexibility in computationally extracting frames after exposure. An overview of how this could be accomplished was presented at EI2014, with an emphasis on frameless sensor technology. In contrast, the current work centers on deriving frameless data from sequences of conventionally captured frames

Temporal shape super-resolution by intra-frame motion encoding using high-fps structured light

One of the solutions of depth imaging of moving scene is to project a static pattern on the object and use just a single image for reconstruction. However, if the motion of the object is too fast with respect to the exposure time of the image sensor, patterns on the captured image are blurred and reconstruction fails. In this paper, we impose multiple projection patterns into each single captured image to realize temporal super resolution of the depth image sequences. With our method, multiple patterns are projected onto the object with higher fps than possible with a camera. In this case, the observed pattern varies depending on the depth and motion of the object, so we can extract temporal information of the scene from each single image. The decoding process is realized using a learning-based approach where no geometric calibration is needed. Experiments confirm the effectiveness of our method where sequential shapes are reconstructed from a single image. Both quantitative evaluations and comparisons with recent techniques were also conducted.Comment: 9 pages, Published at the International Conference on Computer Vision (ICCV 2017

Efficient Space-Time Sampling with Pixel-wise Coded Exposure for High Speed Imaging

Cameras face a fundamental tradeoff between spatial and temporal resolution. Digital still cameras can capture images with high spatial resolution, but most high-speed video cameras have relatively low spatial resolution. It is hard to overcome this tradeoff without incurring a significant increase in hardware costs. In this paper, we propose techniques for sampling, representing and reconstructing the space-time volume in order to overcome this tradeoff. Our approach has two important distinctions compared to previous works: (1) we achieve sparse representation of videos by learning an over-complete dictionary on video patches, and (2) we adhere to practical hardware constraints on sampling schemes imposed by architectures of current image sensors, which means that our sampling function can be implemented on CMOS image sensors with modified control units in the future. We evaluate components of our approach - sampling function and sparse representation by comparing them to several existing approaches. We also implement a prototype imaging system with pixel-wise coded exposure control using a Liquid Crystal on Silicon (LCoS) device. System characteristics such as field of view, Modulation Transfer Function (MTF) are evaluated for our imaging system. Both simulations and experiments on a wide range of scenes show that our method can effectively reconstruct a video from a single coded image while maintaining high spatial resolution

UAV-BASED GEOTECHNICAL MODELING AND MAPPING OF AN INACCESSIBLE UNDERGROUND SITE

Digital photogrammetry is becoming a more common method used for mapping geological and structural rock mass features in underground mining. The issue of capturing geological and structural data in inaccessible, unsupported areas of mines remains even when utilizing terrestrial photogrammetric methods; thus, geotechnical models of mines are left with incomplete datasets. Large unsupported underground voids, like stopes, have the potential to cause major failures, but by filling in the geotechnical data gaps in inaccessible areas, potential failures can be predicted through kinematic analysis of the area’s mapped discontinuities. Implementation of Unmanned Aerial Vehicles (UAVs) in underground mines and recent advances in obstacle detection systems have allowed for greater experimentation with photogrammetry conducted from a UAV platform in mines. For this study, a UAV-based underground photogrammetry system was developed to manually capture imagery in an inaccessible stope at Barrick Gold Corporation’s Golden Sunlight Mine (GSM) in Whitehall, Montana, to see whether or not the approach is a viable remote sensing technique for gathering georeferenced geotechnical data. Development of the system involved selecting an appropriate UAV platform, identifying a lighting system capable of providing adequate illumination, acquiring a sensor system that consistently avoids obstacles, and choosing the appropriate UAV camera (and its respective settings) for underground UAV-based imaging. In order to georeference the data collected in the inaccessible stope, paintballs were shot into the stope to create ground control points that were then surveyed in laser range detection. These paintball marks had to be in visual line-of-sight and visible in the imagery captured via UAV camera in order to georeferenced them. Using the imagery collected in the stope at GSM, models were constructed and structural features were mapped on those models. Bentley ContextCapture software was able to successfully construct a stope model from the video frame imagery collected via UAV in the stope, while ADAM Technology was not. Split-Engineering’s Split-FX and ADAM Technology were used separately to map the discontinuity planes present within the model. A comparison of underground discontinuity mapping was performed using the UAV-based photogrammetry captured in the stope and hand mapping data collected around the entrance to the stope. It was found that northeasterly striking discontinuity planes were identified using the digital mapping, but not in hand mapping. Using UAV-based photogrammetry for geotechnical data collection creates a quick and thorough mapping process with time-stamped imagery that can potentially create a safer mine. The lessons learned during this study may help guide future efforts using UAVs to capture geologic data and to help monitor stability in areas that are inaccessible

An instrumentation system for the measurement and display of the dynamic force distribution under the foot during locomotion

Bibliography: pages 81-83.The clinical assessment of the weight bearing foot during locomotion is normally based on subjective judgement rather than on quantitative measurement. The techniques which have been proposed for recording the dynamic forces acting on the foot are either too complex for clinical practise or there is difficulty in relating the measured force distribution to the physical surface of the foot. The system that has been developed measures the vertical foot/ground forces during gait and immediately displays the data in a manner which can be readily assimilated. The instrumentation system consists of a segmented force plate constructed from 16 transparent beams mounted so that the total load as well as the centre of pressure on any beam can be ascertained. When the foot contacts the plate, its plantar surface is photographed through the transparent force plate by a television camera while a second television camera photographs a lateral aspect of the legs and feet. A composite video display is then generated consisting of (i) a lateral view of the legs and feet (ii) a view of the plantar surface of the planted foot with centre of pressure lines superimposed (iii) a bar chart display of the load carried by each beam. The system output is recorded on a video tape recorder which has a stop motion facility. This enables a frame by frame analysis to be made subsequently and selected stills to be photographed as a permanent record. Three series of photographs are presented which clearly show the differences between normal and abnormal gait