Contextual cropping and scaling of TV productions

This is the author's accepted manuscript. The final publication is available at Springer via http://dx.doi.org/10.1007/s11042-011-0804-3. Copyright @ Springer Science+Business Media, LLC 2011.In this paper, an application is presented which automatically adapts SDTV (Standard Definition Television) sports productions to smaller displays through intelligent cropping and scaling. It crops regions of interest of sports productions based on a smart combination of production metadata and systematic video analysis methods. This approach allows a context-based composition of cropped images. It provides a differentiation between the original SD version of the production and the processed one adapted to the requirements for mobile TV. The system has been comprehensively evaluated by comparing the outcome of the proposed method with manually and statically cropped versions, as well as with non-cropped versions. Envisaged is the integration of the tool in post-production and live workflows

An Improved Observation Model for Super-Resolution under Affine Motion

Super-resolution (SR) techniques make use of subpixel shifts between frames in an image sequence to yield higher-resolution images. We propose an original observation model devoted to the case of non isometric inter-frame motion as required, for instance, in the context of airborne imaging sensors. First, we describe how the main observation models used in the SR literature deal with motion, and we explain why they are not suited for non isometric motion. Then, we propose an extension of the observation model by Elad and Feuer adapted to affine motion. This model is based on a decomposition of affine transforms into successive shear transforms, each one efficiently implemented by row-by-row or column-by-column 1-D affine transforms. We demonstrate on synthetic and real sequences that our observation model incorporated in a SR reconstruction technique leads to better results in the case of variable scale motions and it provides equivalent results in the case of isometric motions

Towards retrieving force feedback in robotic-assisted surgery: a supervised neuro-recurrent-vision approach

Robotic-assisted minimally invasive surgeries have gained a lot of popularity over conventional procedures as they offer many benefits to both surgeons and patients. Nonetheless, they still suffer from some limitations that affect their outcome. One of them is the lack of force feedback which restricts the surgeon's sense of touch and might reduce precision during a procedure. To overcome this limitation, we propose a novel force estimation approach that combines a vision based solution with supervised learning to estimate the applied force and provide the surgeon with a suitable representation of it. The proposed solution starts with extracting the geometry of motion of the heart's surface by minimizing an energy functional to recover its 3D deformable structure. A deep network, based on a LSTM-RNN architecture, is then used to learn the relationship between the extracted visual-geometric information and the applied force, and to find accurate mapping between the two. Our proposed force estimation solution avoids the drawbacks usually associated with force sensing devices, such as biocompatibility and integration issues. We evaluate our approach on phantom and realistic tissues in which we report an average root-mean square error of 0.02 N.Peer ReviewedPostprint (author's final draft

Enabling arbitrary rotation camera-motion using multi-sprites with minimum coding cost

Object-oriented coding in the MPEG-4 standard enables the separate processing of foreground objects and the scene background (sprite). Since the background sprite only has to be sent once, transmission bandwidth can be saved.We have found that the counter-intuitive approach of splitting the background into several independent parts can reduce the overall amount of data. Furthermore, we show that in the general case, the synthesis of a single background sprite is even impossible and that the scene background must be sent as multiple sprites instead. For this reason, we propose an algorithm that provides an optimal partitioning of a video sequence into independent background sprites (a multisprite), resulting in a significant reduction of the involved coding cost. Additionally, our sprite-generation algorithm ensures that the sprite resolution is kept high enough to preserve all details of the input sequence, which is a problem especially during camera zoom-in operations. Even though our sprite generation algorithm creates multiple sprites instead of only a single background sprite, it is fully compatible with the existing MPEG-4 standard. The algorithm has been evaluated with several test sequences, including the well-known Table-tennis and Stefan sequences. The total coding cost for the sprite VOP is reduced by a factor of about 2.6 or even higher, depending on the sequence

Biologically Inspired Dynamic Textures for Probing Motion Perception

Perception is often described as a predictive process based on an optimal inference with respect to a generative model. We study here the principled construction of a generative model specifically crafted to probe motion perception. In that context, we first provide an axiomatic, biologically-driven derivation of the model. This model synthesizes random dynamic textures which are defined by stationary Gaussian distributions obtained by the random aggregation of warped patterns. Importantly, we show that this model can equivalently be described as a stochastic partial differential equation. Using this characterization of motion in images, it allows us to recast motion-energy models into a principled Bayesian inference framework. Finally, we apply these textures in order to psychophysically probe speed perception in humans. In this framework, while the likelihood is derived from the generative model, the prior is estimated from the observed results and accounts for the perceptual bias in a principled fashion.Comment: Twenty-ninth Annual Conference on Neural Information Processing Systems (NIPS), Dec 2015, Montreal, Canad

On the evolution of elastic properties during laboratory stick-slip experiments spanning the transition from slow slip to dynamic rupture

The physical mechanisms governing slow earthquakes remain unknown, as does the relationship between slow and regular earthquakes. To investigate the mechanism(s) of slow earthquakes and related quasi-dynamic modes of fault slip we performed laboratory experiments on simulated fault gouge in the double direct shear configuration. We reproduced the full spectrum of slip behavior, from slow to fast stick slip, by altering the elastic stiffness of the loading apparatus (k) to match the critical rheologic stiffness of fault gouge (kc). Our experiments show an evolution from stable sliding, when k>kc, to quasi-dynamic transients when k ~ kc, to dynamic instabilities when k<kc. To evaluate the microphysical processes of fault weakening we monitored variations of elastic properties. We find systematic changes in P wave velocity (Vp) for laboratory seismic cycles. During the coseismic stress drop, seismic velocity drops abruptly, consistent with observations on natural faults. In the preparatory phase preceding failure, we find that accelerated fault creep causes a Vp reduction for the complete spectrum of slip behaviors. Our results suggest that the mechanics of slow and fast ruptures share key features and that they can occur on same faults, depending on frictional properties. In agreement with seismic surveys on tectonic faults our data show that their state of stress can be monitored by Vp changes during the seismic cycle. The observed reduction in Vp during the earthquake preparatory phase suggests that if similar mechanisms are confirmed in nature high-resolution monitoring of fault zone properties may be a promising avenue for reliable detection of earthquake precursors

Downscaling of fracture energy during brittle creep experiments

We present mode 1 brittle creep fracture experiments along fracture surfaces that contain strength heterogeneities. Our observations provide a link between smooth macroscopic time-dependent failure and intermittent microscopic stress-dependent processes. We find the large-scale response of slow-propagating subcritical cracks to be well described by an Arrhenius law that relates the fracture speed to the energy release rate. At the microscopic scale, high-resolution optical imaging of the transparent material used (PMMA) allows detailed description of the fracture front. This reveals a local competition between subcritical and critical propagation (pseudo stick-slip front advances) independently of loading rates. Moreover, we show that the local geometry of the crack front is self-affine and the local crack front velocity is power law distributed. We estimate the local fracture energy distribution by combining high-resolution measurements of the crack front geometry and an elastic line fracture model. We show that the average local fracture energy is significantly larger than the value derived from a macroscopic energy balance. This suggests that homogenization of the fracture energy is not straightforward and should be taken cautiously. Finally, we discuss the implications of our results in the context of fault mechanics