Multiresolution-based reconstruction for compressive spectral video sensing using a spectral multiplexing sensor

Spectral multiplexing sensors based on compressive sensing attempt to break the Nyquist barrier to acquire high spectral resolution scenes. Particularly, the colored coded aperture-based compressive spectral imager extended to video, or video C-CASSI, is a spectral multiplexing sensor that allows capturing spectral dynamic scenes by projecting each spectral frame onto a bidimensional detector using a 3D coded aperture. Afterwards, the compressed signal reconstruction is performed iteratively by finding a sparse solution to an undetermined linear system of equations. Even though the acquired signal can be recovered from much fewer observations by an  −  -norm recovery algorithm than using conventional sensors, the reconstruction exhibits diverse challenges originated by the temporal variable or motion. The motion during the reconstruction produces artifacts that damages the entire data. In this work, a multiresolution-based reconstruction method for compressive spectral video sensing is proposed. In this way, it obtains the temporal information from the measurements at a low computational cost. Thereby, the optimization problem to recover the signal is extended by adding temporal information in order to correct the errors originated by the scene motion. Computational experiments performed over four different spectral videos show an improvement up to 4dB in terms of peak-signal to noise ratio (PSNR) in the reconstruction quality using the multiresolution approach applied to the spectral video reconstruction with respect to the traditional inverse problem.  Los sensores de multiplexación espectral basados en muestreo compresivo intentan romper la barrera de Nyquist para adquirir escenas de alta resolución espectral. Particularmente, el sistema de imágenes espectrales de única captura basado en aperturas codificadas de color extendido a vídeo, o video – CCASSI, es un sensor de multiplexación espectral que permite la adquisición de imágenes espectrales dinámicas proyectando cada fotograma espectral sobre un detector bidimensional usando un apertura de codificación 3D. Posteriormente, la reconstrucción de la señal comprimida se realiza iterativamente encontrando una solución escasa a un sistema lineal de ecuaciones indeterminado. Si bien la señal adquirida puede ser recuperada desde un algoritmo basado en la norma l_2 − l_1, con  muchas menos observaciones en comparación a los sistemas convencionales, dicha reconstrucción presenta diversos desafíos originados por la variable temporal o el movimiento. El movimiento durante la reconstrucción produce artefactos que dañan la totalidad de los datos. En este trabajo, se propone un método de reconstrucción basado en múltiples resoluciones para la adquisición compresiva de video espectral. De este modo, el problema de optimización para recuperar la señal se amplía añadiendo la información temporal como restricción, con el objetivo de corregir los errores originados por el movimiento de la escena. Los experimentos computacionales realizados en cuatro videos espectrales diferentes muestran una mejora de hasta 4dB en términos de relación pico-señal a ruido (PSNR) en la calidad de reconstrucción usando el enfoque multirresolución aplicado a la reconstrucción de video espectral con respecto al problema de optimización que no considera el movimiento

Multiresolution-based reconstruction for compressive spectral video sensing using a spectral multiplexing sensor

Spectral multiplexing sensors based on compressive sensing attempt to break the Nyquist barrier to acquire high spectral resolution scenes. Particularly, the colored coded aperture-based compressive spectral imager extended to video, or video C-CASSI, is a spectral multiplexing sensor that allows capturing spectral dynamic scenes by projecting each spectral frame onto a bidimensional detector using a 3D coded aperture. Afterwards, the compressed signal reconstruction is performed iteratively by finding a sparse solution to an undetermined linear system of equations. Even though the acquired signal can be recovered from much fewer observations by an  −  -norm recovery algorithm than using conventional sensors, the reconstruction exhibits diverse challenges originated by the temporal variable or motion. The motion during the reconstruction produces artifacts that damages the entire data. In this work, a multiresolution-based reconstruction method for compressive spectral video sensing is proposed. In this way, it obtains the temporal information from the measurements at a low computational cost. Thereby, the optimization problem to recover the signal is extended by adding temporal information in order to correct the errors originated by the scene motion. Computational experiments performed over four different spectral videos show an improvement up to 4dB in terms of peak-signal to noise ratio (PSNR) in the reconstruction quality using the multiresolution approach applied to the spectral video reconstruction with respect to the traditional inverse problem.  Los sensores de multiplexación espectral basados en muestreo compresivo intentan romper la barrera de Nyquist para adquirir escenas de alta resolución espectral. Particularmente, el sistema de imágenes espectrales de única captura basado en aperturas codificadas de color extendido a vídeo, o video – CCASSI, es un sensor de multiplexación espectral que permite la adquisición de imágenes espectrales dinámicas proyectando cada fotograma espectral sobre un detector bidimensional usando un apertura de codificación 3D. Posteriormente, la reconstrucción de la señal comprimida se realiza iterativamente encontrando una solución escasa a un sistema lineal de ecuaciones indeterminado. Si bien la señal adquirida puede ser recuperada desde un algoritmo basado en la norma l_2 − l_1, con  muchas menos observaciones en comparación a los sistemas convencionales, dicha reconstrucción presenta diversos desafíos originados por la variable temporal o el movimiento. El movimiento durante la reconstrucción produce artefactos que dañan la totalidad de los datos. En este trabajo, se propone un método de reconstrucción basado en múltiples resoluciones para la adquisición compresiva de video espectral. De este modo, el problema de optimización para recuperar la señal se amplía añadiendo la información temporal como restricción, con el objetivo de corregir los errores originados por el movimiento de la escena. Los experimentos computacionales realizados en cuatro videos espectrales diferentes muestran una mejora de hasta 4dB en términos de relación pico-señal a ruido (PSNR) en la calidad de reconstrucción usando el enfoque multirresolución aplicado a la reconstrucción de video espectral con respecto al problema de optimización que no considera el movimiento

Foreground segmentation in depth imagery using depth and spatial dynamic models for video surveillance applications

Low-cost systems that can obtain a high-quality foreground segmentation almostindependently of the existing illumination conditions for indoor environments are verydesirable, especially for security and surveillance applications. In this paper, a novelforeground segmentation algorithm that uses only a Kinect depth sensor is proposedto satisfy the aforementioned system characteristics. This is achieved by combininga mixture of Gaussians-based background subtraction algorithm with a new Bayesiannetwork that robustly predicts the foreground/background regions between consecutivetime steps. The Bayesian network explicitly exploits the intrinsic characteristics ofthe depth data by means of two dynamic models that estimate the spatial and depthevolution of the foreground/background regions. The most remarkable contribution is thedepth-based dynamic model that predicts the changes in the foreground depth distributionbetween consecutive time steps. This is a key difference with regard to visible imagery,where the color/gray distribution of the foreground is typically assumed to be constant.Experiments carried out on two different depth-based databases demonstrate that theproposed combination of algorithms is able to obtain a more accurate segmentation of theforeground/background than other state-of-the art approaches

Enhancement of ELDA Tracker Based on CNN Features and Adaptive Model Update

Appearance representation and the observation model are the most important components in designing a robust visual tracking algorithm for video-based sensors. Additionally, the exemplar-based linear discriminant analysis (ELDA) model has shown good performance in object tracking. Based on that, we improve the ELDA tracking algorithm by deep convolutional neural network (CNN) features and adaptive model update. Deep CNN features have been successfully used in various computer vision tasks. Extracting CNN features on all of the candidate windows is time consuming. To address this problem, a two-step CNN feature extraction method is proposed by separately computing convolutional layers and fully-connected layers. Due to the strong discriminative ability of CNN features and the exemplar-based model, we update both object and background models to improve their adaptivity and to deal with the tradeoff between discriminative ability and adaptivity. An object updating method is proposed to select the “good” models (detectors), which are quite discriminative and uncorrelated to other selected models. Meanwhile, we build the background model as a Gaussian mixture model (GMM) to adapt to complex scenes, which is initialized offline and updated online. The proposed tracker is evaluated on a benchmark dataset of 50 video sequences with various challenges. It achieves the best overall performance among the compared state-of-the-art trackers, which demonstrates the effectiveness and robustness of our tracking algorithm

A Coded Aperture Compressive Imaging Array and Its Visual Detection and Tracking Algorithms for Surveillance Systems

In this paper, we propose an application of a compressive imaging system to the problem of wide-area video surveillance systems. A parallel coded aperture compressive imaging system is proposed to reduce the needed high resolution coded mask requirements and facilitate the storage of the projection matrix. Random Gaussian, Toeplitz and binary phase coded masks are utilized to obtain the compressive sensing images. The corresponding motion targets detection and tracking algorithms directly using the compressive sampling images are developed. A mixture of Gaussian distribution is applied in the compressive image space to model the background image and for foreground detection. For each motion target in the compressive sampling domain, a compressive feature dictionary spanned by target templates and noises templates is sparsely represented. An l1 optimization algorithm is used to solve the sparse coefficient of templates. Experimental results demonstrate that low dimensional compressed imaging representation is sufficient to determine spatial motion targets. Compared with the random Gaussian and Toeplitz phase mask, motion detection algorithms using a random binary phase mask can yield better detection results. However using random Gaussian and Toeplitz phase mask can achieve high resolution reconstructed image. Our tracking algorithm can achieve a real time speed that is up to 10 times faster than that of the l1 tracker without any optimization