Video Compressive Sensing for Dynamic MRI

We present a video compressive sensing framework, termed kt-CSLDS, to accelerate the image acquisition process of dynamic magnetic resonance imaging (MRI). We are inspired by a state-of-the-art model for video compressive sensing that utilizes a linear dynamical system (LDS) to model the motion manifold. Given compressive measurements, the state sequence of an LDS can be first estimated using system identification techniques. We then reconstruct the observation matrix using a joint structured sparsity assumption. In particular, we minimize an objective function with a mixture of wavelet sparsity and joint sparsity within the observation matrix. We derive an efficient convex optimization algorithm through alternating direction method of multipliers (ADMM), and provide a theoretical guarantee for global convergence. We demonstrate the performance of our approach for video compressive sensing, in terms of reconstruction accuracy. We also investigate the impact of various sampling strategies. We apply this framework to accelerate the acquisition process of dynamic MRI and show it achieves the best reconstruction accuracy with the least computational time compared with existing algorithms in the literature.Comment: 30 pages, 9 figure

Extrinsic Methods for Coding and Dictionary Learning on Grassmann Manifolds

Sparsity-based representations have recently led to notable results in various visual recognition tasks. In a separate line of research, Riemannian manifolds have been shown useful for dealing with features and models that do not lie in Euclidean spaces. With the aim of building a bridge between the two realms, we address the problem of sparse coding and dictionary learning over the space of linear subspaces, which form Riemannian structures known as Grassmann manifolds. To this end, we propose to embed Grassmann manifolds into the space of symmetric matrices by an isometric mapping. This in turn enables us to extend two sparse coding schemes to Grassmann manifolds. Furthermore, we propose closed-form solutions for learning a Grassmann dictionary, atom by atom. Lastly, to handle non-linearity in data, we extend the proposed Grassmann sparse coding and dictionary learning algorithms through embedding into Hilbert spaces. Experiments on several classification tasks (gender recognition, gesture classification, scene analysis, face recognition, action recognition and dynamic texture classification) show that the proposed approaches achieve considerable improvements in discrimination accuracy, in comparison to state-of-the-art methods such as kernelized Affine Hull Method and graph-embedding Grassmann discriminant analysis.Comment: Appearing in International Journal of Computer Visio

이동 물체 감지 및 분진 영상 복원의 연구

학위논문 (박사) -- 서울대학교 대학원 : 자연과학대학 수리과학부, 2021. 2. 강명주.Robust principal component analysis(RPCA), a method used to decom- pose a matrix into the sum of a low-rank matrix and a sparse matrix, has been proven effective in modeling the static background of videos. However, because a dynamic background cannot be represented by a low-rank matrix, measures additional to the RPCA are required. In this thesis, we propose masked RPCA to process backgrounds containing moving textures. First- order Marcov random field (MRF) is used to generate a mask that roughly labels moving objects and backgrounds. To estimate the background, the rank minimization process is then applied with the mask multiplied. During the iteration, the background rank increases as the object mask expands, and the weight of the rank constraint term decreases, which increases the accuracy of the background. We compared the proposed method with state- of-art, end-to-end methods to demonstrate its advantages. Subsequently, we suggest novel dedusting method based on dust-optimized transmission map and deep image prior. This method consists of estimating atmospheric light and transmission in that order, which is similar to dark channel prior-based dehazing methods. However, existing atmospheric light estimating methods widely used in dehazing schemes give an overly bright estimation, which results in unrealistically dark dedusting results. To ad- dress this problem, we propose a segmentation-based method that gives new estimation in atmospheric light. Dark channel prior based transmission map with new atmospheric light gives unnatural intensity ordering and zero value at low transmission regions. Therefore, the transmission map is refined by scattering model based transformation and dark channel adaptive non-local total variation (NLTV) regularization. Parameter optimizing steps with deep image prior(DIP) gives the final dedusting result.강건 주성분 분석은 배경 감산을 통한 동영상의 전경 추출의 방법으로 이 용되어왔으나, 동적배경은저계수행렬로표현될수없기때문에동적배경 감산에성능적한계를가지고있었다. 우리는전경과배경을구분하는일계마 르코프연쇄를도입해정적배경을나타내는항과곱하고이것을이용한새로 운형태의강건주성분분석을제안하여동적배경감산문제를해결한다. 해당 최소화문제는반복적인교차최적화를통하여해결한다. 이어서대기중의미세 먼지에의해오염된영상을복원한다. 영상분할과암흑채널가정에기반하여 깊이지도를구하고, 비국소총변동최소화를통하여정제한다. 이후깊은영상 가정에기반한영상생성기를통하여최종적으로복원된영상을구한다. 실험을 통하여제안된방법을다른방법들과비교하고질적인측면과양적인측면모 두에서우수함을확인한다.Abstract i 1 Introduction 1 1.1 Moving Object Detection In Dynamic Backgrounds 1 1.2 Image Dedusting 2 2 Preliminaries 4 2.1 Moving Object Detection In Dynamic Backgrounds 4 2.1.1 Literature review 5 2.1.2 Robust principal component analysis(RPCA) and their application status 7 2.1.3 Graph cuts and α-expansion algorithm 14 2.2 Image Dedusting 16 2.2.1 Image dehazing methods 16 2.2.2 Dust model 18 2.2.3 Non-local total variation(NLTV) 19 3 Dynamic Background Subtraction With Masked RPCA 21 3.1 Motivation 21 3.1.1 Motivation of background modeling 21 3.1.2 Mask formulation 23 3.1.3 Model 24 3.2 Optimization 25 3.2.1 L-Subproblem 25 3.2.2 L˜-Subproblem 26 3.2.3 M-Subproblem 27 3.2.4 p-Subproblem 28 3.2.5 Adaptive parameter control 28 3.2.6 Convergence 29 3.3 Experimental results 31 3.3.1 Benchmark Algorithms And Videos 31 3.3.2 Implementation 32 3.3.3 Evaluation 32 4 Deep Image Dedusting With Dust-Optimized Transmission Map 41 4.1 Transmission estimation 41 4.1.1 Atmospheric light estimation 41 4.1.2 Transmission estimation 43 4.2 Scene radiance recovery 47 4.3 Experimental results 51 4.3.1 Implementation 51 4.3.2 Evaluation 52 5 Conclusion 58 Abstract (in Korean) 69 Acknowledgement (in Korean) 70Docto

Anomaly detection in moving-camera videos with sparse and low-rank matrix decompositions

This work presents two methods based on sparse decompositions that can detect anomalies in video sequences obtained from moving cameras. The first method starts by computing the union of subspaces (UoS) that best represents all the frames from a reference (anomaly-free) video as a low-rank projection plus a sparse residue. Then it performs a low-rank representation of the target (possibly anomalous) video by taking advantage of both the UoS and the sparse residue computed from the reference video. The anomalies are extracted after post-processing this video with these residual data. Such algorithm provides good detection results while at the same time obviating the need for previous video synchronization. However, this technique looses its detection efficiency when target and reference videos presents more severe misalignments. This may happen due to small uncontrolled camera moviment and shaking during the acquisition phase, which is often common in realworld situations. To extend its applicability, a second contribution is proposed in order to cope with these possible pose misalignments. This is done by modeling the target-reference pose discrepancy as geometric transformations acting on the domain of frames of the target video. A complete matrix decomposition algorithm is presented in order to perform a sparse representation of the target video as a sparse combination of the reference video plus a sparse residue, while taking into account the transformation acting on it. Our method is then verified and compared against state-of-the-art techniques using a challenging video dataset, that comprises recordings presenting the described misalignments. Under the evaluation metrics used, the second proposed method exhibits an improvement of at least 16% over the first proposed one, and 22% over the next best rated method.Apresentamos dois métodos baseados em decomposições esparsas que podem detectar anomalias em sequências de vídeo obtidas por câmeras em movimento. O primeiro método estima a união de subespaços (UdS) que melhor representa todos os quadros de um vídeo de referência (livre de anomalias) como uma projeção de baixo-posto mais um resíduo esparso. Em seguida, é realizada uma representação de baixo-posto do vídeo alvo (possivelmente anômalo) aproveitando a UdS e o resíduo esparso calculado a partir do vídeo de referência. As anomalias são extraídas após o pós-processamento destas informações residuais. Esse algoritmo fornece bons resultados de detecção, além de eliminar a necessidade de uma sincronização prévia dos vídeos. No entanto, essa técnica perde eficiência quando os vídeos de referência e alvo apresentam desalinhamentos mais graves entre si. Isso pode ocorrer devido a pequenos movimentos descontrolados da câmera e tremores durante a fase de aquisição. Para estender sua aplicabilidade, uma segunda contribuição é proposta a fim de lidar com esse possível desalinhamento. Isso é feito modelando a discrepância de pose de câmera entre os vídeos de referência e alvo com transformações geométricas agindo no domínio dos quadros do vídeo alvo. Um algoritmo completo de decomposição de matrizes é apresentado para realizar uma representação esparsa do vídeo alvo como uma combinação esparsa do vídeo de referência, levando em consideração as transformações que atuam sobre seus quadros. Nosso método é então verificado e comparado com técnicas de última geração com auxílio de vídeos de uma base desafiadora, apresentando os desalinhamentos em questão. Sob as métricas de avaliação usadas, o segundo método proposto exibe uma melhoria de pelo menos 16% em relação ao primeiro, e 22% sobre o método melhor avaliado logo em seguida

Learning Deep Representations of Appearance and Motion for Anomalous Event Detection

We present a novel unsupervised deep learning framework for anomalous event detection in complex video scenes. While most existing works merely use hand-crafted appearance and motion features, we propose Appearance and Motion DeepNet (AMDN) which utilizes deep neural networks to automatically learn feature representations. To exploit the complementary information of both appearance and motion patterns, we introduce a novel double fusion framework, combining both the benefits of traditional early fusion and late fusion strategies. Specifically, stacked denoising autoencoders are proposed to separately learn both appearance and motion features as well as a joint representation (early fusion). Based on the learned representations, multiple one-class SVM models are used to predict the anomaly scores of each input, which are then integrated with a late fusion strategy for final anomaly detection. We evaluate the proposed method on two publicly available video surveillance datasets, showing competitive performance with respect to state of the art approaches.Comment: Oral paper in BMVC 201