Video modeling via implicit motion representations

Video modeling refers to the development of analytical representations for explaining the intensity distribution in video signals. Based on the analytical representation, we can develop algorithms for accomplishing particular video-related tasks. Therefore video modeling provides us a foundation to bridge video data and related-tasks. Although there are many video models proposed in the past decades, the rise of new applications calls for more efficient and accurate video modeling approaches.;Most existing video modeling approaches are based on explicit motion representations, where motion information is explicitly expressed by correspondence-based representations (i.e., motion velocity or displacement). Although it is conceptually simple, the limitations of those representations and the suboptimum of motion estimation techniques can degrade such video modeling approaches, especially for handling complex motion or non-ideal observation video data. In this thesis, we propose to investigate video modeling without explicit motion representation. Motion information is implicitly embedded into the spatio-temporal dependency among pixels or patches instead of being explicitly described by motion vectors.;Firstly, we propose a parametric model based on a spatio-temporal adaptive localized learning (STALL). We formulate video modeling as a linear regression problem, in which motion information is embedded within the regression coefficients. The coefficients are adaptively learned within a local space-time window based on LMMSE criterion. Incorporating a spatio-temporal resampling and a Bayesian fusion scheme, we can enhance the modeling capability of STALL on more general videos. Under the framework of STALL, we can develop video processing algorithms for a variety of applications by adjusting model parameters (i.e., the size and topology of model support and training window). We apply STALL on three video processing problems. The simulation results show that motion information can be efficiently exploited by our implicit motion representation and the resampling and fusion do help to enhance the modeling capability of STALL.;Secondly, we propose a nonparametric video modeling approach, which is not dependent on explicit motion estimation. Assuming the video sequence is composed of many overlapping space-time patches, we propose to embed motion-related information into the relationships among video patches and develop a generic sparsity-based prior for typical video sequences. First, we extend block matching to more general kNN-based patch clustering, which provides an implicit and distributed representation for motion information. We propose to enforce the sparsity constraint on a higher-dimensional data array signal, which is generated by packing the patches in the similar patch set. Then we solve the inference problem by updating the kNN array and the wanted signal iteratively. Finally, we present a Bayesian fusion approach to fuse multiple-hypothesis inferences. Simulation results in video error concealment, denoising, and deartifacting are reported to demonstrate its modeling capability.;Finally, we summarize the proposed two video modeling approaches. We also point out the perspectives of implicit motion representations in applications ranging from low to high level problems

Robust density modelling using the student's t-distribution for human action recognition

The extraction of human features from videos is often inaccurate and prone to outliers. Such outliers can severely affect density modelling when the Gaussian distribution is used as the model since it is highly sensitive to outliers. The Gaussian distribution is also often used as base component of graphical models for recognising human actions in the videos (hidden Markov model and others) and the presence of outliers can significantly affect the recognition accuracy. In contrast, the Student's t-distribution is more robust to outliers and can be exploited to improve the recognition rate in the presence of abnormal data. In this paper, we present an HMM which uses mixtures of t-distributions as observation probabilities and show how experiments over two well-known datasets (Weizmann, MuHAVi) reported a remarkable improvement in classification accuracy. © 2011 IEEE

Habitat Selection of Female Desert Bighorn Sheep: Tradeoffs Associated with Reproduction

Animals select habitat types that enhance their ability to survive, reproduce, and therefore enhance reproductive fitness. Selection for specific habitat types often varies within populations based on season, habitat availability, sex, age, reproductive status, and other characteristics. Therefore, we expect habitat selection to be changing throughout the lifetime of an individual to meet the metabolic and nutritional constraints of specific periods. For instance, reproductive status, especially provisioning dependent young, is commonly linked to changes in behavior of female ungulates that results in variation in resource selection around parturition. Around parturition, female ungulates are commonly linked to tradeoffs between maternal nutritional condition and survival of offspring. These tradeoffs are hypothesized to take place because they allow females to increase their reproductive fitness by enhancing the likelihood that their offspring survives to recruitment. Understanding these shifts in habitat selection are essential to proper management and the conservation of ungulates.I was interested in documenting the variation in habitat selection of female bighorn sheep (Ovis canadensis nelsoni) around parturition, in addition to characterizing composition and quality of diets of females based on time of year and the provisioning status of individuals. This period is critical to female ungulates because their choice of habitat components also influences survival of their offspring and ultimately, the female’s reproductive fitness. I used 2 populations of desert bighorn sheep in west-central Nevada, to study how ungulates adjust habitat selection around parturition. I investigated habitat selection of female desert bighorn sheep from the beginning of their third trimester until weaning of offspring. Furthermore, I investigated how females selected parturition sites and neonates selected bed sites immediately following birth. To accomplish this goal, I captured adult and neonatal bighorn sheep and equipped individuals with very high frequency (VHF) and global positioning system (GPS) radio-collars from 2016 to 2018. Additionally, I collected fecal samples from female bighorn sheep throughout the year and within the parturition season, based on pregnancy and provisioning status of individuals. I also used unmanned aerial vehicles and publicly available remote sensing data to characterize birth sites of parturient females and bed sites of female-offspring pairs. I found that females adjusted habitat selection, based on provisioning status, by trading off maternal nutritional condition for survival of offspring. Prior to parturition, females selected areas with higher forage availability, however, following parturition, females adjusted selection to areas with habitat features that are commonly associated with increased survival of young (i.e., steep slopes, rugged terrain, and more open habitats). Furthermore, as neonates aged, females adjusted habitat selection back to pre-parturition selection levels, where females would be able to fulfill nutritional needs and offspring were less vulnerable to predation. These apparent tradeoffs were also supported by my analyses of diet composition and quality. Females that were not provisioning offspring tended to have higher percentages of nitrogen in fecal pellets indicating that they were consuming higher quality diets than those females that were provisioning offspring. Diet quality of females also varied throughout the year, where spring and summer months had higher fecal nitrogen content than winter months. Additionally, females adjusted diet composition based on provisioning status and season. I found that female bighorn sheep differed in selection strategies for birth and bed sites at each study area. Overall, females tended to select parturition sites at broad scales with steep slopes, high visibility, and close to ridgelines. Microhabitat selection for birth sites was similar to broad scale selection, other than females selected for low downslope visibility and for less steep slopes. Finally, parent-offspring pairs selected bed sites with higher amounts of concealment cover than birth sites

Recent Advances in Image Restoration with Applications to Real World Problems

In the past few decades, imaging hardware has improved tremendously in terms of resolution, making widespread usage of images in many diverse applications on Earth and planetary missions. However, practical issues associated with image acquisition are still affecting image quality. Some of these issues such as blurring, measurement noise, mosaicing artifacts, low spatial or spectral resolution, etc. can seriously affect the accuracy of the aforementioned applications. This book intends to provide the reader with a glimpse of the latest developments and recent advances in image restoration, which includes image super-resolution, image fusion to enhance spatial, spectral resolution, and temporal resolutions, and the generation of synthetic images using deep learning techniques. Some practical applications are also included

Darknet Traffic Analysis A Systematic Literature Review

The primary objective of an anonymity tool is to protect the anonymity of its users through the implementation of strong encryption and obfuscation techniques. As a result, it becomes very difficult to monitor and identify users activities on these networks. Moreover, such systems have strong defensive mechanisms to protect users against potential risks, including the extraction of traffic characteristics and website fingerprinting. However, the strong anonymity feature also functions as a refuge for those involved in illicit activities who aim to avoid being traced on the network. As a result, a substantial body of research has been undertaken to examine and classify encrypted traffic using machine learning techniques. This paper presents a comprehensive examination of the existing approaches utilized for the categorization of anonymous traffic as well as encrypted network traffic inside the darknet. Also, this paper presents a comprehensive analysis of methods of darknet traffic using machine learning techniques to monitor and identify the traffic attacks inside the darknet.Comment: 35 Pages, 13 Figure

Mathematical Approaches for Image Enhancement Problems

This thesis develops novel techniques that can solve some image enhancement problems using theoretically and technically proven and very useful mathematical tools to image processing such as wavelet transforms, partial differential equations, and variational models. Three subtopics are mainly covered. First, color image denoising framework is introduced to achieve high quality denoising results by considering correlations between color components while existing denoising approaches can be plugged in flexibly. Second, a new and efficient framework for image contrast and color enhancement in the compressed wavelet domain is proposed. The proposed approach is capable of enhancing both global and local contrast and brightness as well as preserving color consistency. The framework does not require inverse transform for image enhancement since linear scale factors are directly applied to both scaling and wavelet coefficients in the compressed domain, which results in high computational efficiency. Also contaminated noise in the image can be efficiently reduced by introducing wavelet shrinkage terms adaptively in different scales. The proposed method is able to enhance a wavelet-coded image computationally efficiently with high image quality and less noise or other artifact. The experimental results show that the proposed method produces encouraging results both visually and numerically compared to some existing approaches. Finally, image inpainting problem is discussed. Literature review, psychological analysis, and challenges on image inpainting problem and related topics are described. An inpainting algorithm using energy minimization and texture mapping is proposed. Mumford-Shah energy minimization model detects and preserves edges in the inpainting domain by detecting both the main structure and the detailed edges. This approach utilizes faster hierarchical level set method and guarantees convergence independent of initial conditions. The estimated segmentation results in the inpainting domain are stored in segmentation map, which is referred by a texture mapping algorithm for filling textured regions. We also propose an inpainting algorithm using wavelet transform that can expect better global structure estimation of the unknown region in addition to shape and texture properties since wavelet transforms have been used for various image analysis problems due to its nice multi-resolution properties and decoupling characteristics