Nonlinear dance motion analysis and motion editing using Hilbert-Huang transform

Human motions (especially dance motions) are very noisy, and it is hard to analyze and edit the motions. To resolve this problem, we propose a new method to decompose and modify the motions using the Hilbert-Huang transform (HHT). First, HHT decomposes a chromatic signal into "monochromatic" signals that are the so-called Intrinsic Mode Functions (IMFs) using an Empirical Mode Decomposition (EMD) [6]. After applying the Hilbert Transform to each IMF, the instantaneous frequencies of the "monochromatic" signals can be obtained. The HHT has the advantage to analyze non-stationary and nonlinear signals such as human-joint-motions over FFT or Wavelet transform. In the present paper, we propose a new framework to analyze and extract some new features from a famous Japanese threesome pop singer group called "Perfume", and compare it with Waltz and Salsa dance. Using the EMD, their dance motions can be decomposed into motion (choreographic) primitives or IMFs. Therefore we can scale, combine, subtract, exchange, and modify those IMFs, and can blend them into new dance motions self-consistently. Our analysis and framework can lead to a motion editing and blending method to create a new dance motion from different dance motions.Comment: 6 pages, 10 figures, Computer Graphics International 2017, Conference short pape

Neural tracking of visual periodic motion

Periodicity is a fundamental property of biological systems, including human movement systems. Periodic movements support displacements of the body in the environment as well as interactions and communication between individuals. Here, we use electroencephalography (EEG) to investigate the neural tracking of visual periodic motion, and more specifically, the relevance of spatiotemporal information contained at and between their turning points. We compared EEG responses to visual sinusoidal oscillations versus nonlinear Rayleigh oscillations, which are both typical of human movements. These oscillations contain the same spatiotemporal information at their turning points but differ between turning points, with Rayleigh oscillations having an earlier peak velocity, shown to increase an individual's capacity to produce accurately synchronized movements. EEG analyses highlighted the relevance of spatiotemporal information between the turning points by showing that the brain precisely tracks subtle differences in velocity profiles, as indicated by earlier EEG responses for Rayleigh oscillations. The results suggest that the brain is particularly responsive to velocity peaks in visual periodic motion, supporting their role in conveying behaviorally relevant timing information at a neurophysiological level. The results also suggest key functions of neural oscillations in the Alpha and Beta frequency bands, particularly in the right hemisphere. Together, these findings provide insights into the neural mechanisms underpinning the processing of visual periodic motion and the critical role of velocity peaks in enabling proficient visuomotor synchronization

Short-Term Power Prediction of a Wind Farm Based on Empirical Mode Decomposition and Mayfly Algorithm–Back Propagation Neural Network

With the improvement of energy consumption structure, the installed capacity of wind power increases gradually. However, the inherent intermittency and instability of wind energy bring severe challenges to the dispatching operation. Wind power forecasting is one of the main solutions. In this work, a new combined wind power prediction model is proposed. First, a quartile method is used for data cleaning, namely, identifying and eliminating the abnormal data. Then, the wind power data sequence is decomposed by empirical mode decomposition to eliminate non-stationary characteristics. Finally, the wind generator data are trained by the MA-BP network to establish the wind power prediction model. Also, the simulation tests verify the prediction effect of the proposed method. Specifically speaking, the average MAPE is decreased to 12.4979% by the proposed method. Also, the average RMSE and MAE are 107.1728 and 71.604 kW, respectively

Study of Climate Variability Patterns at Different Scales – A Complex Network Approach

Das Klimasystem der Erde besteht aus zahlreichen interagierenden Teilsystemen, die sich über verschiedene Zeitskalen hinweg verändern, was zu einer äußerst komplizierten räumlich-zeitlichen Klimavariabilität führt. Das Verständnis von Prozessen, die auf verschiedenen räumlichen und zeitlichen Skalen ablaufen, ist ein entscheidender Aspekt bei der numerischen Wettervorhersage. Die Variabilität des Klimas, ein sich selbst konstituierendes System, scheint in Mustern auf großen Skalen organisiert zu sein. Die Verwendung von Klimanetzwerken hat sich als erfolgreicher Ansatz für die Erkennung der räumlichen Ausbreitung dieser großräumigen Muster in der Variabilität des Klimasystems erwiesen. In dieser Arbeit wird mit Hilfe von Klimanetzwerken gezeigt, dass die Klimavariabilität nicht nur auf größeren Skalen (Asiatischer Sommermonsun, El Niño/Southern Oscillation), sondern auch auf kleineren Skalen, z.B. auf Wetterzeitskalen, in Mustern organisiert ist. Dies findet Anwendung bei der Erkennung einzelner tropischer Wirbelstürme, bei der Charakterisierung binärer Wirbelsturm-Interaktionen, die zu einer vollständigen Verschmelzung führen, und bei der Untersuchung der intrasaisonalen und interannuellen Variabilität des Asiatischen Sommermonsuns. Schließlich wird die Anwendbarkeit von Klimanetzwerken zur Analyse von Vorhersagefehlern demonstriert, was für die Verbesserung von Vorhersagen von immenser Bedeutung ist. Da korrelierte Fehler durch vorhersagbare Beziehungen zwischen Fehlern verschiedener Regionen aufgrund von zugrunde liegenden systematischen oder zufälligen Prozessen auftreten können, wird gezeigt, dass Fehler-Netzwerke helfen können, die räumlich kohärenten Strukturen von Vorhersagefehlern zu untersuchen. Die Analyse der Fehler-Netzwerk-Topologie von Klimavariablen liefert ein erstes Verständnis der vorherrschenden Fehlerquelle und veranschaulicht das Potenzial von Klimanetzwerken als vielversprechendes Diagnoseinstrument zur Untersuchung von Fehlerkorrelationen.The Earth’s climate system consists of numerous interacting subsystems varying over a multitude of time scales giving rise to highly complicated spatio-temporal climate variability. Understanding processes occurring at different scales, both spatial and temporal, has been a very crucial problem in numerical weather prediction. The variability of climate, a self-constituting system, appears to be organized in patterns on large scales. The climate networks approach has been very successful in detecting the spatial propagation of these large scale patterns of variability in the climate system. In this thesis, it is demonstrated using climate network approach that climate variability is organized in patterns not only at larger scales (Asian Summer Monsoon, El Niño-Southern Oscillation) but also at shorter scales, e.g., weather time scales. This finds application in detecting individual tropical cyclones, characterizing binary cyclone interaction leading to a complete merger, and studying the intraseasonal and interannual variability of the Asian Summer Monsoon. Finally, the applicability of the climate network framework to understand forecast error properties is demonstrated, which is crucial for improvement of forecasts. As correlated errors can arise due to the presence of a predictable relationship between errors of different regions because of some underlying systematic or random process, it is shown that error networks can help to analyze the spatially coherent structures of forecast errors. The analysis of the error network topology of a climate variable provides a preliminary understanding of the dominant source of error, which shows the potential of climate networks as a very promising diagnostic tool to study error correlations

Motion capture data processing, retrieval and recognition.

Character animation plays an essential role in the area of featured film and computer games. Manually creating character animation by animators is both tedious and inefficient, where motion capture techniques (MoCap) have been developed and become the most popular method for creating realistic character animation products. Commercial MoCap systems are expensive and the capturing process itself usually requires an indoor studio environment. Procedural animation creation is often lacking extensive user control during the generation progress. Therefore, efficiently and effectively reusing MoCap data can brings significant benefits, which has motivated wider research in terms of machine learning based MoCap data processing. A typical work flow of MoCap data reusing can be divided into 3 stages: data capture, data management and data reusing. There are still many challenges at each stage. For instance, the data capture and management often suffer from data quality problems. The efficient and effective retrieval method is also demanding due to the large amount of data being used. In addition, classification and understanding of actions are the fundamental basis of data reusing. This thesis proposes to use machine learning on MoCap data for reusing purposes, where a frame work of motion capture data processing is designed. The modular design of this framework enables motion data refinement, retrieval and recognition. The first part of this thesis introduces various methods used in existing motion capture processing approaches in literature and a brief introduction of relevant machine learning methods used in this framework. In general, the frameworks related to refinement, retrieval, recognition are discussed. A motion refinement algorithm based on dictionary learning will then be presented, where kinematical structural and temporal information are exploited. The designed optimization method and data preprocessing technique can ensure a smooth property for the recovered result. After that, a motion refinement algorithm based on matrix completion is presented, where the low-rank property and spatio-temporal information is exploited. Such model does not require preparing data for training. The designed optimization method outperforms existing approaches in regard to both effectiveness and efficiency. A motion retrieval method based on multi-view feature selection is also proposed, where the intrinsic relations between visual words in each motion feature subspace are discovered as a means of improving the retrieval performance. A provisional trace-ratio objective function and an iterative optimization method are also included. A non-negative matrix factorization based motion data clustering method is proposed for recognition purposes, which aims to deal with large scale unsupervised/semi-supervised problems. In addition, deep learning models are used for motion data recognition, e.g. 2D gait recognition and 3D MoCap recognition. To sum up, the research on motion data refinement, retrieval and recognition are presented in this thesis with an aim to tackle the major challenges in motion reusing. The proposed motion refinement methods aim to provide high quality clean motion data for downstream applications. The designed multi-view feature selection algorithm aims to improve the motion retrieval performance. The proposed motion recognition methods are equally essential for motion understanding. A collection of publications by the author of this thesis are noted in publications section

Image Analysis Applications of the Maximum Mean Discrepancy Distance Measure

The need to quantify distance between two groups of objects is prevalent throughout the signal processing world. The difference of group means computed using the Euclidean, or L2 distance, is one of the predominant distance measures used to compare feature vectors and groups of vectors, but many problems arise with it when high data dimensionality is present. Maximum mean discrepancy (MMD) is a recent unsupervised kernel-based pattern recognition method which may improve differentiation between two distinct populations over many commonly used methods such as the difference of means, when paired with the proper feature representations and kernels. MMD-based distance computation combines many powerful concepts from the machine learning literature, such as data distribution-leveraging similarity measures and kernel methods for machine learning. Due to this heritage, we posit that dissimilarity-based classification and changepoint detection using MMD can lead to enhanced separation between different populations. To test this hypothesis, we conduct studies comparing MMD and the difference of means in two subareas of image analysis and understanding: first, to detect scene changes in video in an unsupervised manner, and secondly, in the biomedical imaging field, using clinical ultrasound to assess tumor response to treatment. We leverage effective computer vision data descriptors, such as the bag-of-visual-words and sparse combinations of SIFT descriptors, and choose from an assessment of several similarity kernels (e.g. Histogram Intersection, Radial Basis Function) in order to engineer useful systems using MMD. Promising improvements over the difference of means, measured primarily using precision/recall for scene change detection, and k-nearest neighbour classification accuracy for tumor response assessment, are obtained in both applications.1 yea

Activity Representation from Video Using Statistical Models on Shape Manifolds

Activity recognition from video data is a key computer vision problem with applications in surveillance, elderly care, etc. This problem is associated with modeling a representative shape which contains significant information about the underlying activity. In this dissertation, we represent several approaches for view-invariant activity recognition via modeling shapes on various shape spaces and Riemannian manifolds. The first two parts of this dissertation deal with activity modeling and recognition using tracks of landmark feature points. The motion trajectories of points extracted from objects involved in the activity are used to build deformation shape models for each activity, and these models are used for classification and detection of unusual activities. In the first part of the dissertation, these models are represented by the recovered 3D deformation basis shapes corresponding to the activity using a non-rigid structure from motion formulation. We use a theory for estimating the amount of deformation for these models from the visual data. We study the special case of ground plane activities in detail because of its importance in video surveillance applications. In the second part of the dissertation, we propose to model the activity by learning an affine invariant deformation subspace representation that captures the space of possible body poses associated with the activity. These subspaces can be viewed as points on a Grassmann manifold. We propose several statistical classification models on Grassmann manifold that capture the statistical variations of the shape data while following the intrinsic Riemannian geometry of these manifolds. The last part of this dissertation addresses the problem of recognizing human gestures from silhouette images. We represent a human gesture as a temporal sequence of human poses, each characterized by a contour of the associated human silhouette. The shape of a contour is viewed as a point on the shape space of closed curves and, hence, each gesture is characterized and modeled as a trajectory on this shape space. We utilize the Riemannian geometry of this space to propose a template-based and a graphical-based approaches for modeling these trajectories. The two models are designed in such a way to account for the different invariance requirements in gesture recognition, and also capture the statistical variations associated with the contour data

Multimedia Forensics

This book is open access. Media forensics has never been more relevant to societal life. Not only media content represents an ever-increasing share of the data traveling on the net and the preferred communications means for most users, it has also become integral part of most innovative applications in the digital information ecosystem that serves various sectors of society, from the entertainment, to journalism, to politics. Undoubtedly, the advances in deep learning and computational imaging contributed significantly to this outcome. The underlying technologies that drive this trend, however, also pose a profound challenge in establishing trust in what we see, hear, and read, and make media content the preferred target of malicious attacks. In this new threat landscape powered by innovative imaging technologies and sophisticated tools, based on autoencoders and generative adversarial networks, this book fills an important gap. It presents a comprehensive review of state-of-the-art forensics capabilities that relate to media attribution, integrity and authenticity verification, and counter forensics. Its content is developed to provide practitioners, researchers, photo and video enthusiasts, and students a holistic view of the field