Multimodal Semantics Extraction from User-Generated Videos

User-generated video content has grown tremendously fast to the point of outpacing professional content creation. In this work we develop methods that analyze contextual information of multiple user-generated videos in order to obtain semantic information about public happenings (e.g., sport and live music events) being recorded in these videos. One of the key contributions of this work is a joint utilization of different data modalities, including such captured by auxiliary sensors during the video recording performed by each user. In particular, we analyze GPS data, magnetometer data, accelerometer data, video- and audio-content data. We use these data modalities to infer information about the event being recorded, in terms of layout (e.g., stadium), genre, indoor versus outdoor scene, and the main area of interest of the event. Furthermore we propose a method that automatically identifies the optimal set of cameras to be used in a multicamera video production. Finally, we detect the camera users which fall within the field of view of other cameras recording at the same public happening. We show that the proposed multimodal analysis methods perform well on various recordings obtained in real sport events and live music performances

Hierarchical deconvolution for incoherent scatter radar data

Abstract We propose a novel method for deconvolving incoherent scatter radar data to recover accurate reconstructions of backscattered powers. The problem is modelled as a hierarchical noise-perturbed deconvolution problem, where the lower hierarchy consists of an adaptive length-scale function that allows for a non-stationary prior and as such enables adaptive recovery of smooth and narrow layers in the profiles. The estimation is done in a Bayesian statistical inversion framework as a two-step procedure, where hyperparameters are first estimated by optimisation and followed by an analytical closed-form solution of the deconvolved signal. The proposed optimisation-based method is compared to a fully probabilistic approach using Markov chain Monte Carlo techniques enabling additional uncertainty quantification. In this paper we examine the potential of the hierarchical deconvolution approach using two different prior models for the length-scale function. We apply the developed methodology to compute the backscattered powers of measured polar mesospheric winter echoes, as well as summer echoes, from the EISCAT VHF radar in Tromsø, Norway. Computational accuracy and performance are tested using a simulated signal corresponding to a typical background ionosphere and a sporadic E layer with known ground truth. The results suggest that the proposed hierarchical deconvolution approach can recover accurate and clean reconstructions of profiles, and the potential to be successfully applied to similar problems

Gaussian Markov random field priors in ionospheric 3-D multi-instrument tomography

Abstract In ionospheric tomography, the atmospheric electron density is reconstructed from different electron density related measurements, most often from ground-based measurements of satellite signals. Typically, ionospheric tomography suffers from two major complications. First, the information provided by measurements is insufficient and additional information is required to obtain a unique solution. Second, with necessary spatial and temporal resolutions, the problem becomes very high dimensional, and hence, computationally infeasible. With Bayesian framework, the required additional information can be given with prior probability distributions. The approach then provides physically quantifiable probabilistic interpretation for all model variables. Here, Gaussian Markov random fields (GMRFs) are used for constructing the prior electron density distribution. The use of GMRF introduces sparsity to the linear system, making the problem computationally feasible. The method is demonstrated over Fennoscandia with measurements from global navigation satellite system (GNSS) and low Earth orbit (LEO) satellite receiver networks, GNSS occultation receivers, LEO satellite Langmuir probes, and ionosonde and incoherent scatter radar measurements