In-loop Feature Tracking for Structure and Motion with Out-of-core Optimization

In this paper, a novel and approach for obtaining 3D models from video sequences captured with hand-held cameras is addressed. We define a pipeline that robustly deals with different types of sequences and acquiring devices. Our system follows a divide and conquer approach: after a frame decimation that pre-conditions the input sequence, the video is split into short-length clips. This allows to parallelize the reconstruction step which translates into a reduction in the amount of computational resources required. The short length of the clips allows an intensive search for the best solution at each step of reconstruction which robustifies the system. The process of feature tracking is embedded within the reconstruction loop for each clip as opposed to other approaches. A final registration step, merges all the processed clips to the same coordinate fram

Classification and Clustering of Shared Images on Social Networks and User Profile Linking

The ever increasing prevalence of smartphones and the popularity of social network platforms have facilitated instant sharing of multimedia content through social networks. However, the ease in taking and sharing photos and videos through social networks also allows privacy-intrusive and illegal content to be widely distributed. As such, images captured and shared by users on their profiles are considered as significant digital evidence for social network data analysis. The Sensor Pattern Noise (SPN) caused by camera sensor imperfections during the manufacturing process mainly consists of the Photo-Response Non-Uniformity (PRNU) noise that can be extracted from taken images without hacking the device. It has been proven to be an effective and robust device fingerprint that can be used for different important digital image forensic tasks, such as image forgery detection, source device identification and device linking. Particularly, by fingerprinting the camera sources captured a set of shared images on social networks, User Profile Linking (UPL) can be performed on social network platforms. The aim of this thesis is to present effective and robust methods and algorithms for better fulfilling shared image analysis based on SPN. We propose clustering and classification based methods to achieve Smartphone Identification (SI) and UPL tasks, given a set of images captured by a known number of smartphones and shared on a set of known user profiles. The important outcome of the proposed methods is UPL across different social networks where the clustered images from one social network are applied to fingerprint the related smartphones and link user profiles on the other social network. Also, we propose two methods for large-scale image clustering of different types of the shared images by users, without prior knowledge about the types and number of the smartphones

Fusion of non-visual and visual sensors for human tracking

Human tracking is an extensively researched yet still challenging area in the Computer Vision field, with a wide range of applications such as surveillance and healthcare. People may not be successfully tracked with merely the visual information in challenging cases such as long-term occlusion. Thus, we propose to combine information from other sensors with the surveillance cameras to persistently localize and track humans, which is becoming more promising with the pervasiveness of mobile devices such as cellphones, smart watches and smart glasses embedded with all kinds of sensors including accelerometers, gyroscopes, magnetometers, GPS, WiFi modules and so on. In this thesis, we firstly investigate the application of Inertial Measurement Unit (IMU) from mobile devices to human activity recognition and human tracking, we then develop novel persistent human tracking and indoor localization algorithms by the fusion of non-visual sensors and visual sensors, which not only overcomes the occlusion challenge in visual tracking, but also alleviates the calibration and drift problems in IMU tracking --Abstract, page iii