Self-supervised Learning of Interpretable Keypoints from Unlabelled Videos

We propose KeypointGAN, a new method for recognizing the pose of objects from a single image that for learning uses only unlabelled videos and a weak empirical prior on the object poses. Video frames differ primarily in the pose of the objects they contain, so our method distils the pose information by analyzing the differences between frames. The distillation uses a new dual representation of the geometry of objects as a set of 2D keypoints, and as a pictorial representation, i.e. a skeleton image. This has three benefits: (1) it provides a tight `geometric bottleneck' which disentangles pose from appearance, (2) it can leverage powerful image-to-image translation networks to map between photometry and geometry, and (3) it allows to incorporate empirical pose priors in the learning process. The pose priors are obtained from unpaired data, such as from a different dataset or modality such as mocap, such that no annotated image is ever used in learning the pose recognition network. In standard benchmarks for pose recognition for humans and faces, our method achieves state-of-the-art performance among methods that do not require any labelled images for training.Comment: CVPR 2020 (oral). Project page: http://www.robots.ox.ac.uk/~vgg/research/unsupervised_pose

Imaging dynamics beneath turbid media via parallelized single-photon detection

Noninvasive optical imaging through dynamic scattering media has numerous important biomedical applications but still remains a challenging task. While standard methods aim to form images based upon optical absorption or fluorescent emission, it is also well-established that the temporal correlation of scattered coherent light diffuses through tissue much like optical intensity. Few works to date, however, have aimed to experimentally measure and process such data to demonstrate deep-tissue imaging of decorrelation dynamics. In this work, we take advantage of a single-photon avalanche diode (SPAD) array camera, with over one thousand detectors, to simultaneously detect speckle fluctuations at the single-photon level from 12 different phantom tissue surface locations delivered via a customized fiber bundle array. We then apply a deep neural network to convert the acquired single-photon measurements into video of scattering dynamics beneath rapidly decorrelating liquid tissue phantoms. We demonstrate the ability to record video of dynamic events occurring 5-8 mm beneath a decorrelating tissue phantom with mm-scale resolution and at a 2.5-10 Hz frame rate

Digital Image Processing

This book presents several recent advances that are related or fall under the umbrella of 'digital image processing', with the purpose of providing an insight into the possibilities offered by digital image processing algorithms in various fields. The presented mathematical algorithms are accompanied by graphical representations and illustrative examples for an enhanced readability. The chapters are written in a manner that allows even a reader with basic experience and knowledge in the digital image processing field to properly understand the presented algorithms. Concurrently, the structure of the information in this book is such that fellow scientists will be able to use it to push the development of the presented subjects even further

SecureAD: A secure video anomaly detection framework on convolutional neural network in edge computing environment

National Research Foundation (NRF) Singapore under Strategic Capability Research Centres Funding Intiatives; Ministry of Education, Singapore under its Academic Research Funding Tier

Toward commercial realisation of whole field interferometric analysis

The objective of this work was to produce an instrument which could undertake wholefield inspection and displacement measurement utilising a non-contacting technology. The instrument has been designed to permit operation by engineers not necessarily familiar with the underlying technology and produce results in a meaningful form. Of the possible techniques considered Holographic Interferometry was originally identified as meeting these objectives. Experimental work undertaken 'provides' data which confirms the potential of the technique for solving problems but also highlights some difficulties. In order to perform a complete three dimensional displacement analysis a number of holographic views must be recorded. Considerable effort is required to extract quantitative data from the holograms. Error analysis of the experimental arrangement has highlighted a number of practical restrictions which lead to data uncertainties. Qualitative analysis of engineering components using Holographic Interferometry has been successfully undertaken and results in useful analytical data which is used in three different engineering design programmes. Unfortunately, attempts to quantify the data to provide strain values relies upon double differentiation of the fringe field, a process that is highly sensitive to fringe position errors. In spite of this, these experiments provided the confidence that optical interferometry is able to produce data of suitable displacement sensitivity, with results acceptable to other engineers.....

Unsupervised Automatic Detection Of Transient Phenomena In InSAR Time-Series using Machine Learning

The detection and measurement of transient episodes of crustal deformation from global InSAR datasets are crucial for a wide range of solid earth and natural hazard applications. But the large volumes of unlabelled data captured by satellites preclude manual systematic analysis, and the small signal-to-noise ratio makes the task difficult. In this thesis, I present a state-of-the-art, unsupervised and event-agnostic deep-learning based approach for the automatic identification of transient deformation events in noisy time-series of unwrapped InSAR images. I adopt an anomaly detection framework that learns the ‘normal’ spatio-temporal pattern of noise in the data, and which therefore identifies any transient deformation phenomena that deviate from this pattern as ‘anomalies’. The deep-learning model is built around a bespoke autoencoder that includes convolutional and LSTM layers, as well as a neural network which acts as a bridge between the encoder and decoder. I train our model on real InSAR data from northern Turkey and find it has an overall accuracy and true positive rate of around 85% when trying to detect synthetic deformation signals of length-scale > 350 m and magnitude > 4 cm. Furthermore, I also show the method can detect (1) a real Mw 5.7 earthquake in InSAR data from an entirely different region- SW Turkey, (2) a volcanic deformation in Domuyo, Argentina, (3) a synthetic slow-slip event and (4) an interseismic deformation around NAF in a descending frame in northern Turkey. Overall I show that my method is suitable for automated analysis of large, global InSAR datasets, and for robust detection and separation of deformation signals from nuisance signals in InSAR data

Development of Portable Diffuse Optical Spectroscopic Systems For Treatment Monitoring

The goal of this dissertation is to demonstrate the utility of portable, small-scale diffuse optical spectroscopic (DOS) systems for the diagnosis and treatment monitoring of various diseases. These systems employ near-infrared light (wavelength range of 650nm to 950nm) to probe human tissue and are sensitive to changes in scattering and absorption properties of tissues. The absorption is mainly influenced by the components of blood, namely oxy- and deoxy-hemoglobin (HbO2 and Hb) and parameters that can be derived from them (e.g. total hemoglobin concentration [THb] and oxygen saturation, StO2). Therefore, I focused on diseases in which these parameters change, which includes vascular diseases such as Peripheral Atrial Disease (PAD) and Infantile Hemangiomas (IH) as well as musculoskeletal autoimmune diseases such as Rheumatoid Arthritis (RA). In each of these specific diseases, current monitoring techniques are limited by their sensitivity to disease progression or simply do not exist as a quantitative metric. As part of this project, I first designed and built a wireless handheld DOS device (WHDD) that can perform DOS measurements at various tissue depths. This device was used in a 15-patient pilot study for infantile hemangiomas (IH) to differentiate diseased skin from normal skin and monitor the vascular changes during intervention. In another study, I compare the ultra-small form- factor WHDD’s ability to monitor synovitis and disease progression during a patient’s treatment of RA against the capabilities of a proven frequency domain optical tomographic (FDOT) system that has shown to differentiate patients with and without RA. Learning from clinical utility of the WHDD from these two studies, I adapted the WHDD technology to develop a compact multi- channel DOS measurement system to monitor perfusion changes in the lower extremities before and after surgical intervention for patients with peripheral artery disease (PAD). Using this multi- channel system, which we called the vascular optical spectroscopic measurement (VOSM) system, our group conducted a 20-subject pilot study to quantify its ability to monitor blood perfusion before and after revascularization of stenotic arteries in the lower extremities. This proof-of- concept study demonstrated how DOS may help vascular surgeons perform revascularization procedures in the operating room and assists in post-operative treatment monitoring of vascular diseases