HUMAN ACTIVITY RECOGNITION FROM EGOCENTRIC VIDEOS AND ROBUSTNESS ANALYSIS OF DEEP NEURAL NETWORKS

In recent years, there has been significant amount of research work on human activity classification relying either on Inertial Measurement Unit (IMU) data or data from static cameras providing a third-person view. There has been relatively less work using wearable cameras, providing egocentric view, which is a first-person view providing the view of the environment as seen by the wearer. Using only IMU data limits the variety and complexity of the activities that can be detected. Deep machine learning has achieved great success in image and video processing in recent years. Neural network based models provide improved accuracy in multiple fields in computer vision. However, there has been relatively less work focusing on designing specific models to improve the performance of egocentric image/video tasks. As deep neural networks keep improving the accuracy in computer vision tasks, the robustness and resilience of the networks should be improved as well to make it possible to be applied in safety-crucial areas such as autonomous driving. Motivated by these considerations, in the first part of the thesis, the problem of human activity detection and classification from egocentric cameras is addressed. First, anew method is presented to count the number of footsteps and compute the total traveled distance by using the data from the IMU sensors and camera of a smart phone. By incorporating data from multiple sensor modalities, and calculating the length of each step, instead of using preset stride lengths and assuming equal-length steps, the proposed method provides much higher accuracy compared to commercially available step counting apps. After the application of footstep counting, more complicated human activities, such as steps of preparing a recipe and sitting on a sofa, are taken into consideration. Multiple classification methods, non-deep learning and deep-learning-based, are presented, which employ both ego-centric camera and IMU data. Then, a Genetic Algorithm-based approach is employed to set the parameters of an activity classification network autonomously and performance is compared with empirically-set parameters. Then, a new framework is introduced to reduce the computational cost of human temporal activity recognition from egocentric videos while maintaining the accuracy at a comparable level. The actor-critic model of reinforcement learning is applied to optical flow data to locate a bounding box around region of interest, which is then used for clipping a sub-image from a video frame. A shallow and deeper 3D convolutional neural network is designed to process the original image and the clipped image region, respectively.Next, a systematic method is introduced that autonomously and simultaneously optimizes multiple parameters of any deep neural network by using a bi-generative adversarial network (Bi-GAN) guiding a genetic algorithm(GA). The proposed Bi-GAN allows the autonomous exploitation and choice of the number of neurons for the fully-connected layers, and number of filters for the convolutional layers, from a large range of values. The Bi-GAN involves two generators, and two different models compete and improve each other progressively with a GAN-based strategy to optimize the networks during a GA evolution.In this analysis, three different neural network layers and datasets are taken into consideration: First, 3D convolutional layers for ModelNet40 dataset. We applied the proposed approach on a 3D convolutional network by using the ModelNet40 dataset. ModelNet is a dataset of 3D point clouds. The goal is to perform shape classification over 40shape classes. LSTM layers for UCI HAR dataset. UCI HAR dataset is composed of InertialMeasurement Unit (IMU) data captured during activities of standing, sitting, laying, walking, walking upstairs and walking downstairs. These activities were performed by 30 subjects, and the 3-axial linear acceleration and 3-axial angular velocity were collected at a constant rate of 50Hz. 2D convolutional layers for Chars74k Dataset. Chars74k dataset contains 64 classes(0-9, A-Z, a-z), 7705 characters obtained from natural images, 3410 hand-drawn characters using a tablet PC and 62992 synthesised characters from computer fonts giving a total of over 74K images. In the final part of the thesis, network robustness and resilience for neural network models is investigated from adversarial examples (AEs) and automatic driving conditions. The transferability of adversarial examples across a wide range of real-world computer vision tasks, including image classification, explicit content detection, optical character recognition(OCR), and object detection are investigated. It represents the cybercriminal’s situation where an ensemble of different detection mechanisms need to be evaded all at once.Novel dispersion Reduction(DR) attack is designed, which is a practical attack that overcomes existing attacks’ limitation of requiring task-specific loss functions by targeting on the “dispersion” of internal feature map. In the autonomous driving scenario, the adversarial machine learning attacks against the complete visual perception pipeline in autonomous driving is studied. A novel attack technique, tracker hijacking, that can effectively fool Multi-Object Tracking (MOT) using AEs on object detection is presented. Using this technique, successful AEs on as few as one single frame can move an existing object in to or out of the headway of an autonomous vehicle to cause potential safety hazards

Optimization of exposure time division for wide field observations

The optical observations of wide fields of view encounter the problem of selection of best exposure time. As there are usually plenty of objects observed simultaneously, the quality of photometry of the brightest ones is always better than of the dimmer ones. Frequently all of them are equally interesting for the astronomers and thus it is desired to have all of them measured with the highest possible accuracy. In this paper we present a novel optimization algorithm dedicated for the division of exposure time into sub-exposures, which allows to perform photometry with more balanced noise budget. Thanks to the proposed technique, the photometric precision of dimmer objects is increased at the expense of the measurement fidelity of the brightest ones. We tested the method on real observations using two telescope setups demonstrating its usefulness and good agreement with the theoretical expectations. The main application of our approach is a wide range of sky surveys, including the ones performed by the space telescopes. The method can be applied for planning virtually any photometric observations, in which the objects of interest show a wide range of magnitudes.Comment: 18 pages, 5 figure

A Study of Several Statistical Methods for Classification with Application to Microbial Source Tracking

With the advent of computers and the information age, vast amounts of data generated in a great deal of science and industry fields require the statisticians to explore further. In particular, statistical and computational problems in biology and medicine have created a new field of bioinformatics, which is attracting more and more statisticians, computer scientists, and biologists. Several procedures have been developed for tracing the source of fecal pollution in water resources based on certain characteristics of certain microorganisms. Use of this collection of techniques has been termed microbial source tracking (MST). Most of the current methods for MST are based on patterns of either phenotypic or genotypic variation in indicator organisms. Studies also suggested that patterns of genotypic variation might be more reliable due to their less association with environmental factors than those of phenotypic variation. Among the genotypic methods for source tracking, fingerprinting via rep-PCR is most common. Thus, identifying the specific pollution sources in contaminated waters based on rep-PCR fingerprinting techniques, viewed as a classification problem, has become an increasingly popular research topic in bioinformatics. In the project, several statistical methods for classification were studied, including linear discriminant analysis, quadratic discriminant analysis, logistic regression, and $k$-nearest-neighbor rules, neural networks and support vector machine. This project report summaries each of these methods and relevant statistical theory. In addition, an application of these methods to a particular set of MST data is presented and comparisons are made

From disease genes to behavioural screen in zebrafish: early onset Alzheimer’s as case study

To design prevention strategies and disease-modifying therapies in Alzheimer’s disease, we must discover biological processes which contribute to disease. Genomic studies can point to such causal processes, but their findings are rarely exploited in a systematic, hypothesis-free manner. In this thesis, we present a strategy in zebrafish to link disease-associated genes to likely causal processes. The first step is to inactivate each gene in zebrafish larvae. For this purpose, we developed a rapid CRISPR-Cas9 method capable of converting wild-type eggs directly into knockout larvae for any gene of interest. The method effectively cuts the experimental time from gene to knockout zebrafish from months to one day. The second step is to monitor the behaviour of the mutated larvae. As a case study, we targeted the three genes associated with early-onset Alzheimer’s disease. We found, for example, that larvae with loss-of-function mutations in presenilin 2 are less active during the day. The third step will be to use predictive pharmacology to identify drugs which cause the same phenotype in wild- type animals, thereby pointing to the defective process. This strategy is both scalable thanks to the knockout method and generalisable beyond Alzheimer’s disease. It can now be employed to screen tens or hundreds of genes associated with other conditions, such as schizophrenia or epilepsy

A Systematic Correlation of Nanoparticle Size with Diffusivity through Biological Fluids

Nanomedicine, the application of nanotechnology for medical purposes, has been widely identified as a potential solution for today‟s healthcare problems. Nanomedicine uses the "bottom-up‟ principles of nanoscale engineering to improve areas of medicine which have previously been considered undevelopable. One of the enduring challenges for medicine is the design of innovative devices able to overcome biological barriers, allowing drugs and therapeutics to effectively reach their correct location of action. Biological barriers are a defence mechanism of the body which are extremely well-evolved to protect the body from foreign and harmful particles. Therapeutic drugs and devices, which are not harmful, are often identified by the body as dangerous because their composition differs from native and accepted entities. The traversal of these biological barriers, such as mucus, remains a bottleneck in the progress of drug delivery and gene therapy. The mucus barrier physically limits the motion of particles due to its complicated mesh structure which obstructs the particles' traversal path. Mucus fibres can also adhere to the particles, entrapping them and restricting their motion. Particle traversal of mucus is carried out by passive diffusion. As diffusion has traditionally been defined by the Stokes-Einstein equation as inversely proportional to particle radius, it follows that reducing particle sizes into the nanoscale would result in increased diffusive ability. These predictions, however, do not consider the obstructive effects of the complicated mesh structure for the case of mucus. The exact effect of reducing particle size into the nanoscale for diffusion through mucus is therefore unknown. Multiple Particle Tracking was used to obtain real-time movies of the diffusion of nanoparticles, ranging from 12nm – 220nm in diameter, through mucus samples. The experimental data generated was used to systematically correlate the relationship between particle size and diffusivity through mucus. This study reveals that nanoparticles, smaller than the average pore size in the mucus mesh structure, can diffuse through lower viscosity pores which pose less resistance to diffusive motion, allowing nanoparticles to travel at up to four times the speed expected from the bulk viscosity of the mucus. This type of information can help researchers understand the importance of size for therapeutic nanoparticles, allowing researchers to decide whether attempts to decrease nanoparticle size at the expense of other functionality are worthwhile