MAC-REALM: A video content feature extraction and modelling framework

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.A consequence of the ‘data deluge’ is the exponential increase in digital video footage, while the ability to find relevant video clips diminishes. Traditional text based search engines are no longer optimal for searching, as they cannot provide a granular search of the content inside video footage. To be able to search the video in a content based manner, the content features of the video need to be extracted and modelled into a content model, which can then act as a searchable proxy for the video content. This thesis focuses on the extraction of syntactic and semantic content features and content modelling, using machine driven processes, with either little or no user interaction. Our abstract framework design extracts syntactic and semantic content features and compiles them into an integrated content model. The framework integrates a four plane strategy that consists of a pre-processing plane that removes redundant data and filters the media to improve the feature extraction properties of the media; a syntactic feature extraction plane that extracts low level syntactic feature and mid-level syntactic features that have semantic attributes; a semantic relationship analysis and linkage plane, where the spatial and temporal relationships of all the content features are defined, and finally a content modelling stage where the syntactic and semantic content features are integrated into a content model. Each of the four planes can be split into three layers namely, the content layer, where the content to be processed is stored; the application layer, where the content is converted into content descriptions, and the MPEG-7 layer, where content descriptions are serialised. Using MPEG-7 standards to produce the content model will provide wide-ranging interoperability, while facilitating granular multi-content type searches. The framework is aiming to ‘bridge’ the semantic gap, by integrating the syntactic and semantic content features from extraction through to modelling. The design of the framework has been implemented into a prototype called MAC-REALM, which has been tested and evaluated for its effectiveness to extract and model content features. Conclusions are drawn about the research output as a whole and whether they have met the objectives. Finally, future work is presented on how concept detection and crowd sourcing can be used with MAC-REALM

Eye quietness and quiet eye in expert and novice golf performance: an electrooculographic analysis

Quiet eye (QE) is the final ocular fixation on the target of an action (e.g., the ball in golf putting). Camerabased eye-tracking studies have consistently found longer QE durations in experts than novices; however, mechanisms underlying QE are not known. To offer a new perspective we examined the feasibility of measuring the QE using electrooculography (EOG) and developed an index to assess ocular activity across time: eye quietness (EQ). Ten expert and ten novice golfers putted 60 balls to a 2.4 m distant hole. Horizontal EOG (2ms resolution) was recorded from two electrodes placed on the outer sides of the eyes. QE duration was measured using a EOG voltage threshold and comprised the sum of the pre-movement and post-movement initiation components. EQ was computed as the standard deviation of the EOG in 0.5 s bins from –4 to +2 s, relative to backswing initiation: lower values indicate less movement of the eyes, hence greater quietness. Finally, we measured club-ball address and swing durations. T-tests showed that total QE did not differ between groups (p = .31); however, experts had marginally shorter pre-movement QE (p = .08) and longer post-movement QE (p < .001) than novices. A group × time ANOVA revealed that experts had less EQ before backswing initiation and greater EQ after backswing initiation (p = .002). QE durations were inversely correlated with EQ from –1.5 to 1 s (rs = –.48 - –.90, ps = .03 - .001). Experts had longer swing durations than novices (p = .01) and, importantly, swing durations correlated positively with post-movement QE (r = .52, p = .02) and negatively with EQ from 0.5 to 1s (r = –.63, p = .003). This study demonstrates the feasibility of measuring ocular activity using EOG and validates EQ as an index of ocular activity. Its findings challenge the dominant perspective on QE and provide new evidence that expert-novice differences in ocular activity may reflect differences in the kinematics of how experts and novices execute skills

Evolutionary design of deep neural networks

Mención Internacional en el título de doctorFor three decades, neuroevolution has applied evolutionary computation to the optimization of the topology of artificial neural networks, with most works focusing on very simple architectures. However, times have changed, and nowadays convolutional neural networks are the industry and academia standard for solving a variety of problems, many of which remained unsolved before the discovery of this kind of networks. Convolutional neural networks involve complex topologies, and the manual design of these topologies for solving a problem at hand is expensive and inefficient. In this thesis, our aim is to use neuroevolution in order to evolve the architecture of convolutional neural networks. To do so, we have decided to try two different techniques: genetic algorithms and grammatical evolution. We have implemented a niching scheme for preserving the genetic diversity, in order to ease the construction of ensembles of neural networks. These techniques have been validated against the MNIST database for handwritten digit recognition, achieving a test error rate of 0.28%, and the OPPORTUNITY data set for human activity recognition, attaining an F1 score of 0.9275. Both results have proven very competitive when compared with the state of the art. Also, in all cases, ensembles have proven to perform better than individual models. Later, the topologies learned for MNIST were tested on EMNIST, a database recently introduced in 2017, which includes more samples and a set of letters for character recognition. Results have shown that the topologies optimized for MNIST perform well on EMNIST, proving that architectures can be reused across domains with similar characteristics. In summary, neuroevolution is an effective approach for automatically designing topologies for convolutional neural networks. However, it still remains as an unexplored field due to hardware limitations. Current advances, however, should constitute the fuel that empowers the emergence of this field, and further research should start as of today.This Ph.D. dissertation has been partially supported by the Spanish Ministry of Education, Culture and Sports under FPU fellowship with identifier FPU13/03917. This research stay has been partially co-funded by the Spanish Ministry of Education, Culture and Sports under FPU short stay grant with identifier EST15/00260.Programa Oficial de Doctorado en Ciencia y Tecnología InformáticaPresidente: María Araceli Sanchís de Miguel.- Secretario: Francisco Javier Segovia Pérez.- Vocal: Simon Luca

Review of Wearable Devices and Data Collection Considerations for Connected Health

Wearable sensor technology has gradually extended its usability into a wide range of well-known applications. Wearable sensors can typically assess and quantify the wearer’s physiology and are commonly employed for human activity detection and quantified self-assessment. Wearable sensors are increasingly utilised to monitor patient health, rapidly assist with disease diagnosis, and help predict and often improve patient outcomes. Clinicians use various self-report questionnaires and well-known tests to report patient symptoms and assess their functional ability. These assessments are time consuming and costly and depend on subjective patient recall. Moreover, measurements may not accurately demonstrate the patient’s functional ability whilst at home. Wearable sensors can be used to detect and quantify specific movements in different applications. The volume of data collected by wearable sensors during long-term assessment of ambulatory movement can become immense in tuple size. This paper discusses current techniques used to track and record various human body movements, as well as techniques used to measure activity and sleep from long-term data collected by wearable technology devices

Usability, Efficiency and Security of Personal Computing Technologies

New personal computing technologies such as smartphones and personal fitness trackers are widely integrated into user lifestyles. Users possess a wide range of skills, attributes and backgrounds. It is important to understand user technology practices to ensure that new designs are usable and productive. Conversely, it is important to leverage our understanding of user characteristics to optimize new technology efficiency and effectiveness. Our work initially focused on studying older users, and personal fitness tracker users. We applied the insights from these investigations to develop new techniques improving user security protections, computational efficiency, and also enhancing the user experience. We offer that by increasing the usability, efficiency and security of personal computing technology, users will enjoy greater privacy protections along with experiencing greater enjoyment of their personal computing devices. Our first project resulted in an improved authentication system for older users based on familiar facial images. Our investigation revealed that older users are often challenged by traditional text passwords, resulting in decreased technology use or less than optimal password practices. Our graphical password-based system relies on memorable images from the user\u27s personal past history. Our usability study demonstrated that this system was easy to use, enjoyable, and fast. We show that this technique is extendable to smartphones. Personal fitness trackers are very popular devices, often worn by users all day. Our personal fitness tracker investigation provides the first quantitative baseline of usage patterns with this device. By exploring public data, real-world user motivations, reliability concerns, activity levels, and fitness-related socialization patterns were discerned. This knowledge lends insight to active user practices. Personal user movement data is captured by sensors, then analyzed to provide benefits to the user. The dynamic time warping technique enables comparison of unequal data sequences, and sequences containing events at offset times. Existing techniques target short data sequences. Our Phase-aware Dynamic Time Warping algorithm focuses on a class of sinusoidal user movement patterns, resulting in improved efficiency over existing methods. Lastly, we address user data privacy concerns in an environment where user data is increasingly flowing to manufacturer remote cloud servers for analysis. Our secure computation technique protects the user\u27s privacy while data is in transit and while resident on cloud computing resources. Our technique also protects important data on cloud servers from exposure to individual users

Optimized Ensemble Approach for Multi-model Event Detection in Big data

Event detection acts an important role among modern society and it is a popular computer process that permits to detect the events automatically. Big data is more useful for the event detection due to large size of data. Multimodal event detection is utilized for the detection of events using heterogeneous types of data. This work aims to perform for classification of diverse events using Optimized Ensemble learning approach. The Multi-modal event data including text, image and audio are sent to the user devices from cloud or server where three models are generated for processing audio, text and image. At first, the text, image and audio data is processed separately. The process of creating a text model includes pre-processing using Imputation of missing values and data normalization. Then the textual feature extraction using integrated N-gram approach. The Generation of text model using Convolutional two directional LSTM (2DCon_LSTM). The steps involved in image model generation are pre-processing using Min-Max Gaussian filtering (MMGF). Image feature extraction using VGG-16 network model and generation of image model using Tweaked auto encoder (TAE) model. The steps involved in audio model generation are pre-processing using Discrete wavelet transform (DWT). Then the audio feature extraction using Hilbert Huang transform (HHT) and Generation of audio model using Attention based convolutional capsule network (Attn_CCNet). The features obtained by the generated models of text, image and audio are fused together by feature ensemble approach. From the fused feature vector, the optimal features are trained through improved battle royal optimization (IBRO) algorithm. A deep learning model called Convolutional duo Gated recurrent unit with auto encoder (C-Duo GRU_AE) is used as a classifier. Finally, different types of events are classified where the global model are then sent to the user devices with high security and offers better decision making process. The proposed methodology achieves better performances are Accuracy (99.93%), F1-score (99.91%), precision (99.93%), Recall (99.93%), processing time (17seconds) and training time (0.05seconds). Performance analysis exceeds several comparable methodologies in precision, recall, accuracy,&nbsp;F1 score, training time, and processing&nbsp;time. This designates that the proposed methodology achieves improved performance than the compared schemes. In addition, the proposed scheme detects the multi-modal events accurately

Predictors of exercise adherence and weight control : an application of self-determination theory

Doutoramento em Motricidade Humana, especialidade em Saúde e Condição FísicaThe present Thesis was set within a longitudinal randomized controlled trial, consisting of a 1-year theory-based behavior change intervention and a 2-year follow-up period with no intervention. This trial included 239 overweight or obese women (age: 37.6±7 y; BMI: 31.5±4.1 kg/m2) who were premenopausal and free from identified disease. The main intervention was designed to increase physical activity and internal motivation, following self-determination theory (SDT). While tested before for other health behaviors with promising results, SDT had never been previously applied to weight management in longterm, controlled designs. Thus, the four studies comprising this Thesis were designed to provide a comprehensive analysis of how a theory-based intervention, implemented to affect specific theory-based mediators, would impact exercise adherence and body weight change both in short and long-term, from a SDT perspective. After a complete description of the study protocol, theoretical framework, and main intervention strategies (study 1), study 2 showed that the intervention was successful in affecting theory-driven mediators, physical activity, and weight change at 12 months (intervention’s end), demonstrating that the socio-contextual characteristics advanced by SDT Abstract -2- are amenable to manipulation, and revealing the potential utility of SDT to shape behavioral interventions targeting the promotion of physically active lifestyles and weight-change. Studies 3 and 4 searched for a more in-depth understanding of the dynamics of exercise motivation by exploring and testing mediational models aimed at outlining theory-based mechanisms and their impact on different types of physical activity at intervention’s end (study 3), and on long-term behavioral exercise regulations and physical activity (24-month) and 36-month weight change (study 4). Convergent with previous research, but extending it into the context of a randomized controlled trial, these studies indicated that perceived needsupportive health care climate, psychological needs for autonomy and competence, and intrinsic motivation mediate the effects of the experimental treatment climate on structured exercise behavior. Furthermore, these variables rested within the causal path of long-term weight loss, providing evidence from a link between experimentally-increased autonomous motivation, long-term physical activity adoption, and 3-year weight management. This application of SDT to physical activity and weight management provides experimental evidence that an autonomy-supportive context facilitates the internalization of regulatory processes, particularly through its effect on exercise-related constructs, which in turn promote long-term positive behavioral and clinical outcomes in overweight/obese women.A presente tese foi desenvolvida no contexto de um estudo longitudinal, controlado e com distribuição aleatória, configurando uma intervenção comportamental teoricamente sustentada, com a duração de um ano, seguida de dois anos de follow-up (sem qualquer intervenção). Este estudo envolveu 239 mulheres com excesso de peso ou obesidade, prémenopausicas e sem patologia diagnosticada. O programa de intervenção foi desenhado de acordo com os princípios base da teoria da auto-determinação (TAD), visando sobretudo o aumento da motivação intrínseca para a actividade física, procurando-se promover a sua adesão a longo prazo. Apesar de este enquadramento conceptual já ter sido testado com resultados promissores na promoção de vários comportamentos de saúde, não há relato de um teste experimental, com avaliações no longo prazo, da TAD no âmbito do controlo do peso. Os 4 artigos que constituem esta tese, no seu conjunto, foram desenhados para permitir a análise do modo como uma intervenção teoricamente sustentada, desenhada e implementada para ter efeito em variáveis-alvo mediadoras poderá influenciar a adesão ao exercício e a alteração do peso corporal, tanto no curto como no longo prazo. Partindo de uma descrição pormenorizada do protocolo experimental, das principais estratégias de intervenção e do racional teórico que as sustenta (estudo 1), o estudo 2 veio demonstrar que, 1 ano após o seu inicio, a intervenção parece ter sido bem sucedida na promoção de actividade física e perda de peso, tendo exercido influência positiva ao nível das principais variáveis-alvo mediadoras (por comparação com o grupo de controlo). Este estudo evidenciou a possibilidade de manipulação experimental das principais características sóciocontextuais identificadas pela TAD como fundamentais na promoção de estilos de vida activos, compatíveis com a eficaz gestão do peso. Os estudos 3 e 4 representam a passagem do estudo do impacto da intervenção para a procura de entendimento dos principais mecanismos motivacionais envolvidos. Com recurso ao teste de modelos mediacionais, procurou-se estudar o efeito diferenciado das variáveis-alvo identificadas pela TAD na adopção de diferentes tipos de actividade física um ano após o inicio da intervenção (estudo 3), bem como o papel das referidas variáveis na adesão continuada à actividade física (follow-up 2º ano), e alteração do peso a longo prazo (follow-up 3º ano). De forma congruente com investigações anteriores, mas estendendo-as a um enquadramento experimental e controlado, os resultados encontrados sustentam a percepção de um clima de suporte à satisfação das necessidades psicológicas básicas (autonomia e competência), e a motivação intrínseca, como mediadoras dos efeitos produzidos pela intervenção na adesão à actividade física estruturada. Estas variáveis funcionaram também como mecanismos associados ao envolvimento continuado em actividades físicas (dois anos após o início da intervenção) e controlo do peso a longo prazo (3 anos após o início da intervenção). Com base nos resultados encontrados, esta aplicação da TAD à promoção da actividade física e do controlo do peso fornece evidência experimental de que um clima de suporte ao desenvolvimento de autonomia e competência pode facilitar a internalização de regulações motivacionais mais autónomas para a actividade física, as quais se constituem como facilitadoras de mudanças comportamentais importantes e com relevância clínica, em mulheres com excesso de peso ou obesidade