Probabilistic movement modeling for intention inference in human-robot interaction.

Intention inference can be an essential step toward efficient humanrobot interaction. For this purpose, we propose the Intention-Driven Dynamics Model (IDDM) to probabilistically model the generative process of movements that are directed by the intention. The IDDM allows to infer the intention from observed movements using Bayes ’ theorem. The IDDM simultaneously finds a latent state representation of noisy and highdimensional observations, and models the intention-driven dynamics in the latent states. As most robotics applications are subject to real-time constraints, we develop an efficient online algorithm that allows for real-time intention inference. Two human-robot interaction scenarios, i.e., target prediction for robot table tennis and action recognition for interactive humanoid robots, are used to evaluate the performance of our inference algorithm. In both intention inference tasks, the proposed algorithm achieves substantial improvements over support vector machines and Gaussian processes.

Advances in Well Control: Early Kick Detection and Automated Control Systems

The devastating impact of the Macondo blowout incident has underscored the need for effective well barriers, early kick detection systems, and increased automation of well-control operations toward successful drilling and completion operations particularly in deep offshore environments. Early kick detection systems should be capable of detecting a gas influx both during drilling and tripping operations regardless of the drilling fluid system with minimal false-negative alarms, while automated control systems regain well-control eliminating delays or omissions due to human error. In this chapter, developments in the deployment of early kick detection and automated control systems in conventional and managed pressure drilling operations are reviewed. We discuss the use and placement of surface sensors such as the Coriolis flowmeter, smart flowback fingerprinting when the rig pumps are off, real-time gas monitoring along the marine riser and downhole measurements complimented with machine learning algorithms for early kick detection. We then focus on the application of automated well-control systems for managed pressure drilling operations for which gas kicks are circulated without stopping the pumps or shutting in the well and in conventional well operations requiring intelligent tool joint space-out prior to well shut in especially for deep offshore operations

Audio-visual football video analysis, from structure detection to attention analysis

Sport video is an important video genre. Content-based sports video analysis attracts great interest from both industry and academic fields. A sports video is characterised by repetitive temporal structures, relatively plain contents, and strong spatio-temporal variations, such as quick camera switches and swift local motions. It is necessary to develop specific techniques for content-based sports video analysis to utilise these characteristics. For an efficient and effective sports video analysis system, there are three fundamental questions: (1) what are key stories for sports videos; (2) what incurs viewer’s interest; and (3) how to identify game highlights. This thesis is developed around these questions. We approached these questions from two different perspectives and in turn three research contributions are presented, namely, replay detection, attack temporal structure decomposition, and attention-based highlight identification. Replay segments convey the most important contents in sports videos. It is an efficient approach to collect game highlights by detecting replay segments. However, replay is an artefact of editing, which improves with advances in video editing tools. The composition of replay is complex, which includes logo transitions, slow motions, viewpoint switches and normal speed video clips. Since logo transition clips are pervasive in game collections of FIFA World Cup 2002, FIFA World Cup 2006 and UEFA Championship 2006, we take logo transition detection as an effective replacement of replay detection. A two-pass system was developed, including a five-layer adaboost classifier and a logo template matching throughout an entire video. The five-layer adaboost utilises shot duration, average game pitch ratio, average motion, sequential colour histogram and shot frequency between two neighbouring logo transitions, to filter out logo transition candidates. Subsequently, a logo template is constructed and employed to find all transition logo sequences. The precision and recall of this system in replay detection is 100% in a five-game evaluation collection. An attack structure is a team competition for a score. Hence, this structure is a conceptually fundamental unit of a football video as well as other sports videos. We review the literature of content-based temporal structures, such as play-break structure, and develop a three-step system for automatic attack structure decomposition. Four content-based shot classes, namely, play, focus, replay and break were identified by low level visual features. A four-state hidden Markov model was trained to simulate transition processes among these shot classes. Since attack structures are the longest repetitive temporal unit in a sports video, a suffix tree is proposed to find the longest repetitive substring in the label sequence of shot class transitions. These occurrences of this substring are regarded as a kernel of an attack hidden Markov process. Therefore, the decomposition of attack structure becomes a boundary likelihood comparison between two Markov chains. Highlights are what attract notice. Attention is a psychological measurement of “notice ”. A brief survey of attention psychological background, attention estimation from vision and auditory, and multiple modality attention fusion is presented. We propose two attention models for sports video analysis, namely, the role-based attention model and the multiresolution autoregressive framework. The role-based attention model is based on the perception structure during watching video. This model removes reflection bias among modality salient signals and combines these signals by reflectors. The multiresolution autoregressive framework (MAR) treats salient signals as a group of smooth random processes, which follow a similar trend but are filled with noise. This framework tries to estimate a noise-less signal from these coarse noisy observations by a multiple resolution analysis. Related algorithms are developed, such as event segmentation on a MAR tree and real time event detection. The experiment shows that these attention-based approach can find goal events at a high precision. Moreover, results of MAR-based highlight detection on the final game of FIFA 2002 and 2006 are highly similar to professionally labelled highlights by BBC and FIFA

Optimization of an Intelligent Autonomous Drilling Rig: Testing and Implementation of Machine Learning and Control Algorithms for Formation Classification, Downhole Vibrations Management and Directional Drilling

Master's thesis in Petroleum EngineeringIn recent years, considerable resources have been invested to explore applications for- and to exploit the vast amount of data that gets collected during exploration, drilling and production of oil and gas. Such data will potentially become a game changer for the industry in terms of reduced costs through improved operational efficiency and fewer accidents, improved HSE through strengthened situational awareness, ensured optimal placement of wells, less wear on equipment and so on. While machine learning algorithms have been around for decades, it is only in the last five to ten years that increased computational power along with heavily digitalized control- and monitoring systems have been made available. Considering the state of art technology that exists today and the significant resources that are being invested into the technology of tomorrow, the idea of intelligent and fully automated machinery on the drill floor that is capable of consistently selecting the best decisions or predictions based on the information available and providing the driller and operator with such recommendations, becomes closer to a reality every day. This thesis is the result of research carried out on the topic of drilling automation. Its basis has been improvements and upgrades conducted on a laboratory-scale drilling rig developed at the University of Stavanger, as part of the multi-disciplinary project; UiS Drillbotics. Main contribution of the thesis is a study on how machine learning can be used to develop models that are capable of accurately predicting what rock formation is being drilled using an autonomous control system, along with detecting some common drilling incidents in real-time on the laboratory rig. Methodology is also applied to field data from the Volve field. Furthermore, research and implementation of search algorithms to ensure optimal drilling speed (ROP), safety to personnel and environment (HSE), and efficiency along with a digitalized drilling program for directional drilling, gets presented. Finally, rig upgrades for directional drilling and research into downhole sensors that get used in a closed-loop steering model is elaborated on.submittedVersio

Aprendizagem automática de comportamentos para futebol robótico

Mestrado em Engenharia de Computadores e TelemáticaA soccer-playing robot must be able to carry out a set of behaviors, whose complexity can vary greatly. Manually programming a robot to accomplish those behaviors may be a difficult and time-consuming process. Automated learning techniques become interesting in this setting, because they allow the learning of behaviors based only on a very high-level description of the task to be completed, leaving the details to be figured out by the learning agent. Reinforcement Learning takes inspiration from nature and animal learning to model agents that interact with an environment, choosing actions that are more likely to lead them to accumulate rewards and avoid punishment. As agents experience the environment and the effect of their actions, they gain experience which is used to derive a policy. Agents can do this instantaneously after they observe the effect of their last action, or after collecting batches of these observations. The latter alternative, called Batch Reinforcement Learning, has been used in real world applications with very promissing results. This thesis explores the use of Batch Reinforcement Learning for learning robotic soccer behaviors, including dribbling the ball and receiving a pass. Practical experiments were undertaken with the CAMBADA simulator, as well as with the CAMBADA robots.Um robô futebolista necessita de executar comportamentos variados, desde os mais simples aos mais complexos e difíceis. Programar manualmente a execução destes comportamentos pode tornar-se uma tarefa bastante morosa e complicada. Neste contexto, os métodos de aprendizagem automática tornam-se interessantes, pois permitem a aprendizagem de comportamentos através de uma especificação a muito alto nível da tarefa a aprender, deixando a responsabilidade ao agente autónomo de lidar com os detalhes. A Aprendizagem por Reforço toma inspiração na natureza e na aprendizagem animal para modelar agentes que interagem com o seu ambiente de forma a escolherem as ações que aumentam a probabilidade de receberem recompensas e evitarem castigos. À medida que os agentes experimentam ações e observam os seus efeitos, ganham experiência e a partir dela derivam uma política. Isto é feito após cada observação do efeito de uma ação, ou após reunir conjuntos destas observações. Esta última alternativa, também chamada Aprendizagem por Reforço Batch, tem sido usada em aplicações reais com resultados promissores. Esta tese explora o uso de Aprendizagem por Reforço Batch para a aprendizagem de comportamentos para futebol robótico, tais como driblar a bola e receber um passe. Os resultados presentes neste documento foram obtidos de experiências realizadas com o simulador da equipa CAMBADA, assim como com os seus robôs

Spectral discontinuity in concatenative speech synthesis – perception, join costs and feature transformations

This thesis explores the problem of determining an objective measure to represent human perception of spectral discontinuity in concatenative speech synthesis. Such measures are used as join costs to quantify the compatibility of speech units for concatenation in unit selection synthesis. No previous study has reported a spectral measure that satisfactorily correlates with human perception of discontinuity. An analysis of the limitations of existing measures and our understanding of the human auditory system were used to guide the strategies adopted to advance a solution to this problem. A listening experiment was conducted using a database of concatenated speech with results indicating the perceived continuity of each concatenation. The results of this experiment were used to correlate proposed measures of spectral continuity with the perceptual results. A number of standard speech parametrisations and distance measures were tested as measures of spectral continuity and analysed to identify their limitations. Time-frequency resolution was found to limit the performance of standard speech parametrisations.As a solution to this problem, measures of continuity based on the wavelet transform were proposed and tested, as wavelets offer superior time-frequency resolution to standard spectral measures. A further limitation of standard speech parametrisations is that they are typically computed from the magnitude spectrum. However, the auditory system combines information relating to the magnitude spectrum, phase spectrum and spectral dynamics. The potential of phase and spectral dynamics as measures of spectral continuity were investigated. One widely adopted approach to detecting discontinuities is to compute the Euclidean distance between feature vectors about the join in concatenated speech. The detection of an auditory event, such as the detection of a discontinuity, involves processing high up the auditory pathway in the central auditory system. The basic Euclidean distance cannot model such behaviour. A study was conducted to investigate feature transformations with sufficient processing complexity to mimic high level auditory processing. Neural networks and principal component analysis were investigated as feature transformations. Wavelet based measures were found to outperform all measures of continuity based on standard speech parametrisations. Phase and spectral dynamics based measures were found to correlate with human perception of discontinuity in the test database, although neither measure was found to contribute a significant increase in performance when combined with standard measures of continuity. Neural network feature transformations were found to significantly outperform all other measures tested in this study, producing correlations with perceptual results in excess of 90%

Bayesian inferencing and deterministic anisotropy for the retrieval of the molecular geometry ∣Ψ(r)∣2|\Psi(\mathbf{r})|^2 in gas-phase diffraction experiments

Currently, our general approach to retrieve the molecular geometry from ultrafast gas-phase diffraction heavily relies on complex geometric simulations to make conclusive interpretations. In this manuscript, we develop a broadly applicable ultrafast gas-phase diffraction method that approximates the molecular frame geometry $|\Psi(\mathbf{r}, t)|^2$ distribution using Bayesian Inferencing. This method does not require complex molecular dynamics simulation and can identify the unique molecular structure. We demonstrate this method's viability by retrieving the ground state geometry distribution $|\Psi(\mathbf{r})|^2$ for both simulated stretched NO$_2$ and measured ground state N$_2$O. Due to our statistical interpretation, we retrieve a coordinate-space resolution on the order of 100~fm, depending on signal quality, an improvement of order 100 compared to commonly used Fourier transform based methods. By directly measuring the width of $|\Psi(\mathbf{r})|^2$, this is generally only accessible through simulation, we open ultrafast gas-phase diffraction capabilities to measurements beyond current analysis approaches. Our method also leverages deterministic ensemble anisotropy; this provides an explicit dependence on the molecular frame angles. This method's ability to retrieve the unique molecular structure with high resolution, and without complex simulations, provides the potential to effectively turn gas-phase ultrafast diffraction into a discovery oriented technique, one that probes systems that are prohibitively difficult to simulate.Comment: 16 pages, 8 figures, 2 tables. Please find the analysis code and templates for new molecules at https://github.com/khegazy/BIG

Multi-Modality Human Action Recognition

Human action recognition is very useful in many applications in various areas, e.g. video surveillance, HCI (Human computer interaction), video retrieval, gaming and security. Recently, human action recognition becomes an active research topic in computer vision and pattern recognition. A number of action recognition approaches have been proposed. However, most of the approaches are designed on the RGB images sequences, where the action data was collected by RGB/intensity camera. Thus the recognition performance is usually related to various occlusion, background, and lighting conditions of the image sequences. If more information can be provided along with the image sequences, more data sources other than the RGB video can be utilized, human actions could be better represented and recognized by the designed computer vision system.;In this dissertation, the multi-modality human action recognition is studied. On one hand, we introduce the study of multi-spectral action recognition, which involves the information from different spectrum beyond visible, e.g. infrared and near infrared. Action recognition in individual spectra is explored and new methods are proposed. Then the cross-spectral action recognition is also investigated and novel approaches are proposed in our work. On the other hand, since the depth imaging technology has made a significant progress recently, where depth information can be captured simultaneously with the RGB videos. The depth-based human action recognition is also investigated. I first propose a method combining different type of depth data to recognize human actions. Then a thorough evaluation is conducted on spatiotemporal interest point (STIP) based features for depth-based action recognition. Finally, I advocate the study of fusing different features for depth-based action analysis. Moreover, human depression recognition is studied by combining facial appearance model as well as facial dynamic model

2015 Summer Research Symposium Abstract Book

2015 Summer volume of abstracts for science research projects conducted by students at Trinity College