Enhanced Augmented Reality Framework for Sports Entertainment Applications

Augmented Reality (AR) superimposes virtual information on real-world data, such as displaying useful information on videos/images of a scene. This dissertation presents an Enhanced AR (EAR) framework for displaying useful information on images of a sports game. The challenge in such applications is robust object detection and recognition. This is even more challenging when there is strong sunlight. We address the phenomenon where a captured image is degraded by strong sunlight. The developed framework consists of an image enhancement technique to improve the accuracy of subsequent player and face detection. The image enhancement is followed by player detection, face detection, recognition of players, and display of personal information of players. First, an algorithm based on Multi-Scale Retinex (MSR) is proposed for image enhancement. For the tasks of player and face detection, we use adaptive boosting algorithm with Haar-like features for both feature selection and classification. The player face recognition algorithm uses adaptive boosting with the LDA for feature selection and nearest neighbor classifier for classification. The framework can be deployed in any sports where a viewer captures images. Display of players-specific information enhances the end-user experience. Detailed experiments are performed on 2096 diverse images captured using a digital camera and smartphone. The images contain players in different poses, expressions, and illuminations. Player face recognition module requires players faces to be frontal or up to ?350 of pose variation. The work demonstrates the great potential of computer vision based approaches for future development of AR applications.COMSATS Institute of Information Technolog

A realidade aumentada no contexto museológico: o caso dos Museus de Portugal

A Realidade Aumentada pertence às tecnologias que incorporam a Indústria 4.0 e é utilizada com muita frequência pelas organizações de diferentes áreas de atuação. No contexto museológico internacional, a RA tem vindo a ser mais aplicada, no entanto, no caso específico de Portugal, existe ainda uma grande escassez no que toca à implementação desta tecnologia. O principal objetivo desta investigação foi determinar quais são os fatores de influência no sucesso ou no insucesso da implementação de Realidade Aumentada nos museus. A investigação teve como base um grupo de 5 museus portugueses, sob tutela da Direção Regional da Cultura do Norte, que integraram um projeto de implmenetação da Realidade Aumentada. De forma a cumprir com o objetivo do estudo foi utilizada a metodologia de estudo de casos múltiplos e o método de inquérito por entrevista. Após a recolha de dados, os mesmos foram tratados e apresentados e procedeu-se ao estudo qualitativo com recurso à análise descritiva e comparativa, com base na análise de conteúdo. Os resultados mostraram que os fatores de maior influência no sucesso ou insucesso da implementação de RA nos museus são a tecnologia de rede, a formação dos recursos humanos e os custos associados à RA. As conclusões deste estudo lançam luzes sobre a complexidade associada à integração da tecnologia de RA no contexto museológico, evidenciando, ainda, os motivos pelos quais a duração de projetos de RA em museus portugueses é tão curta. Estas perceções fornecem uma base para esforços futuros no aproveitamente da RA para melhorar as experiências museológicas, ao mesmo tempo que enfatizam a importância de uma compreensão abrangente dos desafios contextuais e das considerações envolvidas em tais implementações.Augmented Reality belongs to the technologies that embody Industry 4.0 and is used very frequently by organisations in different fields. In the international museum context, AR has been more widely applied, but in the specific case of Portugal, there is still a great lack of implementation of this technology. The main objective of this investigation was to determine what factores influence the success or failure of the implementation of Augmented Reality in museums. The investigation was based on a group of 5 Portuguese museums, under the supervision of the Direção Regional da Cultura do Norte, which were part of a project to implement Augmented Reality. In order to fulfil these objectives, the multiple case study methodology and the interview survey method were used. After collecting the data, they were processed and presented and a qualitative study was carried out using descriptive and comparative analysis, based on content analysis. The results showed that the factors that most influence the success or failure of implementing AR in museums are network technology, human resource training and the costs associated with AR. The conclusions of this study shed light on the complexity associated with integrating AR technology into the museum context, and also hilight the reasons why the duraction of the project in the different museums was so short. These insights provide a basis for future efforts to harness AR to enhance museum experiences, while emphasising the importance of a comprehensive understanding of the contextual challenges and considerations involved in such implementations

Machine learning y realidad aumentada para el reconocimiento de recursos turísticos

Apurímac, a pesar de contar con gran cantidad de recursos turísticos, no ha podido di-fundirlos de manera adecuada, por lo que en esta investigación se pretende aplicar ma-chine learning y realidad aumentada para la detección y geolocalización de recursos tu-rísticos. Construyendo una aplicación móvil que integre todas estas tecnologías y permi-ta mejorar la experiencia del visitante en tiempo real. Para lograr el objetivo, se conside-raron 25 recursos turísticos de la región, 5 para el entrenamiento del modelo machine learning y 20 para la ubicación en tiempo real por geolocalización. En cuanto a machine learning, se entrenó con un dataset construido exclusivamente para esta investigación, mediante YOLOv3 sobre Darknet, a continuación, el modelo entrenado se incluyó en un servidor web con Flask sobre Python, que estará a la espera de imágenes. Además, se implementó una aplicación web para la gestión de recursos turísticos que serán mostra-dos al usuario final. En lo referente a realidad aumentada esta se implementó sobre una aplicación móvil la cual envía imágenes captadas por la cámara del móvil al detector, esta app móvil también permite mostrar puntos de interés cercanos basado en la geoloca-lización y orientación actual; ya sean reconocidos o geolocalizados, la app permite mos-trar la información del recurso turístico mediante realidad aumentada. Como resultados se logró una precisión del modelo en el reconocimiento de imágenes superior al 90%, se logró determinar los puntos de interés turístico cercanos al móvil basándose en su geopo-sicionamiento y orientación, finalmente, se logró definir una arquitectura que intercomu-nique estos tres sistemas que trabajan con tecnologías diferentes.Tesi

Multi-target tracking using appearance models for identity maintenance

This thesis considers perception systems for urban environments. It focuses on the task of tracking dynamic objects and in particular on methods that can maintain the identities of targets through periods of ambiguity. Examples of such ambiguous situations occur when targets interact with each other, or when they are occluded by other objects or the environment. With the development of self driving cars, the push for autonomous delivery of packages, and an increasing use of technology for security, surveillance and public-safety applications, robust perception in crowded urban spaces is more important than ever before. A critical part of perception systems is the ability to understand the motion of objects in a scene. Tracking strategies that merge closely-spaced targets together into groups have been shown to offer improved robustness, but in doing so sacrifice the concept of target identity. Additionally, the primary sensor used for the tracking task may not provide the information required to reason about the identity of individual objects. There are three primary contributions in this work. The first is the development of 3D lidar tracking methods with improved ability to track closely-spaced targets and that can determine when target identities have become ambiguous. Secondly, this thesis defines appearance models suitable for the task of determining the identities of previously-observed targets, which may include the use of data from additional sensing modalities. The final contribution of this work is the combination of lidar tracking and appearance modelling, to enable the clarification of target identities in the presence of ambiguities caused by scene complexity. The algorithms presented in this work are validated on both carefully controlled and unconstrained datasets. The experiments show that in complex dynamic scenes with interacting targets, the proposed methods achieve significant improvements in tracking performance