Exploring the Touch and Motion Features in Game-Based Cognitive Assessments

Early detection of cognitive decline is important for timely intervention and treatment strategies to prevent further deterioration or development of more severe cognitive impairment, as well as identify at risk individuals for research. In this paper, we explore the feasibility of using data collected from built-in sensors of mobile phone and gameplay performance in mobile-game-based cognitive assessments. Twenty-two healthy participants took part in the two-session experiment where they were asked to take a series of standard cognitive assessments followed by playing three popular mobile games in which user-game interaction data were passively collected. The results from bivariate analysis reveal correlations between our proposed features and scores obtained from paper-based cognitive assessments. Our results show that touch gestural interaction and device motion patterns can be used as supplementary features on mobile game-based cognitive measurement. This study provides initial evidence that game related metrics on existing off-the-shelf games have potential to be used as proxies for conventional cognitive measures, specifically for visuospatial function, visual search capability, mental flexibility, memory and attention

Audio-visual football video analysis, from structure detection to attention analysis

Sport video is an important video genre. Content-based sports video analysis attracts great interest from both industry and academic fields. A sports video is characterised by repetitive temporal structures, relatively plain contents, and strong spatio-temporal variations, such as quick camera switches and swift local motions. It is necessary to develop specific techniques for content-based sports video analysis to utilise these characteristics. For an efficient and effective sports video analysis system, there are three fundamental questions: (1) what are key stories for sports videos; (2) what incurs viewer’s interest; and (3) how to identify game highlights. This thesis is developed around these questions. We approached these questions from two different perspectives and in turn three research contributions are presented, namely, replay detection, attack temporal structure decomposition, and attention-based highlight identification. Replay segments convey the most important contents in sports videos. It is an efficient approach to collect game highlights by detecting replay segments. However, replay is an artefact of editing, which improves with advances in video editing tools. The composition of replay is complex, which includes logo transitions, slow motions, viewpoint switches and normal speed video clips. Since logo transition clips are pervasive in game collections of FIFA World Cup 2002, FIFA World Cup 2006 and UEFA Championship 2006, we take logo transition detection as an effective replacement of replay detection. A two-pass system was developed, including a five-layer adaboost classifier and a logo template matching throughout an entire video. The five-layer adaboost utilises shot duration, average game pitch ratio, average motion, sequential colour histogram and shot frequency between two neighbouring logo transitions, to filter out logo transition candidates. Subsequently, a logo template is constructed and employed to find all transition logo sequences. The precision and recall of this system in replay detection is 100% in a five-game evaluation collection. An attack structure is a team competition for a score. Hence, this structure is a conceptually fundamental unit of a football video as well as other sports videos. We review the literature of content-based temporal structures, such as play-break structure, and develop a three-step system for automatic attack structure decomposition. Four content-based shot classes, namely, play, focus, replay and break were identified by low level visual features. A four-state hidden Markov model was trained to simulate transition processes among these shot classes. Since attack structures are the longest repetitive temporal unit in a sports video, a suffix tree is proposed to find the longest repetitive substring in the label sequence of shot class transitions. These occurrences of this substring are regarded as a kernel of an attack hidden Markov process. Therefore, the decomposition of attack structure becomes a boundary likelihood comparison between two Markov chains. Highlights are what attract notice. Attention is a psychological measurement of “notice ”. A brief survey of attention psychological background, attention estimation from vision and auditory, and multiple modality attention fusion is presented. We propose two attention models for sports video analysis, namely, the role-based attention model and the multiresolution autoregressive framework. The role-based attention model is based on the perception structure during watching video. This model removes reflection bias among modality salient signals and combines these signals by reflectors. The multiresolution autoregressive framework (MAR) treats salient signals as a group of smooth random processes, which follow a similar trend but are filled with noise. This framework tries to estimate a noise-less signal from these coarse noisy observations by a multiple resolution analysis. Related algorithms are developed, such as event segmentation on a MAR tree and real time event detection. The experiment shows that these attention-based approach can find goal events at a high precision. Moreover, results of MAR-based highlight detection on the final game of FIFA 2002 and 2006 are highly similar to professionally labelled highlights by BBC and FIFA

Combining local features and region segmentation: methods and applications

Tesis doctoral inédita leída en la Universidad Autónoma de Madrid, Escuela Politécnica Superior, Departamento de Tecnología Electrónica y de las Comunicaciones. Fecha de lectura: 23-01-2020Esta tesis tiene embargado el acceso al texto completo hasta el 23-07-2021Muchas y muy diferentes son las propuestas que se han desarrollado en el área de la visión artificial para la extracción de información de las imágenes y su posterior uso. Entra las más destacadas se encuentran las conocidas como características locales, del inglés local features, que detectan puntos o áreas de la imagen con ciertas características de interés, y las describen usando información de su entorno (local). También destacan las regiones en este área, y en especial este trabajo se ha centrado en los segmentadores en regiones, cuyo objetivo es agrupar la información de la imagen atendiendo a diversos criterios. Pese al enorme potencial de estas técnicas, y su probado éxito en diversas aplicaciones, su definición lleva implícita una serie de limitaciones funcionales que les han impedido exportar sus capacidades a otras áreas de aplicación. Se pretende impulsar el uso de estas herramientas en dichas aplicaciones, y por tanto mejorar los resultados del estado del arte, mediante la propuesta de un marco de desarrollo de nuevas soluciones. En concreto, la hipótesis principal del proyecto es que las capacidades de las características locales y los segmentadores en regiones son complementarias, y que su combinación, realizada de la forma adecuada, las maximiza a la vez que minimiza sus limitaciones. El principal objetivo, y por tanto la principal contribución del proyecto, es validar dicha hipótesis mediante la propuesta de un marco de desarrollo de nuevas soluciones combinando características locales y segmentadores para técnicas con capacidades mejoradas. Al tratarse de un marco de combinación de dos técnicas, el proceso de validación se ha llevado a cabo en dos pasos. En primer lugar se ha planteado el caso del uso de segmentadores en regiones para mejorar las características locales. Para verificar la viabilidad y el éxito de esta combinación se ha desarrollado una propuesta específica, SP-SIFT, que se ha validado tanto a nivel experimental como a nivel de aplicación real, en concreto como técnica principal de algoritmos de seguimiento de objetos. En segundo lugar, se ha planteado el caso de uso de características locales para mejorar los segmentadores en regiones. Para verificar la viabilidad y el éxito de esta combinación se ha desarrollado una propuesta específica, LF-SLIC, que se ha validado tanto a nivel experimental como a nivel de aplicación real, en concreto como técnica principal de un algoritmo de segmentación de lesiones pigmentadas de la piel. Los resultados conceptuales han probado que las técnicas mejoran a nivel de capacidades. Los resultados aplicados han probado que estas mejoras permiten el uso de estas técnicas en aplicaciones donde antes no tenían éxito. Con ello, se ha considerado la hipótesis validada, y por tanto exitosa la definición de un marco para el desarrollo de nuevas técnicas específicas con capacidades mejoradas. En conclusión, la principal aportación de la tesis es el marco de combinación de técnicas, plasmada en sus dos propuestas específicas: características locales mejoradas con segmentadores y segmentadores mejorados con características locales, y en el éxito conseguido en sus aplicaciones.A huge number of proposals have been developed in the area of computer vision for information extraction from images, and its further use. One of the most prevalent solutions are those known as local features. They detect points or areas of the image with certain characteristics of interest, and describe them using information from their (local) environment. The regions also stand out in the area, and especially this work has focused on the region segmentation algorithms, whose objective is to group the information of the image according to di erent criteria. Despite the enormous potential of these techniques, and their proven success in a number of applications, their de nition implies a series of functional limitations that have prevented them from exporting their capabilities to other application areas. In this thesis, it is intended to promote the use of these tools in these applications, and therefore improve the results of the state of the art, by proposing a framework for developing new solutions. Speci cally, the main hypothesis of the project is that the capacities of the local features and the region segmentation algorithms are complementary, and thus their combination, carried out in the right way, maximizes them while minimizing their limitations. The main objective, and therefore the main contribution of the thesis, is to validate this hypothesis by proposing a framework for developing new solutions combining local features and region segmentation algorithms, obtaining solutions with improved capabilities. As the hypothesis is proposing to combine two techniques, the validation process has been carried out in two steps. First, the use case of region segmentation algorithms enhancing local features. In order to verify the viability and success of this combination, a speci c proposal, SP-SIFT, was been developed. This proposal was validated both experimentally and in a real application scenario, speci cally as the main technique of object tracking algorithms. Second, the use case of enhancing region segmentation algorithm with local features. In order to verify the viability and success of this combination, a speci c proposal, LF-SLIC, was developed. The proposal was validated both experimentally and in a real application scenario, speci cally as the main technique of a pigmented skin lesions segmentation algorithm. The conceptual results proved that the techniques improve at the capabilities level. The application results proved that these improvements allow the use of this techniques in applications where they were previously unsuccessful. Thus, the hypothesis can be considered validated, and therefore the de nition of a framework for the development of new techniques with improved capabilities can be considered successful. In conclusion, the main contribution of the thesis is the framework for the combination of techniques, embodied in the two speci c proposals: enhanced local features with region segmentation algorithms, and region segmentation algorithms enhanced with local features; and in the success achieved in their applications.The work described in this Thesis was carried out within the Video Processing and Understanding Lab at the Department of Tecnología Electrónica y de las Comunicaciones, Escuela Politécnica Superior, Universidad Autónoma de Madrid (from 2014 to 2019). It was partially supported by the Spanish Government (TEC2014-53176-R, HAVideo)

Analyzing the Strengths, Weaknesses, Opportunities, and Threats of AI in Libraries

This article provides a comprehensive analysis of the strengths, weaknesses, opportunities, and threats (SWOT) associated with the integration of Artificial Intelligence (AI) in libraries. AI has the potential to transform library and information science, revolutionizing processes, and services. The strengths of AI in libraries include efficient information retrieval and management, enhanced user experiences through personalization, automation of routine tasks, and improved decision-making through data analysis. However, the weaknesses of AI in libraries encompass ethical considerations and biases, the potential lack of human touch and personalized assistance, technical challenges, and concerns about job displacement. The article also explores the opportunities presented by AI, such as advanced search capabilities, expanded accessibility of digital collections, support for diverse user needs, and collaboration among libraries. On the other hand, the threats and challenges of AI in libraries involve privacy and security risks, dependence on technology and potential system failures, user acceptance and trust issues, and the impact on traditional library services and roles. By considering these factors, libraries can make informed decisions and strategically implement AI to maximize its benefits while addressing the associated challenges. The findings of this analysis emphasize the importance of thoughtful implementation and human-AI collaboration to ensure the best outcomes for library users and stakeholders in the future

空間的なテクスチャ解析によるコンプレックスネットワークに基づくテクスチャ解析の改善

This thesis proposes a new texture analysis model which enhanced from traditional complex network-based model for texture characterization via spatial texture analysis. The conceptual framework of the proposed model is to synergize between pattern recognition and graph theory research areas. The results of experiment show that the proposed model can capture robust textural information under various uncontrolled environments using standard texture databases. Texture analysis has played an important role in the last few decades. There are a growing number of techniques described in the literature, one of new area research is a complex network for texture characterization, which has developed in recent years. Inspired by the human brain system, the relation among structure texture elements on an image can be derived using the complex network model. Compared to the task of texture classification, development of the original complex network model is required in order to improve classification performance in environment variations. To fulfill this requirement, the enhancing complex network by spatial texture analysis (i.e., spatial distribution and spatial relation) has been achieved in this thesis. The proposed approach addresses the above requirement by investigating and modifying the original complex network model by extracting more discriminative information. A new graph connectivity measurement has been devised, including local spatial pattern mapping, which is denoted as a LSPM, to encode and describe local spatial arrangement of pixels. To the best of the author\u27s knowledge, as investigated in this thesis, the encoding spatial information which has been adapted within the original complex network model presented here were first proposed and reported by the author. The essence of this proposed graph connectivity measurement describes the spatial structure of local image texture cause it can effectively capture and detect micro-structures (e.g., edges, lines, spots) information which is critical being used to distinguish various pattern structures and invariant uncontrolled environments. Moreover, the graph-based representation has been investigated for improving the performance of texture classification. Spatial vector property has been comprised of deterministic graph modeling which decomposing the two component of the magnitude and the direction. Then, the proposed hybrid-based complex network comprises the enhancing graph-based representation, and the new graph connectivity measurement has been devised as an enhancing complex network-based model for texture characterization in this thesis. The experiments are evaluated by using four standard texture databases include Brodatz, UIUC, KTH-TIPS, and UMD. The experimental results are presented in terms of classification rate in this thesis to demonstrate that: firstly, the proposed graph connectivity measurement (LSPM) approach achieved on-average 86.25%, 77.25%, 89.38% and 94.06% respectively based on four databases. Secondly, the proposed graph-based spatial property approach achieved on-average 90.92%, 87.92%, 96.56% and 92.65%, respectively; finally, the hybrid-based complex network model achieved on-average 88.92%, 85.46%, 95.14% and 95.52% respectively. Accordingly, this thesis has advanced the original complex network-based model for texture characterization.九州工業大学博士学位論文 学位記番号：生工博甲第329号 学位授与年月日：平成30年9月21日1 Introduction|2 Literature Review|3 Complex Network Model and Spatial Information|4 Graph-based Representation in Texture Analysis|5 Hybrid-based Complex Network Model|6 Conclusions九州工業大学平成30年