Semantically enhancing multimedia lifelog events

Lifelogging is the digital recording of our everyday behaviour in order to identify human activities and build applications that support daily life. Lifelogs represent a unique form of personal multimedia content in that they are temporal, synchronised, multi-modal and composed of multiple media. Analysing lifelogs with a view to supporting content-based access, presents many challenges. These include the integration of heterogeneous input streams from different sensors, structuring a lifelog into events, representing events, and interpreting and understanding lifelogs. In this paper we demonstrate the potential of semantic web technologies for analysing lifelogs by automatically augmenting descriptions of lifelog events. We report on experiments and demonstrate how our re- sults yield rich descriptions of multi-modal, multimedia lifelog content, opening up even greater possibilities for managing and using lifelogs

IAPMA 2011: 2nd Workshop on information access to personal media archives

Towards e-Memories: challenges of capturing, summarising, presenting, understanding, using, and retrieving relevant information from heterogeneous data contained in personal media archives. Welcome to IAPMA 2011, the second international workshop on "Information Access for Personal Media Archives". It is now possible to archive much of our life experiences in digital form using a variety of sources, e.g. blogs written, tweets made, social network status updates, photographs taken, videos seen, music heard, physiological monitoring, locations visited and environmentally sensed data of those places, details of people met, etc. Information can be captured from a myriad of personal information devices including desktop computers, PDAs, digital cameras, video and audio recorders, and various sensors, including GPS, Bluetooth, and biometric devices

Report on the Workshop on Personal Knowledge Graphs (PKG 2021) at AKBC 2021

The term personal knowledge graph (PKG) has been broadly used to refer to structured representation of information about a given user, primarily in the form of entities that are personally related to the user. The potential of personal knowledge graphs as a means of managing and organizing personal data, as well as a source of background knowledge for personalizing downstream services, has recently gained increasing attention from researchers in multiple fields, including that of Information Retrieval, Natural Language Processing, and the Semantic Web. The goal of the PKG’21 workshop was to create a forum for researchers and practitioners from diverse areas to present and discuss methods, tools, techniques, and experiences related to the construction and use of personal knowledge graphs, identify open questions, and create a shared research agenda. It successfully brought about a diverse workshop program, comprising an invited keynote, paper presentations, and breakout discussions, as a half-day event at the 3rd Automated Knowledge Base Construction (AKBC’21) conference. The workshop demonstrated that while the concept and research field of personal knowledge graphs is still in its early stages, there are many promising avenues of future development and research that already, and independently, have attracted the interest of several different communities.publishedVersio

New Frontiers of Quantified Self 3: Exploring Understudied Categories of Users

Quantified Self (QS) field needs to start thinking of how situated needs may affect the use of self-tracking technologies. In this workshop we will focus on the idiosyncrasies of specific categories of users

A Semantic Web approach to ontology-based system: integrating, sharing and analysing IoT health and fitness data

With the rapid development of fitness industry, Internet of Things (IoT) technology is becoming one of the most popular trends for the health and fitness areas. IoT technologies have revolutionised the fitness and the sport industry by giving users the ability to monitor their health status and keep track of their training sessions. More and more sophisticated wearable devices, fitness trackers, smart watches and health mobile applications will appear in the near future. These systems do collect data non-stop from sensors and upload them to the Cloud. However, from a data-centric perspective the landscape of IoT fitness devices and wellness appliances is characterised by a plethora of representation and serialisation formats. The high heterogeneity of IoT data representations and the lack of common accepted standards, keep data isolated within each single system, preventing users and health professionals from having an integrated view of the various information collected. Moreover, in order to fully exploit the potential of the large amounts of data, it is also necessary to enable advanced analytics over it, thus achieving actionable knowledge. Therefore, due the above situation, the aim of this thesis project is to design and implement an ontology based system to (1) allow data interoperability among heterogeneous IoT fitness and wellness devices, (2) facilitate the integration and the sharing of information and (3) enable advanced analytics over the collected data (Cognitive Computing). The novelty of the proposed solution lies in exploiting Semantic Web technologies to formally describe the meaning of the data collected by the IoT devices and define a common communication strategy for information representation and exchange

Requirement Analysis and Implementation of Smart Emergency Medical Services

Emergency medical service (EMS) occurs in a high-pressure and error-prone environment, where paramedics must provide prompt decisions in care while recording information with limited time, incomplete data, restricted resources, and competing priorities. The EMS requires cooperative workflows between patients or caregivers, paramedics and medical centers in the community. In a conventional EMS, they have difficulties in obtaining causes of emergencies and personal medical histories, which are important for a rapid and proper response. We analyzed the requirement of a smart EMS (SEMS) system and derived the key components in connected care environments leveraging information and communication technology. A survey of paramedics (n=113) revealed that a SEMS system using IoT technology should integrate personal lifelogs, electronic medical records, and patient monitoring in ambulances into pre-hospital care recording systems. It also addressed context-awareness in the EMS accelerates first responder's activities, while supporting personalized care not only at the scene of the emergency but also during the entire hospital stay. Based on requirement analysis, we designed and implemented SEMS using health information standards to provide interoperability between devices and systems. As an application of SEMS, an example service is introduced: lifelog-connected EMS for stroke patients with a real-time location service for managing timeline of treatment.ope

LifeLogging: personal big data

We have recently observed a convergence of technologies to foster the emergence of lifelogging as a mainstream activity. Computer storage has become significantly cheaper, and advancements in sensing technology allows for the efficient sensing of personal activities, locations and the environment. This is best seen in the growing popularity of the quantified self movement, in which life activities are tracked using wearable sensors in the hope of better understanding human performance in a variety of tasks. This review aims to provide a comprehensive summary of lifelogging, to cover its research history, current technologies, and applications. Thus far, most of the lifelogging research has focused predominantly on visual lifelogging in order to capture life details of life activities, hence we maintain this focus in this review. However, we also reflect on the challenges lifelogging poses to an information retrieval scientist. This review is a suitable reference for those seeking a information retrieval scientist’s perspective on lifelogging and the quantified self

A Smartphone-based System for Personal Data Management and Personality Analysis

Abstract— The data from or about an individual, called personal data, is continuously increasing due to popularity of smart phones, wearables and other ubiquitous devices. Such personal data can be used to model a user and even digitally clone a person, e.g., Cyber-I (cyber individual) that aims at creating a unique and comprehensive description for every individual to support various personalized services and applications. Due to heterogeneity and sensitivity of personal data, one important issue is how to effectively collect and manage person data with sufficient security protection. Another important issue is how to figure out an individual’s character, i.e., personality from personal data. Therefore, this research is focused on personal data management and personality analysis in a smartphone based client-server system. The smartphone functions as not only a source of personal data but also a gateway to manage other wearables and communicate with a server that keeps personal data in a larger amount and a longer period. A multi-security mechanism is implemented to ensure data security in collection, transmission and storage. Personality analysis is made from data normalization, feature extraction and clustering, to personality computation based on sociological personality theories. Keywords—personal data; personality; system security; cluster algorithm; smartphone; mong

Recuperação e identificação de momentos em imagens

In our modern society almost anyone is able to capture moments and record events due to the ease accessibility to smartphones. This leads to the question, if we record so much of our life how can we easily retrieve specific moments? The answer to this question would open the door for a big leap in human life quality. The possibilities are endless, from trivial problems like finding a photo of a birthday cake to being capable of analyzing the progress of mental illnesses in patients or even tracking people with infectious diseases. With so much data being created everyday, the answer to this question becomes more complex. There is no stream lined approach to solve the problem of moment localization in a large dataset of images and investigations into this problem have only started a few years ago. ImageCLEF is one competition where researchers participate and try to achieve new and better results in the task of moment retrieval. This complex problem, along with the interest in participating in the ImageCLEF Lifelog Moment Retrieval Task posed a good challenge for the development of this dissertation. The proposed solution consists in developing a system capable of retriving images automatically according to specified moments described in a corpus of text without any sort of user interaction and using only state-of-the-art image and text processing methods. The developed retrieval system achieves this objective by extracting and categorizing relevant information from text while being able to compute a similarity score with the extracted labels from the image processing stage. In this way, the system is capable of telling if images are related to the specified moment in text and therefore able to retrieve the pictures accordingly. In the ImageCLEF Life Moment Retrieval 2020 subtask the proposed automatic retrieval system achieved a score of 0.03 in the F1-measure@10 evaluation methodology. Even though this scores are not competitve when compared to other teams systems scores, the built system presents a good baseline for future work.Na sociedade moderna, praticamente qualquer pessoa consegue capturar momentos e registar eventos devido à facilidade de acesso a smartphones. Isso leva à questão, se registamos tanto da nossa vida, como podemos facilmente recuperar momentos específicos? A resposta a esta questão abriria a porta para um grande salto na qualidade da vida humana. As possibilidades são infinitas, desde problemas triviais como encontrar a foto de um bolo de aniversário até ser capaz de analisar o progresso de doenças mentais em pacientes ou mesmo rastrear pessoas com doenças infecciosas. Com tantos dados a serem criados todos os dias, a resposta a esta pergunta torna-se mais complexa. Não existe uma abordagem linear para resolver o problema da localização de momentos num grande conjunto de imagens e investigações sobre este problema começaram há apenas poucos anos. O ImageCLEF é uma competição onde investigadores participam e tentam alcançar novos e melhores resultados na tarefa de recuperação de momentos a cada ano. Este problema complexo, em conjunto com o interesse em participar na tarefa ImageCLEF Lifelog Moment Retrieval, apresentam-se como um bom desafio para o desenvolvimento desta dissertação. A solução proposta consiste num sistema capaz de recuperar automaticamente imagens de momentos descritos em formato de texto, sem qualquer tipo de interação de um utilizador, utilizando apenas métodos estado da arte de processamento de imagem e texto. O sistema de recuperação desenvolvido alcança este objetivo através da extração e categorização de informação relevante de texto enquanto calcula um valor de similaridade com os rótulos extraídos durante a fase de processamento de imagem. Dessa forma, o sistema consegue dizer se as imagens estão relacionadas ao momento especificado no texto e, portanto, é capaz de recuperar as imagens de acordo. Na subtarefa ImageCLEF Life Moment Retrieval 2020, o sistema de recuperação automática de imagens proposto alcançou uma pontuação de 0.03 na metodologia de avaliação F1-measure@10. Mesmo que estas pontuações não sejam competitivas quando comparadas às pontuações de outros sistemas de outras equipas, o sistema construído apresenta-se como uma boa base para trabalhos futuros.Mestrado em Engenharia Eletrónica e Telecomunicaçõe