Stress detection in lifelog data for improved personalized lifelog retrieval system

Stress can be categorized into acute and chronic types, with acute stress having short-term positive effects in managing hazardous situations, while chronic stress can adversely impact mental health. In a biological context, stress elicits a physiological response indicative of the fight-or-flight mechanism, accompanied by measurable changes in physiological signals such as blood volume pulse (BVP), galvanic skin response (GSR), and skin temperature (TEMP). While clinical-grade devices have traditionally been used to measure these signals, recent advancements in sensor technology enable their capture using consumer-grade wearable devices, providing opportunities for research in acute stress detection. Despite these advancements, there has been limited focus on utilizing low-resolution data obtained from sensor technology for early stress detection and evaluating stress detection models under real-world conditions. Moreover, the potential of physiological signals to infer mental stress information remains largely unexplored in lifelog retrieval systems. This thesis addresses these gaps through empirical investigations and explores the potential of utilizing physiological signals for stress detection and their integration within the state-of-the-art (SOTA) lifelog retrieval system. The main contributions of this thesis are as follows. Firstly, statistical analyses are conducted to investigate the feasibility of using low-resolution data for stress detection and emphasize the superiority of subject-dependent models over subject-independent models, thereby proposing the optimal approach to training stress detection models with low-resolution data. Secondly, longitudinal stress lifelog data is collected to evaluate stress detection models in real-world settings. It is proposed that training lifelog models on physiological signals in real-world settings is crucial to avoid detection inaccuracies caused by differences between laboratory and free-living conditions. Finally, a state-of-the-art lifelog interactive retrieval system called \lifeseeker is developed, incorporating the stress-moment filter function. Experimental results demonstrate that integrating this function improves the overall performance of the system in both interactive and non-interactive modes. In summary, this thesis contributes to the understanding of stress detection applied in real-world settings and showcases the potential of integrating stress information for enhancing personalized lifelog retrieval system performance

Replay detection and multi-stream synchronization in CS:GO game streams using content-based Image retrieval and Image signature matching

In GameStory: The 2019 Video Game Analytics Challenge, two main tasks are nominated to solve in the challenge, which are replay detection - multi-stream synchronization, and game story summarization. In this paper, we propose a data-driven based approach to solve the first task: replay detection - multi-stream synchronization. Our solution aims to determine the replays which lie between two logo-transitional endpoints and synchronize them with their sources by extracting frames from videos, then applying image processing and retrieval remedies. In detail, we use the Bag of Visual Words approach to detect the logo-transitional endpoints, which contains multiple replays in between, then employ an Image Signature Matching algorithm for multi-stream synchronization and replay boundaries refinement. The best configuration of our proposed solution manages to achieve the second-highest scores in all evaluation metrics of the challenge

LIFER 2.0: discovering personal lifelog insights using an interactive lifelog retrieval system

This paper describes the participation of the Organiser Team in the ImageCLEFlifelog 2019 Solve My Life Puzzle (Puzzle) and Lifelog Moment Retrieval (LMRT) tasks. We proposed to use LIFER 2.0, an enhanced version of LIFER, which was an interactive retrieval system for personal lifelog data. We utilised LIFER 2.0 with some additional visual features, obtained by using traditional visual bag-of-words, to solve the Puzzle task, while with the LMRT, we applied LIFER 2.0 only with the provided information. The results on both tasks confirmed that by using faceted filter and context browsing, a user can gain insights from their personal lifelog by employing very simple interactions. These results also serve as baselines for other approaches in the ImageCLEFlifelog 2019 challenge to compare with

Dialogue-to-Video Retrieval

Recent years have witnessed an increasing amount of dialogue/conversation on the web especially on social media. That inspires the development of dialogue-based retrieval, in which retrieving videos based on dialogue is of increasing interest for recommendation systems. Different from other video retrieval tasks, dialogue-to-video retrieval uses structured queries in the form of user-generated dialogue as the search descriptor. We present a novel dialogue-to-video retrieval system, incorporating structured conversational information. Experiments conducted on the AVSD dataset show that our proposed approach using plain-text queries improves over the previous counterpart model by 15.8% on R@1. Furthermore, our approach using dialogue as a query, improves retrieval performance by 4.2%, 6.2%, 8.6% on R@1, R@5 and R@10 and outperforms the state-of-the-art model by 0.7%, 3.6% and 6.0% on R@1, R@5 and R@10 respectively

Analysing the performance of stress detection models on consumer-grade wearable devices

Identifying stress levels can provide valuable data for mental health analytics as well as labels for annotation systems. Although much research has been conducted into stress detection models using heart rate variability at a higher cost of data collection, there is a lack of research on the potential of using low-resolution Electrodermal Activity (EDA) signals from consumer-grade wearable devices to identify stress patterns. In this paper, we concentrate on performing statistical analyses on the stress detection capability of two popular approaches of training stress detection models with stress-related biometric signals: user-dependent and userindependent models. Our research manages to show that user-dependent models are statistically more accurate for stress detection. In terms of effectiveness assessment, the balanced accuracy (BA) metric is employed to evaluate the capability of distinguishing stress and non-stress conditions of the models trained on either low-resolution or high-resolution Electrodermal Activity (EDA) signals. The results from the experiment show that training the model with (comparatively lowcost) low-resolution EDA signal does not affect the stress detection accuracy of the model significantly compared to using a high-resolution EDA signal. Our research results demonstrate the potential of attaching the user-dependent stress detection model trained on personal low-resolution EDA signal recorded to collect data in daily life to provide users with personal stress level insight and analysis

Chemical constituents from the leaves of Styrax argentifolius H.L. Li and their antioxidative activity

Searching for bioactive agents from medicinal plants, the phytochemical investigation on the EtOAc extract of the Vietnamese Styrax argentifolius leaves has resulted in the isolation and structural determination of five compounds, including one nor-neolignan egonoic acid (1), one lignan (+)-pinoresinol (2), one sterol (20R)-3β-hydroxysitgmasta-5,22-dien-7-one (3), and two triterpenoids lupeol (4), and 2α,3α,24-trihydroxy-urs-12-en-28-oic acid (5). The chemical structures of these secondary metabolites were elucidated by NMR and MS spectral data. All isolated compounds were first observed in S. argentifolius species. Sterol 3 and triterpenoid 5 were detected in genus Styrax for the first time. With the IC50 value of 19.10 µg/mL, compound 2 possessed the strong activity in DPPH radical scavenging assay

DCU team at the NTCIR-15 micro-activity retrieval task

The growing attention to lifelogging research has led to the creation of many retrieval systems, most of which employed event segmentation as core functionality. While previous literature focused on splitting lifelog data into broad segments of daily living activities, less attention was paid to micro-activities which last for short periods of time, yet carry valuable information for building a high-precision retrieval engine. In this paper, we present our efforts in addressing the NTCIR-15 MART challenge, in which the participants were asked to retrieve micro-activities from a multi-modal dataset. We proposed five models which investigate imagery and sensory data, both jointly and separately using various Deep Learn- ing and Machine Learning techniques, and achieved a maximum mAP score of 0.901 using an Image Tabular Pair-wise Similarity model, and overall ranked second in the competition. Our model not only captures the information coming from the temporal visual data combined with sensor signal, but also works as a Siamese network to discriminate micro-activities

Cytotoxic naphthoquinones from Diospyros fleuryana leaves

In the search for anti-cancer plants in Vietnam, the leaves of Diospyros fleuryana were selected for chemical investigation. Phytochemical analysis of the ethyl acetate (EtOAc) extract led to the isolation of two naphthoquinones isodiospyrin (1), and 8'-hydroxyisodiospyrin (2), and one isoflavone 7-O-methylbiochanin A (3). The chemical structures of isolated compounds were determined by 1D-NMR (1H, and 13C-NMR), 2D-NMR spectra (HSQC, and HMBC), and MS spectroscopy. Compound 3 was isolated from genus Diospyros for the first time. Regarding the strong IC50 values of 2.27, and 8.0 µM against KB, and Hep cell lines respectively, cytotoxic examination suggested that compound 2 is a promising agent in anti-cancer treatment.Â

chemical constituents from methanolic extract of Garcinia mackeaniana leaves and their antioxidant activity

A phytochemical investigation of the methanolic extract of Garcinia mackeaniana leaves led to the isolation, and determination of five secondary metabolites, including one benzophenone 4,3',4'-trihydroxy-2,6-dimethoxybenzophenone (1), two flavone C-glucosides vitexin (2) and its 2''-O-acetyl derivative (3), one biflavone amentoflavone (4), and one mono-phenol methyl protocatechuate (5). The chemical structures of these compounds were characterized by the NMR-spectroscopic method. These isolated compounds were isolated from G. mackeaniana species for the first time. Benzophenone derivative 1 has shown to be associated with a significant IC50 value of 14.97±0.8 µg/mL in the DPPH-antioxidant assay

LifeSeeker 3.0 : an interactive lifelog search engine for LSC’21

In this paper, we present the interactive lifelog retrieval engine developed for the LSC’21 comparative benchmarking challenge. The LifeSeeker 3.0 interactive lifelog retrieval engine is an enhanced version of our previous system participating in LSC’20 - LifeSeeker 2.0. The system is developed by both Dublin City University and the Ho Chi Minh City University of Science. The implementation of LifeSeeker 3.0 focuses on searching and filtering by text query using a weighted Bag-of-Words model with visual concept augmentation and three weighted vocabularies. The visual similarity search is improved using a bag of local convolutional features; while improving the previous version’s performance, enhancing query processing time, result displaying, and browsing support