Evaluating the impact of physical activity apps and wearables: interdisciplinary review

Background: Although many smartphone apps and wearables have been designed to improve physical activity, their rapidly evolving nature and complexity present challenges for evaluating their impact. Traditional methodologies, such as randomized controlled trials (RCTs), can be slow. To keep pace with rapid technological development, evaluations of mobile health technologies must be efficient. Rapid alternative research designs have been proposed, and efficient in-app data collection methods, including in-device sensors and device-generated logs, are available. Along with effectiveness, it is important to measure engagement (ie, users’ interaction and usage behavior) and acceptability (ie, users’ subjective perceptions and experiences) to help explain how and why apps and wearables work. Objectives: This study aimed to (1) explore the extent to which evaluations of physical activity apps and wearables: employ rapid research designs; assess engagement, acceptability, as well as effectiveness; use efficient data collection methods; and (2) describe which dimensions of engagement and acceptability are assessed. Method: An interdisciplinary scoping review using 8 databases from health and computing sciences. Included studies measured physical activity, and evaluated physical activity apps or wearables that provided sensor-based feedback. Results were analyzed using descriptive numerical summaries, chi-square testing, and qualitative thematic analysis. Results: A total of 1829 abstracts were screened, and 858 articles read in full. Of 111 included studies, 61 (55.0%) were published between 2015 and 2017. Most (55.0%, 61/111) were RCTs, and only 2 studies (1.8%) used rapid research designs: 1 single-case design and 1 multiphase optimization strategy. Other research designs included 23 (22.5%) repeated measures designs, 11 (9.9%) nonrandomized group designs, 10 (9.0%) case studies, and 4 (3.6%) observational studies. Less than one-third of the studies (32.0%, 35/111) investigated effectiveness, engagement, and acceptability together. To measure physical activity, most studies (90.1%, 101/111) employed sensors (either in-device [67.6%, 75/111] or external [23.4%, 26/111]). RCTs were more likely to employ external sensors (accelerometers: P=.005). Studies that assessed engagement (52.3%, 58/111) mostly used device-generated logs (91%, 53/58) to measure the frequency, depth, and length of engagement. Studies that assessed acceptability (57.7%, 64/111) most often used questionnaires (64%, 42/64) and/or qualitative methods (53%, 34/64) to explore appreciation, perceived effectiveness and usefulness, satisfaction, intention to continue use, and social acceptability. Some studies (14.4%, 16/111) assessed dimensions more closely related to usability (ie, burden of sensor wear and use, interface complexity, and perceived technical performance). Conclusions: The rapid increase of research into the impact of physical activity apps and wearables means that evaluation guidelines are urgently needed to promote efficiency through the use of rapid research designs, in-device sensors and user-logs to assess effectiveness, engagement, and acceptability. Screening articles was time-consuming because reporting across health and computing sciences lacked standardization. Reporting guidelines are therefore needed to facilitate the synthesis of evidence across disciplines

CHORUS Deliverable 2.1: State of the Art on Multimedia Search Engines

Based on the information provided by European projects and national initiatives related to multimedia search as well as domains experts that participated in the CHORUS Think-thanks and workshops, this document reports on the state of the art related to multimedia content search from, a technical, and socio-economic perspective. The technical perspective includes an up to date view on content based indexing and retrieval technologies, multimedia search in the context of mobile devices and peer-to-peer networks, and an overview of current evaluation and benchmark inititiatives to measure the performance of multimedia search engines. From a socio-economic perspective we inventorize the impact and legal consequences of these technical advances and point out future directions of research

Impact of video summary viewing on episodic memory recall:design guidelines for video summarizations

Reviewing lifelogging data has been proposed as a useful tool to support human memory. However, the sheer volume of data (particularly images) that can be captured by modern lifelogging systems makes the selection and presentation of material for review a challenging task. We present the results of a five-week user study involving 16 participants and over 69,000 images that explores both individual requirements for video summaries and the differences in cognitive load, user experience, memory experience, and recall experience between review using video summarisations and non-summary review techniques. Our results can be used to inform the design of future lifelogging data summarisation systems for memory augmentation

Personalizable edge services for Web accessibility

Web Content Accessibility guidelines by W3C (W3C Recommendation, May 1999. http://www.w3.org/TR/WCAG10/) provide several suggestions for Web designers regarding how to author Web pages in order to make them accessible to everyone. In this context, this paper proposes the use of edge services as an efficient and general solution to promote accessibility and breaking down the digital barriers that inhibit users with disabilities to actively participate to any aspect of society. The idea behind edge services mainly affect the advantages of a personalized navigation in which contents are tailored according to different issues, such as client’s devices capabilities, communication systems and network conditions and, finally, preferences and/or abilities of the growing number of users that access the Web. To meet these requirements, Web designers have to efficiently provide content adaptation and personalization functionalities mechanisms in order to guarantee universal access to the Internet content. The so far dominant paradigm of communication on theWWW, due to its simple request/responsemodel, cannot efficiently address such requirements. Therefore, it must be augmented with new components that attempt to enhance the scalability, the performances and the ubiquity of the Web. Edge servers, acting on the HTTP data flow exchanged between client and server, allow on-the-fly content adaptation as well as other complex functionalities beyond the traditional caching and content replication services. These value-added services are called edge services and include personalization and customization, aggregation from multiple sources, geographical personalization of the navigation of pages (with insertion/emphasis of content that can be related to the user’s geographical location), translation services, group navigation and awareness for social navigation, advanced services for bandwidth optimization such as adaptive compression and format transcoding, mobility, and ubiquitous access to Internet content. This paper presents Personalizable Accessible Navigation (PAN) that is a set of edge services designed to improveWeb pages accessibility, developed and deployed on top of a programmable intermediary framework. The characteristics and the location of the services, i.e., provided by intermediaries, as well as the personalization and the opportunities to select multiple profiles make PAN a platform that is especially suitable for accessing the Web seamlessly also from mobile terminals

Photo Wallet : interface design for simple mobile photo albums

Tese de mestrado. Multimédia (Perfil Tecnologias). Universidade do Porto. Faculdade de Engenharia. 201

Providing effective memory retrieval cues through automatic structuring and augmentation of a lifelog of images

Lifelogging is an area of research which is concerned with the capture of many aspects of an individual's life digitally, and within this rapidly emerging field is the significant challenge of managing images passively captured by an individual of their daily life. Possible applications vary from helping those with neurodegenerative conditions recall events from memory, to the maintenance and augmentation of extensive image collections of a tourist's trips. However, a large lifelog of images can quickly amass, with an average of 700,000 images captured each year, using a device such as the SenseCam. We address the problem of managing this vast collection of personal images by investigating automatic techniques that: 1. Identify distinct events within a full day of lifelog images (which typically consists of 2,000 images) e.g. breakfast, working on PC, meeting, etc. 2. Find similar events to a given event in a person's lifelog e.g. "show me other events where I was in the park" 3. Determine those events that are more important or unusual to the user and also select a relevant keyframe image for visual display of an event e.g. a "meeting" is more interesting to review than "working on PC" 4. Augment the images from a wearable camera with higher quality images from external "Web 2.0" sources e.g. find me pictures taken by others of the U2 concert in Croke Park In this dissertation we discuss novel techniques to realise each of these facets and how effective they are. The significance of this work is not only of benefit to the lifelogging community, but also to cognitive psychology researchers studying the potential benefits of lifelogging devices to those with neurodegenerative diseases

헬스케어 서비스에서 데이터 기반 커뮤니케이션을 위한 디자인 연구

학위논문(박사)--서울대학교 대학원 :융합과학기술대학원 융합과학부(디지털정보융합전공),2020. 2. 이중식.스마트폰과 웨어러블 기기의 보급으로 인해 환자 생성 건강 데이터(Patient-Generated Health Data; PGHD)가 크게 증가하였고, 이는 의사-환자 의사 소통을 개선하여 데이터 중심으로 발전 할 수있는 새로운 기회를 제공했다. PGHD를 사용한 데이터 중심 커뮤니케이션을 통해 환자와 의사는 기존 임상 데이터를 보완하여 이해의 차이를 메울 수 있으며, 환자 건강에 대한 포괄적인 관점도 획득할 수 있다. 그러나, 이러한 새로운 유형의 데이터와 기술을 기존 의료 커뮤니케이션에 통합하는 데에는 여전히 어려움이 남아 있다. 환자는 종종 데이터 수집에 대한 참여와 동기를 잃어버리며, 이에 따라 수집한 데이터는 불완전해지는 문제가 발생한다. 또한 PGHD가 온전하게 수집 되더라도 의사와 환자는 의료 관행에서 이러한 데이터를 활용하는 데 어려움을 겪게 된다. 또한, 시간과 정보의 부족으로 인해 현재 워크 플로우에서 환자와 의사 모두가 PGHD를 통해 협업하는 것은 매우 어려운 일로 알려져 있다. HCI 연구 관점에서, PGHD를 활용 한 데이터 중심 통신을 지원하는 시스템을 설계하면 이러한 과제를 해결할 수 있는 잠재력이 있으며, 이는 데이터 수집(collection), 표현(representation), 해석(interpretation) 및 협업(collaboration)의 네 가지 설계 공간(design space)에서 추가적인 탐색을 요구한다. 따라서, 이 논문에서는 시스템 설계 및 현장 배포 연구를 수행하여, 각 설계 공간에서 해결되지 않은 질문을 탐색하고 경험적 연구 결과 및 설계 지침을 제공하는 것을 목표로 한다. 먼저, 데이터 수집에 대한 설계 공간의 연구로서, 접근성 높은 데이터 추적 도구가 환자가 다양한 유형의 PGHD, 특히 식사 데이터를 수집하는 데 어떤 도움을 줄 수 있는지에 대해 연구하고자 하였다. 이를 위해, 접근성 높은 데이터 추적 도구인 mFood Logger을 디자인한 후, 20 명의 환자와 6 명의 임상의를 대상으로 실증적 연구를 수행했다. 그 결과, 환자와 임상의가 데이터 기반 커뮤니케이션을 위해 원하는 데이터 유형이 무엇인지 파악할 수 있었고, 임상적 맥락에서 데이터를 수집 할 때의 난점과 기회를 발견했다. 둘째, 임상의를 위한 데이터 표현을 파악하기 위해, 18명의 다양한 이해 관계자(e.g., 임상의, EMR 개발자)와 참여적 디자인(participatory design) 프로세스를 통해 PGHD를 표시하는 DataMD를 설계하고 구현했다. 참여적 디자인 워크샵을 통해 알아낸 것은, 의료적 상황의 제약 때문에 임상의가 원하는 데이터 표현 방식이 효율성과 친숙함으로 수렴된다는 점이었다. 임상의는 학습에 걸리는 시간 문제로 인해 새로운 시각화 방법을 사용하지 않았고, 한 번에 많은 양의 데이터를 보고 싶어했다. 이러한 요구 사항을 고려하여, 다양한 유형의 PGHD가 한 눈에 보여지며, 여러 가지 임상 상황을 고려한, DataMD를 설계하고 구현했다. 셋째, 데이터 기반 커뮤니케이션의 중요한 측면으로서, 환자를 위한 데이터 해석 전략을 제시하여 효과적인 데이터 해석을 돕는 설계 지침을 제공합니다. 20명의 만성 질환 환자와의 인터뷰를 통해, 환자들이 PGHD를 해석할 때, 논리적 증거가 아닌 자신의 과거 경험에 강하게 의존한다는 점을 밝혀냈다. 환자들은 자신의 신념과 경험에 따라 여러 데이터 사이의 관계를 가정하며, 이를 확인하기 위해 네 가지의 데이터 해석 전략을 구사했다. 이러한 이해는 설계자와 연구원이 데이터 해석을 지원하는 시스템 설계를 발전시키는 데 도움이 될 수 있다. 마지막으로, 데이터를 통한 협업을 지원하기 위해 앞선 연구에서 디자인한 시스템을 기반으로 PGHD를 공유하고 활용함으로써, 임상의와 환자가 어떻게 협업하는지를 조사하고자 했다. 환자의 데이터 수집 및 해석을 돕는 앱인 MyHealthKeeper와 임상의를 위한 인터페이스인 DataMD로 구성된 통합 시스템을 임상 현장에 배포했다. 80명의 외래환자와의 임상시험 결과에 따르면 PGHD를 통한 협력으로 환자가 행동 변화에 성공할 수있었다. 또한, 앱 사용 로그에 따르면 환자는 직접적인 상호 작용 없이도 임상의와 원격으로 협업 할 수도 있는 것으로 나타났다. 이러한 결과를 바탕으로, 이 연구에서는 임상의와 환자 사이의 협력을 지원할 수있는 주요 기회가 기존 임상 워크플로우에 PGHD 사용을 통합하는 것에 있음을 제시한다. 앞선 연구들을 통해, 데이터 기반 커뮤니케이션을 위한 디자인이 환자와 의사가 PGHD를 통해 협업하는 데 도움이 될 수 있음을 발견했다. PGHD가 네 개의 설계 공간 내에서 기존 의사-환자 통신을 데이터 중심 통신으로 개선 할 수있는 방법을 개념화함으로써, 이 연구는 환자와 의사 간의 데이터 기반 커뮤니케이션을 위한 디자인이 어떻게 도출되어야 하는지에 대한 새로운 시각을 제공할 것으로 기대한다. 이 작업은 HCI, CSCW과 건강 정보학 커뮤니티의 경험적 이해를 높이고, 실용적인 설계 지침을 제공하며, 이론적 확장에 기여한다. 또한, 이 연구는 향후 다른 분야에서 데이터 기반 커뮤니케이션을 지원하는 시스템의 설계가 어떻게 이뤄져야 하는지에 대한 기초를 제공한다.The prevalence of smartphones and wearable devices has led to a dramatic increase in patient-generated health data (PGHD). The growing interest in PGHD has offered new opportunities to improve doctor-patient communication to become more data-driven. Data-driven communication using PGHD enables patients and physicians to fill in gaps between understandings by supplementing existing clinical data, as well as providing a more comprehensive picture of ongoing patient health. However, challenges in integrating such new types of data and technologies into existing healthcare communications remain. Patients often lose their engagement and motivation in data collection, resulting in incomplete data. Even if PGHD is wholly collected, physicians and patients encounter challenges in utilizing such data--representation and interpretation--in healthcare practices. Furthermore, it is challenging for both patients and physicians to collaborate through PGHD in the current workflow due to the lack of time and information overload. From the HCI research perspective, designing a system supporting data-driven communication utilizing PGHD has the potential to address such challenges, which calls for further exploration in four design spaces: data collection, representation, interpretation, and collaboration. Therefore, in this dissertation work, I aim to explore unsolved questions in each design space by conducting a series of design and deployment studies and provide empirical findings and design guidelines. In the design space of data collection, I investigated how the semi-automated tracking tool can support patients to track various types of PGHD, especially food journaling. With the design of mFood Logger, a semi-automated data tracking tool, I conducted an empirical study with 20 patients and 6 clinicians. I identified desired data types for data-driven communication from the patients' and clinicians' sides and uncovered the challenges and opportunities in collecting data within clinical contexts. I was able to understand the feasibility and acceptability of PGHD in clinical practices, as well as clinicians' presence--either remotely or in-person--as an enabler that encourages patients to keep tracking PGHD in the longer-term. Incorporating critical topics regarding data collection from the literature and findings from my work, I discuss the applicability of PGHD and data tracking modes. To support data representation for clinicians, I designed and implemented DataMD that displays PGHD, considering situational constraints through a participatory design process with 18 various stakeholders (e.g., clinicians, EMR developers). Through the participatory design workshop, I found that the ways of data representation that clinicians desired converged to efficiency and familiarity due to the situational constraints. Clinicians wanted to see a large amount of data at once, avoiding using novel visualization methods due to the issue of learnability. Considering those requirements, I designed and implemented DataMD, in which various types of PGHD are represented with considerations of clinical contexts. I discussed the role of data representation in data-driven communication. As the critical aspect of data-driven communication, I present different data-interpretation strategies from patients, providing design guidelines to help effective data-interpretation. By conducting interviews with 20 chronic disease patients, I found that they shaped their interests and assumptions by incorporating prior experiences rather than logical evidence. I also identified four data-interpretation strategies: finding evidence to confirm assumptions, discrediting data to preserve initial assumptions, discovering new insights, and deferring drawing hasty conclusions from data. These understandings help designers and researchers advance the design of systems to support data-interpretation. Lastly, to support collaboration via data, I demonstrate how clinicians and patients collaborate by sharing and utilizing PGHD based on the system I designed. I deployed the integrated system consisting of a patient app, MyHealthKeeper, and a clinician interface, DataMD. I investigated how the system could support collaboration via data. Clinical outcomes revealed that collaboration via PGHD led patients to succeed in behavior change. App usage log also showed that patients could even remotely collaborate with clinicians without direct interactions. Findings from these studies indicate that the key opportunities to facilitate collaboration between clinicians and patients are the integration of data prescriptions into the clinician's workflow and intervention based on natural language feedback generated within clinical contexts. Across these studies, I found that the design for data-driven communication can support patients and physicians to collaborate through PGHD. By conceptualizing how PGHD could improve the existing doctor-patient communication to data-driven communication within four design spaces, I expect that this work will shed new light on how the design should be derived for data-driven communication between patients and physicians in the real world. Taken together, I believe this work contributes to empirical understandings, design guidelines, theoretical extensions, and artifacts in human-computer interaction, computer-supported cooperative work, and health informatics communities. This work also provides a foundation for future researchers to study how the design of the system supporting data-driven communication can empower various users situated in different contexts to communicate through data in other domains, such as learning, beyond the context of healthcare services.1 Introduction 1 1.1 Background 1 1.2 Motivation 4 1.3 Topics of Interest 5 1.3.1 Design Spaces 5 1.3.2 Research Scope 11 1.4 Thesis Statements and Research Questions 13 1.5 Thesis Overview 15 1.6 Contribution 18 1.6.1 Empirical research contributions 18 1.6.2 Artifacts contributions 18 1.6.3 Theoretical contributions 19 2 Conceptual Background & Related Work 20 2.1 Data-driven Communication in Healthcare Services 20 2.1.1 Concept of Doctor-Patient Communication 21 2.1.2 Brief History of Patient-Centered Approach 25 2.1.3 Emergence of Patient-Generated Health Data 27 2.2 Four Design Spaces for Data-Driven Communication 30 2.2.1 Data collection 34 2.2.2 Data Representation 41 2.2.3 Data Interpretation 47 2.2.4 Collaboration via Data 50 3 Data Collection: Study of mFood Logger 54 3.1 Motivation 55 3.2 Preliminary Work & Tool Design 57 3.2.1 Clinical Requirements for Data Collection 57 3.2.2 Design of Data Collection Tool: mFood Logger 60 3.3 Study Design 63 3.3.1 Participants 63 3.3.2 StudyProcedure 64 3.4 Results 69 3.4.1 PatientSide 69 3.4.2 ClinicianSide 76 3.5 Limitations & Conclusion 80 3.6 Chapter 3 Summary 81 4 Data Representation: Design of DataMD 83 4.1 Motivation 84 4.2 Preliminary Work 86 4.2.1 Workflow Journey Maps 87 4.2.2 DesignGoals 89 4.3 Study Design 90 4.3.1 Participants 91 4.3.2 ParticipatoryDesignworkshop 91 4.4 Results 92 4.4.1 DesignRequirements 92 4.4.2 Implementation: DataMD 98 4.5 Limitations & Conclusion 102 4.6 Summary of Chapter4 102 5 Data Interpretation: Data-Interpretation Strategies 103 5.1 Motivation 103 5.2 Study Design 106 5.2.1 Participants 106 5.2.2 Study Procedure 108 5.2.3 Data Analysis 110 5.3 Results 111 5.3.1 Change of Interest in Data 111 5.3.2 Assumptions on Relationships between Data Types 113 5.3.3 Data-InterpretationStrategy 117 5.4 Limitations & Conclusion 124 5.5 Summary of Chapter5 125 6 Collaboration via Data: Deployment Study 126 6.1 Motivation 127 6.2 System Design 128 6.2.1 MyHealthKeeper: Patient App 128 6.2.2 DataMD: Clinician Interface 132 6.3 Study Design 133 6.3.1 Participants 134 6.3.2 Procedure 135 6.4 Data Analysis 138 6.4.1 Statistical Analysis of Clinical Outcomes 139 6.4.2 App Usage Log 139 6.4.3 Observation Data Analysis 139 6.5 Results 140 6.5.1 Behavior Change 140 6.5.2 Data-Collection & Journaling Rate 144 6.5.3 Workflow Integration & Communication Support 146 6.6 Limitations & Conclusion 150 6.7 Summary of Chapter6 151 7 Discussion 152 7.1 Towards a Design for Data-Driven Communication 152 7.1.1 Improve Data Quality for Clinical Applicability 153 7.1.2 Support Accessibility of Data Collection 154 7.1.3 Understand Clinicians Preference for Familiar Data Representation. 157 7.1.4 Embrace Lived Experience for Rich Data Interpretation 158 7.1.5 Prioritize Workflow Integration for Successful Data-Driven Communication 163 7.1.6 Consider Risks of Using Patient-Generated Health Data in Clinical Settings 165 7.2 Opportunities for Future Work 166 7.2.1 Leverage Ubiquitous Technology to Design Data CollectionTools 166 7.2.2 Provide Data-Interpretation Guidelines for People with Different Levels of Literacy and Goals 169 7.2.3 Consider Cultural Differences in Data-Driven Communication 170 8 Conclusion 173 8.1 Summary of Contributions 173 8.2 Future Directions 175 8.3 Final Remarks 176Docto

Digital life stories: Semi-automatic (auto)biographies within lifelog collections

Our life stories enable us to reflect upon and share our personal histories. Through emerging digital technologies the possibility of collecting life experiences digitally is increasingly feasible; consequently so is the potential to create a digital counterpart to our personal narratives. In this work, lifelogging tools are used to collect digital artifacts continuously and passively throughout our day. These include images, documents, emails and webpages accessed; texts messages and mobile activity. This range of data when brought together is known as a lifelog. Given the complexity, volume and multimodal nature of such collections, it is clear that there are significant challenges to be addressed in order to achieve coherent and meaningful digital narratives of our events from our life histories. This work investigates the construction of personal digital narratives from lifelog collections. It examines the underlying questions, issues and challenges relating to construction of personal digital narratives from lifelogs. Fundamentally, it addresses how to organize and transform data sampled from an individual’s day-to-day activities into a coherent narrative account. This enquiry is enabled by three 20-month long-term lifelogs collected by participants and produces a narrative system which enables the semi-automatic construction of digital stories from lifelog content. Inspired by probative studies conducted into current practices of curation, from which a set of fundamental requirements are established, this solution employs a 2-dimensional spatial framework for storytelling. It delivers integrated support for the structuring of lifelog content and its distillation into storyform through information retrieval approaches. We describe and contribute flexible algorithmic approaches to achieve both. Finally, this research inquiry yields qualitative and quantitative insights into such digital narratives and their generation, composition and construction. The opportunities for such personal narrative accounts to enable recollection, reminiscence and reflection with the collection owners are established and its benefit in sharing past personal experience experiences is outlined. Finally, in a novel investigation with motivated third parties we demonstrate the opportunities such narrative accounts may have beyond the scope of the collection owner in: personal, societal and cultural explorations, artistic endeavours and as a generational heirloom