Towards Enhanced Human Activity Recognition through Natural Language Generation and Pose Estimation

Vision-based human activity recognition (HAR) has made substantial progress in recognizing predefined gestures but lacks adaptability for emerging activities. This paper introduces a paradigm shift by harnessing generative modeling and large language models (LLMs) to enhance vision-based HAR. We propose utilizing LLMs to generate descriptive textual representations of activities using pose keypoints as an intermediate representation. Incorporating pose keypoints adds contextual depth to the recognition process, allowing for sequences of vectors resembling text chunks, compatible with LLMs. This innovative fusion of computer vision and natural language processing holds significant potential for revolutionizing activity recognition. A proof of concept study on a Kinetics700 dataset subset validates the approach's efficacy, highlighting improved accuracy and interpretability. Future implications encompass enhanced accuracy, novel research avenues, model generalization, and ethical considerations for transparency. This framework has real-world applications, including personalized gym workout feedback and nuanced sports training insights. By connecting visual cues to interpretable textual descriptions, the proposed framework advances HAR accuracy and applicability, shaping the landscape of pervasive computing and activity recognition research. As this approach evolves, it promises a more insightful understanding of human activities across diverse contexts, marking a significant step towards a better world.Comment: Presented at the Symposium on Generative AI for Pervasive Computing (GenAI4PC) held at UbiComp/ISWC 202

A learning health systems approach to integrating electronic patient-reported outcomes across the health care organization

Introduction: Foundational to a learning health system (LHS) is the presence of a data infrastructure that can support continuous learning and improve patient outcomes. To advance their capacity to drive patient-centered care, health systems are increasingly looking to expand the electronic capture of patient data, such as electronic patient-reported outcome (ePRO) measures. Yet ePROs bring unique considerations around workflow, measurement, and technology that health systems may not be poised to navigate. We report on our effort to develop generalizable learnings that can support the integration of ePROs into clinical practice within an LHS framework. Methods: Guided by action research methodology, we engaged in iterative cycles of planning, acting, observing, and reflecting around ePRO use with two primary goals: (1) mobilize an ePRO community of practice to facilitate knowledge sharing, and (2) establish guidelines for ePRO use in the context of LHS practice. Multiple, emergent data collection activities generated generalizable guidelines that document the tangible best practices for ePRO use in clinical care. We organized guidelines around thematic areas that reflect LHS structures and stakeholders. Results: Three core thematic areas (and 24 guidelines) emerged. The theme of governance reflects the importance of leadership, knowledge management, and facilitating organizational learning around best practice models for ePRO use. The theme of integration considers the intersection of workflow, technology, and human factors for ePROs across areas of care delivery. Lastly, the theme of reporting reflects critical considerations for curating data and information, designing system functions and interactions, and presentation of ePRO data to support the translation of knowledge to action. Conclusions: The guidelines produced from this work highlight the complex, multidisciplinary nature of implementing change within LHS contexts, and the value of action research approaches to enable rapid, iterative learning that leverages the knowledge and experience of communities of practice

CSER and eMERGE: current and potential state of the display of genetic information in the electronic health record

Objective Clinicians’ ability to use and interpret genetic information depends upon how those data are displayed in electronic health records (EHRs). There is a critical need to develop systems to effectively display genetic information in EHRs and augment clinical decision support (CDS)

Proceedings of the 3rd Biennial Conference of the Society for Implementation Research Collaboration (SIRC) 2015: advancing efficient methodologies through community partnerships and team science

It is well documented that the majority of adults, children and families in need of evidence-based behavioral health interventionsi do not receive them [1, 2] and that few robust empirically supported methods for implementing evidence-based practices (EBPs) exist. The Society for Implementation Research Collaboration (SIRC) represents a burgeoning effort to advance the innovation and rigor of implementation research and is uniquely focused on bringing together researchers and stakeholders committed to evaluating the implementation of complex evidence-based behavioral health interventions. Through its diverse activities and membership, SIRC aims to foster the promise of implementation research to better serve the behavioral health needs of the population by identifying rigorous, relevant, and efficient strategies that successfully transfer scientific evidence to clinical knowledge for use in real world settings [3]. SIRC began as a National Institute of Mental Health (NIMH)-funded conference series in 2010 (previously titled the “Seattle Implementation Research Conference”; $150,000 USD for 3 conferences in 2011, 2013, and 2015) with the recognition that there were multiple researchers and stakeholdersi working in parallel on innovative implementation science projects in behavioral health, but that formal channels for communicating and collaborating with one another were relatively unavailable. There was a significant need for a forum within which implementation researchers and stakeholders could learn from one another, refine approaches to science and practice, and develop an implementation research agenda using common measures, methods, and research principles to improve both the frequency and quality with which behavioral health treatment implementation is evaluated. SIRC’s membership growth is a testament to this identified need with more than 1000 members from 2011 to the present.ii SIRC’s primary objectives are to: (1) foster communication and collaboration across diverse groups, including implementation researchers, intermediariesi, as well as community stakeholders (SIRC uses the term “EBP champions” for these groups) – and to do so across multiple career levels (e.g., students, early career faculty, established investigators); and (2) enhance and disseminate rigorous measures and methodologies for implementing EBPs and evaluating EBP implementation efforts. These objectives are well aligned with Glasgow and colleagues’ [4] five core tenets deemed critical for advancing implementation science: collaboration, efficiency and speed, rigor and relevance, improved capacity, and cumulative knowledge. SIRC advances these objectives and tenets through in-person conferences, which bring together multidisciplinary implementation researchers and those implementing evidence-based behavioral health interventions in the community to share their work and create professional connections and collaborations

Participant-Centric Initiatives: Tools to Facilitate Engagement In Research.

Clinical genomic research faces increasing challenges in establishing participant privacy and consent processes that facilitate meaningful choice and communication capacity for longitudinal and secondary research uses. There are an evolving range of participant-centric initiatives that combine web-based informatics tools with new models of engagement and research collaboration. These emerging initiatives may become valuable approaches to support large-scale and longitudinal research studies. We highlight and discuss four types of emerging initiatives for engaging and sustaining participation in research