Data Analysis Processes and Techniques for Validation of Wearable Technology: An Example

Topics in Exercise Science and Kinesiology Volume 3: Issue 1, Article 10, 2022. With wearable technology growing in popularity and sophistication, there remains a need to determine the validity of these devices by independent observers. Validation studies of wearable technology can involve large amounts of data, with data preparation techniques that are not always clearly established. This can make attempts to reproduce the results difficult and does not allow researchers to gain guidance in how to perform their own analyses if they wanted to perform a similar study. Therefore, this paper details the process that was utilized to prepare and analyze the accuracy of several heart rate monitors during mountain biking and can be used as a possible guide to researchers looking to perform similar analyses. We also detail the software used and discuss possible alternatives

Establishing a Methodology for Conducting a Rapid Review on Wearable Technology Reliability and Validity in Applied Settings

Wearable technology is becoming increasingly popular with new devices entering the market frequently, but with disproportionality less research being conducted on the reliability and validity in field settings, outside of the laboratory. This paper details the methodology of gathering research based on the principals of a rapid review, using these points of application: inclusion criteria of studies (databases), search terms (data to be extracted), and how research will be conducted (narrowing articles and assessing bias). Point of application #1: Inclusion criteria are described well as potential databases used, allowing a means of not only systematically gathering information, but also the ability to check the status of the literature regarding wearable technology. Point of application #2: Specific search terms are outlined, and the data fields to be extracted are described. Point of application #3: Mechanism of the rapid review search explained, how articles are to be included and excluded, and how bias will be assessed

Lessons Learned from Sandia National Laboratories' Operational Readiness Review of the Annular Core Research Reactor (ACRR)

The Sandia ACRR (a Hazard Category 2 Nuclear Reactor Facility) was defueled in June 1997 to modify the reactor core and control system to produce medical radioisotopes for the Department of Energy (DOE) Isotope Production Program. The DOE determined that an Operational Readiness Review (ORR) was required to confirm readiness to begin operations within the revised safety basis. This paper addresses the ORR Process, lessons learned from the Sandia and DOE ORRS of the ACRR, and the use of the ORR to confirm authorization basis implementation

A Systematic Review of the Effects of Meditative and Mindful Walking on Mental and Cardiovascular Health

International Journal of Exercise Science 15(2): 1692-1734, 2022. Meditative and mindful exercise are types of physical exercise during which people pay attention, on purpose, to each new present moment without judging their experience. The goal is to apply an accepting awareness of the environment, bodily sensations, thoughts, and emotions without labeling them (e.g., good or bad). The literature centers on qigong, tai chi, and yoga, which are types of mindful exercise that improve mental and cardiovascular health. It is unclear if meditative and mindful walking also improve these health domains. To the authors’ knowledge, this question has not been addressed by a published systematic review. The purpose of this systematic review without a meta-analysis was to synthesize the literature on meditative and mindful walking to determine their effects on mental and cardiovascular health. The protocol follows the PRISMA guidelines, is registered in PROSPERO (CRD42021241180), and is published elsewhere in a peer-reviewed journal. The systematic review contains 14 studies that had various populations, interventions, and outcomes. In 13 studies, the interventions statistically significantly improved scores on at least one outcome of mental or cardiovascular health (e.g., affect, anxiety, depression, distress, state mindfulness, stress, blood pressure, and six-minute walk distance). The improved outcomes should be interpreted cautiously because their clinical meaningfulness is unclear, and the studies had severe methodological limitations. Determining if meditative and mindful walking meaningfully improve mental and cardiovascular health will require randomized controlled trials that use rigorous designs, transparent protocols, and clinically meaningful outcomes that indicate physical function, mental wellbeing, morbidity, and mortality

Efficient Method of Delivery for Powdered Supplement or Placebo for an Outdoor Exercise Investigation

Researchers often encounter issues while attempting to deliver complete doses of a desired supplement/placebo when conducting research in outdoor environments. The problem faced by our research team was how to efficiently deliver 6.4 grams of Beta Alanine (BA) powder to subjects while conducting a hiking study on a windy trail in Southern Utah. To minimize the potential impact of the weather, and to maximize the efficiency of delivery, we determined a premixed bolus of BA contained in individual commercially available 8oz water bottles was the most efficient delivery method. The purpose of this article is to detail the development and implementation of this method. Point of Application 1: Testing solubility, and efficacy of dissolved BA Point of Application 2: Producing a large volume of pre-mixed supplement-bottles and placebo-bottles Point of Application 3: On site administration of supplement and placeb

The Risk of Bias in Validity and Reliability Studies Testing Physiological Variables using Consumer-Grade Wearable Technology: A Systematic Review and WEAR-BOT Analysis

INTRODUCTION: Wearable technology is a quickly evolving field, and new devices with new features to measure/estimate physiological variables are being released constantly. Despite their use, the validity of the devices are largely unknown to the users or researchers, and the quality of the studies that do test validity and reliability vary widely. PURPOSE: Therefore, the purpose of this systematic review was to review the current validity and reliability literature concerning consumer-grade wearable technology measurements/estimates of physiological variables during exercise. Additionally, we sought to perform risk of bias assessments utilizing the novel WEArable technology Risk of Bias and Objectivity Tool (WEAR-BOT). METHODS: This review was conducted following PRISMA guidelines, searching 3 databases: Google Scholar, Scopus, and SPORTDiscus. After screening, 46 papers were identified that met the pre-determined criteria. Then data was extracted and risk of bias assessment performed by independent researchers. Descriptive statistics, weighted averages of mean absolute percentage error (MAPE) and Pearson correlations were calculated. Sample size statistics were performed utilizing the lower 95% confidence interval of the weighted correlation average. RESULTS: Of the 46 papers reviewed, 44 performed validity testing, while 9 performed reliability. The weighted average for MAPE was 12.48% for heart rate (HR) and 30.70% for energy expenditure (EE). The weighted average for Pearson correlations was 0.737 for HR and 0.672 for EE. Risk of bias assessment of validity studies resulted in 30/44 studies being classified as having a “High Risk of Bias”, and 14/44 having “Some Risk of Bias”. None had a “Low Risk of Bias”, according to the novel WEAR-BOT. For reliability studies, 7/9 were classified as “High Risk of Bias”, 2 as “Some Risk of Bias”, and 0 as “Low Risk of Bias”. CONCLUSION: The risk of bias assessment and descriptive statistics paint a troubling picture of the overall state of validity and reliability studies. Statistical analyses, methods, and reporting vary excessively. This review and associated WEAR-BOT analysis can be used by researchers to help standardize methodology, analytics, and reporting of validation and reliability studies of consumer-grade wearable technology

The Validity and Reliability of the Garmin Instinct in Measuring Heart Rate, Energy Expenditure, and Steps During Skipping

As the use of wearable technology to monitor physical activity increases, assessment of the validity and reliability of these devices are needed. A popular device brand is Garmin. Although not a common physical activity, skipping can be included in dynamic warm-ups. PURPOSE: Therefore, the purpose of this study was to determine the validity and reliability of the Garmin Instinct in measuring caloric energy expenditure (EE), average heart rate (HR), and steps while skipping. METHODS: Ten participants (5 female, age: 27±9 years) skipped at a self-selected pace for five minutes. During that time, HR, EE, and steps were measured by the Polar H10, Cosmed K5, and manual counting, respectively. Two Garmin Instincts simultaneously tracked all three variables. A step was defined as any time the foot leaves and hits the ground (stride x 4). Data was input into Google Sheets and summary statistics, t-test with Bonferonni corrections, and mean absolute percentage error (MAPE) were calculated. Additional validity and reliability tests were run in jamovi, including Lin’s concordance correlation coefficient (CCC), TOST tests, Bland-Altman bias, coefficient of variation (CV), and intraclass correlation coefficient (ICC). The pre-established validity criteria are as follows: CCC \u3e 0.7 and MAPE \u3c 10%. The pre-established reliability criteria are as follows: CV \u3c 10% and ICC \u3e 0.7. RESULTS: The Garmin Instinct had a MAPE of 19.2%, 28.5%, and 53.2% for HR, EE, and steps, respectively. It had a CCC of .06, .21, and .01 for HR, EE, and steps, respectively. The 2-tailed paired t-tests with corrections for multiple comparisons was significant for HR and steps. The TOST tests were violated for all 3 measurements (HR, EE, and steps). Bland-Altman analysis produced a bias estimate of 34.0, 0.6, and 1100 for HR, EE, and steps, respectively. The Garmin produced a CV of 11.2%, 14.8%, and 6.6% for HR, EE, and steps, respectively. It produced an ICC of .51, .64, and .81 for HR, EE, and steps, respectively. CONCLUSION: The Garmin Instinct did not meet the pre-established validity criteria for any measure (HR, EE, or steps). However, it did meet the pre-established reliability criteria for steps but not for HR or EE. Therefore, the Garmin Instinct cannot be expected to produce accurate estimates of HR, EE, or steps during skipping

Assessing the Validity of Several Heart Rate Monitors in Wearable Technology While Mountain Biking

International Journal of Exercise Science 16(7): 1440-1450, 2023. Purpose: This study sought to assess the validity of several heart rate (HR) monitors in wearable technology during mountain biking (MTB), compared to the Polar H7® HR monitor, used as the criterion device. Methods: A total of 20 participants completed two MTB trials while wearing six HR monitors (5 test devices, 1 criterion). HR was recorded on a second-by-second basis for all devices analyzed. After data processing, validity measures were calculated, including 1. error analysis: mean absolute percentage errors (MAPE), mean absolute error (MAE), and mean error (ME), and 2. Correlation analysis: Lin’s concordance correlation coefficient (CCC) and Pearson’s correlation coefficient (r). Thresholds for validity were set at MAPE \u3c 10% and CCC \u3e 0.7. Results: The only device that was found to be valid during mountain biking was the Suunto Spartan Sport watch with accompanying HR monitor, with a MAPE of 0.66% and a CCC of 0.99 for the overall, combined data. Conclusion: If a person would like to track their HR during mountain biking, for pacing, training, or other reasons, the devices best able to produce valid results are chest-based, wireless electrocardiogram (ECG) monitors, secured by elastic straps to minimize the movement of the device, such as the Suunto chest-based HR monitor

A Protocol and Novel Tool for Systematically Reviewing the Effects of Mindful Walking on Mental and Cardiovascular Health

To our knowledge, no published systematic review has described the effects of mindful walking on mental and cardiovascular health. We have aimed to fill this gap by first establishing our systematic review protocol. Our protocol was adapted from the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and is registered in PROSPERO (Registration Number: CRD42021241180). The protocol is described step-by-step in this paper, which we wrote to achieve three objectives: to adhere to the best practices stated in the PRISMA guidelines, to ensure procedural transparency, and to enable readers to co-opt our protocol for future systematic reviews on mindful walking and related topics. To achieve our third objective, we provide and explain a novel tool we created to track the sources we will find and screen for our review. Ultimately, the protocol and novel tool will lead to the first published systematic review about mindful walking and will also facilitate future systematic reviews

Validity and Reliability of Physiological Data in Applied Settings Measured by Wearable Technology: A Rapid Systematic Review

The purpose of this review was to evaluate the current state of the literature and to identify the types of study designs, wearable devices, statistical tests, and exercise modes used in validation and reliability studies conducted in applied settings/outdoor environments. This was performed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines. We identified nine articles that fit our inclusion criteria, eight of which tested for validity and one tested for reliability. The studies tested 28 different devices with exercise modalities of running, walking, cycling, and hiking. While there were no universally common analytical techniques used to measure accuracy or validity, correlative measures were used in 88% of studies, mean absolute percentage error (MAPE) in 75%, and Bland–Altman plots in 63%. Intra-class correlation was used to determine reliability. There were not any universally common thresholds to determine validity, however, of the studies that used MAPE and correlation, there were only five devices that had a MAPE of... (see full abstract in article)