An ideal model of an assistive technology assessment and delivery process

The purpose of the present work is to present some aspects of the Assistive Technology Assessment (ATA) process model compatible with the Position Paper 2012 by AAATE/EASTIN. Three aspects of the ATA process will be discussed in light of three topics of the Position Paper 2012: (i) The dimensions and the measures of the User eXperience (UX) evaluation modelled in the ATA process as a way to verify the efficient and the evidence-based practices of an AT service delivery centre; (ii) The relevance of the presence of the psychologist in the multidisciplinary team of an AT service delivery centre as necessary for a complete person-centred assistive solution empowering users to make their own choices; (iii) The new profession of the psychotechnologist, who explores users needs by seeking a proper assistive solution, leading the multidisciplinary team to observe critical issues and problems. Through the foundation of the Position Paper 2012, the 1995 HEART study, the Matching Person and Technology model, the ICF framework, and the pillars of the ATA process, this paper sets forth a concept and approach that emphasise the personal factors of the individual consumer and UX as key to positively impacting a successful outcome and AT solution

Is the LITE version of the usability metric for user experience (UMUX-LITE) a reliable tool to support rapid assessment of new healthcare technology?

Objective To ascertain the reliability of a standardised, short-scale measure of satisfaction in the use of new healthcare technology i.e., the LITE version of the usability metric for user experience (UMUX-LITE). Whilst previous studies have demonstrated the reliability of UMUX-LITE, and its relationship with measures of likelihood to recommend a product, such as the Net Promoter Score (NPS) in other sectors no such testing has been undertaken with healthcare technology. Materials and methods Six point-of-care products at different stages of development were assessed by 120 healthcare professionals. UMUX-LITE was used to gather their satisfaction in use, and NPS to declare their intention to promote the product. Inferential statistics were used to: i) ascertain the reliability of UMUX-LITE, and ii) assess the relationship between UMUX-LITE and NPS at different stages of products development. Results UMUX-LITE showed an acceptable reliability (α = 0.7) and a strong positive correlation with NPS (r = 0.455, p < .001). This is similar to findings in other fields of application. The level of product development did not affect the UMUX-LITE scores, while the stage of development was a significant predictor (R2 = 0.49) of the intention to promote. Discussion and conclusion Practitioners may apply UMUX-LITE alone, or in combination with the NPS, to complement interview and ‘homemade’ scales to investigate the quality of new products at different stages of development. This shortened scale is appropriate for use in the context of healthcare in which busy professionals have a minimal amount of time to support innovation

Why you need to include human factors in clinical and empirical studies of in vitro point of care devices? Review and future perspectives

Use of in-vitro point of care devices - intended as tests performed out of laboratories and near patient - is increasing in clinical environments. International standards indicate that interaction assessment should not end after the product release, yet human factors methods are frequently not included in clinical and empirical studies of these devices. Whilst the literature confirms some advantages of bed-side tests compared to those in laboratories there is a lack of knowledge of the risks associated with their use. This article provides a review of approaches applied by clinical researchers to model the use of in-vitro testing. Results suggest that only a few studies have explored human factor approaches. Furthermore, when researchers investigated people-device interaction these were predominantly limited to qualitative and not standardised approaches. The methodological failings and limitations of these studies, identified by us, demonstrate the growing need to integrate human factors methods in the medical field

Reviewing and extending the five-user assumption: A grounded procedure for interaction evaluation

" © ACM, 2013. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in ACM Transactions on Computer-Human Interaction (TOCHI), {VOL 20, ISS 5, (November 2013)} http://doi.acm.org/10.1145/2506210 "The debate concerning how many participants represents a sufficient number for interaction testing is well-established and long-running, with prominent contributions arguing that five users provide a good benchmark when seeking to discover interaction problems. We argue that adoption of five users in this context is often done with little understanding of the basis for, or implications of, the decision. We present an analysis of relevant research to clarify the meaning of the five-user assumption and to examine the way in which the original research that suggested it has been applied. This includes its blind adoption and application in some studies, and complaints about its inadequacies in others. We argue that the five-user assumption is often misunderstood, not only in the field of Human-Computer Interaction, but also in fields such as medical device design, or in business and information applications. The analysis that we present allows us to define a systematic approach for monitoring the sample discovery likelihood, in formative and summative evaluations, and for gathering information in order to make critical decisions during the interaction testing, while respecting the aim of the evaluation and allotted budget. This approach – which we call the ‘Grounded Procedure’ – is introduced and its value argued.The MATCH programme (EPSRC Grants: EP/F063822/1 EP/G012393/1

Ciao AI: the Italian adaptation and validation of the Chatbot Usability Scale

Chatbot-based tools are becoming pervasive in multiple domains from commercial websites to rehabilitation applications. Only recently, an eleven-item satisfaction inventory was developed (the ChatBot Usability Scale, BUS-11) to help designers in the assessment process of their systems. The BUS-11 has been validated in multiple contexts and languages, i.e., English, German, Dutch, and Spanish. This scale forms a solid platform enabling designers to rapidly assess chatbots both during and after the design process. The present work aims to adapt and validate the BUS-11 inventory in Italian. A total of 1360 questionnaires were collected which related to a total of 10 Italian chatbot-based systems using the BUS-11 inventory and also using the lite version of the Usability Metrics for User eXperience for convergent validity purposes. The Italian version of the BUS-11 was adapted in terms of the wording of one item, and a Multi-Group Confirmatory Factorial Analysis was performed to establish the factorial structure of the scale and compare the effects of the wording adaptation. Results indicate that the adapted Italian version of the scale matches the expected factorial structure of the original scale. The Italian BUS-11 is highly reliable (Cronbach alpha: 0.921), and it correlates to other measures of satisfaction (e.g., UMUX-Lite, τb = 0.67; p &lt; .001) by also offering specific insights regarding the chatbots’ characteristics. The Italian BUS-11 can be confidently used by chatbot designers to assess the satisfaction of their users during formative or summative tests

Evaluation of a hub-and-spoke model for the delivery of femtosecond laser-assisted cataract surgery within the context of a large randomised controlled trial

AIMS: To test a hypothesis that cataract operating room (OR) productivity can be improved with a femtosecond laser (FL) using a hub-and-spoke model and whether any increase in productivity can offset additional costs relating to the FL. METHODS: 400 eyes of 400 patients were enrolled in a randomised-controlled trial comparing FL-assisted cataract surgery (FLACS) with conventional phacoemulsification surgery (CPS). 299 of 400 operations were performed on designated high-volume theatre lists (FLACS=134, CPS=165), where a hub-and-spoke FLACS model (1×FL, 2×ORs=2:1) was compared with independent CPS theatre lists. Details of operative timings and OR utilisation were recorded. Differences in productivity between hub-and-spoke FLACS and CPS sessions were compared using an economic model including testing hypothetical 3:1 and 4:1 models. RESULTS: The duration of the operation itself was 12.04±4.89 min for FLACS compared with CPS of 14.54±6.1 min (P<0.001). Total patient time in the OR was reduced from 23.39±6.89 min with CPS to 20.34±5.82 min with FLACS (P<0.001)(reduction of 3.05 min per case). There was no difference in OR turnaround time between the models. Average number of patients treated per theatre list was 9 for FLACS and 8 for CPS. OR utilisation was 92.08% for FLACS and 95.83% for CPS (P<0.001). Using a previously established economic model, the FLACS service cost £144.60 more than CPS per case. This difference would be £131 and £125 for 3:1 and 4:1 models, respectively. CONCLUSION: The FLACS hub-and-spoke model was significantly faster than CPS, with patients spending less time in the OR. This enabled an improvement in productivity, but insufficient to meaningfully offset the additional costs relating to FLACS

Development and validation of ester impregnated pH strips for locating nasogastric feeding tubes in the stomach-a multicentre prospective diagnostic performance study.

BACKGROUND: NG (nasogastric) tubes are used worldwide as a means to provide enteral nutrition. Testing the pH of tube aspirates prior to feeding is commonly used to verify tube location before feeding or medication. A pH at or lower than 5.5 was taken as evidence for stomach intubation. However, the existing standard pH strips lack sensitivity, especially in patients receiving feeding and antacids medication. We developed and validated a first-generation ester-impregnated pH strip test to improve the accuracy towards gastric placements in adult population receiving routine NG-tube feeding. The sensitivity was improved by its augmentation with the action of human gastric lipase (HGL), an enzyme specific to the stomach. METHODS: We carried out a multi-centred, prospective, two-gate diagnostic accuracy study on patients who require routine NG-tube feeding in 10 NHS hospitals comparing the sensitivity of the novel pH strip to the standard pH test, using either chest X-rays or, in its absence, clinical observation of the absence of adverse events as the reference standard. We also tested the novel pH strips in lung aspirates from patients undergoing oesophageal cancer surgeries using visual inspection as the reference standard. We simulated health economics using a decision analytic model and carried out adoption studies to understand its route to commercialisation. The primary end point is the sensitivity of novel and standard pH tests at the recommended pH cut-off of 5.5. RESULTS: A total of 6400 ester-impregnated pH strips were prepared based on an ISO13485 quality management system. A total of 376 gastric samples were collected from adult patients in 10 NHS hospitals who were receiving routine NG-tube feeding. The sensitivities of the standard and novel pH tests were respectively 49.2% (95% CI 44.1‑54.3%) and 70.2% (95% CI 65.6‑74.8%) under pH cut-off of 5.5 and the novel test has a lung specificity of 89.5% (95% CI 79.6%, 99.4%). Our simulation showed that using the novel test can potentially save 132 unnecessary chest X-rays per check per every 1000 eligible patients, or direct savings of £4034 to the NHS. CONCLUSIONS: The novel pH test correctly identified significantly more patients with tubes located inside the stomach compared to the standard pH test used widely by the NHS. TRIAL REGISTRATION: http://www.isrctn.com/ISRCTN11170249 , Registered 21 June 2017-retrospectively registered

Attitudes towards trusting artificial intelligence insights and factors to prevent the passive adherence of GPs: a pilot study

Artificial Intelligence (AI) systems could improve system efficiency by supporting clinicians in making appropriate referrals. However, they are imperfect by nature and misdiagnoses, if not correctly identified, can have consequences for patient care. In this paper, findings from an online survey are presented to understand the aptitude of GPs (n = 50) in appropriately trusting or not trusting the output of a fictitious AI-based decision support tool when assessing skin lesions, and to identify which individual characteristics could make GPs less prone to adhere to erroneous diagnostics results. The findings suggest that, when the AI was correct, the GPs’ ability to correctly diagnose a skin lesion significantly improved after receiving correct AI information, from 73.6% to 86.8% (X2 (1, N = 50) = 21.787, p < 0.001), with significant effects for both the benign (X2 (1, N = 50) = 21, p < 0.001) and malignant cases (X2 (1, N = 50) = 4.654, p = 0.031). However, when the AI provided erroneous information, only 10% of the GPs were able to correctly disagree with the indication of the AI in terms of diagnosis (d-AIW M: 0.12, SD: 0.37), and only 14% of participants were able to correctly decide the management plan despite the AI insights (d-AIW M:0.12, SD: 0.32). The analysis of the difference between groups in terms of individual characteristics suggested that GPs with domain knowledge in dermatology were better at rejecting the wrong insights from AI. View Full-Tex