The Revival of the Notes Field: Leveraging the Unstructured Content in Electronic Health Records

Problem: Clinical practice requires the production of a time- and resource-consuming great amount of notes. They contain relevant information, but their secondary use is almost impossible, due to their unstructured nature. Researchers are trying to address this problems, with traditional and promising novel techniques. Application in real hospital settings seems not to be possible yet, though, both because of relatively small and dirty dataset, and for the lack of language-specific pre-trained models.Aim: Our aim is to demonstrate the potential of the above techniques, but also raise awareness of the still open challenges that the scientific communities of IT and medical practitioners must jointly address to realize the full potential of unstructured content that is daily produced and digitized in hospital settings, both to improve its data quality and leverage the insights from data-driven predictive models.Methods: To this extent, we present a narrative literature review of the most recent and relevant contributions to leverage the application of Natural Language Processing techniques to the free-text content electronic patient records. In particular, we focused on four selected application domains, namely: data quality, information extraction, sentiment analysis and predictive models, and automated patient cohort selection. Then, we will present a few empirical studies that we undertook at a major teaching hospital specializing in musculoskeletal diseases.Results: We provide the reader with some simple and affordable pipelines, which demonstrate the feasibility of reaching literature performance levels with a single institution non-English dataset. In such a way, we bridged literature and real world needs, performing a step further toward the revival of notes fields

Investigating Visual Perception Impairments through Serious Games and Eye Tracking to Anticipate Handwriting Difficulties

Dysgraphia is a learning disability that causes handwritten production below expectations. Its diagnosis is delayed until the completion of handwriting development. To allow a preventive training program, abilities not directly related to handwriting should be evaluated, and one of them is visual perception. To investigate the role of visual perception in handwriting skills, we gamified standard clinical visual perception tests to be played while wearing an eye tracker at three difficulty levels. Then, we identified children at risk of dysgraphia through the means of a handwriting speed test. Five machine learning models were constructed to predict if the child was at risk, using the CatBoost algorithm with Nested Cross-Validation, with combinations of game performance, eye-tracking, and drawing data as predictors. A total of 53 children participated in the study. The machine learning models obtained good results, particularly with game performances as predictors (F1 score: 0.77 train, 0.71 test). SHAP explainer was used to identify the most impactful features. The game reached an excellent usability score (89.4 +/- 9.6). These results are promising to suggest a new tool for dysgraphia early screening based on visual perception skills

Medium-term patient's satisfaction after primary total knee arthroplasty: enhancing prediction for improved care

Background: Patient-reported satisfaction after total knee arthroplasty (TKA) is low compared to other orthopedic procedures. Although several factors have been reported to influence TKA outcomes, it is still challenging to identify patients who will experience dissatisfaction five years after surgery, thereby improving their management. Indeed, both perioperative information and follow-up questionnaires seem to lack statistical predictive power. Hypothesis: This study aims to demonstrate that machine learning can improve the prediction of patient satisfaction, especially when classical statistics fail to identify complex patterns that lead to dissatisfaction. Patients and methods: Patients who underwent primary TKA were included in a Registry that collected baseline data and clinical outcomes at different follow-ups. The patients were divided into satisfied and dissatisfied groups based on a satisfaction questionnaire administered five years after surgery. Satisfaction was predicted using linear statistical models compared to machine learning algorithms. Results: A total of 147 subjects were analyzed. Regarding statistics, significant differences between satisfaction levels started emerging only six months after the intervention, and the classification was close to random guessing. However, machine learning algorithms could improve the prediction by 72% soon after the intervention, and an improvement of 178% was possible when including follow-ups up to one year. Discussion: This study demonstrates the feasibility of a registry-based approach for monitoring and predicting satisfaction using ML algorithms. Level of evidence: III

Identification and characterization of learning weakness from drawing analysis at the pre-literacy stage

: Handwriting learning delays should be addressed early to prevent their exacerbation and long-lasting consequences on whole children's lives. Ideally, proper training should start even before learning how to write. This work presents a novel method to disclose potential handwriting problems, from a pre-literacy stage, based on drawings instead of words production analysis. Two hundred forty-one kindergartners drew on a tablet, and we computed features known to be distinctive of poor handwriting from symbols drawings. We verified that abnormal features patterns reflected abnormal drawings, and found correspondence in experts' evaluation of the potential risk of developing a learning delay in the graphical sphere. A machine learning model was able to discriminate with 0.75 sensitivity and 0.76 specificity children at risk. Finally, we explained why children were considered at risk by the algorithms to inform teachers on the specific weaknesses that need training. Thanks to this system, early intervention to train specific learning delays will be finally possible

Play-Draw-Write: usability and acceptance of a tablet app for the early screening of handwriting difficulties in kindergartners

Dysgraphia is a Learning Disability that prevents from mastering handwriting. It is belatedly diagnosed, with negative consequences on children’s life. To anticipate Dysgraphia screening to a pre-literacy age, we present Play-Draw-Write, a tablet-based application designed to assess handwriting-related features, starting from drawing. It focuses on different aspects of graphical gesture production, such as rhythmicity and speed-accuracy tradeoff, but also to the possible alteration which might occur in gesture production itself, in free drawing. Preliminary inspection of quantitative parameters extracted from the app games suggests their potential in detecting children considered at risk of developing delays in graphical abilities, according to their teachers’ judgement. In this work, we focus on children’s opinion in terms of system acceptance and usability, to enable a longitudinal monitoring through our app and a better evaluation of the direction of possible corrections. Results from usability and acceptance questionnaires on 177 children revealed that they liked playing with the app, and wish to use it again, even when encountering some difficulties. These results are a first step toward an early, easy, and broad screening of Dysgraphia, before handwriting is learnt

The Revival of the Notes Field: Leveraging the Unstructured Content in Electronic Health Records

Problem: Clinical practice requires the production of a time- and resource-consuming great amount of notes. They contain relevant information, but their secondary use is almost impossible, due to their unstructured nature. Researchers are trying to address this problems, with traditional and promising novel techniques. Application in real hospital settings seems not to be possible yet, though, both because of relatively small and dirty dataset, and for the lack of language-specific pre-trained models.Aim: Our aim is to demonstrate the potential of the above techniques, but also raise awareness of the still open challenges that the scientific communities of IT and medical practitioners must jointly address to realize the full potential of unstructured content that is daily produced and digitized in hospital settings, both to improve its data quality and leverage the insights from data-driven predictive models.Methods: To this extent, we present a narrative literature review of the most recent and relevant contributions to leverage the application of Natural Language Processing techniques to the free-text content electronic patient records. In particular, we focused on four selected application domains, namely: data quality, information extraction, sentiment analysis and predictive models, and automated patient cohort selection. Then, we will present a few empirical studies that we undertook at a major teaching hospital specializing in musculoskeletal diseases.Results: We provide the reader with some simple and affordable pipelines, which demonstrate the feasibility of reaching literature performance levels with a single institution non-English dataset. In such a way, we bridged literature and real world needs, performing a step further toward the revival of notes fields

A Tablet App for Handwriting Skill Screening at the Preliteracy Stage: Instrument Validation Study

Background: Difficulties in handwriting, such as dysgraphia, impact several aspects of a child’s everyday life. Current methodologies for the detection of such difficulties in children have the following three main weaknesses: (1) they are prone to subjective evaluation; (2) they can be administered only when handwriting is mastered, thus delaying the diagnosis and the possible adoption of countermeasures; and (3) they are not always easily accessible to the entire community. Objective: This work aims at developing a solution able to: (1) quantitatively measure handwriting features whose alteration is typically seen in children with dysgraphia; (2) enable their study in a preliteracy population; and (3) leverage a standard consumer technology to increase the accessibility of both early screening and longitudinal monitoring of handwriting difficulties. Methods: We designed and developed a novel tablet-based app Play Draw Write to assess potential markers of dysgraphia through the quantification of the following three key handwriting laws: isochrony, homothety, and speed-accuracy tradeoff. To extend such an approach to a preliteracy age, the app includes the study of the laws in terms of both word writing and symbol drawing. The app was tested among healthy children with mastered handwriting (third graders) and those at a preliterate age (kindergartners). Results: App testing in 15 primary school children confirmed that the three laws hold on the tablet surface when both writing words and drawing symbols. We found significant speed modulation according to size (P<.001), no relevant changes to fraction time for 67 out of 70 comparisons, and significant regression between movement time and index of difficulty for 44 out of 45 comparisons (P<.05, R2>0.28, 12 degrees of freedom). Importantly, the three laws were verified on symbols among 19 kindergartners. Results from the speed-accuracy exercise showed a significant evolution with age of the global movement time (circle: P=.003, square: P<.001, word: P=.001), the goodness of fit of the regression between movement time and accuracy constraints (square: P<.001, circle: P=.02), and the index of performance (square: P<.001). Our findings show that homothety, isochrony, and speed-accuracy tradeoff principles are present in children even before handwriting acquisition; however, some handwriting-related skills are partially refined with age. Conclusions: The designed app represents a promising solution for the screening of handwriting difficulties, since it allows (1) anticipation of the detection of alteration of handwriting principles at a preliteracy age and (2) provision of broader access to the monitoring of handwriting principles. Such a solution potentially enables the selective strengthening of lacking abilities before they exacerbate and affect the child’s whole life

Investigating Visual Perception Impairments through Serious Games and Eye Tracking to Anticipate Handwriting Difficulties

Dysgraphia is a learning disability that causes handwritten production below expectations. Its diagnosis is delayed until the completion of handwriting development. To allow a preventive training program, abilities not directly related to handwriting should be evaluated, and one of them is visual perception. To investigate the role of visual perception in handwriting skills, we gamified standard clinical visual perception tests to be played while wearing an eye tracker at three difficulty levels. Then, we identified children at risk of dysgraphia through the means of a handwriting speed test. Five machine learning models were constructed to predict if the child was at risk, using the CatBoost algorithm with Nested Cross-Validation, with combinations of game performance, eye-tracking, and drawing data as predictors. A total of 53 children participated in the study. The machine learning models obtained good results, particularly with game performances as predictors (F1 score: 0.77 train, 0.71 test). SHAP explainer was used to identify the most impactful features. The game reached an excellent usability score (89.4 ± 9.6). These results are promising to suggest a new tool for dysgraphia early screening based on visual perception skills

A CO-DESIGNED PLATFORM FOR A TECHNOLOGY-BASED EARLY IDENTIFICATION TO SUPPORT SPECIFIC LEARNING DISORDERS SCREENING IN SCHOOLS

In the first years of schooling, children build the basis of their knowledge and life skills; this development may be hindered by specific learning disorders (SLDs) that impact learning and, consequently, self-esteem, with effects that last throughout life. Their screening is difficult because of the complexity in distinguishing a learning delay from an actual disorder at an early stage. Still, it is essential to detect as soon as possible the presence of SLDs to effectively treat them. This is the rationale of the IndiPote(dn)S project. It addresses children from the last year of kindergarten to the second year of primary school; first, teachers observe children in school activities and report their abilities following grids of learning weaknesses precursors, designed by psychologists and child neuropsychiatrists (CNP); then, weak children are trained with a Vademecum of pen-and-paper or manual activities; finally, they are observed again. If the problems are not solved, further insight from a CNP is asked with priority. The process is performed by trained teachers, who however introduce subjectivity. In this context, technology can be beneficial to detecting signs not visible to the naked eye, structuring data collection, and standardizing the observation and training, both fundamental to gaining insights on children's learning trajectory in the screening path. This work aimed to co-design and develop a complete instrument to systematize data collection (Aim 1) and provide technological support to the screening and training (Aim 2). As for Aim 1, after brainstorming with stakeholders to understand the paper-based process, a web-app platform was devised. Specifically, the web-app choice was made for its capability of running on different devices (e.g. PCs and tablets), requiring only Internet access to work. The platform provides basic functions to input data on class composition; grade-specific questionnaires on children’s weaknesses and their training; and outcome of CNPs’ visits. The platform was proposed to schools in iterative testing that lasted from 2019 to 2023 and involved more than 130 schools and 15 thousand children on average per year. Yearly enhancements have been made thanks to users’ feedback and prompts. During this period, the focus was to assess adherence by measuring compliance, and the effectiveness of the screening by measuring the percentage of true positives in the reporting to CNP. Since its potential in the pilot phase, a final scalable version of the platform was produced to enable widespread adoption. The mean compliance obtained during iterative testing was 87.5%, whereas the true positives in CNP reporting resulted to be 75.8%. As for Aim 2, the final platform was also enriched with a module able to connect the observation and training activities with technologies like serious games or smart objects, paired with a reasoner that will be used to provide suggestions on training, describe the learning trajectory, and predict the outcome. The use case of a smart ink pen for screening will be presented. In conclusion, the platform is a promising tool for weakness identification, fundamental for the SLDs screening