Google Translate Error Analysis for Mental Healthcare Information: Evaluating Accuracy, Comprehensibility, and Implications for Multilingual Healthcare Communication

This study explores the use of Google Translate (GT) for translating mental healthcare (MHealth) information and evaluates its accuracy, comprehensibility, and implications for multilingual healthcare communication through analysing GT output in the MHealth domain from English to Persian, Arabic, Turkish, Romanian, and Spanish. Two datasets comprising MHealth information from the UK National Health Service website and information leaflets from The Royal College of Psychiatrists were used. Native speakers of the target languages manually assessed the GT translations, focusing on medical terminology accuracy, comprehensibility, and critical syntactic/semantic errors. GT output analysis revealed challenges in accurately translating medical terminology, particularly in Arabic, Romanian, and Persian. Fluency issues were prevalent across various languages, affecting comprehension, mainly in Arabic and Spanish. Critical errors arose in specific contexts, such as bullet-point formatting, specifically in Persian, Turkish, and Romanian. Although improvements are seen in longer-text translations, there remains a need to enhance accuracy in medical and mental health terminology and fluency, whilst also addressing formatting issues for a more seamless user experience. The findings highlight the need to use customised translation engines for Mhealth translation and the challenges when relying solely on machine-translated medical content, emphasising the crucial role of human reviewers in multilingual healthcare communication

Work package 2: Mapping and Assessing Existing Multilingual Resources in Mental Healthcare

The objective of WP2 is to create a validated resource repository for healthcare providers and LLP TCNs of qualityassessed language resources in healthcare, among which guidelines/policies/regulation, translation and interpreting services, and multilingual treatment possibilities. Each partner will make an inventory of language support resources available in different mental healthcare settings in liaison with relevant organisations during the first 6 months of the action. The resources will be classified and assessed in terms of their availability in different languages, their nature and purpose, their addressees, usability and their overall quality through an Estimate-Talk- Estimate (ETE) Delphi technique, a validated method to ensure good quality decision-making (Moher et al., 2010). The database resulting from this study can be found here: https://mentalhealth4all.eu

Mhealth4All Project

During a three-year multidisciplinary trajectory, leading academics and non-academics (e.g. linguists, clinical and social psychologists, communication scientists, migrant and healthcare provider organisations, public service interpreters), will develop, implement and evaluate an evidence-based multilingual culturally-sensitive sustainable digital information and communication platform to enhance access to mental healthcare for third-country nationals (TCNs) with low language proficiency (LLP) in the host country’s language across various European countries. The following target groups will be included: refugees, asylum seekers, other migrant patient groups suffering from mental health issues, healthcare providers, interpreters and cultural mediators, policy makers and NGOs

Work package 4: Co-creation of patient educational narratives

The objective of this WP is to co-create a minimum of four multilingual culturally-sensitive patient educational narrative videos in liaison with migrant healthcare organisations in order to educate and support LLP TCNs’ access to mental healthcare services. The rationale for including audiovisual educational material is twofold. First, a relatively high proportion of LLP TCNs has low literacy skills and will not be able to comprehend written materials and are thus in need of audio materials. In addition, audiovisual information is often better understood and recalled than written information, also among literate populations. Second, videos are more easily disseminated through online media channels beyond country borders than written materials and can therefore reach more LLP TCNs in Europe

Work package 6: Evaluation of the digital multilingual information and communication platform in practice

The objective of this WP is to evaluate the open access digital platform in terms of enhancing access to mental healthcare for LLP TCNs, through increasing LLP TCNs’ and mental healthcare providers’ knowledge, understanding and competencies regarding (communication) strategies to overcome the language barrier in mental healthcare. Based on the materials created in WP2 (resource repository), WP4 (patient educational videos) and WP5 (educational videos for healthcare providers), the design of the digital multilingual information and communication platform will be finalised and subsequently evaluated in healthcare practice to ensure readiness of the platform for implementation for any LLP TCN group in European mental healthcare settings. A pretest-posttest design will be employed to evaluate the open access digital platform with regard to the process of implementing it in healthcare practice, and its effects in increasing access to mental healthcare services among LLP TCNs. A training manual will be developed to train healthcare providers how to make use of the digital platform in practice and a training will be delivered to participating healthcare providers. The training manual will be uploaded on the platform and will be developed in such a manner that the platform can also be easily used without face-to-face training (e.g. by using visuals/screenshots, etc.). The effects of the digital platform will be tested among approximately 52 healthcare providers (see section 3.2.1 for power analysis). For each provider, we strive to include at least 5 LLP TCN patients (total N=260). For the process-evaluation we will include the following measures that will be assessed among all healthcare providers and LLP TCNs, either by written surveys or orally administered surveys in the mother tongue of the patients: ease of use, comprehensibility of information and usefulness of the contents. In addition, tracking statistics will be used to assess the frequency with which the digital platform is visited. For the effect-evaluation, validated scales will be used (either written or orally administered) to assess changes in the following four dimensions of access before and after having accessed the digital platform (i.e. pretest measures will be collected just before the training, posttest measures about 1 month after the training; process measures will be collected at the same time as posttest effect measures): 1. availability of healthcare and language services (e.g. increased ease of access to existing services through the resource repository). 2. approachability of healthcare and language services (e.g. increased knowledge, awareness and understanding about language services and rights available). 3. acceptability of healthcare and language services (e.g. increased intercultural and communicative competencies of healthcare providers). 4. appropriateness of healthcare and language services (e.g. increased match between LLP TCNs needs and actual services provided)

Work package 3: Barriers, needs and communication strategies

The first objective of this WP is to identify the most severe barriers to accessing mental healthcare services for LLP TCNs and to conduct an analysis of the communication, educational/training and practical needs arising for LLP TCNs and healthcare providers to promote this access. Building on this analysis, the second objective is to identify different communication strategies that can potentially mitigate the identified barriers and fulfil the identified needs effectively. Consideration will be given to (1) strategies addressing LLP TCNs’ and providers’ educational needs; (2) macro-strategies enabling access (e.g. use of a second language or lingua franca, individuals providing language support, translation tools); and (3) micro-strategies supporting effective communication and interaction within the afore-mentioned macro-options, in different communicative events and at different stages in the clinical care process. Given the increasing use of technology in healthcare settings as an emerging macro-strategy to access language support, especially automated translation such as Google Translate, our work on the second objective will include a small-scale simulated examination of the use of this technology in supporting access to mental healthcare services for LLP TCNs

Work package 5: Co-creation of educational videos for healthcare providers

The objective of this WP is to co-create - in close collaboration with stakeholders (e.g. healthcare providers, interpreters/intercultural mediators, migrant organisations) - online educational videos for healthcare professionals to enhance their intercultural and communicative competencies, thereby promoting access to mental healthcare for LLP TCNs. As training in this field is often limited to providing narrow knowledge without actual clinical work with the migrant patient community or a link to broader patient management (Chiarenza et al. 2019), we will develop and present our educational videos as a tool to solve problems healthcare providers encounter in everyday clinical practice, based on the outcomes of WP3 (i.e. salient needs, severest barriers, and recommended communication strategies). Furthermore, we will base the content of the videos on the concept of storytelling and address real-life issues through providers’ personal testimonies, because these techniques have shown to be effective in generating changes in knowledge, skills and behaviour (Singhal et al., 2004)