Automatic captions on video calls: a must for the older adults

Abstract

Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. Work co-financed by the Canary Islands Agency for Research, Innovation and Information Society of the Ministry of Economy, Knowledge and Employment and by the European Social Fund (ESF) Integrated Operational Program of the Canary Islands 2014–2020, Axis 3 Priority Topic 74 (85%). This work has been supported by the PERGAMEX ACTIVE project, Ref. RTI2018- 096986-B-C32, funded by the Ministry of Science and Innovation. Spain. Agencia Canaria de Investigación, Innovación y Sociedad de la Información (TESIS2020010113).In recent years, the use of video call or video conference tools has not stopped increasing, and especially due to the COVID- 19 pandemic, the use of video calls increased in the educational and work spheres, but also in the family sphere, due to the risks of contagion in face-to-face meetings. Throughout the world, many older people are affected by hearing loss. Auditory functional diversity can make it difficult to enjoy video calls. Using automatic captions might help these people, but not all video calling tools offer this functionality, and some offer it in some languages. We developed an automatic conversation captioning tool using Automatic Speech Recognition and Speech to Text, using the free software tool Coqui STT. This automatic captioning tool is independent of the video call platform used and allows older adults or anyone with auditory functional diversity to enjoy video calls in a simple way. A transparent user interface was designed for our tool that overlays the video call window, and the tool allows us to easily change the text size, color, and background settings. It is also important to remember that many older people have visual functional diversity, so they could have problems reading the texts, thus it is important that each person can adapt the text to their needs. An analysis has been carried out that includes older people to analyze the benefits of the interface, as well as some configuration preferences, and a proposal to improve the way the text is displayed on the screen. Spanish and English were tested during the investigation, but the tool allows us to easily install dozens of new languages based on models trained for Coqui STT.Open Access funding provided thanks to the CRUE-CSIC agreement with Springer NatureCanary Islands Agency for Research, Innovation and Information Society of the Ministry of Economy, Knowledge and Employment and by the European Social Fund (ESF) Integrated Operational Program of the Canary Islands 2014–2020, Axis 3 Priority Topic 74 (85%)PERGAMEX ACTIVE project, Ref. RTI2018- 096986-B-C32, funded by the Ministry of Science and Innovation. SpainAgencia Canaria de Investigación, Innovación y Sociedad de la Información (TESIS2020010113

    Similar works