Advances on the Transcription of Historical Manuscripts based on Multimodality, Interactivity and Crowdsourcing

Natural Language Processing (NLP) is an interdisciplinary research field of Computer Science, Linguistics, and Pattern Recognition that studies, among others, the use of human natural languages in Human-Computer Interaction (HCI). Most of NLP research tasks can be applied for solving real-world problems. This is the case of natural language recognition and natural language translation, that can be used for building automatic systems for document transcription and document translation. Regarding digitalised handwritten text documents, transcription is used to obtain an easy digital access to the contents, since simple image digitalisation only provides, in most cases, search by image and not by linguistic contents (keywords, expressions, syntactic or semantic categories). Transcription is even more important in historical manuscripts, since most of these documents are unique and the preservation of their contents is crucial for cultural and historical reasons. The transcription of historical manuscripts is usually done by paleographers, who are experts on ancient script and vocabulary. Recently, Handwritten Text Recognition (HTR) has become a common tool for assisting paleographers in their task, by providing a draft transcription that they may amend with more or less sophisticated methods. This draft transcription is useful when it presents an error rate low enough to make the amending process more comfortable than a complete transcription from scratch. Thus, obtaining a draft transcription with an acceptable low error rate is crucial to have this NLP technology incorporated into the transcription process. The work described in this thesis is focused on the improvement of the draft transcription offered by an HTR system, with the aim of reducing the effort made by paleographers for obtaining the actual transcription on digitalised historical manuscripts. This problem is faced from three different, but complementary, scenarios: · Multimodality: The use of HTR systems allow paleographers to speed up the manual transcription process, since they are able to correct on a draft transcription. Another alternative is to obtain the draft transcription by dictating the contents to an Automatic Speech Recognition (ASR) system. When both sources (image and speech) are available, a multimodal combination is possible and an iterative process can be used in order to refine the final hypothesis. · Interactivity: The use of assistive technologies in the transcription process allows one to reduce the time and human effort required for obtaining the actual transcription, given that the assistive system and the palaeographer cooperate to generate a perfect transcription. Multimodal feedback can be used to provide the assistive system with additional sources of information by using signals that represent the whole same sequence of words to transcribe (e.g. a text image, and the speech of the dictation of the contents of this text image), or that represent just a word or character to correct (e.g. an on-line handwritten word). · Crowdsourcing: Open distributed collaboration emerges as a powerful tool for massive transcription at a relatively low cost, since the paleographer supervision effort may be dramatically reduced. Multimodal combination allows one to use the speech dictation of handwritten text lines in a multimodal crowdsourcing platform, where collaborators may provide their speech by using their own mobile device instead of using desktop or laptop computers, which makes it possible to recruit more collaborators.El Procesamiento del Lenguaje Natural (PLN) es un campo de investigación interdisciplinar de las Ciencias de la Computación, Lingüística y Reconocimiento de Patrones que estudia, entre otros, el uso del lenguaje natural humano en la interacción Hombre-Máquina. La mayoría de las tareas de investigación del PLN se pueden aplicar para resolver problemas del mundo real. Este es el caso del reconocimiento y la traducción del lenguaje natural, que se pueden utilizar para construir sistemas automáticos para la transcripción y traducción de documentos. En cuanto a los documentos manuscritos digitalizados, la transcripción se utiliza para facilitar el acceso digital a los contenidos, ya que la simple digitalización de imágenes sólo proporciona, en la mayoría de los casos, la búsqueda por imagen y no por contenidos lingüísticos. La transcripción es aún más importante en el caso de los manuscritos históricos, ya que la mayoría de estos documentos son únicos y la preservación de su contenido es crucial por razones culturales e históricas. La transcripción de manuscritos históricos suele ser realizada por paleógrafos, que son personas expertas en escritura y vocabulario antiguos. Recientemente, los sistemas de Reconocimiento de Escritura (RES) se han convertido en una herramienta común para ayudar a los paleógrafos en su tarea, la cual proporciona un borrador de la transcripción que los paleógrafos pueden corregir con métodos más o menos sofisticados. Este borrador de transcripción es útil cuando presenta una tasa de error suficientemente reducida para que el proceso de corrección sea más cómodo que una completa transcripción desde cero. Por lo tanto, la obtención de un borrador de transcripción con una baja tasa de error es crucial para que esta tecnología de PLN sea incorporada en el proceso de transcripción. El trabajo descrito en esta tesis se centra en la mejora del borrador de transcripción ofrecido por un sistema RES, con el objetivo de reducir el esfuerzo realizado por los paleógrafos para obtener la transcripción de manuscritos históricos digitalizados. Este problema se enfrenta a partir de tres escenarios diferentes, pero complementarios: · Multimodalidad: El uso de sistemas RES permite a los paleógrafos acelerar el proceso de transcripción manual, ya que son capaces de corregir en un borrador de la transcripción. Otra alternativa es obtener el borrador de la transcripción dictando el contenido a un sistema de Reconocimiento Automático de Habla. Cuando ambas fuentes están disponibles, una combinación multimodal de las mismas es posible y se puede realizar un proceso iterativo para refinar la hipótesis final. · Interactividad: El uso de tecnologías asistenciales en el proceso de transcripción permite reducir el tiempo y el esfuerzo humano requeridos para obtener la transcripción correcta, gracias a la cooperación entre el sistema asistencial y el paleógrafo para obtener la transcripción perfecta. La realimentación multimodal se puede utilizar en el sistema asistencial para proporcionar otras fuentes de información adicionales con señales que representen la misma secuencia de palabras a transcribir (por ejemplo, una imagen de texto, o la señal de habla del dictado del contenido de dicha imagen de texto), o señales que representen sólo una palabra o carácter a corregir (por ejemplo, una palabra manuscrita mediante una pantalla táctil). · Crowdsourcing: La colaboración distribuida y abierta surge como una poderosa herramienta para la transcripción masiva a un costo relativamente bajo, ya que el esfuerzo de supervisión de los paleógrafos puede ser drásticamente reducido. La combinación multimodal permite utilizar el dictado del contenido de líneas de texto manuscrito en una plataforma de crowdsourcing multimodal, donde los colaboradores pueden proporcionar las muestras de habla utilizando su propio dispositivo móvil en lugar de usar ordenadores,El Processament del Llenguatge Natural (PLN) és un camp de recerca interdisciplinar de les Ciències de la Computació, la Lingüística i el Reconeixement de Patrons que estudia, entre d'altres, l'ús del llenguatge natural humà en la interacció Home-Màquina. La majoria de les tasques de recerca del PLN es poden aplicar per resoldre problemes del món real. Aquest és el cas del reconeixement i la traducció del llenguatge natural, que es poden utilitzar per construir sistemes automàtics per a la transcripció i traducció de documents. Quant als documents manuscrits digitalitzats, la transcripció s'utilitza per facilitar l'accés digital als continguts, ja que la simple digitalització d'imatges només proporciona, en la majoria dels casos, la cerca per imatge i no per continguts lingüístics (paraules clau, expressions, categories sintàctiques o semàntiques). La transcripció és encara més important en el cas dels manuscrits històrics, ja que la majoria d'aquests documents són únics i la preservació del seu contingut és crucial per raons culturals i històriques. La transcripció de manuscrits històrics sol ser realitzada per paleògrafs, els quals són persones expertes en escriptura i vocabulari antics. Recentment, els sistemes de Reconeixement d'Escriptura (RES) s'han convertit en una eina comuna per ajudar els paleògrafs en la seua tasca, la qual proporciona un esborrany de la transcripció que els paleògrafs poden esmenar amb mètodes més o menys sofisticats. Aquest esborrany de transcripció és útil quan presenta una taxa d'error prou reduïda perquè el procés de correcció siga més còmode que una completa transcripció des de zero. Per tant, l'obtenció d'un esborrany de transcripció amb un baixa taxa d'error és crucial perquè aquesta tecnologia del PLN siga incorporada en el procés de transcripció. El treball descrit en aquesta tesi se centra en la millora de l'esborrany de la transcripció ofert per un sistema RES, amb l'objectiu de reduir l'esforç realitzat pels paleògrafs per obtenir la transcripció de manuscrits històrics digitalitzats. Aquest problema s'enfronta a partir de tres escenaris diferents, però complementaris: · Multimodalitat: L'ús de sistemes RES permet als paleògrafs accelerar el procés de transcripció manual, ja que són capaços de corregir un esborrany de la transcripció. Una altra alternativa és obtenir l'esborrany de la transcripció dictant el contingut a un sistema de Reconeixement Automàtic de la Parla. Quan les dues fonts (imatge i parla) estan disponibles, una combinació multimodal és possible i es pot realitzar un procés iteratiu per refinar la hipòtesi final. · Interactivitat: L'ús de tecnologies assistencials en el procés de transcripció permet reduir el temps i l'esforç humà requerits per obtenir la transcripció real, gràcies a la cooperació entre el sistema assistencial i el paleògraf per obtenir la transcripció perfecta. La realimentació multimodal es pot utilitzar en el sistema assistencial per proporcionar fonts d'informació addicionals amb senyals que representen la mateixa seqüencia de paraules a transcriure (per exemple, una imatge de text, o el senyal de parla del dictat del contingut d'aquesta imatge de text), o senyals que representen només una paraula o caràcter a corregir (per exemple, una paraula manuscrita mitjançant una pantalla tàctil). · Crowdsourcing: La col·laboració distribuïda i oberta sorgeix com una poderosa eina per a la transcripció massiva a un cost relativament baix, ja que l'esforç de supervisió dels paleògrafs pot ser reduït dràsticament. La combinació multimodal permet utilitzar el dictat del contingut de línies de text manuscrit en una plataforma de crowdsourcing multimodal, on els col·laboradors poden proporcionar les mostres de parla utilitzant el seu propi dispositiu mòbil en lloc d'utilitzar ordinadors d'escriptori o portàtils, la qual cosa permet ampliar el nombrGranell Romero, E. (2017). Advances on the Transcription of Historical Manuscripts based on Multimodality, Interactivity and Crowdsourcing [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/86137TESI

Digital annotations: an exploration of experiences

Digital texts and learning platforms introduce possibilities of forms of reading and writing that can be contrasted with pre-digital understandings of how readers and writers interact with texts. In current Higher education contexts, there is a requirement to embrace the use of digital technologies to access study materials and engage with academic practices; these technologies are often selected and supported by university computing support, or staff creating the course of study, and those participating are expected to accept and grasp the potential for their own work. At the same time students and staff, who can be from diverse language and cultural contexts, are expected to conform to the visible academic, linguistic and cultural practices for writing, submitting texts, and taking part in learning discussions. Study practices also include various forms of notes, comments and annotations to texts that are sometimes private and sometimes exchanged in various ways, including digital formats. Although constraints are placed on what is acceptable in the visible academic settings, the digital choices available to staff and students are extensive. Concurrent to this, changes in course design, resources and support (for staff and students) are being subtly changed in a way that may seem routine (Goodfellow & Lea, 2013) but are gradually and significantly changing the way reading and writing are regarded. This study explores the use of modifications to texts which are variously labelled as digital notes, comments or annotations, with a focus on how these are valued and how they can change perceptions of reader, writer and text in Higher education study practices. These modifications often (but not necessarily) take the form of additions that are marked, separated, or indicated by colour/emphasis to indicate that they are not part of the original text; however, the original digital text has been changed by these modifications, and the resulting text now incorporates the original with layers of new text. This creates a new digital text, which can, of course, undergo further transformation if the process is repeated. In the context of this study, the term “digital annotations” is used for modifications that are created digitally (using different modalities, so could include graphic, photographic as well as written and audio texts) and therefore become part of the creation of new texts. The study draws on theories of literacy, applied linguistics, and social semiotics. The main research questions for the study are “How do users evaluate, use and contribute to digital annotations?” and “what perceived value is placed on modified texts following the creation of digital annotations?” In answering these questions, the conclusions lead to greater understanding of the practical concerns as well as the theoretical questions connected to the process of interacting with digital texts. Using digital annotations to make sense and meaning from digital texts implicates the reader as a writer but also involves the form or mode of the text in a way that demonstrates this is more than an arbitrary choice. Activity Theory (Engestrom, 2000) was used to identify the tensions and contradictions in these choices. A survey and conversations (semi-structured interviews) were used to provide data, and analysis was done using thematic and narrative enquiry. Conclusions show that the choices made by users are subject to the affordances offered by the digital tools, but also their own familiarity with the digital tools, their perceptions of public and private study practices, and the languages they can utilize to probe and create meaning. This has implications for the ways in which digital technologies are promoted in educational contexts, and for the ways in which digital innovations guide and steer institutions, staff and students in an increasingly global world

Towards the collaborative museum? Social media, participation, disciplinary experts and the public in the contemporary museum

This thesis examines the use of social media by museums aiming to establish collaborative relationships with the public. Social media platforms have been widely espoused as transformative in allowing diverse, new or previously excluded audiences to enter into egalitarian, participatory relationships with museums. This thesis deconstructs the concepts of participation and collaboration and identifies the various factors that constrain the extent to which social media enables participatory relationships between previously unequal actors. These factors include the historical disciplinary aims and cultural authority of museums, persistent social inequalities, and the motivations of social media followers. It elucidates crucial questions such as, are various publics enabled to participate on an equal level with each other and with museums? Who benefits from collaborative projects in general and which parties benefit from the use of social media in particular? What are the factors that limit the establishment of collaborative practice? And, conversely, what are the factors that define truly collaborative practice? This research examines museums’ use of and discourses surrounding social media as well as social media followers’ motivations for engaging with museums online. A large body of quantitative and qualitative data gained through in-depth web-based surveys is analysed, primarily using critical discourse analysis, and informed by other critical orientations including media archaeology and the sociology of expertise. The analysis indicates that museums consider social media to be a transformative, democratising technology. However, museums’ acceptance of technologically determinist arguments significantly inhibits positive societal change and the extent to which collaborative relationships can be established with various publics. This research contributes significantly to the existing archaeological and museum studies literature by providing a theoretically and empirically informed critical analysis of the prevailing positive discourses surrounding social media and participation. It has important practical implications for museums in arguing that targeted, critically informed and ethically aware projects are necessary to achieve situations resembling ‘collaboration’. It provides a significant body of data that will inform the formulation and continuation of collaborative projects in museums. Furthermore, it informs broader archaeological debates on involving various publics in archaeological practice. This thesis also demonstrates the importance and effectiveness of critical discourse analysis and related critical approaches for analysing large bodies of qualitative data

Moving Boundaries in Translation Studies

Translation is in motion. Both translation practice and translation studies (TS) have seen considerable innovation in recent decades, and we are currently witnessing a wealth of new approaches and concepts, some of which refect new translation phenomena, whereas others mirror new scholarly foci. Volunteer translation, crowdsourcing, virtual translator networks, transediting, and translanguaging are only some examples of practices and notions that are emerging on the scene alongside a renewed focus on well-established concepts that have traditionally been considered peripheral to the practice and study of translation: intralingual and intersemiotic translation are cases in point. At the same time, technological innovation and global developments such as the spread of English as a lingua franca are affecting wide areas of translation and, with it, translation studies. These trends are currently pushing or even crossing our traditional understandings of translation (studies) and its boundaries. The question is: how to deal with these developments? Some areas of the translation profession seem to respond by widening its borders, adding new practices such as technical writing, localisation, transcreation, or post-editing to their job portfolios, whereas others seem to be closing ranks. The same trend can be observed in the academic discipline: some branches of translation studies are eager to embrace all new developments under the TS umbrella, whereas others tend to dismiss (some of) them as irrelevant or as merely refecting new names for age-old practices. Translation is in motion. Technological developments, digitalisation and globalisation are among the many factors affecting and changing translation and, with it, translation studies. Moving Boundaries in Translation Studies offers a bird’s-eye view of recent developments and discusses their implications for the boundaries of the discipline. With 15 chapters written by leading translation scholars from around the world, the book analyses new translation phenomena, new practices and tools, new forms of organisation, new concepts and names as well as new scholarly approaches and methods. This is key reading for scholars, researchers and advanced students of translation and interpreting studies. The Open Access version of this book, available at http://www.taylorfrancis.com, has been made available under a Creative Commons Attribution-Non Commercial-No Derivatives 4.0 licens

Multimodal translation analysis: Arab Spring speeches in Arabic and English

In the contemporary globalized world, translation plays a key role in sharing news across the globe, in particular in the age of multimedia, where meaning is transferred through various modes and genres. This study focuses on two Arab Spring speeches of Mummar Algaddafi’s and Hosni Mubarak’s and their translations in different media. The thesis initially conducts a comparative study of the source texts (STs), including a textual/contextual analysis drawing on Norman Fairclough’s Critical Discourse Analysis, and on Gunther Kress’ multimodal analysis. This is followed by examining the target texts (TTs) to investigate the inventible changes that occur during the translation process, particularly if the translation involves not only a transfer of meaning from Arabic to English but also from mode to mode (such as, speaking to writing) and genre to genre (a political speech to a newspaper article). The thesis introduces the Multimodal Translation Analysis model to investigate the following aspects of the TTs: linguistic aspects of the TTs, the TT’s multimodal qualities, and, drawing on Mona Baker’s narrative theory, the role of dominant narratives in the shaping of the TT

Moving Boundaries in Translation Studies

Translation is in motion. Both translation practice and translation studies (TS) have seen considerable innovation in recent decades, and we are currently witnessing a wealth of new approaches and concepts, some of which refect new translation phenomena, whereas others mirror new scholarly foci. Volunteer translation, crowdsourcing, virtual translator networks, transediting, and translanguaging are only some examples of practices and notions that are emerging on the scene alongside a renewed focus on well-established concepts that have traditionally been considered peripheral to the practice and study of translation: intralingual and intersemiotic translation are cases in point. At the same time, technological innovation and global developments such as the spread of English as a lingua franca are affecting wide areas of translation and, with it, translation studies. These trends are currently pushing or even crossing our traditional understandings of translation (studies) and its boundaries. The question is: how to deal with these developments? Some areas of the translation profession seem to respond by widening its borders, adding new practices such as technical writing, localisation, transcreation, or post-editing to their job portfolios, whereas others seem to be closing ranks. The same trend can be observed in the academic discipline: some branches of translation studies are eager to embrace all new developments under the TS umbrella, whereas others tend to dismiss (some of) them as irrelevant or as merely refecting new names for age-old practices. Translation is in motion. Technological developments, digitalisation and globalisation are among the many factors affecting and changing translation and, with it, translation studies. Moving Boundaries in Translation Studies offers a bird’s-eye view of recent developments and discusses their implications for the boundaries of the discipline. With 15 chapters written by leading translation scholars from around the world, the book analyses new translation phenomena, new practices and tools, new forms of organisation, new concepts and names as well as new scholarly approaches and methods. This is key reading for scholars, researchers and advanced students of translation and interpreting studies. The Open Access version of this book, available at http://www.taylorfrancis.com, has been made available under a Creative Commons Attribution-Non Commercial-No Derivatives 4.0 licens

Electronic Literature as Digital Humanities

This book is available as open access through the Bloomsbury Open programme and is available on www.bloomsburycollections.com. Electronic Literature as Digital Humanities: Contexts, Forms & Practices is a volume of essays that provides a detailed account of born-digital literature by artists and scholars who have contributed to its birth and evolution. Rather than offering a prescriptive definition of electronic literature, this book takes an ontological approach through descriptive exploration, treating electronic literature from the perspective of the digital humanities (DH)––that is, as an area of scholarship and practice that exists at the juncture between the literary and the algorithmic. The domain of DH is typically segmented into the two seemingly disparate strands of criticism and building, with scholars either studying the synthesis between cultural expression and screens or the use of technology to make artifacts in themselves. This book regards electronic literature as fundamentally DH in that it synthesizes these two constituents. Electronic Literature as Digital Humanities provides a context for the development of the field, informed by the forms and practices that have emerged throughout the DH moment, and finally, offers resources for others interested in learning more about electronic literature

Intelligence, Creativity and Fantasy

UID/HIS/04666/2019 This is the 2nd volume of PHI series, published by CRC Press, the 4th published by CRC Press and the 5th volume of PHI proceedings.The texts presented in Proportion Harmonies and Identities (PHI) - INTELLIGENCE, CREATIVITY AND FANTASY were compiled with the intent to establish a multidisciplinary platform for the presentation, interaction and dissemination of research. The aim is also to foster the awareness and discussion on the topics of Harmony and Proportion with a focus on different visions relevant to Architecture, Arts and Humanities, Design, Engineering, Social and Natural Sciences, and their importance and benefits for the sense of both individual and community identity. The idea of modernity has been a significant motor for development since the Western Early Modern Age. Its theoretical and practical foundations have become the working tools of scientists, philosophers, and artists, who seek strategies and policies to accelerate the development process in different contexts.authorsversionpublishe