Connecting question answering and conversational agents: Contextualizing German questions for interactive question answering systems

Waltinger U, Breuing A, Wachsmuth I. Connecting question answering and conversational agents: Contextualizing German questions for interactive question answering systems. KI - Künstliche Intelligenz. 2012;26(4):381-390.Research results in the field of Question Answering (QA) have shown that the classification of natural language questions significantly contributes to the accuracy of the generated answers. In this paper we present an approach which extends the prevalent question classification techniques by additionally considering further contextual information provided by the questions. Thereby we focus on improving the conversational abilities of existing interactive interfaces by enhancing their underlying QA systems in terms of response time and correctness. As a result, we are able to introduce a method based on a tripartite contextualization. First, we present a comprehensive question classification experiment based on machine learning using two different datasets and various feature sets for the German language. Second, we propose a method for detecting the focus chunk of a given question, that is, for identifying which part of the question is fundamentally relevant to the answer and which part refers to a specification of it. Third, we investigate how to identify and label the topic of a given question by means of a human-judgment experiment. We show that the resulting contextualization method contributes to an improvement of existing question answering systems and enhances their application within interactive scenarios

Image-Enabled Discourse: Investigating the Creation of Visual Information as Communicative Practice

Anyone who has clarified a thought or prompted a response during a conversation by drawing a picture has exploited the potential of image making as an interactive tool for conveying information. Images are increasingly ubiquitous in daily communication, in large part due to advances in visually enabled information and communication technologies (ICT), such as information visualization applications, image retrieval systems and visually enabled collaborative work tools. Human abilities to use images to communicate are however far more sophisticated and nuanced than these technologies currently support. In order to learn more about the practice of image making as a specialized form of information and communication behavior, this study examined face-to-face conversations involving the creation of ad hoc visualizations (i.e., napkin drawings ). A model of image-enabled discourse is introduced, which positions image making as a specialized form of communicative practice. Multimodal analysis of video-recorded conversations focused on identifying image-enabled communicative activities in terms of interactional sociolinguistic concepts of conversational involvement and coordination, specifically framing, footing and stance. The study shows that when drawing occurs in the context of an ongoing dialogue, the activity of visual representation performs key communicative tasks. Visualization is a form of social interaction that contributes to the maintenance of conversational involvement in ways that are not often evident in the image artifact. For example, drawing enables us to coordinate with each other, to introduce alternative perspectives into a conversation and even to temporarily suspend the primary thread of a discussion in order to explore a tangential thought. The study compares attributes of the image artifact with those of the activity of image making, described as a series of contrasting affordances. Visual information in complex systems is generally represented and managed based on the affordances of the artifact, neglecting to account for all that is communicated through the situated action of creating. These finding have heuristic and best-practice implications for a range of areas related to the design and evaluation of virtual collaboration environments, visual information extraction and retrieval systems, and data visualization tools

A Multimodal Approach to Metadiscourse as an Organizational Tool in Lectures

This thesis explores the uses of organizational metadiscourse in lectures from a multimodal perspective, thus providing a holistic view of its use. Moreover, this study explores how the use of organizational metadiscourse, both at a linguistic and at non-verbal level, is influenced by the lecturing style chosen by the lecturers (conversational, rhetorical or reading styles).Esta tesis explora los usos del metadiscurso organizativo en clases universitarias desde una perspectiva multimodal que permite obtener una visión holística del mismo. Además, se describe como el discurso organizativo es influenciado tanto a nivel lingüístico como a nivel no verbal por el estilo de enseñanza del profesorado (conversacional, retórico o lector).Programa de Doctorat en Llengües Aplicades, Literatura i Traducci

Geographic information extraction from texts

A large volume of unstructured texts, containing valuable geographic information, is available online. This information – provided implicitly or explicitly – is useful not only for scientific studies (e.g., spatial humanities) but also for many practical applications (e.g., geographic information retrieval). Although large progress has been achieved in geographic information extraction from texts, there are still unsolved challenges and issues, ranging from methods, systems, and data, to applications and privacy. Therefore, this workshop will provide a timely opportunity to discuss the recent advances, new ideas, and concepts but also identify research gaps in geographic information extraction

Evaluating Machine Intelligence with Question Answering

Humans ask questions to learn about the world and to test knowledge understanding. The ability to ask questions combines aspects of intelligence unique to humans: language understanding, knowledge representation, and reasoning. Thus, building systems capable of intelligent question answering (QA) is a grand goal of natural language processing (NLP). To measure progress in NLP, we create "exams" for computer systems and compare their effectiveness against a reference point---often based on humans. How precisely we measure progress depends on whether we are building computer systems that optimize human satisfaction in information-seeking tasks or that measure progress towards intelligent QA. In the first part of this dissertation, we explore each goal in turn, how they differ, and describe their relationship to QA formats. As an example of an information-seeking evaluation, we introduce a new dialog QA task paired with a new evaluation method. Afterward, we turn our attention to using QA to evaluate machine intelligence. A good evaluation should be able to discriminate between lesser and more capable QA models. This dissertation explores three ways to improve the discriminative power of QA evaluations: (1) dynamic weighting of test questions, (2) a format that by construction tests multiple levels of knowledge, and (3) evaluation data that is created through human-computer collaboration. By dynamically weighting test questions, we challenge a foundational assumption of the de facto standard in QA evaluation---the leaderboard. Namely, we contend that contrary to nearly all QA and NLP evaluations which implicitly assign equal weights to examples by averaging scores, that examples are not equally useful for estimating machine (or human) QA ability. As any student may tell you, not all questions on an exam are equally difficult and in the worst-case questions are unsolvable. Drawing on decades of research in educational testing, we propose adopting an alternative evaluation methodology---Item Response Theory---that is widely used to score human exams (e.g., the SAT). By dynamically weighting questions, we show that this improves the reliability of leaderboards in discriminating between models of differing QA ability while also being helpful in the construction of new evaluation datasets. Having improved the scoring of models, we next turn to improving the format and data in QA evaluations. Our idea is simple. In most QA tasks (e.g., Jeopardy!), each question tests a single level of knowledge; in our task (the trivia game Quizbowl), we test multiple levels of knowledge with each question. Since each question tests multiple levels of knowledge, this decreases the likelihood that we learn nothing about the difference between two models (i.e., they are both correct or both wrong), which substantially increases discriminative power. Despite the improved format, we next show that while our QA models defeat accomplished trivia players, that they are overly reliant on brittle pattern matching, which indicates a failure to intelligently answer questions. To mitigate this problem, we introduce a new framework for building evaluation data where humans and machines cooperatively craft trivia questions that are difficult to answer through clever pattern matching tricks alone---while being no harder for humans. We conclude by sketching a broader vision for QA evaluation that combines the three components of evaluation we improve---scoring, format, and data---to create living evaluations and re-imagine the role of leaderboards

AI in Museums: Reflections, Perspectives and Applications

Artificial intelligence is becoming an increasingly important topic in the cultural sector. While museums have long focused on building digital object databases, the existing data can now become a field of application for machine learning, deep learning and foundation model approaches. This goes hand in hand with new artistic practices, curation tools, visitor analytics, chatbots, automatic translations and tailor-made text generation. With a decidedly interdisciplinary approach, the volume brings together a wide range of critical reflections, practical perspectives and concrete applications of artificial intelligence in museums, and provides an overview of the current state of the debate

Reading Visual Texts as a Community of Readers: Developing Historical Thinking Skills Among Adolescents Using Historical Photographs

The purpose of this qualitative study was to explore the ways 9th grade students analyzed historical photographs and shared authority over the interpretation of those images. Students are seen as “digital natives” (Prensky, 2001); but most students are not visually literate and are left with the crucial task of making sense of visual sources on their own. There is limited research on student ability to read and interpret historical visual images. As suggested by Werner (2002), agency to read visual texts emerges when students have the capacities to read and interpret visual texts and share authority over meaning making. Data collection consisted of semi-structured interviews, analyses of student annotated visual texts and other documents, video recorded observations, as well as student audio blog reflections. Data obtained in this study addressed the following questions: How do 9th grade U.S. history students build the capacities to read and interpret historical photographs using the heuristics of sourcing, contextualization, and corroboration? How is shared authority demonstrated when a group of 9th grade U.S. history students have multiple opportunities to read and interpret historical photographs? The results of this study suggest that the participants’ continuous reading and subsequent analysis of visual texts was an ongoing, scaffolded, iterative process, one intertwined with a growing understanding of and ability to use heuristics to interpret historical photographs. Asking factual questions made composing complex conceptual questions possible. Conceptual questioning, and to a lesser extent factual questioning, reflects a student’s emerging ability to interpret visual evidence, frame theories, and think like a historian. As students become more proficient questioning and using the three heuristics of sourcing, contextualization, and corroboration, the authority over meaning making may shift to the students. As a result, students learned to control overly imaginative thinking, to rely on themselves to make meaning and to trust their own interpretations. Findings indicate that agency emerges through an iterative process as students ask conceptual questions, solve problems, and intuitively read visual texts with more authority to make meaning for themselves and for their peers

Towards a pedagogical framework for construction of historicity: a case of using Wikis among pre-service teachers at Makerere University

This thesis originates from the realization that the pedagogy of history is becoming dangerously obsolete, as it does not always relate to the contemporary needs of 21st century learners, who often find learning history irrelevant to their present situation. This challenge is attributed to, among other reasons, the way history is taught employing largely behaviorist pedagogies with significantly reduced active learner engagement and little alignment to the way today's students learn. Gadamer's historical hermeneutic theory was employed to advocate for a dialogical approach between the past (part) and the present (whole) mediated by Emerging Technologies, specifically Wikis. Thus, the study is guided by three research questions: firstly, how is historicity constructed on the Wiki platform among pre-service teachers at Makerere University? Secondly, how is authenticity of history meanings constructed among pre-service teachers? Thirdly, what design principles guide a pedagogical framework for construction of historicity? A Design Based Research Methodology (DBR), with theoretically informed solutions aligned to the study problem, was used among pre-service teachers enrolled at Makerere University, Uganda, for the period 2013-2016. Consequently, four phases of DBR were employed: identification of the problem by the researcher in collaboration with practitioners; development of solutions informed by existing design principles and technological innovation; iterative cycles of testing and refinement of solutions and finally, reflection to produce design principles and enhance solutions (Reeves, 2006). Data from questionnaires, interviews and observations on the Wiki was gathered and analyzed through a hermeneutic cycle-driven analysis during DBR phase three. Key findings demonstrated that historicity is constructed through dialogical engagements between educator/researcher and students mediated on the Wiki. Authenticity of history meanings is achieved through collaborative editing, reviewing and sharing understandings on a Wiki. The practical contribution of this research lies in the creation of design principles (i.e. connecting with the present, appreciating heritage, dialogue in history, doing history, validating history and applying history) and a pedagogical framework to be used for the construction of historicity mediated by Wikis, while the theoretical contribution lies in the methodological approach of using DBR to systematically implement and operationalize historical hermeneutics theoretical constructs in History Education in the Ugandan context

The Domestication of Voice Activated -Technology & EavesMining: Surveillance, Privacy and Gender Relations at Home

This thesis develops a case study analysis of the Amazon Echo, the first-ever voice-activated smart speaker. The domestication of the devices feminine conversational agent, Alexa, and the integration of its microphone and digital sensor technology in home environments represents a moment of radical change in the domestic sphere. This development is interpreted according to two primary force relations: historical gender patterns of domestic servitude and eavesmining (eavesdropping + datamining) processes of knowledge extraction and analysis. The thesis is framed around three pillars of study that together demonstrate: how routinization with voice-activated technology affects acoustic space and ones experiences of home; how online warm experts initiate a dialogue about the domestication of technology that disregards and ignores Amazons corporate privacy framework; and finally, how the technologys conditions of use silently result in the deployment of ever-intensifying surveillance mechanisms in home environments. Eavesmining processes are beginning to construct a new world of media and surveillance where every spoken word can potentially be heard and recorded, and speaking is inseparable from identification