Staging Transformations for Multimodal Web Interaction Management

Multimodal interfaces are becoming increasingly ubiquitous with the advent of mobile devices, accessibility considerations, and novel software technologies that combine diverse interaction media. In addition to improving access and delivery capabilities, such interfaces enable flexible and personalized dialogs with websites, much like a conversation between humans. In this paper, we present a software framework for multimodal web interaction management that supports mixed-initiative dialogs between users and websites. A mixed-initiative dialog is one where the user and the website take turns changing the flow of interaction. The framework supports the functional specification and realization of such dialogs using staging transformations -- a theory for representing and reasoning about dialogs based on partial input. It supports multiple interaction interfaces, and offers sessioning, caching, and co-ordination functions through the use of an interaction manager. Two case studies are presented to illustrate the promise of this approach.Comment: Describes framework and software architecture for multimodal web interaction managemen

Reading Next: A Vision for Action and Research in Middle and High School Literacy

Outlines fifteen key elements that educators can use to develop an effective adolescent literacy intervention program. Focuses on elements of interventions that are most promising for students that struggle with reading and writing after third grade

A comparative study of visual cues for annotation-based navigation support in adaptive educational hypermedia

Adaptive link annotation is one of the most well-known adaptive navigation support technologies that aims to guide hypermedia users to the most relevant information by personalizing the appearance of hyperlinks. Past work assumed no difference between different interface implementations of personalization approaches that are conceptually the same. The goal of the current study was to determine whether the choice of visual cues does matter by conducting a user study with several alternative designs for link annotation in interactive code examples

Analyzing Digital Literacy Demands, Practices, and Discourses within a Library Computer Programming Club for Children

abstract: Among researchers, educators, and other stakeholders in literacy education, there has been a growing emphasis on developing literacy pedagogies that are more responsive to the ways young people experience literacy in their everyday lives, which often make use of digital media and other technologies for exchanging meaning. This dissertation project sought to explore the nature of these digital-age literacies in the context of children learning through and about new technologies. Conducting a year-long, multimethod observational study of an out-of-school library-based program designed to engage students in self-directed learning around the domain of computer programming, this project was framed around an analysis of digital-age literacies in design, discourse, and practice. To address each of these areas, the project developed a methodology grounded in interpretive, naturalistic, and participant-observation methodologies in collaboration with a local library Code Club in a metropolitan area of the Southwestern U.S between September 2016 and December 2017. Participants in the project included a total of 47 students aged 8-14, 3 librarians, and 3 parents. Data sources for the project included (1) artifactual data, such as the designed interfaces of the online platforms students regularly engaged with, (2) observational data such as protocol-based field notes taken during and after each Code Club meeting, and (3) interview data, collected during qualitative interviews with students, parents, and library facilitators outside the program. These data sources were analyzed through a multi-method interpretive framework, including the multimodal analysis of digital artifacts, qualitative coding, and discourse analysis. The findings of the project illustrate the multidimensional nature of digital-age literacy experiences as they are rendered “on the screen” at the content level, “behind the screen” at the procedural level, and “beyond the screen” at the contextual level. The project contributes to the literature on literacy education by taking an multi-method, interdisciplinary approach to expand analytical perspectives on digital media and literacy in a digital age, while also providing an empirical account of this approach in a community-embedded context of implementation.Dissertation/ThesisDoctoral Dissertation Learning, Literacies and Technologies 201

The Roles of Video in the Design, Development, and Use of Interactive Electronic Conference Proceedings

In this paper, we discuss the design and development of a particular type of electronic publication that has gained recent popularity: electronic conference proceedings. We suggest that modern electronic proceedings should provide a high degree of interactivity. To support such interactivity, proceedings should include an extensive collection of features and diverse multimedia components. Features appropriate for electronic proceedings include annotation, presentation, and retrieval mechanisms. Conference papers and multimedia reproductions of conference presentations with features that allow readers to manipulate these reproductions particularly enhance the interactivity of electronic proceedings. Experience from interactive proceedings the authors have designed is also discussed. Special attention is given to the multiple roles video elements can and should play in interactive proceedings

Sequence and emphasis in automated domain-independent discourse generation

For humans to gain comprehensive views of large amounts of repository contents, they need to have insight into the relations among information objects. It is a challenge to automatically generate presentations of repository contents, through, for example, search results, which reveal such relations to readers. Such presentations must reflect properties of information objects such that large sets of information objects appear as a coherent whole. An approach to this is generation of discourse structures that convey such properties of information objects in presentations. Semantic Web technology provides a conceptual basis for generation of discourse in Web-based information environments. This paper describes automatic generation of sequence and emphasis in presentations of information objects. It shows generation of object sequences and emphasi

Design of Participatory Virtual Reality System for visualizing an intelligent adaptive cyberspace

The concept of 'Virtual Intelligence' is proposed as an intelligent adaptive interaction between the simulated 3-D dynamic environment and the 3-D dynamic virtual image of the participant in the cyberspace created by a virtual reality system. A system design for such interaction is realised utilising only a stereoscopic optical head-mounted LCD display with an ultrasonic head tracker, a pair of gesture-controlled fibre optic gloves and, a speech recogni(ion and synthesiser device, which are all connected to a Pentium computer. A 3-D dynamic environment is created by physically-based modelling and rendering in real-time and modification of existing object description files by afractals-based Morph software. It is supported by an extensive library of audio and video functions, and functions characterising the dynamics of various objects. The multimedia database files so created are retrieved or manipulated by intelligent hypermedia navigation and intelligent integration with existing information. Speech commands control the dynamics of the environment and the corresponding multimedia databases. The concept of a virtual camera developed by ZeIter as well as Thalmann and Thalmann, as automated by Noma and Okada, can be applied for dynamically relating the orientation and actions of the virtual image of the participant with respect to the simulated environment. Utilising the fibre optic gloves, gesture-based commands are given by the participant for controlling his 3-D virtual image using a gesture language. Optimal estimation methods and dataflow techniques enable synchronisation between the commands of the participant expressed through the gesture language and his 3-D dynamic virtual image. Utilising a framework, developed earlier by the author, for adaptive computational control of distribute multimedia systems, the data access required for the environment as well as the virtual image of the participant can be endowed with adaptive capability

A practical approach to language complexity: a wikipedia case study

In this paper we present statistical analysis of English texts from Wikipedia. We try to address the issue of language complexity empirically by comparing the simple English Wikipedia (Simple) to comparable samples of the main English Wikipedia (Main). Simple is supposed to use a more simplified language with a limited vocabulary, and editors are explicitly requested to follow this guideline, yet in practice the vocabulary richness of both samples are at the same level. Detailed analysis of longer units (n-grams of words and part of speech tags) shows that the language of Simple is less complex than that of Main primarily due to the use of shorter sentences, as opposed to drastically simplified syntax or vocabulary. Comparing the two language varieties by the Gunning readability index supports this conclusion. We also report on the topical dependence of language complexity, that is, that the language is more advanced in conceptual articles compared to person-based (biographical) and object-based articles. Finally, we investigate the relation between conflict and language complexity by analyzing the content of the talk pages associated to controversial and peacefully developing articles, concluding that controversy has the effect of reducing language complexity