A Video Analysis of Eye Movements during Typing: How Effective is Handwriting during Note-Taking Tasks?

Keyboard input for non-alphabetical languages, such as Chinese and Japanese, is problematic because it is labor intensive and imposes a high cognitive load. In our previous work, we measured the effectiveness of handwriting during a note-taking task in Japanese, and found that the input speed during note-taking was higher by hand than by keyboard. The results also showed that the quality of notes taken by hand was higher than that of notes taken by keyboard, and this might have been due to the higher cognitive load during typing. In addition, observation during the experiment revealed several problems subjects faced in the keyboard input task. To evaluate the significance of these observations, we had to obtain quantitative evidence through further study of participant behavior. Therefore, we repeated the experiment, this time with video analysis of the keyboard subtask. By analyzing the participants’ eye movements and their behavior throughout the keyboard subtask we obtained quantitative evidence to support our findings from the previous study. Here, we describe this experiment and our findings in detail

Making Sense of Document Collections with Map-Based Visualizations

As map-based visualizations of documents become more ubiquitous, there is a greater need for them to support intellectual and creative high-level cognitive activities with collections of non-cartographic materials -- documents. This dissertation concerns the conceptualization of map-based visualizations as tools for sensemaking and collection understanding. As such, map-based visualizations would help people use georeferenced documents to develop understanding, gain insight, discover knowledge, and construct meaning. This dissertation explores the role of graphical representations (such as maps, Kohonen maps, pie charts, and other) and interactions with them for developing map-based visualizations capable of facilitating sensemaking activities such as collection understanding. While graphical representations make document collections more perceptually and cognitively accessible, interactions allow users to adapt representations to users’ contextual needs. By interacting with representations of documents or collections and being able to construct representations of their own, people are better able to make sense of information, comprehend complex structures, and integrate new information into their existing mental models. In sum, representations and interactions may reduce cognitive load and consequently expedite the overall time necessary for completion of sensemaking activities, which typically take much time to accomplish. The dissertation proceeds in three phases. The first phase develops a conceptual framework for translating ontological properties of collections to representations and for supporting visual tasks by means of graphical representations. The second phase concerns the cognitive benefits of interaction. It conceptualizes how interactions can help people during complex sensemaking activities. Although the interactions are explained on the example of a prototype built with Google Maps, they are independent iv of Google Maps and can be applicable to various other technologies. The third phase evaluates the utility, analytical capabilities and usability of the additional representations when users interact with a visualization prototype – VIsual COLlection EXplorer. The findings suggest that additional representations can enhance understanding of map-based visualizations of library collections: specifically, they can allow users to see trends, gaps, and patterns in ontological properties of collections

Understanding the “Galakei” Appeal; A Study of Language and Interaction

Qualitative research methodologies were used in order to better understand how well currently available mobile phone interfaces are designed to match the way Japanese is thought of by native speakers. A conceptual design is presented at the end as a proposal for how to better design a mobile interface for writing Japanese

Role of Semantic web in the changing context of Enterprise Collaboration

In order to compete with the global giants, enterprises are concentrating on their core competencies and collaborating with organizations that compliment their skills and core activities. The current trend is to develop temporary alliances of independent enterprises, in which companies can come together to share skills, core competencies and resources. However, knowledge sharing and communication among multidiscipline companies is a complex and challenging problem. In a collaborative environment, the meaning of knowledge is drastically affected by the context in which it is viewed and interpreted; thus necessitating the treatment of structure as well as semantics of the data stored in enterprise repositories. Keeping the present market and technological scenario in mind, this research aims to propose tools and techniques that can enable companies to assimilate distributed information resources and achieve their business goals

Genie: A Generator of Natural Language Semantic Parsers for Virtual Assistant Commands

To understand diverse natural language commands, virtual assistants today are trained with numerous labor-intensive, manually annotated sentences. This paper presents a methodology and the Genie toolkit that can handle new compound commands with significantly less manual effort. We advocate formalizing the capability of virtual assistants with a Virtual Assistant Programming Language (VAPL) and using a neural semantic parser to translate natural language into VAPL code. Genie needs only a small realistic set of input sentences for validating the neural model. Developers write templates to synthesize data; Genie uses crowdsourced paraphrases and data augmentation, along with the synthesized data, to train a semantic parser. We also propose design principles that make VAPL languages amenable to natural language translation. We apply these principles to revise ThingTalk, the language used by the Almond virtual assistant. We use Genie to build the first semantic parser that can support compound virtual assistants commands with unquoted free-form parameters. Genie achieves a 62% accuracy on realistic user inputs. We demonstrate Genie's generality by showing a 19% and 31% improvement over the previous state of the art on a music skill, aggregate functions, and access control.Comment: To appear in PLDI 201

Report on the 2015 NSF Workshop on Unified Annotation Tooling

On March 30 & 31, 2015, an international group of twenty-three researchers with expertise in linguistic annotation convened in Sunny Isles Beach, Florida to discuss problems with and potential solutions for the state of linguistic annotation tooling. The participants comprised 14 researchers from the U.S. and 9 from outside the U.S., with 7 countries and 4 continents represented, and hailed from fields and specialties including computational linguistics, artificial intelligence, speech processing, multi-modal data processing, clinical & medical natural language processing, linguistics, documentary linguistics, sign-language linguistics, corpus linguistics, and the digital humanities. The motivating problem of the workshop was the balkanization of annotation tooling, namely, that even though linguistic annotation requires sophisticated tool support to efficiently generate high-quality data, the landscape of tools for the field is fractured, incompatible, inconsistent, and lacks key capabilities. The overall goal of the workshop was to chart the way forward, centering on five key questions: (1) What are the problems with current tool landscape? (2) What are the possible benefits of solving some or all of these problems? (3) What capabilities are most needed? (4) How should we go about implementing these capabilities? And, (5) How should we ensure longevity and sustainability of the solution? I surveyed the participants before their arrival, which provided significant raw material for ideas, and the workshop discussion itself resulted in identification of ten specific classes of problems, five sets of most-needed capabilities. Importantly, we identified annotation project managers in computational linguistics as the key recipients and users of any solution, thereby succinctly addressing questions about the scope and audience of potential solutions. We discussed management and sustainability of potential solutions at length. The participants agreed on sixteen recommendations for future work. This technical report contains a detailed discussion of all these topics, a point-by-point review of the discussion in the workshop as it unfolded, detailed information on the participants and their expertise, and the summarized data from the surveys

CoMMA Corporate Memory Management through Agents Corporate Memory Management through Agents: The CoMMA project final report

This document is the final report of the CoMMA project. It gives an overview of the different search activities that have been achieved through the project. First, a description of the general requirements is proposed through the definition of two scenarios. Then it shows the different technical aspects of the projects and the solution that has been proposed and implemented