Estimating Residual Error Rate in Recognized Handwritten Documents Using Artificial Error Injection

ABSTRACT Both handwriting recognition systems and their users are error prone. Handwriting recognizers make recognition errors, and users may miss those errors when verifying output. As a result, it is common for recognized documents to contain residual errors. Unfortunately, in some application domains (e.g. health informatics), tolerance for residual errors in recognized handwriting may be very low, and a desire might exist to maximize user accuracy during verification. In this paper, we present a technique that allows us to measure the performance of a user verifying recognizer output. We inject artificial errors into a set of recognized handwritten forms and show that the rate of injected errors and recognition errors caught is highly correlated in real time. Systems supporting user verification can make use of this measure of user accuracy in a variety of ways. For example, they can force users to slow down or can highlight injected errors that were missed, thus encouraging users to take more care

Providing integrated toolkit-level support for ambiguity in recognition-based interfaces

Recognition technologies are being used extensively in both the commercial and research worlds. But recognizers are still error-prone, and this results in performance problems and brittle dialogues. These problems are a barrier to acceptance and usefulness of recognition systems. Better interfaces to recognition systems, which can help to reduce the burden of recognition errors, are difficult to build because of lack of knowledge about the ambiguity inherent in recognition. We have extended a user interface toolkit in order to model and to provide structured support for ambiguity at the input event level [7]. This makes it possible to build re-usable interface components for resolving ambiguity and dealing with recognition errors. These interfaces can help to reduce the negative effects of recognition errors. By providing these components at a toolkit level, we make it easier for application writers to provide good support for error handling. And we can explore new types of interfaces for resolving a more varied range of ambiguity

Providing Integrated Toolkit-Level Support for Ambiguity in Recognition-Based Interfaces

Interfaces based on recognition technologies are used extensively in both the commercial and research worlds. But recognizers are still error-prone, and this results in human performance problems, brittle dialogues, and other barriers to acceptance and utility of recognition systems. Interface techniques specialized to recognition systems can help reduce the burden of recognition errors, but building these interfaces depends on knowledge about the ambiguity inherent in recognition. We have extended a user interface toolkit in order to model and to provide structured support for ambiguity at the input event level. This makes it possible to build re-usable interface components for resolving ambiguity and dealing with recognition errors. These interfaces can help to reduce the negative effects of recognition errors. By providing these components at a toolkit level, we make it easier for application writers to provide good support for error handling. Further, with this robust support, we a..

Conceptual model for usable multi-modal mobile assistance during Umrah

Performing Umrah is very demanding and to be performed in very crowded environments. In response to that, many efforts have been initiated to overcome the difficulties faced by pilgrims. However, those efforts focus on acquiring initial perspective and background knowledge before going to Mecca. Findings of preliminary study show that those efforts do not support multi-modality for user interaction. Nowadays the computational capabilities in mobile phones enable it to serve people in various aspects of daily life. Consequently, the mobile phone penetration has increased dramatically in the last decade. Hence, this study aims to propose a comprehensive conceptual model for usable multimodal mobile assistance during Umrah called Multi-model Mobile Assistance during Umrah (MMA-U). Thus, four (4) supporting objectives are formulated, and the Design Science Research Methodology has been adopted. For the usability of MMA-U, Systematic Literature Review (SLR) indicates ten (10) attributes: usefulness, errors rate, simplicity, reliability, ease of use, safety, flexibility, accessibility, attitude, and acceptability. Meanwhile, the content and comparative analysis result in five (5) components that construct the conceptual model of MMA-U: structural, content composition, design principles, development approach, technology, and the design and usability theories. Then, the MMA-U has been reviewed and well-accepted by 15 experts. Later, the MMA-U was incorporated into a prototype called Personal Digital Mutawwif (PDM). The PDM was developed for the purpose of user test in the field. The findings indicate that PDM facilitates the execution of Umrah and successfully meet pilgrims’ needs and expectations. Also, the pilgrims were satisfied and felt that they need to have PDM. In fact, they would recommend PDM to their friends, which mean that use of PDM is safe and suitable while performing Umrah. As a conclusion, the theoretical contribution; the conceptual model of MMA-U; provides guidelines for developing multimodal content mobile applications during Umrah

Today’s voice-based user interfaces:analysis of six paradoxes

Abstract. This master’s thesis was about voice-based user interfaces, including the history of voice as an interactive medium, acceptance of the medium among users, and modern implementations of the interfaces. Initial problems of early implementations of voice-based user interfaces were about human-machine communication, as natural language of humans is ambiguous and highly interpretable, causing difficulties to machines when interpreting the given queries. Other topics of interest with the interfaces included implementing human-like cues to machines in order to make them more appealing to users, and how humans see inanimate objects when communicating with them. The problems of machines’ understanding of humans have been reduced greatly, and modern voice-based conversational virtual assistants emerged in the early 2010s. These assistants were examined: what they are, what they have achieved and what does the future hold for them. To evaluate voice as an interactive medium and the assistants, conceptual framework of six distinct paradoxes was used. Prior research, from history to today’s issues, was used to reflect against the conducted research

Handling ambiguous user input on touchscreen kiosks

Thesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2005.Includes bibliographical references (p. 87-94).Touchscreen kiosks are becoming an increasingly popular means of providing a wide arrange of services to the public. However, the principal drawback of these types of systems lies within the elevated error rates due to finger imprecision and screen miscalibration. These issues become worrisome, considering the greater responsibilities and reliance placed upon touchscreens. This thesis investigates two novel techniques that attempt to alleviate these interaction problems. The first technique, predictive pointing, incorporates information regarding past interactions and an area cursor (which maps the user's touch to a circular area rather than a single point) to provide a better estimate of the intended selection. The second technique, gestural drawing, allows users to draw particular shapes onscreen to execute actions as an alternative means of input that is largely unaffected by issues of miscalibration. Results from a user study indicated that both techniques provided significant advantages in not only lowering error rates, but also improving task completion times over traditional tasks of target selection.by Christopher K. Leung.M.Eng

The WOZ Recognizer: A Tool For Understanding User Perceptions of Sketch-Based Interfaces

Sketch recognition has the potential to be an important input method for computers in the coming years; however, designing and building an accurate and sophisticated sketch recognition system is a time consuming and daunting task. Since sketch recognition is still at a level where mistakes are common, it is important to understand how users perceive and tolerate recognition errors and other user interface elements with these imperfect systems. A problem in performing this type of research is that we cannot easily control aspects of recognition in order to rigorously study the systems. We performed a study examining user perceptions of three pen-based systems for creating logic gate diagrams: a sketch-based interface, a WIMP-based interface, and a hybrid interface that combined elements of sketching and WIMP. We found that users preferred the sketch-based interface and we identified important criteria for pen-based application design. This work exposed the issue of studying recognition systems without fine-grained control over accuracy, recognition mode, and other recognizer properties. In order to solve this problem, we developed a Wizard of Oz sketch recognition tool, the WOZ Recognizer, that supports controlled symbol and position accuracy and batch and streaming recognition modes for a variety of sketching domains. We present the design of the WOZ Recognizer, modeling recognition domains using graphs, symbol alphabets, and grammars; and discuss the types of recognition errors we included in its design. Further, we discuss how the WOZ Recognizer simulates sketch recognition, controlling the WOZ Recognizer, and how users interact with it. In addition, we present an evaluative user study of the WOZ Recognizer and the lessons we learned. We have used the WOZ Recognizer to perform two user studies examining user perceptions of sketch recognition; both studies focused on mathematical sketching. In the first study, we examined whether users prefer recognition feedback now (real-time recognition) or later (batch recognition) in relation to different recognition accuracies and sketch complexities. We found that participants displayed a preference for real-time recognition in some situations (multiple expressions, low accuracy), but no statistical preference in others. In our second study, we examined whether users displayed a greater tolerance for recognition errors when they used mathematical sketching applications they found interesting or useful compared to applications they found less interesting. Participants felt they had a greater tolerance for the applications they preferred, although our statistical analysis did not positively support this. In addition to the research already performed, we propose several avenues for future research into user perceptions of sketch recognition that we believe will be of value to sketch recognizer researchers and application designers

Internet on mobiles: evolution of usability and user experience

The mobile Internet is no longer a new phenomenon; the first mobile devices supporting web access were introduced over 10 years ago. During the past ten years technology and business infrastructure have evolved and the number of mobile Internet users has increased all over the world. Service user interface, technology and business infrastructure have built a framework for service adaptation: they can act as enablers or as barriers. Users evaluate how the new technology adds value to their life based on multiple factors. This dissertation has its focus in the area of human-computer interaction research and practices. The overall goal of my research has been to improve the usability and the user experience of mobile Internet services. My research has sought answers to questions relevant in service development process. Questions have varied during the years, the main question being: How to design and create mobile Internet services that people can use and want to use? I have sought answers mostly from a human factors perspective, but have also taken the elements form technology and business infrastructure into consideration. In order to answer the questions raised in service development projects, we have investigated the mobile Internet services in the laboratory and in the field. My research has been conducted in various countries in 3 continents: Asia, Europe and North America. These studies revealed differences in mobile Internet use in different countries and between user groups. Studies in this dissertation were conducted between years 1998 and 2007 and show how questions and research methods have evolved during the time. Good service creation requires that all three factors: technology, business infrastructure and users are taken in consideration. When using knowledge on users in decision making, it is important to understand that the different phases of the service development cycle require the different kind of information on users. It is not enough to know about the users, the knowledge about users has to be transferred into decisions. The service has to be easy to use so that people can use it. This is related to usability. Usability is a very important factor in service adoption, but it is not enough. The service has to have relevant content from user perspective. The content is the reason why people want to use the service. In addition to the content and the ease of use, people evaluate the goodness of the service based on many other aspects: the cost, the availability and the reliability of the system for example. A good service is worth trying and after the first experience, is it worth using. These aspects are considered to influence the 'user experience' of the system. In this work I use lexical analysis to evaluate how the words "usability" and "user experience" are used in mobile HCI conference papers during the past 10 years. The use of both words has increased during the period and reflects the evolution of research questions and methodology over time

Rethinking Pen Input Interaction: Enabling Freehand Sketching Through Improved Primitive Recognition

Online sketch recognition uses machine learning and artificial intelligence techniques to interpret markings made by users via an electronic stylus or pen. The goal of sketch recognition is to understand the intention and meaning of a particular user's drawing. Diagramming applications have been the primary beneficiaries of sketch recognition technology, as it is commonplace for the users of these tools to rst create a rough sketch of a diagram on paper before translating it into a machine understandable model, using computer-aided design tools, which can then be used to perform simulations or other meaningful tasks. Traditional methods for performing sketch recognition can be broken down into three distinct categories: appearance-based, gesture-based, and geometric-based. Although each approach has its advantages and disadvantages, geometric-based methods have proven to be the most generalizable for multi-domain recognition. Tools, such as the LADDER symbol description language, have shown to be capable of recognizing sketches from over 30 different domains using generalizable, geometric techniques. The LADDER system is limited, however, in the fact that it uses a low-level recognizer that supports only a few primitive shapes, the building blocks for describing higher-level symbols. Systems which support a larger number of primitive shapes have been shown to have questionable accuracies as the number of primitives increase, or they place constraints on how users must input shapes (e.g. circles can only be drawn in a clockwise motion; rectangles must be drawn starting at the top-left corner). This dissertation allows for a significant growth in the possibility of free-sketch recognition systems, those which place little to no drawing constraints on users. In this dissertation, we describe multiple techniques to recognize upwards of 18 primitive shapes while maintaining high accuracy. We also provide methods for producing confidence values and generating multiple interpretations, and explore the difficulties of recognizing multi-stroke primitives. In addition, we show the need for a standardized data repository for sketch recognition algorithm testing and propose SOUSA (sketch-based online user study application), our online system for performing and sharing user study sketch data. Finally, we will show how the principles we have learned through our work extend to other domains, including activity recognition using trained hand posture cues