Gesture-based Object Recognition using Histograms of Guiding Strokes

Sadeghipour A, Morency L-P, Kopp S. Gesture-based Object Recognition using Histograms of Guiding Strokes. In: Bowden R, Collomosse J, Mikolajczyk K, eds. Proceedings of the British Machine Vision Conference. BMVA Press; 2012: 44.1-44.11

Drawing on the World: sketch in context

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2009.Cataloged from PDF version of thesis.Includes bibliographical references (p. 109-111).This thesis introduces the idea that combining sketch recognition with contextual data-information about what is being drawn on-can improve the recognition of meaning in sketch and enrich the user interaction experience. I created a language called StepStool that facilitates the description of the relationship between digital ink and contextual data, and wrote the corresponding interpreter that enables my system to distinguish between gestural commands issued to an autonomous forklift. A user study was done to compare the correctness of a sketch interface with and without context on the canvas. This thesis coins the phrase "Drawing on the World" to mean contextual sketch recognition, describes the implementation and methodology behind "Drawing on the World", describes the forklift's interface, and discusses other possible uses for a contextual gesture recognizer. Sample code is provided that describes the specifics of the StepStool engine's implementation and the implementation of the forklift's interface.by Andrew Correa.S.M

Toward semantic model generation from sketch and multi-touch interactions

Designers usually start their design process by exploring and evolving their ideas rapidly through sketching since this helps them to make numerous attempts at creating, practicing, simulating, and representing ideas. Creativity inherent in solving the ill-defined problems (Eastman, 1969) often emerges when designers explore potential solutions while sketching in the design process (Schön, 1992). When using computer programs such as CAD or Building Information Modeling (BIM) tools, designers often preplan the tasks prior to executing commands instead of engaging in the process of designing. Researchers argue that these programs force designers to focus on how to use a tool (i.e. how to execute series of commands) rather than how to explore a design, and thus hinder creativity in the early stages of the design process (Goel, 1995; Dorta, 2007). Since recent design and documentation works have been computer-generated using BIM software, transitions between ideas in sketches and those in digital CAD systems have become necessary. By employing sketch interactions, we argue that a computer system can provide a rapid, flexible, and iterative method to create 3D models with sufficient data for facilitating smooth transitions between designers’ early sketches and BIM programs. This dissertation begins by describing the modern design workflows and discussing the necessary data to be exchanged in the early stage of design. It then briefly introduces the modern cognitive theories, including embodiment (Varela, Rosch, & Thompson, 1992), situated action (Suchman, 1986), and distributed cognition (Hutchins, 1995). It continues by identifying problems in current CAD programs used in the early stage of the design process, using these theories as lenses. After reviewing modern attempts, including sketch tools and design automation tools, we describe the design and implementation of a sketch and multi-touch program, SolidSketch, to facilitate and augment our abilities to work on ill-defined problems in the early stage of design. SolidSketch is a parametric modeling program that enables users to construct 3D parametric models rapidly through sketch and multi-touch interactions. It combines the benefits of traditional design tools, such as physical models and pencil sketches (i.e. rapid, low-cost, and flexible methods), with the computational power offered by digital modeling tools, such as CAD. To close the gap between modern BIM and traditional sketch tools, the models created with SolidSketch can be read by other BIM programs. We then evaluate the programs with comparisons to the commercial CAD programs and other sketch programs. We also report a case study in which participants used the system for their design explorations. Finally, we conclude with the potential impacts of this new technology and the next steps for ultimately bringing greater computational power to the early stages of design.Ph.D

Automated Objective Surgical Skill Assessment and Visualization in the Operating Room Using Unstructured Tool Motion for Improved Surgical Training

Past attempts at surgical skill assessment using tool motion in the operating room have focused on highly-structured surgical tasks such as suturing. These methods considered only generic descriptive metrics such as the operating time and the number of movements made, which are of limited instructional value. In this thesis, we develop and evaluate an automated method of surgical skill assessment of flap elevation in nasal septoplasty in the operating room. The obstructed field of view and highly unstructured nature of septoplasty hinders trainees from efficiently learning how to effectively perform the procedure. Thus, we also present the development of a real-time visualization system that allows trainees and instructors to better observe tool motion with respect to patient anatomy during the operation. In this work, we propose a descriptive structure of septoplasty that consists of the following two activity types: (1) the brushing activity directed away from the septum plane that characterizes the consistency of the surgeon’s wrist motion and (2) the activity along the septal plane that characterizes the surgeon’s coverage pattern. We computed features related to these activity types that allow classification of a surgeon’s level of training with an average accuracy of about 72%. Further, as opposed to previously-measured generic motion metrics, the presented features provide surgeons with personalized, actionable feedback regarding their tool motion

Drawing, Handwriting Processing Analysis: New Advances and Challenges

International audienceDrawing and handwriting are communicational skills that are fundamental in geopolitical, ideological and technological evolutions of all time. drawingand handwriting are still useful in defining innovative applications in numerous fields. In this regard, researchers have to solve new problems like those related to the manner in which drawing and handwriting become an efficient way to command various connected objects; or to validate graphomotor skills as evident and objective sources of data useful in the study of human beings, their capabilities and their limits from birth to decline