Freehand Interaction with a Paper-Based Input Interface

Depending on their structure, graphical user interfaces may sometimes be not fully comfortable, because they require oculomotor coordination and relatively precise mouse movement. While physical interfaces may be the right choice for purpose-specific appliances, they become less practicable in the case of interfaces for software tools, since each program would need its own physical interface. Moreover, there may be situations in which the input interface is used in dirty environments (such as industrial plants), and needs to be replaced very often. Starting from these premises, this article studies the feasibility of a gesture interaction input approach based on a paper interface, also appraising its efficiency and investigating the user acceptance level. Through a finger, the user can directly operate interface controls either drawn by him or herself or provided in ready-to-use pre-printed sheets. The implemented prototype, based on computer vision, relies on a simple and cheap webcam, and controls consist of sliders, buttons, knobs and trackpads. Experiments have shown that the proposed approach is practicable in terms of both performance and positive reception by users, although some inherent problems of vision-based systems must be taken into account. In addition, users’ opinions have provided feedback about potential application scenarios and possible system improvements. The obtained results can be the starting point for deeper and more focused investigations, driving further research on the topic