WEBCAM-BASED LASER DOT DETECTION TECHNIQUE IN COMPUTER REMOTE CONTROL

ABSTRACTIn this paper, the authors propose a method to detect the laser dot in an interactive system using laser pointers. The method is designed for presenters who need to interact with the computer during the presentation by using the laserpointer. The detection technique is developed by using a camera to capture the presentation screen and processing every frames transferred to the ara computer. This paper focuses on the detection and tracking of laser dots, based on their characteristics to distinguish a laser dotfrom other areas on the captured frames. Experimental results showed that the proposed method could reduce the rate of misdetection by light noises of a factor of 10 and achieve an average accuracy of 82% of detection in normal presentation environments. The results point out that the better way to describe the laser dots’ features based on visual concept is to use the HSI color space instead of the normal RGB space.Keywords.  laser pointer; laser dot/spot; laser pointer interaction; control; mouse; computer screen/display

Query-by-Pointing: Algorithms and Pointing Error Compensation

People typically communicate by pointing, talking, sketching, writing, and typing. Pointing can be used to visualize or exchange information about an object when there is no other mutually understood way of communication. Despite its proven expressiveness, however, it has not yet become a frequently used modality to interact with computer systems. With the rapid move towards the adoption of mobile technologies, geographic information systems (GISs) have a particular need for advanced forms of interaction that enable users to query the geographic world directly. To enable pointing-based query system on a handheld device, a number of fundamental technical challenges have to be overcome. For such a system to materialize we need models stored in the device\u27s knowledge base that can be used as surrogate of real world objects. These computations, however, assume that (1) the pointing direction matches with the line-of-sight and (2) the observations about location and direction are precise enough so that a computational model will determine the same object as what the user points at. Both assumptions are not true. This thesis, therefore, develops an efficient error compensation model to reduce the discrepancy between the line-of-sight of the eye and the pointer direction. The model is based on a coordinate system centered at the neck and distances measured from neck to eye, neck to shoulder, shoulder to handheld pointer, and the pointing direction. An experiment was conducted using a gyro-enhanced sensor and three subjects who pointed at marked targets in a given room. It showed that the error compensation algorithm significantly reduces errors in pointing with arms outstretched

Physical Selection in Ubiquitous Computing

Jokapaikan tietotekniikassa (ubiquitous computing) tietotekniset laitteet sulautuvat fyysiseen ympäristöön siten että niiden käyttäjät voivat olla yhtä aikaa vuorovaikutuksessa näiden laitteiden kanssa ja toimia fyysisessä ympäristössään. Laitteet ovat yhteydessä toisiinsa, ne ovat eri kokoisia ja niillä on erilaisia syöttö- ja tulostusmahdollisuuksia tarkoituksestaan riippuen. Nämä jokapaikan tietotekniikan ominaisuudet luovat tarpeen vuorovaikutustavoille, jotka eroavat huomattavasti tavanomaisten työpöytätietokoneiden vuorovaikutustavoista. Fyysinen valinta (physical selection) on jokapaikan tietotekniikan vuorovaikutustehtävä, jota käytetään kertomaan käyttäjän kannettavalle päätelaitteelle minkä fyysisen esineen kanssa käyttäjä haluaa olla vuorovaikutuksessa. Fyysinen valinta perustuu tunnisteisiin (tag), jotka yksilöivät fyysiset esineet tai sisältävät fyysisen hyperlinkin digitaalisessa muodossa olevaan tietoon, joka liittyy esineeseen, johon kyseinen tunniste on liitetty. Käyttäjä valitsee fyysisen hyperlinkin koskettamalla, osoittamalla tai skannaamalla tunnistetta sopivalla lukulaitteella varustetulla päätelaitteellaan. Fyysinen valinta voidaan toteuttaa erilaisilla teknologioilla, kuten sähköisesti luettavilla tunnisteilla ja niiden lukijoilla, infrapunalähettimillä sekä optisesti luettavilla tunnisteilla ja matkapuhelinten kameroilla. Tässä väitöskirjassa analysoidaan fyysistä valintaa vuorovaikutustehtävänä ja toteutusteknisestä näkökulmasta sekä esitellään eri valintatavat ­ kosketus, osoitus ja skannaus. Koskeusta ja osoitusta on tutkittu toteuttamalla prototyyppi ja tutkimalla sen avulla valintatapoja kokeellisesti. Tämän väitöskirjan tuloksiin kuuluu fyysisen valinnan analysointi jokapaikan tietotekniikan kontekstissa, ehdotuksia fyysisten hyperlinkkien visualisoinnista sekä fyysisessä ympäristössä että päätelaitteessa, ja käyttäjävaatimuksia fyysiselle valinnalle osana jokapaikan tietotekniikan arkkitehtuuria.In ubiquitous computing, the computing devices are embedded into the physical environment so that the users can interact with the devices at the same time as they interact with the physical environment. The various devices are connected to each other, and have various sizes and input and output capabilities depending on their purpose. These features of ubiquitous computing create a need for interaction methods that are radically different from the desktop computer interactions. Physical selection is an interaction task for ubiquitous computing and it is used to tell the user s mobile terminal which physical object the user wants to interact with. It is based on tags that identify physical objects or store a physical hyperlink to digital information related to the object the tag is attached to. The user selects the physical hyperlink by touching, pointing or scanning the tag with the mobile terminal that is equipped with an appropriate reader. Physical selection has been implemented with various technologies, such as radio-frequency tags and readers, infrared transceivers, and optically readable tags and mobile phone cameras. In this dissertation, physical selection is analysed as a user interaction task, and from the implementation viewpoint. Different selection methods ­ touching, pointing and scanning ­ are presented. Touching and pointing have been studied by implementing a prototype and conducting user experiments with it. The contributions of this dissertation include an analysis of physical selection in the ubiquitous computing context, suggestions for visualising the physical hyperlinks in both the physical environment and in the mobile terminal, and user requirements for physical selection as a part of an ambient intelligence architecture

Physical Selection in Ubiquitous Computing

Jokapaikan tietotekniikassa (ubiquitous computing) tietotekniset laitteet sulautuvat fyysiseen ympäristöön siten että niiden käyttäjät voivat olla yhtä aikaa vuorovaikutuksessa näiden laitteiden kanssa ja toimia fyysisessä ympäristössään. Laitteet ovat yhteydessä toisiinsa, ne ovat eri kokoisia ja niillä on erilaisia syöttö- ja tulostusmahdollisuuksia tarkoituksestaan riippuen. Nämä jokapaikan tietotekniikan ominaisuudet luovat tarpeen vuorovaikutustavoille, jotka eroavat huomattavasti tavanomaisten työpöytätietokoneiden vuorovaikutustavoista. Fyysinen valinta (physical selection) on jokapaikan tietotekniikan vuorovaikutustehtävä, jota käytetään kertomaan käyttäjän kannettavalle päätelaitteelle minkä fyysisen esineen kanssa käyttäjä haluaa olla vuorovaikutuksessa. Fyysinen valinta perustuu tunnisteisiin (tag), jotka yksilöivät fyysiset esineet tai sisältävät fyysisen hyperlinkin digitaalisessa muodossa olevaan tietoon, joka liittyy esineeseen, johon kyseinen tunniste on liitetty. Käyttäjä valitsee fyysisen hyperlinkin koskettamalla, osoittamalla tai skannaamalla tunnistetta sopivalla lukulaitteella varustetulla päätelaitteellaan. Fyysinen valinta voidaan toteuttaa erilaisilla teknologioilla, kuten sähköisesti luettavilla tunnisteilla ja niiden lukijoilla, infrapunalähettimillä sekä optisesti luettavilla tunnisteilla ja matkapuhelinten kameroilla. Tässä väitöskirjassa analysoidaan fyysistä valintaa vuorovaikutustehtävänä ja toteutusteknisestä näkökulmasta sekä esitellään eri valintatavat ­ kosketus, osoitus ja skannaus. Koskeusta ja osoitusta on tutkittu toteuttamalla prototyyppi ja tutkimalla sen avulla valintatapoja kokeellisesti. Tämän väitöskirjan tuloksiin kuuluu fyysisen valinnan analysointi jokapaikan tietotekniikan kontekstissa, ehdotuksia fyysisten hyperlinkkien visualisoinnista sekä fyysisessä ympäristössä että päätelaitteessa, ja käyttäjävaatimuksia fyysiselle valinnalle osana jokapaikan tietotekniikan arkkitehtuuria.In ubiquitous computing, the computing devices are embedded into the physical environment so that the users can interact with the devices at the same time as they interact with the physical environment. The various devices are connected to each other, and have various sizes and input and output capabilities depending on their purpose. These features of ubiquitous computing create a need for interaction methods that are radically different from the desktop computer interactions. Physical selection is an interaction task for ubiquitous computing and it is used to tell the user s mobile terminal which physical object the user wants to interact with. It is based on tags that identify physical objects or store a physical hyperlink to digital information related to the object the tag is attached to. The user selects the physical hyperlink by touching, pointing or scanning the tag with the mobile terminal that is equipped with an appropriate reader. Physical selection has been implemented with various technologies, such as radio-frequency tags and readers, infrared transceivers, and optically readable tags and mobile phone cameras. In this dissertation, physical selection is analysed as a user interaction task, and from the implementation viewpoint. Different selection methods ­ touching, pointing and scanning ­ are presented. Touching and pointing have been studied by implementing a prototype and conducting user experiments with it. The contributions of this dissertation include an analysis of physical selection in the ubiquitous computing context, suggestions for visualising the physical hyperlinks in both the physical environment and in the mobile terminal, and user requirements for physical selection as a part of an ambient intelligence architecture

Interaktion mit Medienfassaden : Design und Implementierung interaktiver Systeme für große urbane Displays

Media facades are a prominent example of the digital augmentation of urban spaces. They denote the concept of turning the surface of a building into a large-scale urban screen. Due to their enormous size, they require interaction at a distance and they have a high level of visibility. Additionally, they are situated in a highly dynamic urban environment with rapidly changing conditions, which results in settings that are neither comparable, nor reproducible. Altogether, this makes the development of interactive media facade installations a challenging task. This thesis investigates the design of interactive installations for media facades holistically. A theoretical analysis of the design space for interactive installations for media facades is conducted to derive taxonomies to put media facade installations into context. Along with this, a set of observations and guidelines is provided to derive properties of the interaction from the technical characteristics of an interactive media facade installation. This thesis further provides three novel interaction techniques addressing the form factor and resolution of the facade, without the need for additionally instrumenting the space around the facades. The thesis contributes to the design of interactive media facade installations by providing a generalized media facade toolkit for rapid prototyping and simulating interactive media facade installations, independent of the media facade’s size, form factor, technology and underlying hardware.Die wachsende Zahl an Medienfassenden ist ein eindrucksvolles Beispiel für die digitale Erweiterung des öffentlichen Raums. Medienfassaden beschreiben die Möglichkeit, die Oberfläche eines Gebäudes in ein digitales Display zu wandeln. Ihre Größe erfordert Interaktion aus einer gewissen Distanz und führt zu einer großen Sichtbarkeit der dargestellten Inhalte. Medienfassaden-Installationen sind bedingt durch ihre dynamische Umgebung nur schwerlich vergleich- und reproduzierbar. All dies macht die Entwicklung von Installationen für Medienfassaden zu einer großen Herausforderung. Diese Arbeit beschäftigt sich mit der Entwicklung interaktiver Installationen für Medienfassaden. Es wird eine theoretische Analyse des Design-Spaces interaktiver Medienfassaden-Installationen durchgeführt und es werden Taxonomien entwickelt, die Medienfassaden-Installationen in Bezug zueinander setzen. In diesem Zusammenhang werden ausgehend von den technischen Charakteristika Eigenschaften der Interaktion erarbeitet. Zur Interaktion mit Medienfassaden werden drei neue Interaktionstechniken vorgestellt, die Form und Auflösung der Fassade berücksichtigen, ohne notwendigerweise die Umgebung der Fassade zu instrumentieren. Die Ergebnisse dieser Arbeit verbessern darüber hinaus die Entwicklung von Installationen für Medienfassaden, indem ein einheitliches Medienfassaden-Toolkit zum Rapid-Prototyping und zur Simulation interaktiver Installationen vorgestellt wird, das unabhängig von Größe und Form der Medienfassade sowie unabhängig von der verwendeten Technologie und der zugrunde liegenden Hardware ist

Cross-display attention switching in mobile interaction with large displays

Mobile devices equipped with features (e.g., camera, network connectivity and media player) are increasingly being used for different tasks such as web browsing, document reading and photography. While the portability of mobile devices makes them desirable for pervasive access to information, their small screen real-estate often imposes restrictions on the amount of information that can be displayed and manipulated on them. On the other hand, large displays have become commonplace in many outdoor as well as indoor environments. While they provide an efficient way of presenting and disseminating information, they provide little support for digital interactivity or physical accessibility. Researchers argue that mobile phones provide an efficient and portable way of interacting with large displays, and the latter can overcome the limitations of the small screens of mobile devices by providing a larger presentation and interaction space. However, distributing user interface (UI) elements across a mobile device and a large display can cause switching of visual attention and that may affect task performance. This thesis specifically explores how the switching of visual attention across a handheld mobile device and a vertical large display can affect a single user's task performance during mobile interaction with large displays. It introduces a taxonomy based on the factors associated with the visual arrangement of Multi Display User Interfaces (MDUIs) that can influence visual attention switching during interaction with MDUIs. It presents an empirical analysis of the effects of different distributions of input and output across mobile and large displays on the user's task performance, subjective workload and preference in the multiple-widget selection task, and in visual search tasks with maps, texts and photos. Experimental results show that the selection of multiple widgets replicated on the mobile device as well as on the large display, versus those shown only on the large display, is faster despite the cost of initial attention switching in the former. On the other hand, a hybrid UI configuration where the visual output is distributed across the mobile and large displays is the worst, or equivalent to the worst, configuration in all the visual search tasks. A mobile device-controlled large display configuration performs best in the map search task and equal to best (i.e., tied with a mobile-only configuration) in text- and photo-search tasks