Estudo de modos de comando em cenários de interacção gestual

Tese de mestrado, Engenharia Informática (Sistemas de Informação), Universidade de Lisboa, Faculdade de Ciências, 2010Recentemente, tem-se assistido a uma “revolução tecnológica” na concepção de dispositivos computacionais que visam a interacção pessoa-máquina. Os periféricos de entrada deixaram de ser a única forma de transmitir intenções às máquinas, sendo agora possível fazê-lo com o próprio corpo. Dispositivos que permitem interacção por toque estão-se a disseminar por locais públicos, mas não é só nestes locais que o fenómeno se verifica. A quantidade de produtos comerciais que permitem este género de interacção também não pára de aumentar, pelo que é necessário compreender as vantagens e desvantagens da interacção gestual e torná-la cada vez mais eficaz. Existem muitas tecnologias que possibilitam a construção de dispositivos tácteis, variando nas suas capacidades e custos. O estudo dessas tecnologias, no decorrer deste trabalho, resultou na construção de uma mesa interactiva multi-toque de “baixo custo”. Nos dispositivos vocacionados para interacção gestual as dimensões da superfície com a qual é possível interagir são iguais às dimensões do ecrã, o que leva à necessidade de ter uma especial atenção na concepção de aplicações para estes dispositivos. As características de uma interface concebida para um ecrã de grandes dimensões poderão não ser adequadas para um ecrã de dimensões mais reduzidas, e vice-versa. Além das dimensões, o género de aplicação também influencia o paradigma de interacção. No caso específico de interacção gestual em aplicações de desenho existe a dificuldade acrescida da aplicação compreender quando o gesto do utilizador tem por objectivo desenhar ou executar um comando. Neste trabalho são apresentados dois conjuntos de gestos de comando com o objectivo de eliminar a ambiguidade existente entre os gestos em aplicações de desenho. São também apresentadas as conclusões de estudos conduzidos para atestar a qualidade dos conjuntos propostos, assim como a sua adequabilidade relativamente a diferentes dimensões de ecrã.Lately we’ve been witnessing a “technologic revolution” in the making of devices that allow human-computer interaction. Input devices are no longer the only way to instruct intentions to computers. It’s now possible to do the same using one's own body. Devices that allow touch interaction are being disseminated in public places, but it’s not only in those places that the phenomenon occurs. The number of commercial products that allow this kind of interaction doesn’t stop growing. So, it’s of vital importance to understand the advantages and disadvantages of gestural interaction and make it more effective. There are a lot of technologies that allow the construction of tactile devices, going through a wide range of capabilities and manufacturing costs. The study of those technologies, during this work, resulted in the construction of a “low-cost” multi-touch interactive table. In devices oriented for gestural interaction, the dimensions of the surface of interaction are equal to the dimensions of the screen, which demands a special attention in the design of applications for those devices. The features of an interface conceived for a large screen may not be suitable for a screen of smaller dimensions, and vice-versa. Apart from the dimensions, the kind of application can also influence the interaction paradigm. In the specific case of gestural interaction in drawing applications there’s also the increased difficulty of making the application understand when the gesture has the objective of drawing or, instead, executing a command. In this work, two sets of command gestures are introduced, with the goal of disambiguating the intent of gestures in drawing applications. Also presented are the conclusions of studies who aimed to test the quality of the proposed sets, as well as their suitability to multi-sized screens

A software based mentor system

This thesis describes the architecture, implementation issues and evaluation of Mentor - an educational support system designed to mentor students in their university studies. Students can ask (by typing) natural language questions and Mentor will use several educational paradigms to present information from its Knowledge Base or from data-mined online Web sites to respond. Typically the questions focus on the student’s assignments or in their preparation for their examinations. Mentor is also pro-active in that it prompts the student with questions such as "Have you started your assignment yet?". If the student responds and enters into a dialogue with Mentor, then, based upon the student’s questions and answers, it guides them through a Directed Learning Path planned by the lecturer, specific to that assessment. The objectives of the research were to determine if such a system could be designed, developed and applied in a large-scale, real-world environment and to determine if the resulting system was beneficial to students using it. The study was significant in that it provided an analysis of the design and implementation of the system as well as a detailed evaluation of its use. This research integrated the Computer Science disciplines of network communication, natural language parsing, user interface design and software agents, together with pedagogies from the Computer Aided Instruction and Intelligent Tutoring System fields of Education. Collectively, these disciplines provide the foundation for the two main thesis research areas of Dialogue Management and Tutorial Dialogue Systems. The development and analysis of the Mentor System required the design and implementation of an easy to use text based interface as well as a hyper- and multi-media graphical user interface, a client-server system, and a dialogue management system based on an extensible kernel. The multi-user Java-based client-server system used Perl-5 Regular Expression pattern matching for Natural Language Parsing along with a state-based Dialogue Manager and a Knowledge Base marked up using the XML-based Virtual Human Markup Language. The kernel was also used in other Dialogue Management applications such as with computer generated Talking Heads. The system also enabled a user to easily program their own knowledge into the Knowledge Base as well as to program new information retrieval or management tasks so that the system could grow with the user. The overall framework to integrate and manage the above components into a usable system employed suitable educational pedagogies that helped in the student’s learning process. The thesis outlines the learning paradigms used in, and summarises the evaluation of, three course-based Case Studies of university students’ perception of the system to see how effective and useful it was, and whether students benefited from using it. This thesis will demonstrate that Mentor met its objectives and was very successful in helping students with their university studies. As one participant indicated: ‘I couldn’t have done without it.

Interactive visualisation tools for supporting taxonomists working practice.

The necessity for scientists and others to use consistent terminology has recently beenregarded as fundamental to advancing scientific research, particularly where data fromdisparate sources must be shared, compared or integrated. One area where there aresignificant difficulties with the quality of collected data is the field of taxonomicdescription. Taxonomic description lies at the heart of the classification of organismsand communication of ideas of biodiversity. As part of their working practice,taxonomists need to gather descriptive data about a number of specimens on aconsistent basis for individual projects. Collecting semantically well-defined structureddata could improve the clarity and comparability of such data. No tools howevercurrently exist to allow taxonomists to do so within their working practice.Ontologies are increasingly used to describe and define complex domain data. As a partof related research an ontology of descriptive terminology for controlling the storageand use of flowering plant description data was developed.This work has applied and extended model-based user interface developmentenvironments to utilise such an ontology for the automatic generation of appropriatedata entry interfaces that support semantically well defined and structured descriptivedata. The approach taken maps the ontology to a system domain model, which ataxonomist can then specialise using their domain expertise, for their data entry needs asrequired for individual projects. Based on this specialised domain knowledge, thesystem automatically generates appropriate data entry interfaces that capture dataconsistent with the original ontology. Compared with traditional model-based userautomatic interface development environments, this approach also has the potential toreduce the labour requirements for the expert developer.The approach has also been successfully tested to generate data entry interfaces basedon an XML schema for the exchange of biodiversity datasets

Interactive visualisation tools for supporting taxonomists working practice.

The necessity for scientists and others to use consistent terminology has recently beenregarded as fundamental to advancing scientific research, particularly where data fromdisparate sources must be shared, compared or integrated. One area where there aresignificant difficulties with the quality of collected data is the field of taxonomicdescription. Taxonomic description lies at the heart of the classification of organismsand communication of ideas of biodiversity. As part of their working practice,taxonomists need to gather descriptive data about a number of specimens on aconsistent basis for individual projects. Collecting semantically well-defined structureddata could improve the clarity and comparability of such data. No tools howevercurrently exist to allow taxonomists to do so within their working practice.Ontologies are increasingly used to describe and define complex domain data. As a partof related research an ontology of descriptive terminology for controlling the storageand use of flowering plant description data was developed.This work has applied and extended model-based user interface developmentenvironments to utilise such an ontology for the automatic generation of appropriatedata entry interfaces that support semantically well defined and structured descriptivedata. The approach taken maps the ontology to a system domain model, which ataxonomist can then specialise using their domain expertise, for their data entry needs asrequired for individual projects. Based on this specialised domain knowledge, thesystem automatically generates appropriate data entry interfaces that capture dataconsistent with the original ontology. Compared with traditional model-based userautomatic interface development environments, this approach also has the potential toreduce the labour requirements for the expert developer.The approach has also been successfully tested to generate data entry interfaces basedon an XML schema for the exchange of biodiversity datasets