ATLAS: A flexible and extensible architecture for linguistic annotation

We describe a formal model for annotating linguistic artifacts, from which we derive an application programming interface (API) to a suite of tools for manipulating these annotations. The abstract logical model provides for a range of storage formats and promotes the reuse of tools that interact through this API. We focus first on ``Annotation Graphs,'' a graph model for annotations on linear signals (such as text and speech) indexed by intervals, for which efficient database storage and querying techniques are applicable. We note how a wide range of existing annotated corpora can be mapped to this annotation graph model. This model is then generalized to encompass a wider variety of linguistic ``signals,'' including both naturally occuring phenomena (as recorded in images, video, multi-modal interactions, etc.), as well as the derived resources that are increasingly important to the engineering of natural language processing systems (such as word lists, dictionaries, aligned bilingual corpora, etc.). We conclude with a review of the current efforts towards implementing key pieces of this architecture.Comment: 8 pages, 9 figure

Collaboration in a multi-user game: impacts of an awareness tool on mutual modeling

This paper presents an experimental research that focuses on collaboration in a multi-player game. The aim of the project is to study the cognitive impacts of awareness tools, i.e., artifacts that allow users of a collaborative system to be aware of what is going on in the joint virtual environment. The focus is on finding an effect on performance as well as on the representation an individual builds of what his partner knows, plans and intends to do (i.e., mutual modeling). We find that using awareness tools has a significant effect by improving task performance. However, the players who were provided with this tool did not show any improvement of their mutual modeling. Further analysis on contrasted groups revealed that there was an effect of the awareness tool on mutual modeling for players who spent a large amount of time using the too

SCOPE FOR USABILITY TESTS IN IS DEVELOPMENT

Despite being a common, established concept in wide usage, usability tests can vary greatly in goals, techniques and results. A usability test purchased and performed for a specific software product, may result in either minor user interface improvements or radical U-turns in the development. Such variation has been discussed as a problem of the scientific reliability and validity of the testing method. In practice it is more important what ‘kind of data’ one can expect of the selected method than whether it is reliably always the same data. This expectation of information content or ‘scope’ is of importance for evaluators, who select and conduct usability tests for a specific purpose. However, the scope is not explicitly stated or even discussed: Too often the premise is that, because a usability test involves users, it brings the (necessary) user-centeredness to the design i.e. takes socio-technical fundamentals as inherently given. Through a literature review of testing practices and analytical considerations, we search for the scope of a usability test, which could deliberately approach the socio-technical tradition and equally develop both the system and the user organization. A case example represents a possible realization of the extended scope of usability test