An infrastructure for the development of Semantic Desktop applications

In einem permanent wachsenden Ausmaß wird unser Leben digital organisiert. Viele tagtägliche Aktivitäten manifestieren sich (auch) in digitaler Form: einerseits explizit, wenn digitale Informationen für Arbeitsaufgaben oder in der Freizeit entstehen und verwendet werden; andererseits auch implizit, wenn Informationen indirekt, als Konsequenz unseres Handelns, erzeugt oder manipuliert wird. Ein großer Teil dieser Informationsbestände ist persönlicher Natur, d.h., diese Information hat einen bestimmten Bezug zu uns als Person. Die Speicher- und Rechenleistung der Geräte, mit denen wir üblicherweise mit solchen persönlichen Daten interagieren, wurde in den letzten Jahren kontinuierlich erhöht, und es besteht Grund zur Annahme, dass sich diese Entwicklung in der Zukunft fortsetzt. Während also die physische Leistung von Datenspeichern enorm erhöht wurde, hat deren logische und organisatorische Leistung seit der Erfindung der ersten Personal Computer praktisch stagniert. Nach wie vor sind hierarchische Dateisysteme der de-facto-Standard für die Organisation von persönlichen Daten. Solche Dateisysteme repräsentieren Daten als diskrete Einheiten (Dateien), die Blätter eines Baums von beschrifteten Knoten (Verzeichnisse) darstellen. Die Unterteilung des persönlichen Datenraums in kleine Einheiten unterstützt die Handhabung solcher Strukturen durch den Menschen, allerdings können viele Arten von Organisationsinformation nicht adäquat in einer Baumstruktur dargestellt werden. Dies wirkt sich negativ auf die Qualität der Datenorganisation aus. Aktuelle Forschung im Bereich Personal Information Management liefert zwar mögliche Ansätze, um hierarchische Systeme zu ersetzen, tendiert jedoch manchmal dazu, die Arbeit mit Information überzuformalisieren. Dies ist insbesondere kritisch, weil der durchschnittliche Anwender von PIM-Systemen über keine Erfahrung mit komplexen logischen Systemen verfügt. Diese Arbeit präsentiert ein alternatives Organisationsmodell für persönliche Daten, die darauf abzielt, eine Balance zwischen der unstrukturierten Charakteristik von Dateisystemen und den formalen Eigenschaften von logik-basierten Systemen zu finden. Nach einer vergleichenden Studie der aktuellen Forschungssituation im Bereich Semantic Desktop und Personal Information Management wird dieses Modell auf drei Ebenen vorgestellt. Zunächst wird ein abstraktes Modell sowie eine Abfrage-Algebra in Form von abstrakten Operationen auf dieses Modell vorgestellt. Dieses Modell erlaubt die Abbildung von im Personal Information Management gebräuchlichen Daten, aber erfordert keine völlige Umstellung auf Seiten des Benutzers. Anschließend wird dieses abstrakte Modell in konkreten Repräsentationen übergeführt, und es wird gezeigt, wie diese Repräsentationen effizient bearbeitet, gespeichert, und ausgetauscht werden können. Schließlich wird die Anwendung dieses Modells anhand von konkreten prototypischen Implementierungen gezeigt.The extent to which our daily lives are digitized is continuously growing. Many of our everyday activities manifest themselves in digital form; either in an explicit way, when we actively use digital information for work or spare time; or in an implicit way, when information is indirectly created or manipulated as a consequence of our action. A large fraction of these data volumes can be considered as personal information, that is, information that has a certain class of relationship to us as human beings. The storage and processing capacity of the devices that we use to interact with these data has been enormously increasing over the last years, and we can expect this development to continue in the future. However, while the power of physical data storage is permanently increasing, the development of logical data organization power of personal devices has been stagnating since the invention of the first personal computers. Still, hierarchical file systems are the de-facto standard for data organization on personal devices. File systems represent information as a set of discrete data units (files) that are arranged as leaves on a tree of labeled nodes (directories). This structure, on the one hand, can be easily understood by humans, since the separation into small information units supports the manual manageability of the personal data space, in comparison to systems that employ continuous data structures. On the other hand, hierarchical structures suffer from a number of deficiencies which have negative impact on the quality of personal information management, and it lacks of expressive mechanisms which in turn would help to improve information retrieval according to user needs. Significant research effort has been invested in order to improve the mechanisms for personal information management. The resulting works represent potential alternatives or supplements for systems in place, but sometimes run the risk of over-formalizing information management; a problem that is especially apparent in situations where a non-expert end user is the direct consumer of such services. The contribution of this thesis is to present an alternative organizational model for management of personal data that strikes a balance between the unstructured nature of file systems and the highly formal characteristics of logic-based systems. After a comparative analysis of the current situation and recent research effort in this direction, it describes this organizational metaphor on three levels: First, on a conceptual level, it discusses an abstract data model, a corresponding query algebra, and a set of abstract operations on this data model. This formal framework is suitable to represent common data structures and usage patterns that can be found in personal information management, but on the same time does not enforce a complete paradigm shift away from established systems. Second, on a representation level, it discusses how this model can be efficiently processed, stored, and exchanged between different systems. Third, on an implementation level, it describes how concrete realizations of this data model can be built and used in various application scenarios

User interfaces for the organisation and representation of unstructured data

Der semantische Ansatz bezüglich der Speicherung, Verwaltung und Extraktion von Daten ist ein vielversprechendes Konzept, das über die letzten Jahre konsequent an Bedeutung gewonnen hat. Die Visualisierung solcher semantsicher Datensysteme weist allerdings immer noch einge Hindernisse auf, sowohl für den Interface-designer als auch für den interagierenden Benutzer. Diese Hindernisse zu überwinden ist die Aufgabe des Interaktion-designers, der durch den Einsatz von intuitiven und effizienten Design-komponenten dem unerfahrenen Benutzer den Einstieg in eine, auf semantischen Daten basierenden, Interaktionsumbegung erleichtern soll. Der Zweck dieser Arbeit ist es einen Überblick zu verschaffen über die Disziplinen des Human Computer Interaction sowie des Interaction Designs, mit der Absicht die grundlegenden menschlichen und technischen Prozesse zu vermitteln, die einen Einfluss auf das Entwerfen von Interaktion haben. Es werden Methoden und Techniken für den eigentlichen Design-prozess vorgestellt, ebenso wie Elemente und Komponenten die eine Rolle in visuellem Interface-design spielen, mit einem schwerpunkt auf die Darstellung semantischer Inhalte. Letztendlich wird ein web-basierendes, graphisches User-interface für SemDAV Repositories vorgestellt, dessen Entwurf auf den gewonnen Erkenntnissen beruht und dessen Implementierung sich nach den aktuellen Entwicklungen im Webdesign richtet.The semantic approach to data storage, management and extraction is a promising concept that has constantly been gaining importance during the past years. However, the visualization of semantic data systems as well as the functionalities of their interfaces still contain some obstacles for both, the interaction designer and the interacting user. In order to accommodate to those obstacles, the interaction designer has to come up with an intuitive and efficient user interface design, introducing inexperienced users to data representation and extraction in a semantic environment. The intention of this work is to provide an overview over the disciplines of Human Computer Interaction and Interaction Design, with the intention to mediate the essential human-based and technical processes influencing the design of interaction. Methods and techniques for the design process will be presented, as well as elements and components of visual design and interaction, with a focus on the representation of semantic content. Finally, a web-based graphical user interface for SemDAV repositories will be introduced, combining the gained insights with the ongoing development in web design

An Abstraction Framework for Tangible Interactive Surfaces

This cumulative dissertation discusses - by the example of four subsequent publications - the various layers of a tangible interaction framework, which has been developed in conjunction with an electronic musical instrument with a tabletop tangible user interface. Based on the experiences that have been collected during the design and implementation of that particular musical application, this research mainly concentrates on the definition of a general-purpose abstraction model for the encapsulation of physical interface components that are commonly employed in the context of an interactive surface environment. Along with a detailed description of the underlying abstraction model, this dissertation also describes an actual implementation in the form of a detailed protocol syntax, which constitutes the common element of a distributed architecture for the construction of surface-based tangible user interfaces. The initial implementation of the presented abstraction model within an actual application toolkit is comprised of the TUIO protocol and the related computer-vision based object and multi-touch tracking software reacTIVision, along with its principal application within the Reactable synthesizer. The dissertation concludes with an evaluation and extension of the initial TUIO model, by presenting TUIO2 - a next generation abstraction model designed for a more comprehensive range of tangible interaction platforms and related application scenarios

Building blocks for semantic data organization on the desktop

Die Organisation von (Multimedia-) Daten auf Desktop-Systemen wird derzeit hauptsächlich durch das Einordnen von Dateien in ein hierarchisches Dateisystem bewerkstelligt. Zusätzlich werden gewisse Inhalte (z.B. Musik oder Fotos) von spezialisierter Software mit Hilfe Datei-bezogener Metadaten verwaltet. Diese Metadaten werden meist direkt im Dateikopf in einer Unzahl verschiedener, vorwiegend proprietärer Formate gespeichert. Allgemein nehmen Metadaten und Links die Schlüsselrollen in fortgeschrittenen Datenorganisationskonzepten ein, ihre eingeschränkte Unterstützung in vorherrschenden Dateisystemen macht die Einführung solcher Konzepte auf dem Desktop jedoch schwierig: Erstens müssen Anwendungen sowohl Dateiformat als auch Metadatenschema verstehen um auf Metadaten zugreifen zu können; zweitens ist ein getrennter Zugriff auf Daten und Metadaten nicht möglich und drittens kann man solche Metadaten nicht mit mehreren Dateien oder mit Dateiordnern assoziieren obgleich letztere die derzeit wichtigsten Konstrukte für die Dateiorganisation darstellen. Dies bedeutet in weiterer Folge: (i) eingeschränkte Möglichkeiten der Datenorganisation, (ii) eingeschränkte Navigationsmöglichkeiten, (iii) schlechte Auffindbarkeit der gespeicherten Daten, und (iv) Fragmentierung von Metadaten. Obschon es Versuche gab, diese Situation (zum Beispiel mit Hilfe semantischer Dateisysteme) zu verbessern, wurden die meisten dieser Probleme bisher vor allem im Web und im Speziellen im semantischen Web adressiert und gelöst. Das Anwenden dort entwickelter Lösungen auf dem Desktop, einer zentralen Plattform der Daten- und Metadatenmanipulation, wäre zweifellos von Vorteil. In der vorliegenden Arbeit wird ein neues, rückwärts-kompatibles Metadatenmodell als Lösungsversuch für die oben genannten Probleme präsentiert. Dieses Modell basiert auf stabilen Datei-Identifikatoren und externen, semantischen, Datei- bezogenen Metadatenbeschreibungen welche im RDF Graphenmodell repräsentiert werden. Diese Beschreibungen sind durch eine einheitliche Linked-Data- Schnittstelle zugänglich und können mit anderen Beschreibungen und Ressourcen verlinkt werden. Im Speziellen erlaubt dieses Modell semantische Links zwischen lokalen Dateisystemobjekten und Netzressourcen im Web sowie im entstehenden “Daten Web” und ermöglicht somit die Integration dieser Datenräume. Das Modell hängt entscheidend von der Stabilität dieser Links ab weshalb zwei Algorithmen präsentiert werden, welche deren Integrität in lokalen und vernetzten Umgebungen erhalten können. Dies bedeutet, dass Links zwischen Dateisystemobjekten, Metadatenbeschreibungen und Netzressourcen nicht brechen wenn sich deren Adressen ändern, z.B. wenn Dateien verschoben oder Linked-Data Ressourcen unter geänderten URIs publiziert werden. Schließlich wird eine prototypische Implementierung des vorgeschlagenen Metadatenmodells präsentiert, welche demonstriert wie die Summe dieser Bausteine eine Metadatenschicht bildet die als Grundlage für semantische Datenorganisation auf dem Desktop verwendet werden kann.The organization of (multimedia) data on current desktop systems is done to a large part by arranging files in hierarchical file systems, but also by specialized applications (e.g., music or photo organizing software) that make use of file-related metadata for this task. These metadata are predominantly stored in embedded file headers, using a magnitude of mainly proprietary formats. Generally, metadata and links play the key roles in advanced data organization concepts. Their limited support in prevalent file system implementations, however, hinders the adoption of such concepts on the desktop: First, non-uniform access interfaces require metadata consuming applications to understand both a file’s format and its metadata scheme; second, separate data/metadata access is not possible, and third, metadata cannot be attached to multiple files or to file folders although the latter are the primary constructs for file organization. As a consequence of this, current desktops suffer, inter alia, from (i) limited data organization possibilities, (ii) limited navigability, (iii) limited data findability, and (iv) metadata fragmentation. Although there were attempts to improve this situation, e.g., by introducing semantic file systems, most of these issues were successfully addressed and solved in the Web and in particular in the Semantic Web and reusing these solutions on the desktop, a central hub of data and metadata manipulation, is clearly desirable. In this thesis a novel, backwards-compatible metadata model that addresses the above-mentioned issues is introduced. This model is based on stable file identifiers and external, file-related, semantic metadata descriptions that are represented using the generic RDF graph model. Descriptions are accessible via a uniform Linked Data interface and can be linked with other descriptions and resources. In particular, this model enables semantic linking between local file system objects and remote resources on the Web or the emerging Web of Data, thereby enabling the integration of these data spaces. As the model crucially relies on the stability of these links, we contribute two algorithms that preserve their integrity in local and in remote environments. This means that links between file system objects, metadata descriptions and remote resources do not break even if their addresses change, e.g., when files are moved or Linked Data resources are re-published using different URIs. Finally, we contribute a prototypical implementation of the proposed metadata model that demonstrates how these building blocks sum up to constitute a metadata layer that may act as a foundation for semantic data organization on the desktop

A Novel, Tag-Based File-System

For decades, computer use has largely focused on managing and manipulating files-- creating and consuming media, browsing the web, software development, and even, with such systems as UNIX and Plan$9$, direct device access can largely be reduced to locating, creating, reading, and writing files. To facilitate these operations, developers have created a vast assortment of file-systems, each presenting a unique framework underlying nearly everything people do with a computer. For various reasons, these file-systems have historically represented only incremental improvements and alterations from their predecessors, leaving the basic design and interaction models relatively unchanged. Because of this, most common file-systems share a similar set of weaknesses and limitations, intrinsic to those models. As an attempt to break with these traditional shortcomings, the author has created STUFFS, a Semantically-Tagged Unstructured Future File-System. It is intended largely as a research platform for investigating fundamentally new ideas in storing, locating, managing, and otherwise manipulating files, their data, and their associated meta-data. As such, STUFFS does not claim to perfectly solve all of these problems -- rather, it serves as a proof-of-concept and testbed for a number of promising new approaches. Of these new features, users are likely most impacted by STUFFS\u27s titular tag-based structure, which spurns the traditional folder hierarchy in favor of a folksonomy inspired, tag-centric approach to file organization. While this change retains backwards compatibility, and is therefore fully usable as a traditional FS, it has profound impact on potential user interaction. In order to support this high level transition, STUFFS is implemented using a relational database for storage and tag-resolution, and, as an exciting side effect, it has gained proper transaction support and full ACID compliance

Ontology based support of small groups

Der Einsatz von Ontologien wird bereits in vielen Anwendungsbereichen als Werkzeug für die Strukturierung und die Verbesserung der Zugänglichkeit von Informationen unterschiedlichster Art genutzt. Sie ermöglichen die explizite Formulierung der Bedeutung von Konzepten und Strukturen beliebiger Domänen. Auch im Rahmen der Zusammenarbeit in und zwischen Gruppen ist der Austausch und die Verarbeitung von Informationen für den Verlauf und den Erfolg der Kooperation von erheblicher Bedeutung. Daher liegt es nahe, auch kollaborative Aktivitäten durch den Einsatz von Ontologien zu unterstützen. Aktuelle Arbeiten in diesem Themenbereich fokussieren jedoch meist auf ausgewählte Aspekte der Zusammenarbeit wie etwa der Kommunikation zwischen den Gruppenmitgliedern oder die Unterstützung durch eine konkrete Softwarekomponente. In dieser Arbeit wird dagegen von einer ganzheitlichen Betrachtung von Kooperationssituationen ausgegangen. Dabei werden die an einer Kooperation beteiligten Personen und die eingesetzten technischen Komponenten als ein gesamtes soziotechnisches System betrachtet, dessen Elemente nicht losgelöst voneinander betrachtet werden können. Das Ziel dieser Arbeit besteht einerseits darin, zu untersuchen, wie sich der Einsatz von Ontologien auf die Unterstützung der Zusammenarbeit auswirkt und andererseits, welche Möglichkeiten sich hieraus für die Gestaltung von Kooperationssystemen ableiten lassen. Einige dieser Möglichkeiten werden im praktischen Teil prototypisch implementiert, um die technische und wirtschaftliche Umsetzbarkeit zu evaluieren.The usage of ontologies is already common in many felds of application to structure diferent kinds of informations and to make them accessible; they further more make it possible to make the meaning of concepts and structures of any domains explicit. In connection with cooperation in and between groups the exchange and processing of informations about the progress and success of cooperation is of considerable meaning. Because of that it seems reasonable to suppose that collaborative activities should be supported by using ontologies. Current works in this feld often focus only on single aspects of cooperation; for example the communication between members of the group or supporting them by using particular software components. This thesis is emanated from the aspect of an integral way of viewing cooperative situations. Especially the cooperation of the involved persons and the technical components should be looked at a sociotechnical system on the whole and not the elements seperated. The aim of this thesis - on the one side - is to determine the impact of using ontologies on supported cooperative work and - on the other side - which possibilities can be derived from that for the design of cooperative systems. Some of these possibilities will be implemented in a prototyp to evaluate the technical and economic feasibility

Detekcija i proaktivna isporuka informacija o saobraćajnim događajjima u mobilnim informacionim sistemima za podršku navigaciji i transportu

The research subject of this dissertation is using traffic participants (drivers, vehicles and mobile devices inside vehicles) as sources of traffic data further used in providing dynamic navigation service and traffic management. Motivation for this approach to detecting and disseminating traffic data is found in inefficiency of adequatelly covering large road network with traditional traffic sensors. Traditional traffic sensors include inductive loops, cameras etc. which are static and permanently integegrated with traffic infrastructure during construction. Participatory sensing concept is not new and is applied in the traffic domain form some years now. Widelly popular social network found their way into traffic domain too, and today there are manz commercially available social networks gathering drivers and allowing them to exchange information. This model of information exchange usually amounts to manual input of messages which are redistributed to other drivers by the system. Floating car Data (FCD) concept eliminates the driver from the information input loop. By anonymously collecting data about current speeds of a large number of vehicles information about traffic congestions can be constructed and used in various types of information systems. Extended Floating car Data (XFCD) concept, this dissertation deals with, expands dataset collected from traffic participants with data from various sensors inside vehicle. This dissertation especially focuses on one particular type of sensor commonly found today integrated with mobile devices used by drivers for navigation, accelerometer. Most important characteristic of this type of sensor is its capability to detect relevant vehicle maneuvers by recording force patterns that act on the vehicle (mobile device) during these maneuvers. GPS receiver traditionally used in FCD systems is typically not capable to detect these types of maneuvers. This dissertation demonstrates accelerometer data analysis methods localized on mobile device tasked with detecting relevant traffic events (vehicle maneuvers). This information collected from participating vehicles is further used in implemented prototype of proactive traffic information service that uses this data to timely warn drivers about hazardous traffic events and conditions existing on their navigation route. Localised analysis of accelerometer device on mobile devices used by the drivers is conditioned by the characteristics of accelerometer data, its sheer volume, and implemented prototype of proactive traffic information service acts preemptively by increasing drivers’ situational awareness of traffic conditions ahead of them

Multimodal Content Delivery for Geo-services

This thesis describes a body of work carried out over several research projects in the area of multimodal interaction for location-based services. Research in this area has progressed from using simulated mobile environments to demonstrate the visual modality, to the ubiquitous delivery of rich media using multimodal interfaces (geo- services). To effectively deliver these services, research focused on innovative solutions to real-world problems in a number of disciplines including geo-location, mobile spatial interaction, location-based services, rich media interfaces and auditory user interfaces. My original contributions to knowledge are made in the areas of multimodal interaction underpinned by advances in geo-location technology and supported by the proliferation of mobile device technology into modern life. Accurate positioning is a known problem for location-based services, contributions in the area of mobile positioning demonstrate a hybrid positioning technology for mobile devices that uses terrestrial beacons to trilaterate position. Information overload is an active concern for location-based applications that struggle to manage large amounts of data, contributions in the area of egocentric visibility that filter data based on field-of-view demonstrate novel forms of multimodal input. One of the more pertinent characteristics of these applications is the delivery or output modality employed (auditory, visual or tactile). Further contributions in the area of multimodal content delivery are made, where multiple modalities are used to deliver information using graphical user interfaces, tactile interfaces and more notably auditory user interfaces. It is demonstrated how a combination of these interfaces can be used to synergistically deliver context sensitive rich media to users - in a responsive way - based on usage scenarios that consider the affordance of the device, the geographical position and bearing of the device and also the location of the device

Interaction with embodied media

Thesis (Ph. D.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2009.Includes bibliographical references (p. 213-222).The graphical user interface has become the de facto metaphor for the majority of our diverse activities using computers, yet the desktop environment provides a one size fits all user interface. This dissertation argues that for the computer to fully realize its potential to significantly extend our intellectual abilities, new interaction techniques must call upon our bodily abilities to manipulate objects, enable collaborative work, and be usable in our everyday physical environment. In this dissertation I introduce a new human-computer interaction concept, embodied media. An embodied media system physically represents digital content such as files, variables, or other program constructs with a collection of self-contained, interactive electronic tokens that can display visual feedback and can be manipulated gesturally by users as a single, coordinated interface. Such a system relies minimally on external sensing infrastructure compared to tabletop or augmented reality systems, and is a more general-purpose platform than most tangible user interfaces. I hypothesized that embodied media interfaces provide advantages for activities that require the user to efficiently arrange and adjust multiple digital content items. Siftables is the first instantiation of an embodied media interface. I built 180 Siftable devices in three design iterations, and developed a programming interface and various applications to explore the possibilities of embodied media.(cont.) In a survey, outside developers reported that Siftables created new user interface possibilities, and that working with Siftables increased their interest in human-computer interaction and expanded their ideas about the field. I evaluated a content organization application with users, finding that Siftables offered an advantage over the mouse+graphical user interface (GUI) for task completion time that was amplified when participants worked in pairs, and a digital image manipulation application in which participants preferred Siftables to the GUI in terms of enjoyability, expressivity, domain learning, and for exploratory/quick arrangement of items.by David Jeffrey Merrill.Ph.D