Formalization and preliminary evaluation of a pipeline for text extraction from infographics

We propose a pipeline for text extraction from infographics that makes use of a novel combination of data mining and computer vision techniques. The pipeline defines a sequence of steps to identify characters, cluster them into text lines, determine their rotation angle, and apply state-of-the-art OCR to recognise the text. In this paper, we formally define the pipeline and present its current implementation. In addition, we have conducted preliminary evaluations over a data corpus of 121 manually annotated infographics from a broad range of illustration types such as bar charts, pie charts, and line charts, maps, and others. We assess the results of our text extraction pipeline by comparing it with two baselines. Finally, we sketch an outline for future work and possibilities for improving the pipeline

THE CROSS-SENSORY GLOBE: Co-Designing a 3D Audio-Tactile Globe Prototype for Blind and Low- Vision Users to Learn Geography

This MRP presents a co-operatively and iteratively designed 3D audio-tactile globe that enables blind and low-vision users to perceive geo-spatial information. Blind and low-vision users rely on learning aids such as 2D-tactile graphics, braille maps and 3D models to learn about geography. I employed co-design as an approach to prototype and evaluate four different iterations of a cross-sensory globe that uses 3D detachable continents to provide geo-spatial haptic information in combination with audio labels. Informed by my co-design and evaluation, I discuss cross-sensory educational aids as an alternative to visually-oriented globes. My findings reveal affordances of 3D-tactile models for conveying concrete features of the Earth (such as varying elevations of landforms) and audio labels for conveying abstract categories about the Earth (such as continent names). I highlight the advantages of longitudinal participatory design that includes the lived experiences and DIY innovations of blind and low-vision users and makers

Ontologies and representation spaces for sketch map interpretation

In this paper, we present a systematic approach to sketch map interpretation. The method decomposes the elements of a sketch map into a hierarchy of categories, from the material sketch map level to the non-material representational sketch map level, and then interprets the sketch map using the five formal representation spaces that we develop. These spaces (set, graph, metric and Euclidean) provide a tiered formal representation based on standard mathematical structures. We take the view that a sketch map bears information about the physical world and systematises this using extensions of existing formal ontologies. The motivation for this work is the partially automatic extraction and integration of information from sketch maps. We propose a set of ontologies and methods as a first step in the direction of a formalisation of partially automatic extraction and integration of sketch map content. We also see this work as a contribution to spatial cognition, where researchers externalise spatial knowledge using sketch mapping. The paper concludes by working through an example that demonstrates the sketch map interpretation at different levels using the underlying method

Towards automated composition of convergent services: a survey

A convergent service is defined as a service that exploits the convergence of communication networks and at the same time takes advantage of features of the Web. Nowadays, building up a convergent service is not trivial, because although there are significant approaches that aim to automate the service composition at different levels in the Web and Telecom domains, selecting the most appropriate approach for specific case studies is complex due to the big amount of involved information and the lack of technical considerations. Thus, in this paper, we identify the relevant phases for convergent service composition and explore the existing approaches and their associated technologies for automating each phase. For each technology, the maturity and results are analysed, as well as the elements that must be considered prior to their application in real scenarios. Furthermore, we provide research directions related to the convergent service composition phases

A Web and Mobile System for Environmental Decision Support

Current field data collection methods for many of today’s scientific and other observer/monitor type applications are still entrenched in the “clipboard age”, requiring manual data transcription to a database management system at some (often much) later date, and only allows for visualisation and analysis of recently captured field data “back in the lab”. This chapter is targeted at progressing today’s pen & paper methodology into the spatially enabled mobile computing age of realtime multi-media data input, integration, visualisation, and analysis simultaneously both in the field and the lab. The system described is customized to the specific needs of the Canadian Great Lakes Laboratory for Fisheries and Aquatic Sciences Fish Habitat Management Group requirements for fish species at risk assessment, but is ready for adaptation to other environmental agency applications (e.g. forestry, health-pesticide monitoring, agriculture, etc.). The chapter is ideally suited to all agencies responsible for collecting field data of any type that have not yet moved to a state-of-the-art mobile and wireless data collection, visualisation, and analysis work methodolog

Tabletop tangible maps and diagrams for visually impaired users

En dépit de leur omniprésence et de leur rôle essentiel dans nos vies professionnelles et personnelles, les représentations graphiques, qu'elles soient numériques ou sur papier, ne sont pas accessibles aux personnes déficientes visuelles car elles ne fournissent pas d'informations tactiles. Par ailleurs, les inégalités d'accès à ces représentations ne cessent de s'accroître ; grâce au développement de représentations graphiques dynamiques et disponibles en ligne, les personnes voyantes peuvent non seulement accéder à de grandes quantités de données, mais aussi interagir avec ces données par le biais de fonctionnalités avancées (changement d'échelle, sélection des données à afficher, etc.). En revanche, pour les personnes déficientes visuelles, les techniques actuellement utilisées pour rendre accessibles les cartes et les diagrammes nécessitent l'intervention de spécialistes et ne permettent pas la création de représentations interactives. Cependant, les récentes avancées dans le domaine de l'adaptation automatique de contenus laissent entrevoir, dans les prochaines années, une augmentation de la quantité de contenus adaptés. Cette augmentation doit aller de pair avec le développement de dispositifs utilisables et abordables en mesure de supporter l'affichage de représentations interactives et rapidement modifiables, tout en étant accessibles aux personnes déficientes visuelles. Certains prototypes de recherche s'appuient sur une représentation numérique seulement : ils peuvent être instantanément modifiés mais ne fournissent que très peu de retour tactile, ce qui rend leur exploration complexe d'un point de vue cognitif et impose de fortes contraintes sur le contenu. D'autres prototypes s'appuient sur une représentation numérique et physique : bien qu'ils puissent être explorés tactilement, ce qui est un réel avantage, ils nécessitent un support tactile qui empêche toute modification rapide. Quant aux dispositifs similaires à des tablettes Braille, mais avec des milliers de picots, leur coût est prohibitif. L'objectif de cette thèse est de pallier les limitations de ces approches en étudiant comment développer des cartes et diagrammes interactifs physiques, modifiables et abordables. Pour cela, nous nous appuyons sur un type d'interface qui a rarement été étudié pour des utilisateurs déficients visuels : les interfaces tangibles, et plus particulièrement les interfaces tangibles sur table. Dans ces interfaces, des objets physiques représentent des informations numériques et peuvent être manipulés par l'utilisateur pour interagir avec le système, ou par le système lui-même pour refléter un changement du modèle numérique - on parle alors d'interfaces tangibles sur tables animées, ou actuated. Grâce à la conception, au développement et à l'évaluation de trois interfaces tangibles sur table (les Tangible Reels, la Tangible Box et BotMap), nous proposons un ensemble de solutions techniques répondant aux spécificités des interfaces tangibles pour des personnes déficientes visuelles, ainsi que de nouvelles techniques d'interaction non-visuelles, notamment pour la reconstruction d'une carte ou d'un diagramme et l'exploration de cartes de type " Pan & Zoom ". D'un point de vue théorique, nous proposons aussi une nouvelle classification pour les dispositifs interactifs accessibles.Despite their omnipresence and essential role in our everyday lives, online and printed graphical representations are inaccessible to visually impaired people because they cannot be explored using the sense of touch. The gap between sighted and visually impaired people's access to graphical representations is constantly growing due to the increasing development and availability of online and dynamic representations that not only give sighted people the opportunity to access large amounts of data, but also to interact with them using advanced functionalities such as panning, zooming and filtering. In contrast, the techniques currently used to make maps and diagrams accessible to visually impaired people require the intervention of tactile graphics specialists and result in non-interactive tactile representations. However, based on recent advances in the automatic production of content, we can expect in the coming years a growth in the availability of adapted content, which must go hand-in-hand with the development of affordable and usable devices. In particular, these devices should make full use of visually impaired users' perceptual capacities and support the display of interactive and updatable representations. A number of research prototypes have already been developed. Some rely on digital representation only, and although they have the great advantage of being instantly updatable, they provide very limited tactile feedback, which makes their exploration cognitively demanding and imposes heavy restrictions on content. On the other hand, most prototypes that rely on digital and physical representations allow for a two-handed exploration that is both natural and efficient at retrieving and encoding spatial information, but they are physically limited by the use of a tactile overlay, making them impossible to update. Other alternatives are either extremely expensive (e.g. braille tablets) or offer a slow and limited way to update the representation (e.g. maps that are 3D-printed based on users' inputs). In this thesis, we propose to bridge the gap between these two approaches by investigating how to develop physical interactive maps and diagrams that support two-handed exploration, while at the same time being updatable and affordable. To do so, we build on previous research on Tangible User Interfaces (TUI) and particularly on (actuated) tabletop TUIs, two fields of research that have surprisingly received very little interest concerning visually impaired users. Based on the design, implementation and evaluation of three tabletop TUIs (the Tangible Reels, the Tangible Box and BotMap), we propose innovative non-visual interaction techniques and technical solutions that will hopefully serve as a basis for the design of future TUIs for visually impaired users, and encourage their development and use. We investigate how tangible maps and diagrams can support various tasks, ranging from the (re)construction of diagrams to the exploration of maps by panning and zooming. From a theoretical perspective we contribute to the research on accessible graphical representations by highlighting how research on maps can feed research on diagrams and vice-versa. We also propose a classification and comparison of existing prototypes to deliver a structured overview of current research

Crowdsourcing Accessibility: Human-Powered Access Technologies

People with disabilities have always engaged the people around them in order to circumvent inaccessible situations, allowing them to live more independently and get things done in their everyday lives. Increasing connectivity is allowing this approach to be extended to wherever and whenever it is needed. Technology can leverage this human work force to accomplish tasks beyond the capabilities of computers, increasing how accessible the world is for people with disabilities. This article outlines the growth of online human support, outlines a number of projects in this space, and presents a set of challenges and opportunities for this work going forward