Developing an interactive overview for non-visual exploration of tabular numerical information

This thesis investigates the problem of obtaining overview information from complex tabular numerical data sets non-visually. Blind and visually impaired people need to access and analyse numerical data, both in education and in professional occupations. Obtaining an overview is a necessary first step in data analysis, for which current non-visual data accessibility methods offer little support. This thesis describes a new interactive parametric sonification technique called High-Density Sonification (HDS), which facilitates the process of extracting overview information from the data easily and efficiently by rendering multiple data points as single auditory events. Beyond obtaining an overview of the data, experimental studies showed that the capabilities of human auditory perception and cognition to extract meaning from HDS representations could be used to reliably estimate relative arithmetic mean values within large tabular data sets. Following a user-centred design methodology, HDS was implemented as the primary form of overview information display in a multimodal interface called TableVis. This interface supports the active process of interactive data exploration non-visually, making use of proprioception to maintain contextual information during exploration (non-visual focus+context), vibrotactile data annotations (EMA-Tactons) that can be used as external memory aids to prevent high mental workload levels, and speech synthesis to access detailed information on demand. A series of empirical studies was conducted to quantify the performance attained in the exploration of tabular data sets for overview information using TableVis. This was done by comparing HDS with the main current non-visual accessibility technique (speech synthesis), and by quantifying the effect of different sizes of data sets on user performance, which showed that HDS resulted in better performance than speech, and that this performance was not heavily dependent on the size of the data set. In addition, levels of subjective workload during exploration tasks using TableVis were investigated, resulting in the proposal of EMA-Tactons, vibrotactile annotations that the user can add to the data in order to prevent working memory saturation in the most demanding data exploration scenarios. An experimental evaluation found that EMA-Tactons significantly reduced mental workload in data exploration tasks. Thus, the work described in this thesis provides a basis for the interactive non-visual exploration of a broad range of sizes of numerical data tables by offering techniques to extract overview information quickly, performing perceptual estimations of data descriptors (relative arithmetic mean) and managing demands on mental workload through vibrotactile data annotations, while seamlessly linking with explorations at different levels of detail and preserving spatial data representation metaphors to support collaboration with sighted users

Etude empirique de l'interaction multimodale mobile en situation naturelle

Ubiquity of mobile interactive devices either in professional, cultural or domestic domain constitutes a new research perspective to both individual and cooperative usage of new communication technologies. Mobile technologies rise, among others, new questioning about mobile usability specific issues, adapted methodologies and field data collection, identification of relevant contextual factors for mobile systems design, superimposition of interaction spaces. We believe that mobility is a new interaction paradigm carrying potential contextual variations, preventing traditional direct manipulation from being the most appropriate interaction style without serious adaptation because it currently requires a high level of visual attention . In this PhD, we first present a review of current knowledge concerning Context Aware Computing (CAC) as well as mobile and multimodal interaction. We then present an empirical study of a mobile multimodal application in the context of a situated collaborative quiz-answering task. Based on quantitative and qualitative results, our research first describes global and individual usage of the available modalities (tactile and vocal). In a second time, we study the influence of some identified internal and external contextual factors on the modalities selection,. For instance, we examinate the impact of factors such as noise context, social context or history of the interaction on the modalities usage. A third part of our results highlights the relation between users choice of a modality depending on the attentional resources required by the mobile interaction. All along the quantitative results, an extensive analysis of qualitative data such as user‘s verbalisation during self confrontation sessions, or activity chronicles examination, allowed us to argue and explain some modality choices and helped us identify some new factors taken into account by the users. Finally, we use these results to propose some models for mobile multimodal applications, as well as some criteria and recommendations for their design.L'ubiquité des dispositifs interactifs mobiles dans les sphères professionnelles, culturelles et domestiques constitue une occasion d'ouverture de nouveaux champs de recherche dans le domaine de l'étude des usages individuels et coopératifs des Nouvelles technologies de l'information et de la communication (NTICs). Les technologies mobiles amènent notamment à renouveler les points de vue sur des questions telles que, les problèmes d'utilisabilité spécifiques posés par la mobilité, les méthodes et les dispositifs de recueil de données, la question de l'identification des facteurs contextuels pertinents pour la conception de systèmes mobiles, la superposition des espaces d'interaction. De ce point de vue, nous posons que la mobilité constitue un nouveau paradigme d'interaction facteur potentiel de variations contextuelles pour lequel le style d'interaction de manipulation directe n'est pas forcément le plus approprié sans des adaptations fortes. Dans cette thèse, nous nous attachons, après avoir posé les bases théoriques concernant le Context Aware Computing (CAC), l'interaction mobile, et multimodale, à étudier empiriquement les usages d'une application multimodale mobile.dans le contexte d'un jeu collaboratif en mobilité. Dans un premier temps, des résultats quantitatifs permettent d'aborder une description des usages globaux et intra individuels des modalités, puis décrivent l'influence d'un certain nombre de facteurs contextuels (internes ou externes) influençant les choix d'usage des modalités, tels que le contexte sonore, social ou encore l'historique de l'interaction. Dans un second temps, une analyse plus fine de données qualitatives telles que les verbalisations des utilisateurs ou l'examen de chroniques d'activités, permet d‘expliquer en partie ces choix d'usages mais également d'identifier de nouveaux facteurs pris en compte par les sujets. Nous nous servons de ces résultats pour proposer des modèles et des recommendations pour la conception d'interfaces multimodales mobiles

Interactive Digital Support for Concept Design Teams

This thesis develops a design method, the ICR (Inform, Create, Reflect) Grid, for improved utilisation of information during concept design. Although concept design is information intensive and critical to project direction, the effective management and use of digital information has not been adequately addressed. The ICR Grid is a prescriptive method which requires design teams to find and build information resources in parallel with creating solutions. As a solution-based approach it allows designers to freely explore ideas, while encouraging flexible thinking by using different modes of conceptual working (analysis, synthesis and evaluation). The output of the method is a linked grid of concepts and information sources. The exploratory phase of the research examined current design process models and concept design methods, with team information use patterns explored through protocol analyses of a design task. This was followed by an examination of literature relating to digital information and a class study on technological support for student designers. The outcome of these explorations was an understanding that to enhance digital information use in concept design, a new approach was necessary. Development began by correlating characteristics of computer games to concept design, with a view to applying new techniques to the structure and management of information. A number of scenarios were subsequently outlined, with one selected and developed using paper-based prototyping. This was eventually formalised as the ICR Grid. Initial evaluation of the new method was carried out through a comparative study with the 6-3-5 Method, which revealed that although fewer concepts were produced with the ICR Grid, they were of a higher quality, variety and detail. Three different companies then used the ICR Grid to address relevant industrial problems, with generally positive feedback obtained on its performance. Several areas are identified for future work and the further enhancement of information use

An Investigation on Benefit-Cost Analysis of Greenhouse Structures in Antalya

Significant population increase across the world, loss of cultivable land and increasing demand for food put pressure on agriculture. To meet the demand, greenhouses are built, which are, light structures with transparent cladding material in order to provide controlled microclimatic environment proper for plant production. Conceptually, greenhouses are similar with manufacturing buildings where a controlled environment for manufacturing and production have been provided and proper spaces for standardized production processes have been enabled. Parallel with the trends in the world, particularly in southern regions, greenhouse structures have been increasingly constructed and operated in Turkey. A significant number of greenhouses are located at Antalya. The satellite images demonstrated that for over last three decades, there has been a continuous invasion of greenhouses on all cultivable land. There are various researches and attempts for the improvement of greenhouse design and for increasing food production by decreasing required energy consumption. However, the majority of greenhouses in Turkey are very rudimentary structures where capital required for investment is low, but maintenance requirements are high when compared with new generation greenhouse structures. In this research paper, life-long capital requirements for construction and operation of greenhouse buildings in Antalya has been investigated by using benefit-cost analysis study

Knowledge Capturing in Design Briefing Process for Requirement Elicitation and Validation

Knowledge capturing and reusing are major processes of knowledge management that deal with the elicitation of valuable knowledge via some techniques and methods for use in actual and further studies, projects, services, or products. The construction industry, as well, adopts and uses some of these concepts to improve various construction processes and stages. From pre-design to building delivery knowledge management principles and briefing frameworks have been implemented across project stakeholders: client, design teams, construction teams, consultants, and facility management teams. At pre-design and design stages, understanding the client’s needs and users’ knowledge are crucial for identifying and articulating the expected requirements and objectives. Due to underperforming results and missed goals and objectives, many projects finish with highly dissatisfied clients and loss of contracts for some organizations. Knowledge capturing has beneficial effects via its principles and methods on requirement elicitation and validation at the briefing stage between user, client and designer. This paper presents the importance and usage of knowledge capturing and reusing in briefing process at pre-design and design stages especially the involvement of client and user, and explores the techniques and technologies that are usable in briefing process for requirement elicitation

DOKY: A Multi-Modal User Interface for Non-Visual Presentation, Navigation and Manipulation of Structured Documents on Mobile and Wearable Devices

There are a large number of highly structured documents available on the Internet. The logical document structure is very important for the reader in order to efficiently handling the document content. In graphical user interfaces, each logical structure element is presented by a specific visualisation, a graphical icon. This representation allows visual readers to recognise the structure at a glance. Another advantage is that it enables direct navigation and manipulation. Blind and visually impaired persons are unable to use graphical user interfaces and for the emerging category of mobile and wearable devices, where there are only small visual displays available or no visual display at all, a non-visual alternative is required too. A multi-modal user interface for non-visual presentation, navigation and manipulation of structured documents on mobile and wearable devices like smart phones, smart watches or smart tablets has been developed as a result of inductive research among 205 blind and visually impaired participants. It enables the user to get a fast overview over the document structure and to efficiently skim and scan over the document content by identifying the type, level, position, length, relationship and content text of each element as well as to focus, select, activate, move, remove and insert structure elements or text. These interactions are presented in a non-visual way using Earcons, Tactons and synthetic speech utterances, serving the auditory and tactile human sense. Navigation and manipulation is provided by using the multitouch, motion (linear acceleration and rotation) or speech recognition input modality. It is a complete solution for reading, creating and editing structured documents in a non-visual way. There is no special hardware required. The name DOKY is derived from a short form of the terms document, and accessibility. A flexible platform-independent and event-driven software architecture implementing the DOKY user interface as well as the automated structured observation research method employed for the investigation into the effectiveness of the proposed user interface has been presented. Because it is platform- and language-neutral, it can be used in a wide variety of platforms, environments and applications for mobile and wearable devices. Each component is defined by interfaces and abstract classes only, so that it can be easily changed or extended, and grouped in a semantically self-containing package. An investigation into the effectiveness of the proposed DOKY user interface has been carried out to see whether the proposed user interface design concepts and user interaction design concepts are effective means for non-visual presentation, navigation and manipulation of structured documents on mobile and wearable devices, by automated structured observations of 876 blind and visually impaired research subjects performing 19 exercises among a highly structured example document using the DOKY Structured Observation App on their own mobile or wearable device remotely over the Internet. The results showed that the proposed user interface design concepts for presentation and navigation and the user interaction design concepts for manipulation are effective and that their effectiveness depends on the input modality and hardware device employed as well as on the use of screen readers