μGraph: Haptic Exploration and Editing of 3D Chemical Diagrams

People with visual impairments or blindness (VIB) encounter diffculties in exploring graphical representations that are widely used for the study of STEM subjects. In particular, graphs are used to represent many different scientifc notations: fowcharts, automata, cognitive maps, and more. Among these, structural chemical formulae are characterized by a complex, often 3-dimensional structure, which makes them hard to access and author with traditional assistive tools. We propose \ub5Graph, a multimodal system that combines haptic and speech feedback to enable people with VIB to explore and edit structural chemical formulae. Two main contributions are presented: (i) a novel, non-visual interaction paradigm for exploring graphs and its implementation in the \ub5Graph system, and (ii) an extensive evaluation of the proposed system with 10 participants with VIB showing that \ub5Graph is thoroughly accessible and that the haptic feedback enhances understanding of the geometric properties of a graph

"Hey Model!" -- Natural User Interactions and Agency in Accessible Interactive 3D Models

While developments in 3D printing have opened up opportunities for improved access to graphical information for people who are blind or have low vision (BLV), they can provide only limited detailed and contextual information. Interactive 3D printed models (I3Ms) that provide audio labels and/or a conversational agent interface potentially overcome this limitation. We conducted a Wizard-of-Oz exploratory study to uncover the multi-modal interaction techniques that BLV people would like to use when exploring I3Ms, and investigated their attitudes towards different levels of model agency. These findings informed the creation of an I3M prototype of the solar system. A second user study with this model revealed a hierarchy of interaction, with BLV users preferring tactile exploration, followed by touch gestures to trigger audio labels, and then natural language to fill in knowledge gaps and confirm understanding.Comment: Paper presented at ACM CHI 2020: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, ACM, New York, April 2020; Replacement: typos correcte

Implications of 3-D Printing for Teaching Geoscience Concepts to Students with Visual Impairments

This article presents the results of a study on the use of 3-D printed models in a science classroom for students with visual impairments and examines whether the use of these models impacts student conceptual understanding and misconceptions related to geosciences concepts, specifically plate tectonics. Data were collected one week prior to instruction, one week after instruction and throughout the 3-week instructional period. Results showed that students with visual impairments held many of the same misconceptions about plate tectonics as students who are typically sighted. All students in this study had fewer misconceptions after the instructional period than they held before instruction began; however, both the 3D group and the TG group continued to hold approximately equal numbers of misconceptions. The adaptations and hands-on experiences in this 3-week curriculum proved effective for students with visual impairments; helping them understand the unifying theory of plate tectonics. Some unique misconceptions held by the students with visual impairments in this research study include plates floating on the ocean, earthquakes moving with the plates, and volcanoes working together with the plates to cause earthquakes. There was a difference between students who had low vision and those with light perception only. The study helps to shed light on the use of 3-D printed models in the science classroom and their effectiveness at helping students with visual impairments learn important geoscience concepts

Indefinite origin: decentralizing knowledge

The RISD Digital Commons notes that “graduate education is focused on producing original contributions to a field, based on research and advance practice. What is collaborative thesis, and what value does it bring to the process? Indefinite Origin, is a thesis advocating for knowledge to be decentralized and treated as a shared resource. Deviating from a focus on originality, it engages in the creation of knowledge as a collaborative effort. Design flexibility is practiced through intellectual humility, negotiating methodologies and welcoming chance. This thesis explores ideas from Furtherfield’s Do-It-With-Other (DIWO) or Do-It-Together (DIT)4 philosophies that promote synergy around commons5, cultural and natural resources that are accessible to all members of society

Computational Frameworks for Multi-Robot Cooperative 3D Printing and Planning

This dissertation proposes a novel cooperative 3D printing (C3DP) approach for multi-robot additive manufacturing (AM) and presents scheduling and planning strategies that enable multi-robot cooperation in the manufacturing environment. C3DP is the first step towards achieving the overarching goal of swarm manufacturing (SM). SM is a paradigm for distributed manufacturing that envisions networks of micro-factories, each of which employs thousands of mobile robots that can manufacture different products on demand. SM breaks down the complicated supply chain used to deliver a product from a large production facility from one part of the world to another. Instead, it establishes a network of geographically distributed micro-factories that can manufacture the product at a smaller scale without increasing the cost. In C3DP, many printhead-carrying mobile robots work together to print a single part cooperatively. While it holds the promise to mitigate issues associated with gantry-based 3D printers, such as lack of scalability in print size and print speed, its realization is challenging because existing studies in the relevant literature do not address the fundamental issues in C3DP that stem from the amalgamation of the mobile nature of the robots, and continuous nature of the manufacturing tasks. To address this challenge, this dissertation asks two fundamental research questions: RQ1) How can the traditional 3D printing process be transformed to enable multi-robot cooperative AM? RQ2) How can cooperative manufacturing planning be realized in the presence of inherent uncertainties in AM and constraints that are dynamic in both space and time? To answer RQ1, we discretize the process of 3D printing into multiple stages. These stages include chunking (dividing a part into smaller chunks), scheduling (assigning chunks to robots and generating print sequences), and path and motion planning. To test the viability of the approach, we conducted a study on the tensile strength of chunk-based parts to examine their mechanical integrity. The study demonstrates that the chunk-based part can be as strong as the conventionally 3D-printed part. Next, we present different computational frameworks to address scheduling issues in C3DP. These include the development of 1) the world-first working strategy for C3DP, 2) a framework for automatic print schedule generation, evaluation, and validation, and 3) a resource-constrained scheduling approach for C3DP that uses a meta-heuristic approach such as a modified Genetic Algorithm (MGA) and a new algorithm that uses a constraint-satisficing approach to obtain collision-free print schedules for C3DP. To answer RQ2, a multi-robot decentralized approach based on a simple set of rules is used to plan for C3DP. The approach is resilient to uncertainties such as variation in printing times and can even outperform the centralized approach that uses MGA with a conflict-based search for large-scale problems. By answering these two fundamental questions, the central objective of the research project to establish computational frameworks to enable multi-robot cooperative manufacturing was achieved. The search for answers to the RQs led to the development of novel concepts that can be used not only in C3DP, but many other manufacturing tasks, in general, requiring cooperation among multiple robots

Touch- and Walkable Virtual Reality to Support Blind and Visually Impaired Peoples‘ Building Exploration in the Context of Orientation and Mobility

Der Zugang zu digitalen Inhalten und Informationen wird immer wichtiger für eine erfolgreiche Teilnahme an der heutigen, zunehmend digitalisierten Zivilgesellschaft. Solche Informationen werden meist visuell präsentiert, was den Zugang für blinde und sehbehinderte Menschen einschränkt. Die grundlegendste Barriere ist oft die elementare Orientierung und Mobilität (und folglich die soziale Mobilität), einschließlich der Erlangung von Kenntnissen über unbekannte Gebäude vor deren Besuch. Um solche Barrieren zu überbrücken, sollten technische Hilfsmittel entwickelt und eingesetzt werden. Es ist ein Kompromiss zwischen technologisch niedrigschwellig zugänglichen und verbreitbaren Hilfsmitteln und interaktiv-adaptiven, aber komplexen Systemen erforderlich. Die Anpassung der Technologie der virtuellen Realität (VR) umfasst ein breites Spektrum an Entwicklungs- und Entscheidungsoptionen. Die Hauptvorteile der VR-Technologie sind die erhöhte Interaktivität, die Aktualisierbarkeit und die Möglichkeit, virtuelle Räume und Modelle als Abbilder von realen Räumen zu erkunden, ohne dass reale Gefahren und die begrenzte Verfügbarkeit von sehenden Helfern auftreten. Virtuelle Objekte und Umgebungen haben jedoch keine physische Beschaffenheit. Ziel dieser Arbeit ist es daher zu erforschen, welche VR-Interaktionsformen sinnvoll sind (d.h. ein angemessenes Verbreitungspotenzial bieten), um virtuelle Repräsentationen realer Gebäude im Kontext von Orientierung und Mobilität berührbar oder begehbar zu machen. Obwohl es bereits inhaltlich und technisch disjunkte Entwicklungen und Evaluationen zur VR-Technologie gibt, fehlt es an empirischer Evidenz. Zusätzlich bietet diese Arbeit einen Überblick über die verschiedenen Interaktionen. Nach einer Betrachtung der menschlichen Physiologie, Hilfsmittel (z.B. taktile Karten) und technologischen Eigenschaften wird der aktuelle Stand der Technik von VR vorgestellt und die Anwendung für blinde und sehbehinderte Nutzer und der Weg dorthin durch die Einführung einer neuartigen Taxonomie diskutiert. Neben der Interaktion selbst werden Merkmale des Nutzers und des Geräts, der Anwendungskontext oder die nutzerzentrierte Entwicklung bzw. Evaluation als Klassifikatoren herangezogen. Begründet und motiviert werden die folgenden Kapitel durch explorative Ansätze, d.h. im Bereich 'small scale' (mit sogenannten Datenhandschuhen) und im Bereich 'large scale' (mit einer avatargesteuerten VR-Fortbewegung). Die folgenden Kapitel führen empirische Studien mit blinden und sehbehinderten Nutzern durch und geben einen formativen Einblick, wie virtuelle Objekte in Reichweite der Hände mit haptischem Feedback erfasst werden können und wie verschiedene Arten der VR-Fortbewegung zur Erkundung virtueller Umgebungen eingesetzt werden können. Daraus werden geräteunabhängige technologische Möglichkeiten und auch Herausforderungen für weitere Verbesserungen abgeleitet. Auf der Grundlage dieser Erkenntnisse kann sich die weitere Forschung auf Aspekte wie die spezifische Gestaltung interaktiver Elemente, zeitlich und räumlich kollaborative Anwendungsszenarien und die Evaluation eines gesamten Anwendungsworkflows (d.h. Scannen der realen Umgebung und virtuelle Erkundung zu Trainingszwecken sowie die Gestaltung der gesamten Anwendung in einer langfristig barrierefreien Weise) konzentrieren.Access to digital content and information is becoming increasingly important for successful participation in today's increasingly digitized civil society. Such information is mostly presented visually, which restricts access for blind and visually impaired people. The most fundamental barrier is often basic orientation and mobility (and consequently, social mobility), including gaining knowledge about unknown buildings before visiting them. To bridge such barriers, technological aids should be developed and deployed. A trade-off is needed between technologically low-threshold accessible and disseminable aids and interactive-adaptive but complex systems. The adaptation of virtual reality (VR) technology spans a wide range of development and decision options. The main benefits of VR technology are increased interactivity, updatability, and the possibility to explore virtual spaces as proxies of real ones without real-world hazards and the limited availability of sighted assistants. However, virtual objects and environments have no physicality. Therefore, this thesis aims to research which VR interaction forms are reasonable (i.e., offering a reasonable dissemination potential) to make virtual representations of real buildings touchable or walkable in the context of orientation and mobility. Although there are already content and technology disjunctive developments and evaluations on VR technology, there is a lack of empirical evidence. Additionally, this thesis provides a survey between different interactions. Having considered the human physiology, assistive media (e.g., tactile maps), and technological characteristics, the current state of the art of VR is introduced, and the application for blind and visually impaired users and the way to get there is discussed by introducing a novel taxonomy. In addition to the interaction itself, characteristics of the user and the device, the application context, or the user-centered development respectively evaluation are used as classifiers. Thus, the following chapters are justified and motivated by explorative approaches, i.e., in the group of 'small scale' (using so-called data gloves) and in the scale of 'large scale' (using an avatar-controlled VR locomotion) approaches. The following chapters conduct empirical studies with blind and visually impaired users and give formative insight into how virtual objects within hands' reach can be grasped using haptic feedback and how different kinds of VR locomotion implementation can be applied to explore virtual environments. Thus, device-independent technological possibilities and also challenges for further improvements are derived. On the basis of this knowledge, subsequent research can be focused on aspects such as the specific design of interactive elements, temporally and spatially collaborative application scenarios, and the evaluation of an entire application workflow (i.e., scanning the real environment and exploring it virtually for training purposes, as well as designing the entire application in a long-term accessible manner)