Description strategies to make an interactive science simulation accessible

Interactive simulations are increasingly important in science education, yet most are inaccessible to blind learners. In developing an accessible prototype of a PhET interactive science simulation, we encountered significant challenges in providing screen reader access, including the need to: 1) describe unpredictable event sequences, 2) cue productive interactions, and 3) to simultaneously convey multiple changes. To address these challenges, we extended existing practices for verbal description of visual interactive content, and we created new strategies for developing rich description for accessible interactive science simulations

μGraph: Haptic Exploration and Editing of 3D Chemical Diagrams

People with visual impairments or blindness (VIB) encounter diffculties in exploring graphical representations that are widely used for the study of STEM subjects. In particular, graphs are used to represent many different scientifc notations: fowcharts, automata, cognitive maps, and more. Among these, structural chemical formulae are characterized by a complex, often 3-dimensional structure, which makes them hard to access and author with traditional assistive tools. We propose \ub5Graph, a multimodal system that combines haptic and speech feedback to enable people with VIB to explore and edit structural chemical formulae. Two main contributions are presented: (i) a novel, non-visual interaction paradigm for exploring graphs and its implementation in the \ub5Graph system, and (ii) an extensive evaluation of the proposed system with 10 participants with VIB showing that \ub5Graph is thoroughly accessible and that the haptic feedback enhances understanding of the geometric properties of a graph

Bluefish: A Relational Framework for Graphic Representations

Complex graphic representations -- such as annotated visualizations, molecular structure diagrams, or Euclidean geometry -- convey information through overlapping perceptual relations. To author such representations, users are forced to use rigid, purpose-built tools with limited flexibility and expressiveness. User interface (UI) frameworks provide only limited relief as their tree-based models are a poor fit for expressing overlaps. We present Bluefish, a diagramming framework that extends UI architectures to support overlapping perceptual relations. Bluefish graphics are instantiated as relational scenegraphs: hierarchical data structures augmented with adjacency relations. Authors specify these relations with scoped references to components found elsewhere in the scenegraph. For layout, Bluefish lazily materializes necessary coordinate transformations. We demonstrate that Bluefish enables authoring graphic representations across a diverse range of domains while preserving the compositional and abstractional affordances of traditional UI frameworks. Moreover, we show how relational scenegraphs capture previously latent semantics that can later be retargeted (e.g., for screen reader accessibility).Comment: 27 pages, 14 figure