What is lost in translation from visual graphics to text for accessibility

Many blind and low-vision individuals are unable to access digital media visually. Currently, the solution to this accessibility problem is to produce text descriptions of visual graphics, which are then translated via text-to-speech screen reader technology. However, if a text description can accurately convey the meaning intended by an author of a visualization, then why did the author create the visualization in the first place? This essay critically examines this problem by comparing the so-called graphic–linguistic distinction to similar distinctions between the properties of sound and speech. It also presents a provisional model for identifying visual properties of graphics that are not conveyed via text-tospeech translations, with the goal of informing the design of more effective sonic translations of visual graphics

Means or end? On the Valuation of Logic Diagrams

From the beginning of the 16th century to the end of the 18th century, there were not less than ten philosophers who focused extensively on Venn’s ostensible analytical diagrams, as noted by modern historians of logic (Venn, Gardner, Baron, Coumet et al.). But what was the reason for early modern philosophers to use logic or analytical diagrams? Among modern historians of logic one can find two theses which are closely connected to each other: M. Gardner states that since the Middle Ages certain logic diagrams were used just in order to teach “dull-witted students”. Therefore, logic diagrams were just a means to an end. According to P. Bernhard, the appreciation of logic diagrams had not started prior to the 1960s, therefore the fact that logic diagrams become an end the point of research arose very late. The paper will focus on the question whether logic resp. analytical diagrams were just means in the history of (early) modern logic or not. In contrast to Gardner, I will argue that logic diagrams were not only used as a tool for “dull-witted students”, but rather as a tool used by didactic reformers in early modern logic. In predating Bernhard’s thesis, I will argue that in the 1820s logic diagrams had already become a value in themselves in Arthur Schopenhauer’s lectures on logic, especially in proof theory

Graphicality: why is there not such a word?

The concept of graphicality first appeared in the work of Edgar Allan Poe. Taking its title from Poe’s painterly metaphor, this paper seeks to understand how graphicality may inform aspects of design thinking that have been neglected. We explore the current use, origins and aspects of graphicality, and contextualise it in some real world scenarios to reaffirm how we live in a graphic age, and how graphicality must be better understood in the way we comprehend other displays of human ability, such as musicality. Poe provides us with a starting point for relating the physical and mental domains of image interpretation. Graphicality is shown to work on a continuum between subjectivity and objectivity, not as something to be measured but appreciated in how it enhances understanding and knowledge. This has implications for many academic disciplines, specifically in how it enhances our appreciation of the graphic in graphic design

Iconic Properties are Lost when Translating Visual Graphics to Text for Accessibility

For many blind and low-vision individuals, accessing charts and graphs often means accessing a text description of the graphics, usually aurally. However, in doing so, parts of the charts that are not originally conveyed textually are lost in the translation into text. By synthesizing ideas from the science and philosophy of perception and cognition, diagrammatic reasoning, and semiotics, this essay makes the case that translating charts into text descriptions results in the loss of iconic properties of the graphics, and proposes that non-linguistic sonification can be recruited to preserve such properties. The essay concludes by proposing how predictions based on this synthesis can inform design

Средство или цель? К истории оценок логических диаграмм

From the beginning of the 16th century to the end of the 18th century, there were not less than ten philosophers who focused extensively on Venn’s ostensible analytical diagrams, as noted by modern historians of logic (Venn, Gardner, Baron, Coumet et al.). But what was the reason for early modern philosophers to use logic or analytical diagrams? Among modern historians of logic one can find two theses which are closely connected to each other: M. Gardner states that since the Middle Ages certain logic diagrams were used just in order to teach “dull-witted students”. Therefore, logic diagrams were just a means to an end. According to P. Bernard, the appreciation of logic diagrams had not started prior to the 1960s, therefore the fact that logic diagrams become an end the point of research arose very late. The paper will focus on the question whether logic resp. analytical diagrams were just means in the history of (early) modern logic or not. In contrast to Gardner, I will argue that logic diagrams were not only used as a tool for “dull-witted students”, but rather as a tool used by didactic reformers in early modern logic. In predating Bernard’s thesis, I will argue that in the 1820s logic diagrams had already become a value in themselves in Arthur Schopenhauer’s lectures on logic, especially in proof theory.С начала XVI в. до конца XVIII в. не менее десятка философов активно работали с наглядными аналитическими диаграммами Венна, что отмечается современными историками логики (Дж. В енном, М. Гарднером, М. Барон, Э. К умэ и др.). Но что заставляло философов Нового времени использовать логические или аналитические диаграммы? Среди современных историков логики распространены два убеждения, тесно связанные между собой: М. Гарднер утверждает, что со времён средневековья некоторые логические диаграммы использовались просто для того, чтобы учить «твердолобых школяров». В таком случае логические диаграммы оказывались всего лишь средством для достижения цели. Согласно П. Бернарду, логические диаграммы не были оценены по достоинству до 1960-х гг., так что целью исследования логические диаграммы стали очень поздно. Настоящая статья посвящена вопросу о том, были ли логические (соответственно, аналитические) диаграммы в истории (ранне)новоевропейской логики всего лишь средством или нет. В противоположность Гарднеру, мы покажем, что логические диаграммы использовались не только как инструмент для «твердолобых школяров», но скорее как инструмент, которым пользовались реформаторы преподавания логики в ранненовоевропейский период. Отодвигая назад временну́ю оценку Бернарда, мы покажем, что уже в 1820-х гг. логические диаграммы стали ценностью сами по себе в лекциях А. Шопенгауэра по логике, особенно в теории доказательств

What are mathematical diagrams?

Although traditionally neglected, mathematical diagrams have recently begun to attract attention from philosophers of mathematics. By now, the literature includes several case studies investigating the role of diagrams both in discovery and justification. Certain preliminary questions have, however, been mostly bypassed. What are diagrams exactly? Are there different types of diagrams? In the scholarly literature, the term “mathematical diagram” is used in diverse ways. I propose a working definition that carves out the phenomena that are of most importance for a taxonomy of diagrams in the context of a practice-based philosophy of mathematics, privileging examples from contemporary mathematics. In doing so, I move away from vague, ordinary notions. I define mathematical diagrams as forming notational systems and as being geometric/topological representations or two-dimensional representations. I also examine the relationship between mathematical diagrams and spatiotemporal intuition. By proposing an explication of diagrams, I explain certain controversies in the existing literature. Moreover, I shed light on why mathematical diagrams are so effective in certain instances, and, at other times, dangerously misleading

Students' use of diagrams for the visualisation of biochemical processes.

Thesis (M.Sc.)-University of Natal, Pietermaritzburg, 2003.Research into the usefulness of scientific diagrams as teaching and learning tools has revealed their great effectiveness in reinforcing and replacing text; summarizing, clarifying, grouping and comparing information; illustrating abstract concepts and spatial relations between concepts; and aiding understanding and integration of knowledge. However, these advantages are not always realised as diagram effectiveness depends on the student's cognitive ability, visual literacy and prior knowledge. In biochemistry, flow diagrams are used as tools for the visualisation of biochemical processes, the abstract nature of which presents problems to students, probably because the depicted content is beyond their perceptual experience. In this study, we define visualisation as the entire process from the perception of an external representation (e.g. diagram), its internal processing, and the expression of a mental model of the represented content. Therefore, visualisation incorporates reasoning processes and interactions with a student's conceptual knowledge, in their construction of a mental model. Students' visualisation difficulties, in terms of conceptual and reasoning difficulties, have been well researched in areas such as physics and chemistry, but neglected in biochemistry, especially with respect to the use of diagrams as visualisation tools. Thus the aim of this study was to investigate students' use of diagrams for the visualisation of biochemical processes, and to identify the nature, and potential sources of students' conceptual, reasoning and diagram-related difficulties revealed during the visualisation process. The study groups ranged from 27 to 95 biochemistry students from the University of Natal and 2 to 13 local and international experts. Propositional knowledge was obtained from textbooks and from a questionnaire to experts. Data on student visualisation of biochemical processes was obtained from their responses to written and interview probes as well as student-generated diagrams. All data was subjected to inductive analysis according to McMillan and Schumacher (1993) and any difficulties that emerged were classified at levels 1- 3 on the framework of Grayson et al. (2001). The possible sources of difficulties were considered in terms of a model by Schonborn et al. (2003 & 2002). The results revealed the following major findings. The meaning of linear, cyclic and cascade biochemical processes was partially resolved by means of an extensive list of generic and distinguishing functional features obtained from experts. Attempts to clarify propositional knowledge of the complement system revealed a deficiency in our understanding of the functional relationship between the complement pathways and highlighted the need for further experimental laboratory work. Several students literally interpreted diagrams of the functional characteristics of biochemical processes (e.g. cyclic) as the spatial arrangement of the intermediates within cells (e.g. occur in "circles"), although in some cases, their verbal responses revealed that they did not hold this difficulty suggesting that they might hold more than one internal model of the process. Some students also showed difficulty using textbook diagrams to visualise the chemistry of glycolytic and complement reactions. In this regard, besides students' conceptual knowledge and reasoning ability, a major source of these difficulties included misleading symbolism and visiospatial characteristics in the diagrams, suggesting the need for improvement of diagram design through the use of clearer symbolism, the standardization of conventions, and improvement of visiospatial properties of diagrams. The results constituted further empirical evidence for the model of Schonbom et al. (2003 & 2002) and led to the proposal of a model of visualisation aimed at clarifying the highly complex and cognitive processes involved in individuals' visualisation of biochemical processes in living systems

A cognitive model of the roles of diagrammatic representation in supporting unpractised reasoning about probability

Cognitive process accounts of the advantages conferred by diagrams in problem solving and reasoning have typically attempted to explain an idealised user or a reasoning system that has equivalent to practised knowledge of the task with the target representation. The thesis investigates the question of how diagrams support users in the process of solving unpractised problems in the domain of probability. The research question is addressed by the design and analysis of an empirical study and cognitive model. The main experiment required participants (N=8) to solve a set of unpractised probability problems presented by combined text and diagram. Think-aloud and eye-movement protocols together with given solutions were used to infer the content and process of problem interpretation, solution interpretation and task execution strategies employed by participants. The data suggested that the diagram was used to facilitate problem solving in three different ways by: (a) supporting sub-problem identification, (b) supporting prior knowledge of diagrammatic sub-schemes used for interpreting a solution and (c) supporting the process of interpreting and testing the specific meaning of given problem instructions and self-generated solution instructions. These empirical data were used to develop cognitive models of canonical strategies of the three identified phenomena: • Sub-problem identification advantages are accounted for by proposing that the spatial semantics of diagrams coupled with competences of the visual-spatial processing system and opportunities for demonstrative interpretation strategies increase the probability of goal-relevant data being made available to central cognition for further processing. • Framing advantages are accounted for by proposing that represented diagrammatic sub-schemes (e.g. part-whole portions, icon-arrays, 2D containers etc.) facilitate access to existing prior knowledge used to frame, derive, and reason about information analogically within that scheme. • Advantages in instruction interpretation are related to the specificity of diagrams which support the opportunity to demonstratively test and evaluate the referential meaning of an instruction. The cognitive model also investigates and evaluates assumptions about the prior knowledge for solving unpractised probability problems; a representational scheme for addressing the co-ordination of sub-goals; a deictic problem representation to support online processing of environmental information, a meta-cognitive processing scheme to address self-argumentation and intention tracking and visual and spatial competences to address the requirements of diagrammatic reasoning. The implications of the cognitive model are discussed with regard to existing accounts of diagrammatic reasoning, probability problem solving (PPS), and unpractised problem solving