Visualization in cyber-geography: reconsidering cartography's concept of visualization in current usercentric cybergeographic cosmologies

This article discusses some epistemological problems of a semiotic and cybernetic character in two current scientific cosmologies in the study of geographic information systems (GIS) with special reference to the concept of visualization in modern cartography. Setting off from Michael Batty’s prolegomena for a virtual geography and Michael Goodchild’s “Human-Computer-Reality-Interaction” as the field of a new media convergence and networking of GIS-computation of geo-data, the paper outlines preliminarily a common field of study, namely that of cybernetic geography, or just “cyber-geography) owing to the principal similarities with second order cybernetics. Relating these geographical cosmologies to some of Science’s dominant, historical perceptions of the exploring and appropriating of Nature as an “inventory of knowledge”, the article seeks to identify some basic ontological and epistemological dimensions of cybernetic geography and visualization in modern cartography. The points made is that a generalized notion of visualization understood as the use of maps, or more precisely as cybergeographic GIS-thinking seems necessary as an epistemological as well as a methodological prerequisite to scientific knowledge in cybergeography. Moreover do these generalized concept seem to lead to a displacement of the positions traditionally held by the scientist and lay-man citizen, that is not only in respect of the perception of the matter studied, i.e. the field of geography, but also of the manner in which the scientist informs the lay-man citizen in the course of action in the public participation in decision making; a displacement that seems to lead to a more critical, or perhaps even quasi-scientific approach as concerns the lay-man user

Semiotically-grounded distant viewing of diagrams : insights from two multimodal corpora

In this article, we argue for the benefits of combining large-scale analyses of visual materials currently pursued within digital humanities with insights from multimodality research, which is an emerging discipline that studies how human communication relies on appropriate combinations of expressive resources. We show that concepts developed within the field of multimodality research provide appropriate metadata schemes for various modes of expression in large corpora and datasets. We illustrate the proposed approach using a common mode of expression, diagrams, and analyse two recent multimodal diagram corpora using statistical and computational methods. Our results suggest that multimodally-motivated metadata schemes can provide a robust foundation for computational analyses of large corpora and datasets. Even if a corpus or dataset is not designed to support full-blown analyses of multimodal communication, our results imply that multimodality theory can still be used to impose tighter analytical control over a variety of visual materials.Peer reviewe

Introducing the diagrammatic semiotic mode

Peer reviewe

Introducing the diagrammatic semiotic mode

Peer reviewe

The functions of visual management

Visual Management has been evolving and effectively employed in some manufacturing and service organisations for a long time. In order to facilitate a cross-industrial learning process and to advance in detailed research the understanding of how the Visual Management concept may serve in an organisation is necessary. The aim of this paper is to identify Visual Management functions and the theoretical base for the construction industry. A detailed literature review and an analysis of the findings were performed accordingly. The necessity of a holistic approach in order to make more use of the Visual Management process and some research opportunities were identified

Meaningful lines : Social semiotic investigations of the graphical line, used as a connector in digital data visualizations

The paper III is not published yet.The dissertation investigates the graphical line, used as a connector in digital data visualizations (DVs), from a social semiotic perspective. It explores the semiotic functions of connecting lines in digital DVs, building empirically on two corpus analyses. These analyses are concerned with how two key semiotic functions are graphically realized in the selected lines and whether graphical conventions regulating these realizations can be identified. In the dissertation the connecting line is researched as a central element of several different DV types – line graphs, timelines, route maps, connection maps, network diagrams, etc. Today, DVs are used in a broad range of social contexts and thus play an essential role in society. New digital technologies make it possible to apply other visual characteristics to graphical lines in DVs than was previously possible, for example, when they were hand-drawn or printed. This change may have influenced the DV design practice, a situation calling for updated research. The dissertation is article-based, comprising three articles and an extended abstract. The extended abstract frames the three articles and includes an introduction, definitions of key terms, more comprehensive sections on the theoretical framework, earlier research and methodology, as well as ethical considerations, reflections on methodological decisions and a conclusion. The two overall research questions addressed in the dissertation are: (1) What characterizes the graphical forms and semiotic functions of connecting lines in current, publicly available digital DVs? (2) What conventions, regulating the relations between these forms and functions, can be observed in such DVs?publishedVersio

Kirjoitetut tunnisteet peruskoulun luonnontieteiden diagrammeissa: kielelliset rakenteet ja diskurssisuhteet

Communication, by nature, is multimodal: it uses various forms (modes) of communication, such as spoken language, written language, illustrations, and many others to create meaning. Multimodality research is the study of communicative situations that rely on such various modes and their combinations. One form of multimodality very commonly seen in everyday life comes in diagrams, which can convey very complex concepts by combining visual expressive resources (such as illustrations or photographs), written language, and diagrammatic elements such as lines and arrows. The primary aim of my thesis is to establish whether the linguistic structures of written labels – that is, textual elements – in diagrams can inform the decomposition of visual expressive resources. Put simply, I seek to find if said visual elements can more accurately be divided into further, more granular units in accordance with linguistic patterns in their accompanying textual elements. To answer my main research question, I posit three sub-questions. First, if certain diagram types (macro-structures), such as tables, cycles, or cross-sections co-occur with specific linguistic patterns; second, if different rhetorical functions found in diagrams employ different structures in their written labels as well; and third, if these functions are signaled by other means in tandem with written language. Answering these questions can help in designing future multimodal corpora and their annotation schemata, increasing annotation accuracy and possibilities for their processing. The theoretical framework used in this thesis synthesizes theories from multimodality theory, discourse studies, and diagrams research. I approach diagrams from the perspective of multimodality, highlighting them as discursive artefacts. This is enabled by the diagrammatic mode, which establishes how discourse semantics can function in the context of diagrams and how their interpretation is dynamic; that is, each element or combination of multiple elements can in turn contextualize or be a part of other elements and their combinations on a different scale. I also discuss the discourse-semantic concepts of coherence and cohesion as they relate to multimodal artefacts: different elements, even if not linguistic, can combine to create semantically meaningful connections between constituents in such an artefact. To exemplify this, I also apply Rhetorical Structure Theory (RST), which seeks to formalize how units of discourse are interconnected and work towards a shared communicative goal. RST employs rhetorical relations such as ELABORATION and IDENTIFICATION to describe how units and their combinations relate to other parts of a text (or other communicative whole). The data I use consists of two interrelated and complementary multimodal corpora: AI2D and AI2D-RST. AI2D is a collection of primary-school textbook science diagrams, annotated for blobs (visual expressive resources), labels, and diagrammatic elements, created for question-answering purposes. It also contains the linguistic data in each of the corpus’s diagrams. AI2D-RST contains a subset of the diagrams in AI2D, expanding them with additional annotation layers for information on macro-structures, visual connectivity, and RST, describing each element’s rhetorical relation in the diagram. I computationally find each rhetorical relation containing a label in AI2D-RST, noting its type, the type of the diagram it appears in, and fetching the labels’ linguistic content from AI2D. I then process each label’s contents with spaCy, a library for natural language processing, for linguistic elements such as phrase types, part-of-speech patterns, and average word counts. The output contains data on each label’s rhetorical relation, the possible macro-structure it is contained in, and said linguistic structures. The results show that there are indeed some differences in how distinct rhetorical relations and macro-groups use language: for example, cycles contain the most verb phrases and highest word count, indicating the use of written language to explicate certain processes to viewers. As linguistic patterns differ across these classes and are contextualized by surrounding diagrammatic elements, approaching diagrams from a discursive standpoint may be beneficial for future empirical multimodality research as well as designing annotation schemata to be more intuitive for annotators. With larger datasets and further research, precise sets of rules containing linguistic structures and layout information may be developed to increase accuracy in probability-based computational analysis of diagrams

Platfospheres and sociocultural explosion of Web 2.0: The commercial centre of the digital semiosphere

This paper explores digital culture with the tools of cultural semiotics in general, and then employing the semiosphere model in particular. Web 2.0 platforms are taken as the major cultural dispositive of our time, as the most representative way in which the internet shapes digital culture. Most of the global population is currently immersed in digital culture. In the first part of the paper the striking similarities between Web 2.0 platforms and the semiosphere are explored and equivalences between the elements of the classic (Lotman’s) semiotic model and these platforms, or platfospheres, are identified. The second part explores the fundamental difference between the “genetic code” at the centre of the semiosphere (as conceived by Lotman), and the computer code and commercial algorithms at the core of the platfospheres that are responsible for their cultural operation. Then the parallels are examined that arise between the past cultural reality, in which the intellectual elite and academics were the driving force of culture, and the contemporary proactive (or even aggressive) core of the platfospheres, in which secret and patent-protected algorithms shape a cultural reality exclusively motivated by the logic of commercial success.  &nbsp

Multimodality of Cultural Content in ELT Materials for Young Learners

Cultural content is incorporated into EL materials to enrich linguistic content. The paper focuses on English materials as cultural artefacts analyzed in terms of multimodality. The aim is to identify the most important multimodal aspects of cultural content offered in English course books for early language education. Following the multimodal discourse analysis, the image–language relations presented in the culture sections are examined. The paper seeks to address two following questions: What is the multimodality of cultural content in English course books for young learners? What are the image-language relations involved in the construction of cultural content? The project involves an analysis of nine course books currently used in teaching English to young learners in Polish primary schools. The data will be collected during the evaluation studies, which here are both quantitative and qualitative in nature. The study is based on a set of universal, content-specific, multimodal, and intermodal criteria. It is hoped that the results from the research project will enrich the process of ELT materials design in terms of multimodality. They will support the need for developing multimodal (visual) literacy through multicultural education in early language education

Diachronic Aaalysis of the visuals in the research paper: a corpus-based study of the strategies and semiotics of visual representation in nutrition biochemistry.

This article reports a diachronic analysis of the visuals as used in the scientific research paper over a 70-year period (1929-1999). The corpus-based (72-articles) study investigates the strategies of specialists in nutrition biochemistry publishing on a single biological theme : essential fatty acids (EFA). It provides data on the various kinds of visuals as well as an inquiry into the size, positioning, nature and function of visual representation within the scientific paper considered as a genre, over the years. Based on the theoretical principle of semiotics, it then proceeds with an analysis of the concept move which underlies the use of visuals, conceived as the hard core of the paper. Readability of a scientific document depends on the number of symbolic and semi-symbolic systems at stake. A scientific visual is constructed as the article itself : It is built as a stratagem, an ambush with no way out. Visuals are used to convince. They are also the only evidence of laboratory work. Semiotic analysis shows that the descriptive nature of the visuals used in the corpus changes after the 60s to become the display of multiple relationships. It also analyzes how the techniques of foregrounding and backgrounding can apply to the study of non-verbal items in the research paper. Social semiotics gives another light on the use of visuals by showing that a scientific paper does not only contain new information. Based on the theme/rheme concepts, borrowed from systemic linguistics, the study of visuals in the corpus and especially that of tables, demonstrates that what is already known by the community (given) tends to be produced on the left-hand side of the tables, whereas the new information is placed on the right. There is also adequation between the title key-words of the tables and the right-hand side results found in these tables. However, due to the development of symbolic representation over the years, the results concerning the research theme in the 80s and 90s tend to follow logical 77 ordering. Thus, the results about essential fatty acids (EFA) are presented or staged in an order according to their chemical composition. Significant results in this case do not appear on the right, but are signalled by special symbols. Specialists can thus very efficiently find the results of interest through this deciphering process, while the layman is more and more at a loss when trying to understand the data represented in visuals. This paper stresses the interest of a diachronic prospect in the study of the visual characteristics of the scientific article, by showing that if visuals are a constant feature, and even constitute the hard core of the article, they should not be considered as stable traits in the research paper contemplated as a genre and their evolution points to changing persuasion strategies pursued by the scientific community represented here