Augmentation of information in educational objects: Effectiveness of arrows and pictures as information for actions in instructional objects

The use of information and communication technology (ICT) in education is now central to facilitating links between learners, resources and instructors. Regardless of whether it is used in distance education or educational objects, ICT enables educators to package education opportunities in an increasing number of alternative ways so as best to meet the varying needs of the end user. Currently, one of the challenges that face instructors is to develop materials that enhance the learner-content interaction by reducing extraneous cognitive load while at the same time facilitating learning. This study explored the effectiveness of pictorial information and augmentation in instructions and educational objects. Dual coding theory is used to suggest that information that can be processed via separate but interconnected systems will facilitate faster processing and deeper learning of the information. University students were randomly assigned to six experimental conditions to perform a novel task using six different instruction manuals. A 3 (text, text-pictorial, text-pictorial-arrows) X 2 (picture of object vs. no picture of object) design was used to test whether augmenting text with pictorial information provided additional valuable information in instructional settings. Results partially support this multimedia effect; participants exhibited superior performance in a Text-Pictorial and Text-Pictorial-Arrows format over Text format. A picture of the object also facilitated superior performance on both the assembly and operating tasks, especially in a text format. Overall, combinations of text-pictorial and text-pictorial-arrows facilitated faster assembly and operation; they reduced errors, extra procedures, and unsuccessful assemblies (uncorrected errors). Results also support the idea that arrows convey unique types of information and function. In particular, arrows may attune people to important information and/or convey information movement that guides actions during tasks. Practical implications are discussed in relation to the type of information combinations that may lead to superior instructional design of instructional objects and research, including how to reduce errors of omission

Arrow Symbols: Theory for Interpretation

People often sketch diagrams when they communicate successfully among each other. Such an intuitive collaboration would also be possible with computers if the machines understood the meanings of the sketches. Arrow symbols are a frequent ingredient of such sketched diagrams. Due to the arrows’ versatility, however, it remains a challenging problem to make computers distinguish the various semantic roles of arrow symbols. The solution to this problem is highly desirable for more effective and user-friendly pen-based systems. This thesis, therefore, develops an algorithm for deducing the semantic roles of arrow symbols, called the arrow semantic interpreter (ASI). The ASI emphasizes the structural patterns of arrow-containing diagrams, which have a strong influence on their semantics. Since the semantic roles of arrow symbols are assigned to individual arrow symbols and sometimes to the groups of arrow symbols, two types of the corresponding structures are introduced: the individual structure models the spatial arrangement of components around each arrow symbol and the inter-arrow structure captures the spatial arrangement of multiple arrow symbols. The semantic roles assigned to individual arrow symbols are classified into orientation, behavioral description, annotation, and association, and the formats of individual structures that correspond to these four classes are identified. The result enables the derivation of the possible semantic roles of individual arrow symbols from their individual structures. In addition, for the diagrams with multiple arrow symbols, the patterns of their inter-arrow structures are exploited to detect the groups of arrow symbols that jointly have certain semantic roles, as well as the nesting relations between the arrow symbols. The assessment shows that for 79% of sample arrow symbols the ASI successfully detects their correct semantic roles, even though the average number of the ASI’s interpretations is only 1.31 per arrow symbol. This result indicates that the structural information is highly useful for deriving the reliable interpretations of arrow symbols

Transforming structured descriptions to visual representations. An automated visualization of historical bookbinding structures.

In cultural heritage, the documentation of artefacts can be both iconographic and textual, i.e. both pictures and drawings on the one hand, and text and words on the other are used for documentation purposes. This research project aims to produce a methodology to transform automatically verbal descriptions of material objects, with a focus on bookbinding structures, into standardized and scholarly-sound visual representations. In the last few decades, the recording and management of documentation data about material objects, including bookbindings, has switched from paper-based archives to databases, but sketches and diagrams are a form of documentation still carried out mostly by hand. Diagrams hold some unique information, but often, also redundant information already secured through verbal means within the databases. This project proposes a methodology to harness verbal information stored within a database and automatically generate visual representations. A number of projects within the cultural heritage sector have applied semantic modelling to generate graphic outputs from verbal inputs. None of these has considered bookbindings and none of these relies on information already recorded within databases. Instead they develop an extra layer of modelling and typically gather more data, specifically for the purpose of generating a pictorial output. In these projects qualitative data (verbal input) is often mixed with quantitative data (measurements, scans, or other direct acquisition methods) to solve the problems of indeterminateness found in verbal descriptions. Also, none of these projects has attempted to develop a general methodology to ascertain the minimum amount ii of information that is required for successful verbal-to-visual transformations for material objects in other fields. This research has addressed these issues. The novel contributions of this research include: (i) a series of methodological recommendations for successful automated verbal-to-visual intersemiotic translations for material objects — and bookbinding structures in particular — which are possible when whole/part relationships, spatial configurations, the object’s logical form, and its prototypical shapes are communicated; (ii) the production of intersemiotic transformations for the domain of bookbinding structures; (iii) design recommendations for the generation of standardized automated prototypical drawings of bookbinding structures; (iv) the application — never considered before — of uncertainty visualization to the field of the archaeology of the book. This research also proposes the use of automatically generated diagrams as data verification tools to help identify meaningless or wrong data, thus increasing data accuracy within databases