Abstraction as a basis for the computational interpretation of creative cross-modal metaphor

Various approaches to computational metaphor interpretation are based on pre-existing similarities between source and target domains and/or are based on metaphors already observed to be prevalent in the language. This paper addresses similarity-creating cross-modal metaphoric expressions. It is shown how the “abstract concept as object” (or reification) metaphor plays a central role in a large class of metaphoric extensions. The described approach depends on the imposition of abstract ontological components, which represent source concepts, onto target concepts. The challenge of such a system is to represent both denotative and connotative components which are extensible, together with a framework of general domains between which such extensions can conceivably occur. An existing ontology of this kind, consistent with some mathematic concepts and widely held linguistic notions, is outlined. It is suggested that the use of such an abstract representation system is well adapted to the interpretation of both conventional and unconventional metaphor that is similarity-creating

From holism to compositionality: memes and the evolution of segmentation, syntax, and signification in music and language

Steven Mithen argues that language evolved from an antecedent he terms “Hmmmmm, [meaning it was] Holistic, manipulative, multi-modal, musical and mimetic”. Owing to certain innate and learned factors, a capacity for segmentation and cross-stream mapping in early Homo sapiens broke the continuous line of Hmmmmm, creating discrete replicated units which, with the initial support of Hmmmmm, eventually became the semantically freighted words of modern language. That which remained after what was a bifurcation of Hmmmmm arguably survived as music, existing as a sound stream segmented into discrete units, although one without the explicit and relatively fixed semantic content of language. All three types of utterance – the parent Hmmmmm, language, and music – are amenable to a memetic interpretation which applies Universal Darwinism to what are understood as language and musical memes. On the basis of Peter Carruthers’ distinction between ‘cognitivism’ and ‘communicativism’ in language, and William Calvin’s theories of cortical information encoding, a framework is hypothesized for the semantic and syntactic associations between, on the one hand, the sonic patterns of language memes (‘lexemes’) and of musical memes (‘musemes’) and, on the other hand, ‘mentalese’ conceptual structures, in Chomsky’s ‘Logical Form’ (LF)

Learning by Seeing by Doing: Arithmetic Word Problems

Learning by doing in pursuit of real-world goals has received much attention from education researchers but has been unevenly supported by mathematics education software at the elementary level, particularly as it involves arithmetic word problems. In this article, we give examples of doing-oriented tools that might promote children\u27s ability to see significant abstract structures in mathematical situations. The reflection necessary for such seeing is motivated by activities and contexts that emphasize affective and social aspects. Natural language, as a representation already familiar to children, is key in these activities, both as a means of mathematical expression and as a link between situations and various abstract representations. These tools support children\u27s ownership of a mathematical problem and its expression; remote sharing of problems and data; software interpretation of children\u27s own word problems; play with dynamically linked representations with attention to children\u27s prior connections; and systematic problem variation based on empirically determined level of difficulty

Artist-Programmers and Programming Languages for the Arts

We consider the artist-programmer, who creates work through its description as source code. The artist-programmer grandstands computer language, giving unique vantage over human-computer interaction in a creative context. We focus on the human in this relationship, noting that humans use an amalgam of language and gesture to express themselves. Accordingly we expose the deep relationship between computer languages and continuous expression, examining how these realms may support one another, and how the artist-programmer may fully engage with both. Our argument takes us up through layers of representation, starting with symbols, then words, language and notation, to consider the role that these representations may play in human creativity. We form a cross-disciplinary perspective from psychology, computer science, linguistics, human-computer interaction, computational creativity, music technology and the arts. We develop and demonstrate the potential of this view to inform arts practice, through the practical introduction of software prototypes, artworks, programming languages and improvised performances. In particular, we introduce works which demonstrate the role of perception in symbolic semantics, embed the representation of time in programming language, include visuospatial arrangement in syntax, and embed the activity of programming in the improvisation and experience of art

Metaphor, theories of concepts and biological reductionism

This thesis is an attempt to present a multidisciplinary approach to adjectival polysemy, particularly adjectival polysemy of a metaphorical type, and its underlying conceptual structure. The last ten years have clearly shown a tendency towards reducing the number of meanings and the idea of metaphor as a mechanism of concept formation has been gaining much force, influencing research in linguistics, psychology and cognitive science. Despite that fact, the long-standing tradition of analytic philosophy did not succumb to the attack. However few contentions are shared in these different fields, one is held unquestioningly by almost everyone. It is the literal-metaphorical distinction which is at the heart of both traditional philosophy and the theory of embodied realism. Drawing extensively on evidence from research on cross-modal perception, synesthesia, double-function terms in cross-cultural studies, child development, psycholinguistic experiments and experiments with brain-damaged subjects, reinterpreting the available data and analyzing in detail theories of concepts contained in cognitive linguistics, lexical semantics and informational semantics, the thesis casts doubt on the validity of the literalmetaphorical distinction, for this class of examples. It stipulates the existence of psychologically primitive concepts, which are likely to be atomic and innate, and offers a no-polysemy view of conceptual structure with implications for linguistic polysemy. It also shows the limits of biological reductionism and emphasizes the need for functional approaches to cognition. The proposed alternative is both unexpected and exciting, and may serve as a basis for bringing together empirical evidence and philosophical coherence in a non-contradictory wa

Dataremix: Aesthetic Experiences of Big Data and Data Abstraction

This PhD by published work expands on the contribution to knowledge in two recent large-scale transdisciplinary artistic research projects: ATLAS in silico and INSTRUMENT | One Antarctic Night and their exhibited and published outputs. The thesis reflects upon this practice-based artistic research that interrogates data abstraction: the digitization, datafication and abstraction of culture and nature, as vast and abstract digital data. The research is situated in digital arts practices that engage a combination of big (scientific) data as artistic material, embodied interaction in virtual environments, and poetic recombination. A transdisciplinary and collaborative artistic practice, x-resonance, provides a framework for the hybrid processes, outcomes, and contributions to knowledge from the research. These are purposefully and productively situated at the objective | subjective interface, have potential to convey multiple meanings simultaneously to a variety of audiences and resist disciplinary definition. In the course of the research, a novel methodology emerges, dataremix, which is employed and iteratively evolved through artistic practice to address the research questions: 1) How can a visceral and poetic experience of data abstraction be created? and 2) How would one go about generating an artistically-informed (scientific) discovery? Several interconnected contributions to knowledge arise through the first research question: creation of representational elements for artistic visualization of big (scientific) data that includes four new forms (genomic calligraphy, algorithmic objects as natural specimens, scalable auditory data signatures, and signal objects); an aesthetic of slowness that contributes an extension to the operative forces in Jevbratt’s inverted sublime of looking down and in to also include looking fast and slow; an extension of Corby’s objective and subjective image consisting of “informational and aesthetic components” to novel virtual environments created from big 3 (scientific) data that extend Davies’ poetic virtual spatiality to poetic objective | subjective generative virtual spaces; and an extension of Seaman’s embodied interactive recombinant poetics through embodied interaction in virtual environments as a recapitulation of scientific (objective) and algorithmic processes through aesthetic (subjective) physical gestures. These contributions holistically combine in the artworks ATLAS in silico and INSTRUMENT | One Antarctic Night to create visceral poetic experiences of big data abstraction. Contributions to knowledge from the first research question develop artworks that are visceral and poetic experiences of data abstraction, and which manifest the objective | subjective through art. Contributions to knowledge from the second research question occur through the process of the artworks functioning as experimental systems in which experiments using analytical tools from the scientific domain are enacted within the process of creation of the artwork. The results are “returned” into the artwork. These contributions are: elucidating differences in DNA helix bending and curvature along regions of gene sequences specified as either introns or exons, revealing nuanced differences in BLAST results in relation to genomics sequence metadata, and cross-correlation of astronomical data to identify putative variable signals from astronomical objects for further scientific evaluation

The metaphorical brain [Research topic]

This Frontiers Special Issue will synthesize current findings on the cognitive neuroscience of metaphor, provide a forum for voicing novel perspectives, and promote new insights into the metaphorical brain

The Grid Sketcher: An AutoCad-based tool for conceptual design processes

Sketching with pencil and paper is reminiscent of the varied, rich, and loosely defined formal processes associated with conceptual design. Architects actively engage such creative paradigms in their exploration and development of conceptual design solutions. The Grid Sketcher, as a conceptual sketching tool, presents one possible computer implementation for enhancing and supporting these processes. It effectively demonstrates the facility with which current technology and the computing environment can enhance and simulate sketching intents and expectations; Typically with respect to design, the position taken is that the two are virtually void of any fundamental commonality. A designer\u27s thoughts are intuitive, at times irrational, and rarely follow consistently identifiable patterns. Conversely, computing requires predictability in just these endeavors. The computing environment, as commonly defined, can not reasonably expect to mimic the typically human domain of creative design. In this context, this thesis accentuates the computer\u27s role as a form generator as opposed to a form evaluator. The computer, under the influence of certain contextual parameters can, however, provide the designer with a rich and elegant set of forms that respond through algorithmics to the designer\u27s creative intents. (Abstract shortened by UMI.)

Abstract expressions of affect

What form should happiness take? And how is disgust shaped? This research investigates how synthetic affective expressions can be designed with minimal reference to the human body. The authors propose that the recognition and attribution of affect expression can be triggered by appropriately presenting the bare essentials used in the mental processes that mediate the recognition and attribution of affect. The novelty of the proposed approach lies in the fact that it is based on mental processes involved in the recognition of affect, independent of the configuration of the human body and face. The approach is grounded in (a) research on the role of abstraction in perception, (b) the elementary processes and features relevant to visual emotion recognition and emotion attribution, and (c) how such features can be used (and combined) to generate a synthetic emotion expression. To further develop the argument for this approach they present a pilot study that shows the feasibility of combining affective features independently of the human configuration by using abstraction to create consistent emotional attributions. Finally, the authors discuss the potential implications of their approach for the design of affective robots. The developed design approach promises a maximization of freedom to integrate intuitively understandable affective expressions with other morphological design factors a technology may require, providing synthetic affective expressions that suit the inherently artificial and applied nature of affective technology