Physikit: Data Engagement Through Physical Ambient Visualizations in the Home

Internet of things (IoT) devices and sensor kits have the potential to democratize the access, use, and appropriation of data. Despite the increased availability of low cost sensors, most of the produced data is "black box" in nature: users often do not know how to access or interpret data. We propose a "human-data design" approach in which end-users are given tools to create, share, and use data through tangible and physical visualizations. This paper introduces Physikit, a system designed to allow users to explore and engage with environmental data through physical ambient visualizations. We report on the design and implementation of Physikit, and present a two-week field study which showed that participants got an increased sense of the meaning of data, embellished and appropriated the basic visualizations to make them blend into their homes, and used the visualizations as a probe for community engagement and social behavior

Tangible user interfaces : past, present and future directions

In the last two decades, Tangible User Interfaces (TUIs) have emerged as a new interface type that interlinks the digital and physical worlds. Drawing upon users' knowledge and skills of interaction with the real non-digital world, TUIs show a potential to enhance the way in which people interact with and leverage digital information. However, TUI research is still in its infancy and extensive research is required in or- der to fully understand the implications of tangible user interfaces, to develop technologies that further bridge the digital and the physical, and to guide TUI design with empirical knowledge. This paper examines the existing body of work on Tangible User In- terfaces. We start by sketching the history of tangible user interfaces, examining the intellectual origins of this field. We then present TUIs in a broader context, survey application domains, and review frame- works and taxonomies. We also discuss conceptual foundations of TUIs including perspectives from cognitive sciences, phycology, and philoso- phy. Methods and technologies for designing, building, and evaluating TUIs are also addressed. Finally, we discuss the strengths and limita- tions of TUIs and chart directions for future research

Computational and Robotic Models of Early Language Development: A Review

We review computational and robotics models of early language learning and development. We first explain why and how these models are used to understand better how children learn language. We argue that they provide concrete theories of language learning as a complex dynamic system, complementing traditional methods in psychology and linguistics. We review different modeling formalisms, grounded in techniques from machine learning and artificial intelligence such as Bayesian and neural network approaches. We then discuss their role in understanding several key mechanisms of language development: cross-situational statistical learning, embodiment, situated social interaction, intrinsically motivated learning, and cultural evolution. We conclude by discussing future challenges for research, including modeling of large-scale empirical data about language acquisition in real-world environments. Keywords: Early language learning, Computational and robotic models, machine learning, development, embodiment, social interaction, intrinsic motivation, self-organization, dynamical systems, complexity.Comment: to appear in International Handbook on Language Development, ed. J. Horst and J. von Koss Torkildsen, Routledg

Conceiving God: Literal and Figurative Prompt for a More Tectonic Distinction

John Sanders’ Theology in the Flesh, the first comprehensive overview of the toolkit that contemporary cognitive linguistics offers for theological appropriation, despite its remarkable success, gives rather minimal attention to blending theory, one of the discipline’s most formidable tools. This paper draws on blending theory to offer an alternative to Sanders’ chapter on conceiving God. Central to the proposal is claim that God-talk, like many of the advances in science, technology, and art, entails a kind of tectonic understanding and conceptual mapping that is neither literal nor figurative

Prompting for Discovery: Flexible Sense-Making for AI Art-Making with Dreamsheets

Design space exploration (DSE) for Text-to-Image (TTI) models entails navigating a vast, opaque space of possible image outputs, through a commensurately vast input space of hyperparameters and prompt text. Minor adjustments to prompt input can surface unexpectedly disparate images. How can interfaces support end-users in reliably steering prompt-space explorations towards interesting results? Our design probe, DreamSheets, supports exploration strategies with LLM-based functions for assisted prompt construction and simultaneous display of generated results, hosted in a spreadsheet interface. The flexible layout and novel generative functions enable experimentation with user-defined workflows. Two studies, a preliminary lab study and a longitudinal study with five expert artists, revealed a set of strategies participants use to tackle the challenges of TTI design space exploration, and the interface features required to support them - like using text-generation to define local "axes" of exploration. We distill these insights into a UI mockup to guide future interfaces.Comment: 13 pages, 14 figures, currently under revie

Which structures are out there? : Learning predictive compositional concepts based on social sensorimotor explorations

How do we learn to think about our world in a flexible, compositional manner? What is the actual content of a particular thought? How do we become language ready? I argue that free energy-based inference processes, which determine the learning of predictive encodings, need to incorporate additional structural learning biases that reflect those structures of our world that are behaviorally relevant for us. In particular, I argue that the inference processes and thus the resulting predictive encodings should enable (i) the distinction of space from entities, with their perceptually and behaviorally relevant properties, (ii) the flexible, temporary activation of relative spatial relations between different entities, (iii) the dynamic adaptation of the involved, distinct encodings while executing, observing, or imagining particular interactions, and (iv) the development of a – probably motor-grounded – concept of forces, which predictively encodes the results of relative spatial and property manipulations dynamically over time. Furthermore, seeing that entity interactions typically have a beginning and an end, free energy-based inference should be additionally biased towards the segmentation of continuous sensorimotor interactions and sensory experiences into events and event boundaries. Thereby, events may be characterized by particular sets of active predictive encodings. Event boundaries, on the other hand, identify those situational aspects that are critical for the commencement or the termination of a particular event, such as the establishment of object contact and contact release. I argue that the development of predictive event encodings naturally lead to the development of conceptual encodings and the possibility of composing these encodings in a highly flexible, semantic manner. Behavior is generated by means of active inference. The addition of internal motivations in the form of homeostatic variables focusses our behavior – including attention and thought – on those environmental interactions that are motivationally-relevant, thus continuously striving for internal homeostasis in a goal-directed manner. As a consequence, behavior focusses cognitive development towards (believed) bodily and cognitively (including socially) relevant aspects. The capacity to integrate tools and other humans into our minds, as well as the motivation to flexibly interact with them, seem to open up the possibility of assigning roles – such as actors, instruments, and recipients – when observing, executing, or imagining particular environmental interactions. Moreover, in conjunction with predictive event encodings, this tool- and socially-oriented mental flexibilization fosters perspective taking, reasoning, and other forms of mentalizing. Finally, I discuss how these structures and mechanisms are exactly those that seem necessary to make our minds language ready