End-user Programming of Ambient Narratives

Ambient Intelligence is a vision on the future of consumer electronics, telecommunications and computing in which devices move into the background while at the same time placing the user experience in the foreground. Producing Ambient Intelligent environments on a large scale is problematic however. First, it is technologically not possible in the foreseeable future to mass produce a product or service that generates Ambient Intelligence, given the current state-of-the-art in machine learning and artificial intelligence. Second, it is economically not feasible to manually design and produce Ambient Intelligence applications for each person individually. One of the main research questions in creating such environments is the design of a system capable of supporting mass customization of ambient experiences by means of end-user programming. A brief outline of the approach taken to address this question is described including future research

Studies in ambient intelligent lighting

The revolution in lighting we are arguably experiencing is led by technical developments in the area of solid state lighting technology. The improved lifetime, efficiency and environmentally friendly raw materials make LEDs the main contender for the light source of the future. The core of the change is, however, not in the basic technology, but in the way users interact with it and the way the quality of the produced effect on the environment is judged. With the new found freedom the users can switch their focus from the confines of the technology to the expression of their needs, regardless of the details of the lighting system. Identifying the user needs, creating an effective language to communicate them to the system, and translating them to control signals that fulfill them, as well as defining the means to measure the quality of the produced result are the topic of study of a new multidisciplinary area of study, Ambient Intelligent Lighting. This thesis describes a series of studies in the field of Ambient Intelligent Lighting, divided in two parts. The first part of the thesis demonstrates how, by adopting a user centric design philosophy, the traditional control paradigms can be superseded by novel, so-called effect driven controls. Chapter 3 describes an algorithm that, using statistical methods and image processing, generates a set of colors based on a term or set of terms. The algorithm uses Internet image search engines (Google Images, Flickr) to acquire a set of images that represent a term and subsequently extracts representative colors from the set. Additionally, an estimate of the quality of the extracted set of colors is computed. Based on the algorithm, a system that automatically enriches music with lyrics based images and lighting was built and is described. Chapter 4 proposes a novel effect driven control algorithm, enabling users easy, natural and system agnostic means to create a spatial light distribution. By using an emerging technology, visible light communication, and an intuitive effect definition, a real time interactive light design system was developed. Usability studies on a virtual prototype of the system demonstrated the perceived ease of use and increased efficiency of an effect driven approach. In chapter 5, using stochastic models, natural temporal light transitions are modeled and reproduced. Based on an example video of a natural light effect, a Markov model of the transitions between colors of a single light source representing the effect is learned. The model is a compact, easy to reproduce, and as the user studies show, recognizable representation of the original light effect. The second part of the thesis studies the perceived quality of one of the unique capabilities of LEDs, chromatic temporal transitions. Using psychophysical methods, existing spatial models of human color vision were found to be unsuitable for predicting the visibility of temporal artifacts caused by the digital controls. The chapters in this part demonstrate new perceptual effects and make the first steps towards building a temporal model of human color vision. In chapter 6 the perception of smoothness of digital light transitions is studied. The studies presented demonstrate the dependence of the visibility of digital steps in a temporal transition on the frequency of change, chromaticity, intensity and direction of change of the transition. Furthermore, a clear link between the visibility of digital steps and flicker visibility is demonstrated. Finally, a new, exponential law for the dependence of the threshold speed of smooth transitions on the changing frequency is hypothesized and proven in subsequent experiments. Chapter 7 studies the discrimination and preference of different color transitions between two colors. Due to memory effects, the discrimination threshold for complete transitions was shown to be larger than the discrimination threshold for two single colors. Two linear transitions in different color spaces were shown to be significantly preferred over a set of other, curved, transitions. Chapter 8 studies chromatic and achromatic flicker visibility in the periphery. A complex change of both the absolute visibility thresholds for different frequencies, as well as the critical flicker frequency is observed. Finally, an increase in the absolute visibility thresholds caused by an addition of a mental task in central vision is demonstrated

End-user Programming of Ambient Narratives

Ambient Intelligence is a vision on the future of consumer electronics, telecommunications and computing in which devices move into the background while at the same time placing the user experience in the foreground. Ambient intelligenc

End-user programming of ambient narratives for smart retail environments

Ambient Intelligence is a vision on the future of the consumer electronics, telecommunications and computer industry that refers to electronic environments that respond to the presence and activity of people and objects. The goal of these intelligent environments is to support the performance of our everyday activities using technology and media in way that puts users in control. Many research prototypes and demonstrations of ambient intelligence systems and applications have been developed since the introduction of this vision, but many of these examples focus on a relatively small application domain and set of functionality. The downside of this reductionist approach is that it surpasses the open-ended dynamic nature and complexity that is inherent to social environments. This thesis aims to find a generic interaction concept to capture the way we form experiences in our everyday life and integrates technology and media into that process. It proposes the design of an end-user programming environment that supports retail designers without special programming skills to create, simulate and deploy smart retail environments within this interaction concept. To derive such a generic interaction concept it is necessary to look at the social, cultural and economical factors that shape the ambient intelligence landscape to better understand how ambient intelligence helps people in performing their everyday life activities and rituals. This analysis shows that notions like play and performance are not just seen on stage in the theatre: Everywhere around us people perform culturally defined social scripts, e.g. in court, in a restaurant or on the street. Social interaction itself in fact can be seen as an improvised performance that takes shape through the interaction of people with their surroundings. This implies technology can be applied to detect these social scripts and in turn affect the environment to improve the performance. This can be found back in shopping malls and shops for example: The shop employees perform a service for the customer in which the shop itself is seen as the stage and the increasingly interactive, dynamic lighting, audio, video the special effects to enhance the shopping experience for customers. In this experience economy, next to the multi-sensory trend also a development towards co-creation environments can be seen in which the consumer becomes an active producer in the construction of the experience that is offered to him. By looking at such co-creation environment from a literary, semiotic perspective they can be considered as interactive narratives consisting of interrelated social scripts with associated device actions. Through interaction with this possibly dynamically changing ambient narrative, i.e. by performing social scripts, players or readers construct their own story or ambient intelligence experience. To apply this ambient narrative concept in practice in the domain of retail, user research was conducted through interviews and workshops with professional users, i.e. retailers, designers and consultants to discover the types of interactive multi-sensory retail experiences these people would want to see and the requirements placed by this group on an end-user programmable ambient narrative system. From this research we learned that designers had a preference for a 3D simulation on a PC in their office to design ambient narratives, while retailers preferred a PDA version to adjust the retail experience on location. Furthermore, a list of thirty requirements was derived that can be grouped into four categories: ambient narrative concept implementation, runtime system performance, functionality end-user programming environment and system extensibility. On the basis of these findings, a formal model was defined to describe the problem of reconstruction ambient intelligence from the (dynamically changing) modular fragments of which a (dynamic) ambient narrative consists. Each fragment consists of a description of the social script and the actions on devices that are coupled to this script. The author of the ambient narrative can specify which fragments may be activated when by setting event triggers in the action section of another fragment. This model can be represented in a hypertext model in which every fragment is a node and each event trigger a link that connects nodes. An ambient narrative engine continuously sequences these fragments based on contextual information, session state and user feedback into a coherent story that is rendered by the devices that surround people. Next to the design and implementation of the ambient narrative engine, the functional user requirements were used to compose a system architecture for an intelligent shop window that supports the entire lifecycle of ambient narratives, from the initial design in a 3D simulation until the modification of fragments in-situ with the PDA. The intelligent shop window prototype with authoring environment was realized and evaluated in ShopLab at the High Tech Campus in Eindhoven on three main criteria: usability, run-time performance and extensibility. To test the usability a user study was conducted in which the participants were asked to perform four different programming tasks with this prototype tool and fill in a questionnaire afterwards with both questions on the intuitiveness, expressive power and efficiency of both authoring tools. From the results of this study we conclude the target user group is able to design intelligent shop window ambient narratives with this prototype system. An observation-in-use experiment revealed the response time of the system was satisfactory in the majority of cases but in some special cases could lead to long response times as a result of the current implementation of the ambient narrative engine. In terms of system architecture and extensibility towards other applications and domains, the strongest restrictions were placed by the assumed fixed sensor infrastructure and partial implementation of the formal ambient narrative model in the prototype