Dix, Alan

Lancaster E-Prints

beyond intentionpushing boundaries with incidental interactionAlan DixLancaster University, Lancaster UKalan@hcibook.comhttp://www.hcibook.com/alan/topics/incidental/Abstract.  Traditional interaction is purposeful, but interactions based onsensing and environmental context are of lower salience and require lessexplicit attention.  The extreme end of this spectrum, but becoming morecommon, is implicit interaction, which occurs when a user action intendedfor one purpose is interpreted to achieve some other goal.  Because it is anextreme point, incidental interaction acts as a touchstone allowing us toexplore the limits of traditional interaction models, task analysis, cognitionand software architectures.  We find that all either break down or requirerethinking in the face of these radically novel interactions.1 . introductionIn traditional computer applications a user was expected to approach the system with aclear intention to perform some activity or achieve some goal.  The actions werepurposeful and direct and the results were explicitly attended to and evaluated.  Thedesign emphasis is on making the affordances of interaction unambiguous and availableand ensuring that system feedback and state are clearly visible.However, in many areas of human computer interaction we have seen a growing numberof systems and interaction paradigms where user attention and intention is lower.  InCSCW the concept of awareness has been central for many years and similarly ambientinterfaces emphasise low salience displays of background information.  Paradigms suchas calm interfaces, tacit and implicit interaction all emphasis output that is non-intrusiveand ecologically natural forms of input/control.Whereas the traditional interface was based around controls and input devices, these lowattention and natural input paradigms are more closely related to sensing technologies andcontextual interpretations.At the extreme end of the spectrum are interactions where actors (users) perform actionsfor some purpose (say opening a door to enter a room) and the system senses this anduses it for some purpose of which the actors are unaware (perhaps adjusting the airconditioning), but which affects their future interactions with the environment or system.I call this incidental interaction.In this paper we'll look briefly at the definition of incidental interaction and someexamples.  We'll then see how this challenges many areas of traditional interactiondesign: the fundamental execution–evaluation cycle implicit in much of HCI, the notionof task and task analysis, the limits of cognitive abilities and the impact of softwarearchitecture.  Incidental interaction is being used to highlight these issues, but it is evidentthat these are problematic in many forms of low attention and low intention interaction.beyond intention 22 . incidental interactiondefinitionwhere actions performed for some other purpose, or unconscious signs,are interpreted in order to influence/improve/facilitatethe actors' future interaction or day-to-day lifeThe above definition is expanded in detail on the incidental interaction web pages(above), but we will briefly review the main issues.The actor has some purpose other than that of the actual incidental interaction (althoughthe results may impact on the user's activity) and may not even be consciously doinganything at all.  As an example of the latter the Commonwealth baton had an electronic'flame' that flashed depending on the bearer's heart rate.The system or application interprets these explicitly.  We are not talking about unintendedside effects.  Although incidental interactions are not intentional from the actor'sperspective they are intended by the designer.This interpretation is then used to in some way improve or facilitate the actor's futureinteractions with the system (e.g. adapting the interface) or the actor's environment(e.g. adjusting the heating).Finally note that the definition is phrased in terms of the plural 'actors'.  This is todeliberately include cases where the property being sensed is related to a group orrelationship (e.g. a meeting), and also to include those where the effect of the incidentalinteraction is to influence some other member of the group rather than those directlysensed (e.g. some awareness mechanisms).We can position incidental interaction at one end of a continuum:intentional press light switchexpected walk into room expecting lights to switch onincidental walk into room and unbeknown to you air conditioning increasesexamplesWe have already seen some examples in the description above.  The automatic lights arequite interesting as a small change would turn them from an expected interaction to anincidental one.  Imagine a house had real light switches, but (for energy saving) alwaysswitches off the lights in rooms that no-one is in. When you approach the room the lightsautomatically turn on, so that you are never aware that they switch off.   Doors thatautomatically unlock when an active badge is near are another example.Car courtesy lights have more complex behaviour.   Depending on the model they mayturn on when the doors are unlocked or when the doors are opened.  They may turn offafter some fixed time, when the doors are closed or when the engine is started.Underlying this is some designer's model of the task of getting into a car – perhapssorting out belongings in the car, looking at a map, etc., before setting off.  The sensorsare unreliable means of detecting the user's intentions, but the incidental interactions withthe lights are designed to support the task.  Note, the driver's purpose is to get into thecar and incidentally the lights come on.In the Pepys project at Xerox EuroPARC , all staff wore an 'active badge' that detectedtheir location in the building using infra-red sensors [[WHFG92]].  The Pepys systemanalysed the logs of people's location at the end of each day and used these to produce abeyond intention 3personal diary for each person [[NEL91]].  Because Pepys knew about the layout of theoffices, and who was where when, it was able to detect when two or more people werein the same location and create a diary entry "had meeting with Allan and Victoria".  Noteagain Allan and Victoria's purpose is to visit Paul's office and incidentally a diary entry iscreated for each of them.The MediaCup [[GBK99,BGS01]] also facilitates incidental interaction.It is a base unit that can be attached to an ordinary coffee mug anddetects movement (when drinking, walking around, etc.), pressure (forfullness), temperature (fresh coffee vs. old) and location.  Theinformation gathered by this gives some indication of the drinker'scurrent activity and location, which can then be used for communityawareness.  Hans' purpose in filling the mug is to have a drink ofcoffee and incidentally his colleagues become aware he is taking abreak.Incidental interactions can also take place entirely within the electronic domain.In many electronic shopping sites the system keeps track of the items you have purchasedor examined and then suggests additional products based on your inferred tastes.  Yourpurpose is to buy product X and incidentally the system infers your tastes and suggestsproduct Y.One system I have worked on personally was onCue.  This is an'intelligent' toolbar that sits on the side of the user's screen.  When theuser cuts or copies any text onCue examines the clipboard.  It analysesthe content and depending on the type of material changes the tool itemsto reflect it.  If the copied text is a post code, onCue suggests internetmapping services; if it looks like a name (initial capitals etc.), onCuesuggests internet directory services; if the text is a list or table ofnumbers, onCue suggests spreadsheet or graphing applications.  Aswell as the triggering event being implicit, the 'suggestion' isdeliberately low salience, the currently suggested services slowly fadein as small icons in the toolbar.  Note again, the user's purpose is tocopy the text somewhere else and incidentally potential useful servicesget offered.fluidityintendedexpectedincidentalco-optioncomprehensionThe continuum from intentional, through expected, to incidentalinteraction is largely about purpose – what the user thinks – notthe actual system itself.  Of course, certain types of system willsuggest more or less strongly one mode of interaction, but there issome fluidity depending on the user's experience, awareness, etc.As the users become more aware of the interactions happeningaround them they may move through the continuum towards morepurposeful interaction.• comprehension: incidental → expected – If users notice,consciously or unconsciously, that incidental interactionsare taking place they may come to expect or rely uponthem.  For example, if I realise that the courtesy  lights come on when I get intothe car I may leave checking my route until I get in knowing that it will be lit.• co-option: expected → intended – When I know that something will happenwhen I perform an action, then I may deliberately perform the action to cause theeffect.  For example, I may deliberately open and close the car door to trigger thecourtesy light mechanism.beyond intention 43 . challenging our modelsAs well as being an interesting interaction paradigm in its own right, incidental interactionreally pushes our fundamental assumptions about interaction and our ways of modellingit.  I will not try and 'solve' these issues within this paper, but instead will raise theproblematic aspects as a challenge for future HCI theory and practice.interaction modelsThe explicit or implicit model behind nearly all interaction is some form of intentionalcycle such as the Norman execution–evaluation loop [[N90]].  The user has some goal(intention), formulates some action that he/she believes will act towards that goal,performs the action and then re-formulates future actions based on the feedback.In traditional cognitive modelling this was seen as very plan driven with goal stacks andhierarchies etc.  In these accounts, the intentional cycle is seen as starting with the user,even to the point that the effects of feedback are often ignored.  In more contextualaccounts of interaction such as situated action [[S87]] or distributed cognition [[H90]],the goals or intentions are more at the level of overall motivations or end-state aspirationsand the focus tends to shift to a cycle of activity starting with the state of the world andrecent system 'responses' with the user acting on the world in response to the currentstate.  However, this is still clearly purposeful activity.Incidental interaction and to a lesser extent expected interactions do not fit this picture.The user and system share the experience of the user's actions, but the purposefulactivity of the user is distinct from the intended outcomes of the system.This is not just a theoretical issue as it is an underlying assumption that cuts throughnearly every usability guideline, principle and method.  For example, the importance ofrapid and visible feedback is based on the assumption that users need to understand fullythe effect of their actions.  In incidental interaction and low awareness applications theopposite is often true.task analysisTraditional task analysis is also highly purposeful, although debatably less wedded tothis than cognitive models.  Certainly to cope with more contextual interactions taskanalysis methods need to evolve to include or link to representations that are more aboutthe physical world and the rich ecology of lived work or domestic life.  In a previouspaper I have examined some of the potential issues and extensions that may be necessaryfor this [[D02]].However, incidental interaction poses a more fundamental question – what task do weneed to model?  In incidental interaction we have two 'tasks' that are occurring.(i) the user's primary task – their purposeful activity.(ii) the task that the incidental interaction is attempting to support or achieveOften, as in the case of the courtesy light, the two are the same task, but it is used indifferent ways.  The user's purposeful activity is assumed to occur, to a large extent,independently of the system's actions.  We need to model it in order to computationallyinterpret the user's actions as activity.  In contrast, we need to model (ii) in order tounderstand how to facilitate or progress it.beyond intention 5cognitionNatural inanimate physical things have a set of properties intrinsic to their physicality:• directness of effect  –  You push something a little and it moves a little, you pushhard and it moves a lot.• locality of effect  –  Effects are here and now.  If you push a rock and then 2 secondslater it moves you would be disturbed.• visibility of state  –  Solid objects have a small number of relevant parameters thatdefine their dynamic state: location, orientation.  We have some difficulty withinvisible properties such as velocity and even more when physical things do havehidden state, for example, the 'joke' balls that have a ball bearing inside and so donot move in a straight line.  Of course this example is not natural but constructed.We have evolved as creatures to cope with physical things and other creatures, nottechnological devices.  Although we have higher level reasoning that enables us to cope –the same reasoning that enables us to create  technology, this is only significant when we'think about' things, our more innate cognitive abilities are shaped by the natural.Computer systems (and other complex technology such as electrical and pneumatic)break these intrinsics of physicality.  Computation creates complexity of effect, networksintroduce non-locality in space, memory non-locality in time and a computer has a vastnumber of invisible variables in its hidden internal state.We cope (just) with this because either we rationalise and use higher-level thinking tomake sense and to make models of these complex non-physical interactions, or we treatthe computer as animate.  In addition, one of the reasons for the development of the GUIinterface style is that it makes the electronic world more like real (inanimate) things.In incidental, expected and low awareness interactions the intention is that the user is notpaying attention to or is unaware of the system's activities.  That is, we are not able tobring our higher cognitive abilities to bear and are dependent on our more innateintelligences, which are of course ill prepared for unphysicality.  To make matters worse,the system activities are often triggered by physical actions and movements of the userand are manifest in the physical world.  In a computer system we are able to re-frameourselves to expect the odd or the apparently magical actions to occur.  In the real worldthese are deeply disturbing.system architectureIn incidental interactions it is very likely that sensors are not used solely for their originalpurpose.  This suggests the need for quite open architectures.  For example, in designingonCue we used a very open blackboard-style architecture for exchanging informationbetween self-discovering and self-configuring components [[DBW00]].  Unfortunately,when we get down to the level of individual applications it is far harder to get contextualinformation without writing special code for each potential application.  This is one of thereasons for using copy/cut to the clipboard as the main trigger – it is one of the fewcross-application standards.Furthermore, many of the contextual interactions envisaged in this area occur in domesticor other private environments.  If we are not careful architectural openness could violateprivacy – imagine if the can of beans (with intelligent food label) you just boughtcommunicated back to the manufacturer the contents of your food cupboard.Highly contextual interactions must also take on board the fact that many of the mostimportant phenomena are not events (things that happen at specific moments), but status(things that always have some measurable value).  My own work on status–eventanalysis has highlighted common phenomena that can be used to understand suchsystems, but this also impacts on the underlying system architectures [[D91,DA96]].beyond intention 64 . so what?Probably the dominant type of context-sensitive interaction is still where theenvironmental context influences the purposeful activity.  In contrast, in incidentalinteraction, the context of the user's current purposeful interaction is used to drive asecondary interaction.  This is perhaps context driven not just context sensitive.The extreme nature of incidental interaction 'breaks' many traditional conceptual, analyticand implementation models.  However, these are not just problems for designingincidental interactions.  To some extent incidental interaction is highlightingcontradictions that apply to other areas of contextual and low-attention interaction styles.So, as well as being an important class of interactions to study in its own right, incidentalinteraction can function as a touchstone for those creating new frameworks, architecturesand models for sensors, context-sensitive applications and ubicomp design frameworks.I am afraid this paper presents few answers, more questions and suggestions about thefuture.  But I hope it also begins to set a framework to guide the process of answeringthese questions.5 . acknowledgementsThis paper draws on discussions with and inspiration from many colleagues andespecially work related to the EPSRC funded CASCO, EQUATOR, and DIRC projects.6 . references[[BGS01]]    Beigl, M., Gellersen, H.-W. and Schmidt, A. MediaCups: Experience with Design and Useof Computer-Augmented Everyday Objects. Computer Networks, Vol. 35, No. 4, Special Issue onPervasive Computing, Elsevier, March 2001, pp. 401-409.[[D91]]  A. J. Dix (1991). Status and events: static and dynamic properties of interactive systems.Proceedings of the Eurographics Seminar: Formal Methods in Computer Graphics, Ed. D. A. Duce.Marina di Carrara, Italy.[[DA96]]  A. Dix and G. Abowd (1996). Modelling status and event behaviour of interactive systems.Software Engineering Journal, 11(6) pp. 334-346.[[DBW00]]  A. Dix, R. Beale and A. Wood (2000). Architectures to make Simple Visualisations usingSimple Systems. Proceedings of Advanced Visual Interfaces - AVI2000, ACM Press, pp. 51-60.[[D02]]  A. Dix (2002).  Managing the Ecology of Interaction. Proceedings of Tamodia 2002 - FirstInternational Workshop on Task Models and User Interface Design, Bucharest, Romania, 18-19 July2002[[GBK99]]    H-W. Gellersen, M. Beigl, H. Krull (1999). The MediaCup: Awareness Technologyembedded in an Everyday Object.  Int. Sym. Handheld and Ubiquitous Computing (HUC99)Karlsruhe, Germany, 1999[[H90]]  Hutchins, E. (1990), The Technology of team navigation.  In Intellectual teamwork: social andtechnical bases of collaborative work. Gallagher, J., Kraut, R. and Egido, C., (eds.), LawrenceErlbaum.[[NEL91]]  Newman, W., Eldridge, M. & Lamming, M. (1991). Pepys: Generating Autobiographies byAutomatic Tracking. In Proceedings of the second European conference on computer supportedcooperative work – ECSCW '91, 25-27 September 1991, Kluwer.Academic Publishers, Amsterdam.pp 175-188.[[N90]]  D. Norman (1990). The Design of Everyday Things. Doubleday, New York.[[S87]]  Suchman, L. (1987). Plans and Situated Actions: The problem of human–machinecommunication. Cambridge University Press,.[[WHFG92]]  Roy Want, Andy Hopper, Veronica Falcao, Jonathon Gibbons.  The Active BadgeLocation System.  ACM Transactions on Information Systems, Vol. 10, No. 1, January 1992, pp91-102.

Beyond intention - pushing boundaries with incidental interaction

Abstract

Similar works

Full text

Available Versions

Lancaster E-Prints