Eftring, Håkan

Magnusson, Charlotte

Rassmus-Gröhn, Kirsten

English

Lund University Publications

LUND UNIVERSITYPO Box 117221 00 Lund+46 46-222 00 00Drawing and guiding gestures in a mathematical task using the AHEAD applicationMagnusson, Charlotte; Rassmus-Gröhn, Kirsten; Eftring, HåkanPublished in:ENACTIVE 072007Link to publicationCitation for published version (APA):Magnusson, C., Rassmus-Gröhn, K., & Eftring, H. (2007). Drawing and guiding gestures in a mathematical taskusing the AHEAD application. In ENACTIVE 07Total number of authors:3General rightsUnless other specific re-use rights are stated the following general rights apply:Copyright and moral rights for the publications made accessible in the public portal are retained by the authorsand/or other copyright owners and it is a condition of accessing publications that users recognise and abide by thelegal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private studyor research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portalRead more about Creative commons licenses: https://creativecommons.org/licenses/Take down policyIf you believe that this document breaches copyright please contact us providing details, and we will removeaccess to the work immediately and investigate your claim.Download date: 07. Oct. 2022Proceedings of ENACTIVE/07 4th International Conference on Enactive Interfaces Grenoble, France, November 19th-22nd, 2007 Drawing and guiding gestures in a mathematical task using the AHEAD application   Charlotte Magnusson*  Kirsten Rassmus-Gröhn* Håkan Eftring*  (*)Department of Design Sciences, Lund University, Sweden E-mail: charlotte.magnusson@certec.lth.se, kirre@certec.lth.se, hakan.eftring@certec.lth.se     Abstract  This paper presents partial results from an evaluation done in practical school work of the audio haptic drawing editor and explorer (AHEAD). In this paper we focus on a mathematical design and reproduce group work task performed by two pupils (one sighted and one visually impaired pupil). Apart from the fact that the pupils were able to do the task, tentative results show some differences in the usage of the program functionality while no significant differences can be seen in the times spent drawing. We also present some discussion of the guiding gesture used by the mouse user to help the PHANToM user understand the drawn image.      1. Introduction  Getting access to 2D graphics is still a large problem for users that are severely visually impaired. Using a haptic display in combination with audio feedback is one way to enable access. General guidelines to create and develop haptic applications and models are collected in [1]. Applications making practical use of non-spoken audio and force-feedback haptics for visually impaired people are e.g. applications supporting mathematical display [2], [3] & [4], games [5-7] and audio-haptic maps [5;8]. As described in [9] and [10], there are indeed people who are blind who have an interest in hand drawing. In [11], a CAD application is presented that enables users to create drawings with the help of audio and keyboard. In [12], a study on a haptic drawing and painting program is presented.   2. The AHEAD application  The AHEAD audio haptic drawing editor and explorer application (see figure 1) is well described in [13] and we give only a short summary here. The virtual environment consists of a virtual sheet of paper that is oriented in the vertical plane (standing up). The application can be used in two different modes: one for editing and one for exploring relief drawings. The application supports two users, one mouse user and one PHANToM user. Both users can draw, edit and explore on the virtual paper. There is also a guiding function where either user can guide the pointer of the other user. The PHANToM user can pull the mouse to the PHANToM position and the mouse user can drag the PHANToM to the mouse position (in this case the PHANToM user experiences a constant force which drags the stylus towards the mouse position).  Figure 1. Two pupils using the AHEAD application.   3. The mathematical design and reproduce group work task  The test was conducted in a ninth grade class in the subject mathematics. The learning task in the particular lesson was to practice using geometrical mathematical language, i.e. words like “rectangle”, “sphere”, “angle” and “diagonal” to describe a composite geometry figure to a fellow pupil. The sighted pupils were instructed to use paper, pencil and a ruler, and the visually impaired pupil and a fellow pupil were instructed to use the AHEAD application. The test setup consisted of a laptop running the program and acting as screen and keyboard for the sighted pupil  ENACTIVE/07   who also was using a mouse for input. The visually impaired pupil had a separate keyboard attached to the same computer, a screen, headphones and the PHANToM OMNI. Half of each screen was blinded by a piece of cardboard to prevent the pupils from seeing the drawing the other person made. The AHEAD application was loaded with a file with a subtle grid in positive relief, and a middle line with the spoken caption “Stop, middle line” (see figure 2).  The pupils were supposed to use one part of the virtual paper/screen each to draw on and the middle line was not to be crossed until the last phase of the task. There were three parts to the task; first, one pupil would design a composite figure in the drawing application (without showing it to the other pupil); second, the same pupil would describe the figure to the other pupil who would try to make a copy based on the description; third, the pupils would together compare the copy to the original figure.   Figure 2. The empty grid for the mathematic task..  4. Results  If we start by looking at the resulting images (figure 3) we see that the pupils have succeeded with the task – i.e both have managed to understand the instructions and show this understanding by drawing a reasonable replica of the original (the drawings are not perfect, but they catch the essence of the design).    Figure 3. The pairs of drawings generated. In the top row, the mouse user (to the right) designed the original, while in the bottom row it was the PHANToM user that did this. The time order in these images are task 1,3 and 5 in the top row and 2,4 and 6 in the bottom row.  Since the AHEAD application logged both sampled PHANToM and mouse positions, together with clicks and keyboard commands it is possible to look a little more in detail at the process behind these drawings. If we start by looking at the PHANToM movements (figure 4) we see that the PHANToM movements are mainly concentrated where they should be, indicating that the user does not have problems with disorientation, something which was supported by observations made during the test – the visually impaired PHANToM user seemed quite well aware of where things were, and did not appear to have any major problems getting lost within the workspace (despite the absence of any limiting box in the current version of the AHEAD application). One problem that did occur was that the user on one occasion accidentally erased the wrong line – something which indicates that more feedback is needed for this type of operations – but apart from this no major problems were experienced. In the top row one can see a few points in the wrong side of the workspace. Since these are not present in the lower row we interpret these to be mainly “resting points” – the user is not really using the PHANToM, but is instead waiting for the mouse user to complete the design.    Figure 4. Sampled PHANToM x-y positions for the different tasks.  In the bottom row one can also see the guided exploration of the mouse generated drawing to the right. It is clear that this guided motion is quite different from the exploration on the left side – the motion on the left side shows more scanning behavior. The PHANToM positions can be compared to the sampled mouse movements shown in figure 5, where the mouse is used not so much for exploration, but mainly for drawing/guiding and quick pointing gestures. The fact that the faster mouse motion is more spread out over the workspace can be partly explained by the fact that the mouse also is used for interaction with the file open dialogs – and of course by the fact that it may be moved unintentionally while the mouse user is waiting for the PHANToM user. For the mouse it is quite clear that the drawing motions are slower than the other gestures (dots closer in the diagrams), but for the PHANToM this is not so obvious. There are some fast explorative moves, but also slower and more detailed ones.  ENACTIVE/07    Figure 5. Sampled mouse x-y positions for the different tasks.  If we look at the amount of erasing actions performed (table 1) we can see a clear learning effect. Both users start by having to erase several times, but after a while they are more familiar with both the tool and the exercise, and no longer need to erase any lines.  Table 1. Nr of erasing actions. The user drawing the original is indicated by *. Task PHANToM Mouse  1 0 6* 2 3* 1 3 3 0* 4 0* 0 5 0 0* 6 0* 0  Looking closer at the drawing gestures for task 1 we can see that the sighted user has experimented quite a lot by drawing and erasing, and this user also used the moving functions to move the lines to the right positions, as well as the shape functions to change lines into straight lines, circles or rectangles. The visually impaired user did not use any of these functions and drew everything where it should be from the beginning (and did not bother to make perfect circles, lines or rectangles). In task 2 the visually impaired user actually succeeded quite well drawing the figure initially, but then accidentally drew a small line and when attempting to erase this unwanted line instead erased the larger drawing. Because of this the other lines were also erased and the user started from the beginning again. The second time this user had more problems positioning the lines correctly, but it is still possible to see the intended figure (see figure 3, bottom left). In task 3 the sighted mouse user immediately gets things the way she wants them, while the visually impaired pupil gets it a bit wrong initially, and instead of using the keys for moving the lines, the lines in the erroneous position are erased and new ones are drawn. For the tasks 4, 5 and 6 the only change made is by the sighted user who uses the straight line function once in task 4 and 5 each. Otherwise the drawings are made the way they are shown in figure 3.  The time spent drawing can be seen in table 2. If we look at the all six tasks and check for differences between the time spent drawing, or the average drawing time per line no significant differences can be found between the PHANToM and the mouse (t-test, t=0,98 and 0,95 respectively). Despite this, some more qualitative observations can still be made. If we look at the total time spent drawing we see that for task 1 and task 2 (the first time each user takes on the different roles in the task) the user designing the original spends longer time drawing than the person drawing the copy. For the following tasks the opposite is true, i.e making the copy takes longer. This should be expected since making the copy involves more talking – the user making the copy has to get the spoken instructions. That things are different the first time is also quite understandable – the pupils have to get used to both the equipment and the exercise. No such differences can be seen for the average time per line, which may indicate that the difference is not so much due to the drawing as it is to the other activities in the tasks. Disregarding the first two tasks it seems that the ratio between the total drawing time of the designer and the copier is smaller when the mouse user is the designer compared to when the PHANToM user is the designer. Since the time spent drawing different lines does not differ this should be due to some other part of the task – the spoken part of the exercise and/or the cognitive/explorative part. Closer examination of the time span between drawing the lines does not reveal significant differences between the PHANToM and the mouse (t=0.76) although one can see that the PHANToM user performs more consistently than the mouse user.  Table 2. The time in seconds spent drawing for the different tasks. For task 1,3 and 5 the mouse user is the one that designs the original, while for the tasks 2,4 and 6 it is the PHANToM user that does this.  Another interesting issue is how the guidance was used.  When the sighted user drew a copy, this was followed by  mouse guiding to present the result to the visually impaired pupil (in the opposite condition there  PHANToM Mouse Ratio, total Task total average per line total average per line Designer/ copier 1 79 7 164 4 2,1 3 141 5 47 6 0,3 5 151 4 42 7 0,3 2 194 6 43 10 4,5 4 42 5 77 6 0,56 86 8 134 8 0,6  ENACTIVE/07   was no guiding – the sighted user just checked the results visually). If we look at the three cases when mouse guidance was used we see that the mouse user tends to start the gesture a bit away (nearer to the position of the PHANToM user) and then guide the PHANToM towards and around the drawing. This strategy appeared to work well – particularly if we compare it to what has happened in other tests where the mouse user sometimes just put the mouse at the desired position and tried to drag the PHANToM from there (no real guiding motion).   5. Conclusion  In this test we find the AHEAD application useful and the students were able to perform the task. The editing functions (apart from erasing) such as moving and changing curves to circles, rectangles or straight lines were only used by the sighted user. For the mouse we see a difference between slower drawing gestures and quicker pointing/moving gestures. For the PHANToM this difference is not so clear – there appears to be both slow and fast exploratory movements. Surprisingly enough there is no significant differences between the time spent drawing for the PHANToM and the mouse, apart from a tendency for the ratio between the time spent drawing for designer and copier to be smaller when the mouse user is the designer (disregarding the first time each user does the designing). Due to the low number of trials and the nature of the task it is hard to say anything definite about this, but since the time spent drawing individual lines appear to be quite similar it is possible the difference is due to the verbal/cognitive part of the task.  We note that our mouse user uses a guiding strategy that works quite well – the mouse user “catches” the PHANToM away from the line and guides it towards and around the drawing. Finally,  we see that the function where the PHANToM user could drag the mouse to the PHANToM position was not used.  6. Acknowledgements  This work is funded by the EU/ IST eInclusion projects MICOLE (IST-2003-511592 STP) and ENACTIVE (IST-2002-002114). The authors are grateful to the EU for the support given in carrying out these activities.  References   [1] Sjöström, Calle, "Non-Visual Haptic Interaction Design - Guidelines and Applications." Ph D Dept. for Design Sciences, Lund University, Faculty of Engineering, 2002. [2] Yu, W. and Brewster, S. A. Comparing Two Haptic Interfaces for Multimodal Graph Rendering.  2002. Florida, USA.  [3] Yu, W., Kangas, K., and Brewster, S., "Web-based haptic applications for blind people to create virtual graphs," Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2003.HAPTICS 2003.Proceedings.11th Symposium on, pp. 318-325, 2003. [4] Bussell, L. Touch Tiles: Elementary Geometry Software with a Haptic and Auditory Interface for Visually Impaired Children.  512-515. 2006. 2003.  [5] Iglesias, R., Casado, S., Gutierrez, T., Barbero, J. I., Avizzano, C. A., Marcheschi, S., and Bergamasco, M., "Computer graphics access for blind people through a haptic and audio virtual environment," Haptic, Audio and Visual Environments and Their Applications, 2004.HAVE 2004.Proceedings.The 3rd IEEE International Workshop on, pp. 13-18, 2004. [6] Magnusson, C., Rassmus-Gröhn, K., Sjöström, C., and Danielsson, H. Navigation and Recognition in Complex Haptic Virtual Environments - Reports from an Extensive Study with Blind Users. Wall, S. A., Riedel, B., Crossan, A., and McGee, M. R.  2002. Edinburgh, UK.  [7] Magnusson, C. and Rassmus-Gröhn, K. Audio haptic tools for navigation in non visual environments.  2005. 11-17-0050.  [8] Magnusson, C. and Rassmus-Gröhn, K., "A Virtual Traffic Environment for People with Visual Impairments," Visual Impairment Research, vol. 7, no. 1, pp. 1-12, 2005. [9] Kennedy, J. M., Drawing and the Blind - Pictures to Touch New Haven and London: Yale University Press, 1993. [10] Art Beyond Sight Art Education for the Blind, Inc. and AFB Press, 2003. [11] Kamel, Hesham M., "The Integrated Communication 2 Draw (IC2D)." Ph D Electrical Engineering and Computer Sciences Department, University of California, 2003. [12] Hansson, Christine, "Haptic Drawing Program." Master Design Sciences, Lund University, Faculty of Engineering, 2003. [13] Rassmus-Gröhn, K., Magnusson, C., and Eftring, H. Iterative Design of an Audio-haptic Drawing Application.  2007.  CHI 2007. 4-28-0070.        ENACTIVE/07    

Drawing and guiding gestures in a mathematical task using the AHEAD application

https://lup.lub.lu.se/search/files/6129554/3045845.pdf

Drawing and guiding gestures in a mathematical task using the AHEAD application

Abstract

Similar works

Full text

Available Versions

Lund University Publications