Issued as Interim reports [nos. 1-2], and Final report, Project no. C-36-62

Badre, Albert Nasib

English

Scholarly Materials And Research @ Georgia Tech

Interacting with Electronic Work Surfaces: Studies in Note Taking and Writing Behavior Principal investigator: Albert N. Badre Georgia Institute of Technology Fi~al Report December 31, 1990 Effects of Long Term Usage Albert N. Badre Paulo Santos Georgia Tech Handwriting recognition systems are to be used by both novice and more experienced users. Every new user of such a system will initially be a novice, with no previous experience of using the system, and will gradually become more and more proficient in using a system by using the acquired expertise. Several experiments have been executed to study the influence of several effects on handwriting rccoguilion. us~..:r satisfaction, and overall system performance. However, due to the short duration of alllltosc experimenL<;, all the data collected thus far pertains exclusively to novice users. In order to uncover effects of long-term usage, another experiment was designed and performed. This report presents the results of such an experiment This experiment is the sixth and last in a series of experiments investigating the effect of several variables on the performance of a handwriting recognition system. Purpose The primary goal of this experiment was to investigate the effects of long term usage of a handwriting recognition system in user behavior and satisfaction. and in system performance. The hypothesis we formulated was that users would change their behavior in order to increase the system recognition accuracy as they become more experienced at using the system. Secondary purposes of the experiment were to identify the ways in which people change the ways they form characters, and to identify possible uses of the timing information to improve recognition accuracy. Description of the experiment The experiment consisted of capturing handwriting samples from subjects, writing on a transparent digitizing tablet. Fourteen subject, all students at Georgia Tech, were asked to visit the usability laboratory at Georgia Tech a total of fourteen times over a period of two months, with no more than a session per day. Laboratory sessions were scheduled by appointment, and most subjects maintained a regular schedule of two visits per week. Two feedback conditions were being studjed: immediate continuous feedback, where the subjects received feedback on recognition concurrently with their writing task; <md postponed feedback. where subjects received feedback at the end of each page of text The pool of subjects consisted of 14 Georgia Tech students, ten males and four females, with ages ranging from 21 to 45 years and averaging 26 years. Thirteen of the subjects were right-handed. and one (male. 23) was left-h~ded. Subjects were paid for their participation in the experiment, and their pay was prorated to the number of characters recognized by the system in each session. They would also be given an extra bonus for completing all fourteen sessions - which they all did - that was also prorated to their average performance over all fourteen sessions. Before the first session their were briefed on the purpose and nature of the experiment. and on their duties as subjects. They read and signed a release form, informing them of their rights (including quitting the experiment at any time) and allowing us to collect the required data. They were also instructed on how to operate the system, and were shown a sheet with guidelines on how to fonn characters. They were allowed to refer to the guideline sheet at any time during the sessions. At the end of the first session they were asked to fill out a questionnaire. with demographic information. Furthermore, at the end of the first. seventh and fourteenth sessions, they were asked to fill out a questionnaire with their opinions on the system. Each session consisted of writing a text. which was projected onto the wall, into a computer, using a transparent digitizer which rested above an LCD. Each subject was instructed to write in all block, uppercase letters (the only type of print recognizable by our algorithm) and to leave a small space between letters. They wrote in the bottom half of the digitizer, which presented a lined display fonna~ with lines spaced about 7mm apart. The text consisted of alphabetic characters. digits and punctuation, and the text length ranged from 419 to 638 characters (mean 563, stand.ml deviation 67). In each session, the text was presented as a sequenced of four transparencies projected onto the wall in front of the subjects. The overall distribution of characters in the paragraphs is shown in table l. A 6.98% B 1.32% c 2.87% D 3.19% E 12.04% F 2.52% G 1.72% H 4.02% I 7.22% J 0.25% K 0.61% L 3.60% M 2.73% N 6.05% 0 7.45% p 1.89% Q 0.25% R6.16% s 6.25% T 7.72% u 3.04% v 1.13% w 1.68% X0.25% y 2.10% z 0.23% 0 0.65% 1 0.65% 2 0.30% 3 0.26% 4 0.26% 5 0.31% 6 0.21% 7 0.34% 8 0.18% 9 0.47% Punctuation 3.1% Table 1 - Distribution of characters over all texts Two independent variables were present in this experiment: session number (1 through 14) and feedback (postponed or continuous). This resulted in a 14x2 2-factor experiment design, with a total of 28 conditions. In the postponed feedback condition, subjects saw what the system recognized at the end of each screen full of text, which generally corresponded to one page of text projected onto the wall. In the continuous feedback condition, the text being recognized by the system was being displayed simultaneously (or with a maximwn delay of one second) with the user's writing. In this case, the users wrote in the bottom half of the screen and the recognized text appeared in the top half of the screen. Equipment and recognition overview Subjects wrote on a Seiko D-Scao digitizer/display device. The display was passive matrix VGA with approximate dimensions of 289mm wide X 157nun high. It was covered by a transparent digitizer and a glass plate, and connected to an NCR 386 PC. A stylus that was tethered to the digitizer was used as the writing device. Stroke tiata collected from the digitizer device were fed to a rule-based character recognition system based on feature detection. Strokes were made up of a series of points sampled from the digitizer at a constant frequency of 143 Hz during pen down states. The recognition software attempted to both segment (identify which strokes should be grouped to form a single character) and recognize (identify which ch~~!er the strokes represent) the stroke data. For recognition to occur. a set of strokes must be segmented correctly and pass all the rules for one and only one character in the system's rulebase. Otherwise, an "unrecognized" response i~etumed for a set of strokes. Once strokes are segmented to contruct a character (either correctly, incorrectly, or yielding an unrecognizable situation). they are considered used and are not included in future attempts to segment and recognize characters. Therefore, an incorre\:t segmentation of a character may lead to a series of incorrect recognitions due to incorrect segmenting. Factors affecting timing performance The effects of the two factors involved in this experiment on the timing performance were analyzed. Two dependent variables were considered: the overall time to complete a session, and the total pen-down time to complete a session. The overall time to complete a session is computed as the sum of t11e times taken to complete a pages, for every page in a session. This method ignores the time spent between pages, but it was nevertheless chosen because the time between pages carries a large variability due to factors that are not object of the study of this experiment As a consequence, it is not possible to detennine the time spent by those subjects receiving postponed feedback, which occurs between pages. The effects of practice on timing performance -Practice at using the system is a direct result of repeated usage. Therefore, higher practice levels can be equated with larger session numbers, and vice versa. The formulated hypothesis is that the time to write decreases with practice. Stated in different tenns, the hypothesis states that there is a negative correlation between the session nwnber (practice) and the tim required to write a fixed number of characters. Figure I shows the average time required to write IOOO characters, and a trend for the negative correlation is evident. Statistical significance is found in the completion time decrease between sessions that are 10 session nwnbers apart, such as between I and 11 or between 2 and 12. Comparing average completion time between continuous and postponed feedback conditions 2500 2000 Normalized time 1500 in seconds per 1 000 chars 1 000 500 0+--+---+----+-t---+--+--+--+----it--+---+--+---1 2 3 4 5 6 7 8 9 10 11 12 13 14 Session Figure 1 • Continous II Postponed A quick glance at figure 1 suggests that users perform faster when using postponed feedback than when using continuous feedback. Let us clarify that this is not necessarily true, and how the above data can not be interpret~_in that way. When users are perfonning their task. with the system, they are either reading text to be written, writing that text, or checking the correctness of their writing. However, in the postponed feedback condition they only check the correctness after the end of each page of input. The way in which the experimental system calculates the total time to complete the task is as the sum of lhe times to complete each page. The time to complete each page is defined as the time that elapses between the instant the user ftrSt puts the stylus down on that page, and the instant the stylus is lifted up after the last character on tbe page. Therefore, the time required to check the feedback in the postponed feedback condition is not being considered, and this may account for a significant time that is not quantified in this experiment. A comparison of the writing times can be made by comparing the total time that the pen is down in each session. Tllis should not be considered as the time to complete a writing task, but it illustrates that there does not seem to be any significant difference in the writing times between postponed and continuous feedback conditions. Figure 2 shows the comparison of total pen-down time for each session, under the . two feedback conditions. Comparing average total pen-down time between continuous and postponed feedback conditions Normalized time in seconds per 1000 chars 1000 800 600 400 200 0+-;-~-+~--~+-~~~-+~--~ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Session Figure 2 * Continous II Postponed By analyzing the data displayed in figure 2. we can not claim that the type of feedback (continuous or postponed) has an effect on total pen-down time: the time required to actually write. We can still claim. however. that there are is a negative correlation between practice and total pen-down time. Statistical significance is again found in the completion time decrease between sessions that are 10 session numbers apat""t such as between I and 11 or between 2 and 12. Timing analysis of the formation of characters An extensive analysis was performed on the temporal infonnation associated with character formation . Each recognized character consisted of a sequence of one to four strokes with the stylus on the digitizer. Of all the IOO.,(X>O+ characters captured during the experiment. 46.2% were formed with a single stroke. 29.4% had two strokes, 20.4% three strokes, and 4.0% four strokes. The average times to form one-stroke characters was 555 milliseconds; for two-stroke characters, 916ms; for three-stroke characters. 1117ms; and for four-stroke characters, 155lms. Pen-up times within and between characters One of the most significant problems in recognition is the proper segmentation of characters. The CIC algorithm used in this experiment perfonns the segmentation of characters based exclusively on spatial characteristics of the characters. In previous experiments we bad empirically verified that one of the most frequeiifcauses of error was poor segmentation. Therefore. we investigated whether stroke timing information could be of any usefulness for the segmentation problem. The problem with segmentation occurs due to the existence of multiple stroke characters. For correct segmentation. an algorithm has to group correctly the set of strokes that forma character. and to separate these strokes from those that belong to the preceding and following characters. Our hypothesis is that shorter pen-up times occur within the same character than between separate characters. To confirm or deny this hypothesis. we analyzed all the pen-up time intervals that occurred between strokes of the same character, and those that occurred between strokes belonging to two characters of the same word. Figure 3 shows the distribution of such times. for intervals shorter than 1 second and with a 55 millisecond granularity. Time between pen-up and pen-down 0.25 0.2 0.15 0.1 0.05 "• f \ -• ' 111181 ! .. \ ' .. ' f \ ~ ' I ; 0 11 0 220 330 440 550 660 770 880 990 Time in milliseconds Figure 3 II Within characters + Between characters Figure 3 shows, as expected, that strokes that occur within tbe same character are closer apart in time than strokes that belong to different characters. The significance of this result is increased when we ask the question from a different perspective: Given an pen-up time t, what isd the probability that the two strokes separated by that pen-up belong to the same character? Figure 4 shows the answer that is derived from the expeerimental data collected. Probability of within/between characters given time between pen-up and pen-down 1.00 0.90 0.80 0.70 0.60 II Within characters Probability 0.50 0.40 + Between characters 0.30 0.20 0.10 0.00 0 220 440 660 880 Time in milliseconds Figure 4 This chart shows that for pen-up times of 110 milliseconds or less, there is approximately a 90% probability that the strokes blong to the same character, and consequentJy a l 0% probability that they belong to different characters. At 500 milliseconds or above, those probabilities are approximately inverted. Between 110 and 500 milliseconds, the probability curves cross in an approximately linear curve. Pen-up velocity within and between characters It may be argued that the results of the previous section (figures 3 and 4) are biased by the fact that strokes belonging to different characters are generally spaced further apart in space, and therefore it takes longer time just to move the stylus due to the increased distance. If this is true, then the pen-up time is of no relevance to the segmentation problem, because it adds no information to the spacial information. In order to prove or disprove this fact, we analyzed the pen-up velocity. For eacb of the pen-up events considered in the previous section, we calculated the distance between the pen-up location and the inunediately following pen~own location, and divided distance by time to obtain the velocity of the pen movement Figure 5 plots the results, as obtained from the data collected in this experiment 0.007 0.006 0.005 0.004 0.003 0.002 0.001 Velocity between pen-up and pen-down Between 0~~4-~~~~~~~~~~~+-0 100 200 300 400 500 600 700 800 900 Velocity in pixels per second Figure 5 Then again, we asked: given a velocity in pixels per second, what is the probability that the two strokes belong to the same character. The data from this experiment is presented in a more appropriate fom1 to answer this question in figure 6. Probability 1.00 0.80 0.60 0.40 0.20 Probability of within/between characters given velocity between pen-up and pen-down 0. 00 -P-+--+--+--+--+--+--t--t--t---1t---1i"--""''U-L.....fuu.tl!-&a. 0 1 00 200 300 400 500 600 700 800 900 Velocity in pixels per second Figure 6 Figure 6 is shows that for low velocities (up to 30 pixels per second), there is a large probability that the strokes are within the same characters. Then, for intennediate velocities, the trend inverts and at around 100 pixels per seocnd there is a 60% probability tbat tbe strokes belong to different characters and about a 40% probability that they are part of the same character. Then the trend inverts once more, and for larger velocities (particularly for velocities greater than 300 pixels per second), there is a fairly constant 70% probability that the strokes belong to the same character. Implications of the results The data and analysis above suggest that stroke temporal infonnation can be used to enhance tbe accuracy of the segmentation task in character recognition. Algorithms should look at character fonnation infonnation based on spatial and temporal iofonnation. Then, they should attempt to recognize characters based on joint probabilities given by the visual properties of tbe characters and by the timing associated with their segmentation . ... 

Interacting with electronic work surfaces : studies in note taking and writing behavior

https://smartech.gatech.edu/bitstream/handle/1853/49805/c-36-627_368010_fr.pdf?sequence=2&isAllowed=y

Interacting with electronic work surfaces : studies in note taking and writing behavior

Abstract

Similar works

Full text

Available Versions

Scholarly Materials And Research @ Georgia Tech