TurboMouse: End-to-end Latency Compensation in Indirect Interaction

International audienceEnd-to-end latency corresponds to the temporal difference between a user input and the corresponding output from a system. It has been shown to degrade user performance in both direct and indirect interaction. If it can be reduced to some extend, latency can also be compensated through software compensation by trying to predict the future position of the cursor based on previous positions, velocities and accelerations. In this paper, we propose a hybrid hardware and software prediction technique specifically designed for partially compensating end-to-end latency in indirect pointing. We combine a computer mouse with a high frequency accelerometer to predict the future location of the pointer using Euler based equations

Characterizing Latency in Touch and Button-Equipped Interactive Systems

International audienceWe present a low cost method to measure and characterize the end-to-end latency when using a touch system (tap la-tency) or an input device equipped with a physical button. Our method relies on a vibration sensor attached to a finger and a photo-diode to detect the screen response. Both are connected to a micro-controller connected to a host computer using a low-latency USB communication protocol in order to combine software and hardware probes to help determine where the latency comes from. We present the operating principle of our method before investigating the main sources of latency in several systems. We show that most of the latency originates from the display side. Our method can help application designers characterize and troubleshoot latency on a wide range of interactive systems

User interface design considerations for emerging input technologies in iTV

Streaming media and interactive television viewing experiences are becoming more commonplace with the introduction of services such as Netflix Streaming, the Apple TV, and Google TV aided by the increase adoption of broadband internet. As these services make their way into the living room, and developers struggle to accommodate more complex interaction requirements, new input methods and interfaces need to be developed. Current interfaces for controlling interactive TV and media management have typically been designed for the desktop and laptop experience, using conventional input devices like a trackpad, mouse and keyboard. These techniques are difficult to reconcile with the typical TV viewing experience. We designed an experiment to test a representative interactive TV interface with a number of emerging input technologies like the Nintendo Wiimote, Microsoft Kinect and tablet applications. We measured user performance with these devices while encumbered by a beverage and plate of food in order to simulate a living room experience. We found that while most of these technologies are suitable for navigating an Interactive TV experience, their use challenges us to rethink the user experience, and places limitations on things like button size and placement, as well as the types of UI widgets we can use. We hope these guidelines and heuristics will help in the design of future interactive TV experiences, as well as the development of novel interaction techniques for the TV viewing experience

Two Touch System Latency Estimators: High Accuracy and Low Overhead

International audienceThe end-to-end latency of interactive systems is well known to degrade user's performance. Touch systems exhibit notable amount of latencies, but it is seldom characterized, probably because latency estimation is a difficult and time consuming undertaking. In this paper, we introduce two novel approaches to estimate the latency of touch systems. Both approaches require an operator to slide a finger on the touch surface, and provide automatic processing of the recorded data. The High Accuracy (HA) approach requires an external camera and careful calibration, but provides a large sample set of accurate latency estimations. The Low Overhead (LO) approach, while not offering as much accuracy as the HA approach, does not require any additional equipment and is implemented in a few lines of code. In a set of experiments, we show that the HA approach can generate a highly detailed picture of the latency distribution of the system, and that the LO approach provides average latency estimates no further than 4~ms from the HA estimate

Modeling and Reducing Spatial Jitter caused by Asynchronous Input and Output Rates

International audienceJitter in interactive systems occurs when visual feedback is perceived as unstable or trembling even though the input signal is smooth or stationary. It can have multiple causes such as sensing noise, or feedback calculations introducing or exacerbating sensing imprecisions. Jitter can however occur even when each individual component of the pipeline works perfectly, as a result of the differences between the input frequency and the display refresh rate. This asynchronicity can introduce rapidly-shifting latencies between the rendered feedbacks and their display on screen, which can result in trembling cursors or viewports. This paper contributes a better understanding of this particular type of jitter. We first detail the problem from a mathematical standpoint, from which we develop a predictive model of jitter amplitude as a function of input and output frequencies, and a new metric to measure this spatial jitter. Using touch input data gathered in a study, we developed a simulator to validate this model and to assess the effects of different techniques and settings with any output frequency. The most promising approach, when the time of the next display refresh is known, is to estimate (via interpolation or extrapolation) the user’s position at a fixed time interval before that refresh. When input events occur at 125 Hz, as is common in touch screens, we show that an interval of 4 to 6 ms works well for a wide range of display refresh rates. This method effectively cancels most of the jitter introduced by input/output asynchronicity, while introducing minimal imprecision or latency

In-Air Un-Instrumented Pointing Performance

I present an analysis of in-air un-instrumented pointing and selection. I look at the performance of these systems and how this performance can be improved, with the eventual goal that their throughput reaches that of the mouse. Many potential limiting factors were explored, such as latency, selection reliability, and elbow stabilization. I found that the un-instrumented in-air pointing as currently implemented performed significantly worse, at less than 75% of mouse throughput. Yet, my research shows that this value can potentially reach mouse-like levels with lower system latencies, user training, and potentially improved finger tracking. Even without these improvements, the large range of applications for un-instrumented 3D hand tracking makes this technology still an attractive option for user interfaces

Next-Point Prediction Metrics for Perceived Spatial Errors

International audienceTouch screens have a delay between user input and corresponding visual interface feedback, called input “latency” (or “lag”). Visual latency is more noticeable during continuous input actions like dragging, so methods to display feedback based on the most likely path for the next few input points have been described in research papers and patents. Designing these “next-point prediction” methods is challenging, and there have been no standard metrics to compare different approaches. We introduce metrics to quantify the probability of 7 spatial error “side-effects” caused by next-point prediction methods. Types of side-effects are derived using a thematic analysis of comments gathered in a 12 participants study covering drawing, dragging, and panning tasks using 5 state-of- the-art next-point predictors. Using experiment logs of actual and predicted input points, we develop quantitative metrics that correlate positively with the frequency of perceived side-effects. These metrics enable practitioners to compare next- point predictors using only input logs

Inspiring End-user Programming Through Exposure to Multi-platform Computing and Improved Integrated Development Environments

The art of software engineering inherently requires high-level problem solving and perseverance, as programmers and designers wrestle with complex design and implementation challenges in the process of turning loose concepts and ideas into working code. In the current developer ecosystem, engineers are commonly incentivized extrinsically by monetary rewards, approval or status. Ironically, extrinsically motivating a person has been proven to decrease complex problem solving performance. On the other hand, motivating a person intrinsically through the Self Deterministic Theory constructs of autonomy, competence and relatedness has been shown to encourage problem solving, creativity and innovation. To determine what motivates different types of developers, and how well their tools support them in their work, we developed and deployed a survey. The validated survey tool measured the intrinsic motivation level of 103 developers of varying personas. Based on the data gathered, we highlight areas for improving the current development environment to foster increased problem solving and creativity

Characterizing the Effects of Local Latency on Aim Performance in First Person Shooters

Real-time games such as first-person shooters (FPS) are sensitive to even small amounts of lag. The effects of network latency have been studied, but less is known about local latency -- that is, the lag caused by local sources such as input devices, displays, and the application. While local latency is important to gamers, we do not know how it affects aiming performance and whether we can reduce its negative effects. To explore these issues, we tested local latency in a variety of real-world gaming systems and carried out a controlled study focusing on targeting and tracking activities in an FPS game with varying degrees of local latency. In addition, we tested the ability of a lag compensation technique (based on aim assistance) to mitigate the negative effects. To motivate the need for these studies, we also examined how aim in FPS differs from pointing in standard 2D tasks, showing significant differences in performance metrics. Our studies found local latencies in the real-world range from 23 to 243~ms that cause significant and substantial degradation in performance (even for latencies as low as 41~ms). The studies also showed that our compensation technique worked well, reducing the problems caused by lag in the case of targeting, and removing the problem altogether in the case of tracking. Our work shows that local latency is a real and substantial problem -- but game developers can mitigate the problem with appropriate compensation methods