TurboMouse: End-to-end Latency Compensation in Indirect Interaction

International audienceEnd-to-end latency corresponds to the temporal difference between a user input and the corresponding output from a system. It has been shown to degrade user performance in both direct and indirect interaction. If it can be reduced to some extend, latency can also be compensated through software compensation by trying to predict the future position of the cursor based on previous positions, velocities and accelerations. In this paper, we propose a hybrid hardware and software prediction technique specifically designed for partially compensating end-to-end latency in indirect pointing. We combine a computer mouse with a high frequency accelerometer to predict the future location of the pointer using Euler based equations

An Inertial-Optical Tracking System for Quantitative, Freehand, 3D Ultrasound

Three dimensional (3D) ultrasound has become an increasingly popular medical imaging tool over the last decade. It offers significant advantages over Two Dimensional (2D) ultrasound, such as improved accuracy, the ability to display image planes that are physically impossible with 2D ultrasound, and reduced dependence on the skill of the sonographer. Among 3D medical imaging techniques, ultrasound is the only one portable enough to be used by first responders, on the battlefield, and in rural areas. There are three basic methods of acquiring 3D ultrasound images. In the first method, a 2D array transducer is used to capture a 3D volume directly, using electronic beam steering. This method is mainly used for echocardiography. In the second method, a linear array transducer is mechanically actuated, giving a slower and less expensive alternative to the 2D array. The third method uses a linear array transducer that is moved by hand. This method is known as freehand 3D ultrasound. Whether using a 2D array or a mechanically actuated linear array transducer, the position and orientation of each image is known ahead of time. This is not the case for freehand scanning. To reconstruct a 3D volume from a series of 2D ultrasound images, assumptions must be made about the position and orientation of each image, or a mechanism for detecting the position and orientation of each image must be employed. The most widely used method for freehand 3D imaging relies on the assumption that the probe moves along a straight path with constant orientation and speed. This method requires considerable skill on the part of the sonographer. Another technique uses features within the images themselves to form an estimate of each image\u27s relative location. However, these techniques are not well accepted for diagnostic use because they are not always reliable. The final method for acquiring position and orientation information is to use a six Degree-of-Freedom (6 DoF) tracking system. Commercially available 6 DoF tracking systems use magnetic fields, ultrasonic ranging, or optical tracking to measure the position and orientation of a target. Although accurate, all of these systems have fundamental limitations in that they are relatively expensive and they all require sensors or transmitters to be placed in fixed locations to provide a fixed frame of reference. The goal of the work presented here is to create a probe tracking system for freehand 3D ultrasound that does not rely on any fixed frame of reference. This system tracks the ultrasound probe using only sensors integrated into the probe itself. The advantages of such a system are that it requires no setup before it can be used, it is more portable because no extra equipment is required, it is immune from environmental interference, and it is less expensive than external tracking systems. An ideal tracking system for freehand 3D ultrasound would track in all 6 DoF. However, current sensor technology limits this system to five. Linear transducer motion along the skin surface is tracked optically and transducer orientation is tracked using MEMS gyroscopes. An optical tracking system was developed around an optical mouse sensor to provide linear position information by tracking the skin surface. Two versions were evaluated. One included an optical fiber bundle and the other did not. The purpose of the optical fiber is to allow the system to integrate more easily into existing probes by allowing the sensor and electronics to be mounted away from the scanning end of the probe. Each version was optimized to track features on the skin surface while providing adequate Depth Of Field (DOF) to accept variation in the height of the skin surface. Orientation information is acquired using a 3 axis MEMS gyroscope. The sensor was thoroughly characterized to quantify performance in terms of accuracy and drift. This data provided a basis for estimating the achievable 3D reconstruction accuracy of the complete system. Electrical and mechanical components were designed to attach the sensor to the ultrasound probe in such a way as to simulate its being embedded in the probe itself. An embedded system was developed to perform the processing necessary to translate the sensor data into probe position and orientation estimates in real time. The system utilizes a Microblaze soft core microprocessor and a set of peripheral devices implemented in a Xilinx Spartan 3E field programmable gate array. The Xilinx Microkernel real time operating system performs essential system management tasks and provides a stable software platform for implementation of the inertial tracking algorithm. Stradwin 3D ultrasound software was used to provide a user interface and perform the actual 3D volume reconstruction. Stradwin retrieves 2D ultrasound images from the Terason t3000 portable ultrasound system and communicates with the tracking system to gather position and orientation data. The 3D reconstruction is generated and displayed on the screen of the PC in real time. Stradwin also provides essential system features such as storage and retrieval of data, 3D data interaction, reslicing, manual 3D segmentation, and volume calculation for segmented regions. The 3D reconstruction performance of the system was evaluated by freehand scanning a cylindrical inclusion in a CIRS model 044 ultrasound phantom. Five different motion profiles were used and each profile was repeated 10 times. This entire test regimen was performed twice, once with the optical tracking system using the optical fiber bundle, and once with the optical tracking system without the optical fiber bundle. 3D reconstructions were performed with and without the position and orientation data to provide a basis for comparison. Volume error and surface error were used as the performance metrics. Volume error ranged from 1.3% to 5.3% with tracking information versus 15.6% to 21.9% without for the version of the system without the optical fiber bundle. Volume error ranged from 3.7% to 7.6% with tracking information versus 8.7% to 13.7% without for the version of the system with the optical fiber bundle. Surface error ranged from 0.319 mm RMS to 0.462 mm RMS with tracking information versus 0.678 mm RMS to 1.261 mm RMS without for the version of the system without the optical fiber bundle. Surface error ranged from 0.326 mm RMS to 0.774 mm RMS with tracking information versus 0.538 mm RMS to 1.657 mm RMS without for the version of the system with the optical fiber bundle. The prototype tracking system successfully demonstrated that accurate 3D ultrasound volumes can be generated from 2D freehand data using only sensors integrated into the ultrasound probe. One serious shortcoming of this system is that it only tracks 5 of the 6 degrees of freedom required to perform complete 3D reconstructions. The optical system provides information about linear movement but because it tracks a surface, it cannot measure vertical displacement. Overcoming this limitation is the most obvious candidate for future research using this system. The overall tracking platform, meaning the embedded tracking computer and the PC software, developed and integrated in this work, is ready to take advantage of vertical displacement data, should a method be developed for sensing it

Functional Aging of the Inner Ear Sensory Systems in Mouse Models of Age-Related Hearing Loss

Age related structural and functional change in the cochlea have been well described and predisposing factors including genetic background, gender, and environmental factors have been identified. To date, nine genetic loci contributing to age related hearing loss (ARHL) have been identified and auditory function has been described in mouse strains carrying these mutations. The effect of these ARHL mutations on the other inner ear modality (vestibular) is poorly understood. The objective of the current study was to characterize and compare age related change in auditory and vestibular function (more specifically macular function) in three strains carrying ARHL mutations (C57BL/6J, CE/J, and NOD NON-H2nb1/LtJ) and one control strain with no known mutations (CBA/CaJ). Auditory brainstem response (ABR) and distortion product otoacoustic emissions (DPOAE) were used to assess cochlear function and vestibular evoked potential (VsEP) was used to assess macular function. The macular organs from young, mid-life, and old animals were harvested, prepared, and imaged using scanning electron microscopy (SEM) and a qualitative comparison of the number and distribution of macular otoconia was made. ABR and VsEP thresholds, peak latencies, and peak to peak amplitudes were quantified. Linear regression, student's T-test, and ANOVA were used to describe and compare auditory and macular function between genders, within strain, and between strains. DPOAE amplitude was plotted as a function of geometric mean frequency for 11 age groups. No statistical analysis was performed on DPOAE data. No significant gender difference was found for auditory or vestibular function in any strain so genders were pooled for further analysis. The intra-strain comparison of auditory and macular sensitivity reveals a significantly different rate of change in the two modalities in all three strains (p<0.0001). Inter-strain comparison of change in macular sensitivity shows that the CBA/CaJ and CE/J strains lose sensitivity at a significantly faster rate than the C57 strain. A comparison of SEM images from young and old mice revealed no apparent qualitative difference in macular otoconia that would explain the observed declines in macular sensitivity.Ph.D

Barehand Mode Switching in Touch and Mid-Air Interfaces

Raskin defines a mode as a distinct setting within an interface where the same user input will produce results different to those it would produce in other settings. Most interfaces have multiple modes in which input is mapped to different actions, and, mode-switching is simply the transition from one mode to another. In touch interfaces, the current mode can change how a single touch is interpreted: for example, it could draw a line, pan the canvas, select a shape, or enter a command. In Virtual Reality (VR), a hand gesture-based 3D modelling application may have different modes for object creation, selection, and transformation. Depending on the mode, the movement of the hand is interpreted differently. However, one of the crucial factors determining the effectiveness of an interface is user productivity. Mode-switching time of different input techniques, either in a touch interface or in a mid-air interface, affects user productivity. Moreover, when touch and mid-air interfaces like VR are combined, making informed decisions pertaining to the mode assignment gets even more complicated. This thesis provides an empirical investigation to characterize the mode switching phenomenon in barehand touch-based and mid-air interfaces. It explores the potential of using these input spaces together for a productivity application in VR. And, it concludes with a step towards defining and evaluating the multi-faceted mode concept, its characteristics and its utility, when designing user interfaces more generally

Design and Application of Wireless Body Sensors

Hörmann T. Design and Application of Wireless Body Sensors. Bielefeld: Universität Bielefeld; 2019

Characterizing the Effects of High-intensity Exercise on Balance and Gait under Dual-task Conditions in Parkinson’s Disease

Parkinson’s disease (PD) is a neurodegenerative disorder, characterized by four cardinal motor symptoms including bradykinesia, tremor, rigidity, and postural instability, and non-motor symptoms including cognitive impairment. Daily activities, such as walking and maintaining balance, are impacted due to impairments in motor function, and are further exacerbated with the addition of cognitive loading, or dual-tasking (DT). High-intensity exercise has demonstrated centrally-mediated improvements of PD symptoms, with additional positive effects on overall health. The goal of this project was to identify changes in dynamic balance recovery and gait function under conditions with and without increased cognitive load after a high-intensity exercise intervention in a PD population. Participants included people with PD who completed an eight-week cycling intervention (PDE), people with Parkinson’s disease who did not complete the intervention (PDC), and healthy age-matched controls (HC), with 14 subjects per group. In Aim 1, while participants underwent a series of destabilizing balance tests, the time taken to regain balance and the center of pressure movement during balance recovery were measured. The PDE group demonstrated greater improvement in balance recovery after exercise compared with the PDC group. In Aim 2, participants completed a series of gait and cognitive tasks, both separately and concurrently. Outcome measures included spatiotemporal and kinematic gait parameters of the lower and upper extremities. The PDE group demonstrated significant improvement in gait measures and DT abilities compared to PDC, while no changes were found in cognitive function for any group. The standard clinical methods of measuring motor function can be subjective, and may not capture subtle motor characteristics. Force plate and motion-capture technologies can provide detailed, objective outcome data, therefore improving the understanding of how exercise affects motor symptoms of Parkinson’s disease. The Motek Computer Assisted Rehabilitation Environment (CAREN) system at the Cleveland Clinic was used to create the testing environment and for data collection. These results of this project suggest global changes in motor function demonstrated by changes in balance recovery and lower and upper extremity gait function. Quantitative gait analysis has shown to be an important metric in assessing effectiveness of an exercise intervention in PD

Characterizing the Effects of High-intensity Exercise on Balance and Gait under Dual-task Conditions in Parkinson’s Disease

Parkinson’s disease (PD) is a neurodegenerative disorder, characterized by four cardinal motor symptoms including bradykinesia, tremor, rigidity, and postural instability, and non-motor symptoms including cognitive impairment. Daily activities, such as walking and maintaining balance, are impacted due to impairments in motor function, and are further exacerbated with the addition of cognitive loading, or dual-tasking (DT). High-intensity exercise has demonstrated centrally-mediated improvements of PD symptoms, with additional positive effects on overall health. The goal of this project was to identify changes in dynamic balance recovery and gait function under conditions with and without increased cognitive load after a high-intensity exercise intervention in a PD population. Participants included people with PD who completed an eight-week cycling intervention (PDE), people with Parkinson’s disease who did not complete the intervention (PDC), and healthy age-matched controls (HC), with 14 subjects per group. In Aim 1, while participants underwent a series of destabilizing balance tests, the time taken to regain balance and the center of pressure movement during balance recovery were measured. The PDE group demonstrated greater improvement in balance recovery after exercise compared with the PDC group. In Aim 2, participants completed a series of gait and cognitive tasks, both separately and concurrently. Outcome measures included spatiotemporal and kinematic gait parameters of the lower and upper extremities. The PDE group demonstrated significant improvement in gait measures and DT abilities compared to PDC, while no changes were found in cognitive function for any group. The standard clinical methods of measuring motor function can be subjective, and may not capture subtle motor characteristics. Force plate and motion-capture technologies can provide detailed, objective outcome data, therefore improving the understanding of how exercise affects motor symptoms of Parkinson’s disease. The Motek Computer Assisted Rehabilitation Environment (CAREN) system at the Cleveland Clinic was used to create the testing environment and for data collection. These results of this project suggest global changes in motor function demonstrated by changes in balance recovery and lower and upper extremity gait function. Quantitative gait analysis has shown to be an important metric in assessing effectiveness of an exercise intervention in PD

Characterization of multiphase flows integrating X-ray imaging and virtual reality

Multiphase flows are used in a wide variety of industries, from energy production to pharmaceutical manufacturing. However, because of the complexity of the flows and difficulty measuring them, it is challenging to characterize the phenomena inside a multiphase flow. To help overcome this challenge, researchers have used numerous types of noninvasive measurement techniques to record the phenomena that occur inside the flow. One technique that has shown much success is X-ray imaging. While capable of high spatial resolutions, X-ray imaging generally has poor temporal resolution. This research improves the characterization of multiphase flows in three ways. First, an X-ray image intensifier is modified to use a high-speed camera to push the temporal limits of what is possible with current tube source X-ray imaging technology. Using this system, sample flows were imaged at 1000 frames per second without a reduction in spatial resolution. Next, the sensitivity of X-ray computed tomography (CT) measurements to changes in acquisition parameters is analyzed. While in theory CT measurements should be stable over a range of acquisition parameters, previous research has indicated otherwise. The analysis of this sensitivity shows that, while raw CT values are strongly affected by changes to acquisition parameters, if proper calibration techniques are used, acquisition parameters do not significantly influence the results for multiphase flow imaging. Finally, two algorithms are analyzed for their suitability to reconstruct an approximate tomographic slice from only two X-ray projections. These algorithms increase the spatial error in the measurement, as compared to traditional CT; however, they allow for very high temporal resolutions for 3D imaging. The only limit on the speed of this measurement technique is the image intensifier-camera setup, which was shown to be capable of imaging at a rate of at least 1000 FPS. While advances in measurement techniques for multiphase flows are one part of improving multiphase flow characterization, the challenge extends beyond measurement techniques. For improved measurement techniques to be useful, the data must be accessible to scientists in a way that maximizes the comprehension of the phenomena. To this end, this work also presents a system for using the Microsoft Kinect sensor to provide natural, non-contact interaction with multiphase flow data. Furthermore, this system is constructed so that it is trivial to add natural, non-contact interaction to immersive visualization applications. Therefore, multiple visualization applications can be built that are optimized to specific types of data, but all leverage the same natural interaction. Finally, the research is concluded by proposing a system that integrates the improved X-ray measurements, with the Kinect interaction system, and a CAVE automatic virtual environment (CAVE) to present scientists with the multiphase flow measurements in an intuitive and inherently three-dimensional manner

On Computer Mouse Pointing Model Online Identification and Endpoint Prediction

International audienceThis paper proposes a new simplified pointing model as a feedback-based dynamical system, including both human and computer sides of the process. It takes into account the commutation between the correction and ballistic phases in pointing tasks. We use the mouse position increment signal from noisy experimental data to achieve our main objectives: to estimate the model parameters online and predict the task endpoint. Some estimation tools and validation results, applying linear regression techniques on the experimental data are presented. We also compare with a similar prediction algorithm to show the potential of our algorithm's implementation