Development of tangible acoustic interfaces for human computer interaction

Tangible interfaces, such as keyboards, mice, touch pads, and touch screens, are widely used in human computer interaction. A common disadvantage with these devices is the presence of mechanical or electronic devices at the point of interaction with the interface. The aim of this work has been to investigate and develop new tangible interfaces that can be adapted to virtually any surface, by acquiring and studying the acoustic vibrations produced by the interaction of the user's finger on the surface. Various approaches have been investigated in this work, including the popular time difference of arrival (TDOA) method, time-frequency analysis of dispersive velocities, the time reversal method, and continuous object tracking. The received signal due to a tap at a source position can be considered the impulse response function of the wave propagation between the source and the receiver. With the time reversal theory, the signals induced by impacts from one position contain the unique and consistent information that forms its signature. A pattern matching method, named Location Template Matching (LTM), has been developed to identify the signature of the received signals from different individual positions. Various experiments have been performed for different purposes, such as consistency testing, acquisition configuration, and accuracy of recognition. Eventually, this can be used to implement HCI applications on any arbitrary surfaces, including those of 3D objects and inhomogeneous materials. The resolution with the LTM method has been studied by different experiments, investigating factors such as optimal sensor configurations and the limitation of materials. On plates of the same material, the thickness is the essential determinant of resolution. With the knowledge of resolution for one material, a simple but faster search method becomes feasible to reduce the computation. Multiple simultaneous impacts are also recognisable in certain cases. The TDOA method has also been evaluated with two conventional approaches. Taking into account the dispersive properties of the vibration propagation in plates, time-frequency analysis, with continuous wavelet transformation, has been employed for the accurate localising of dispersive signals. In addition, a statistical estimation of maximum likelihood has been developed to improve the accuracy and reliability of acoustic localisation. A method to measure and verify the dispersive velocities has also been introduced. To enable the commonly required "drag & drop" function in the operation of graphical user interface (GUI) software, the tracking of a finger scratching on a surface needs to be implemented. To minimise the tracking error, a priori knowledge of previous measurements of source locations is needed to linearise the state model that enables prediction of the location of the contact point and the direction of movement. An adaptive Kalman filter has been used for this purpose.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Development of tangible acoustic interfaces for human computer interaction

Tangible interfaces, such as keyboards, mice, touch pads, and touch screens, are widely used in human computer interaction. A common disadvantage with these devices is the presence of mechanical or electronic devices at the point of interaction with the interface. The aim of this work has been to investigate and develop new tangible interfaces that can be adapted to virtually any surface, by acquiring and studying the acoustic vibrations produced by the interaction of the user's finger on the surface. Various approaches have been investigated in this work, including the popular time difference of arrival (TDOA) method, time-frequency analysis of dispersive velocities, the time reversal method, and continuous object tracking. The received signal due to a tap at a source position can be considered the impulse response function of the wave propagation between the source and the receiver. With the time reversal theory, the signals induced by impacts from one position contain the unique and consistent information that forms its signature. A pattern matching method, named Location Template Matching (LTM), has been developed to identify the signature of the received signals from different individual positions. Various experiments have been performed for different purposes, such as consistency testing, acquisition configuration, and accuracy of recognition. Eventually, this can be used to implement HCI applications on any arbitrary surfaces, including those of 3D objects and inhomogeneous materials. The resolution with the LTM method has been studied by different experiments, investigating factors such as optimal sensor configurations and the limitation of materials. On plates of the same material, the thickness is the essential determinant of resolution. With the knowledge of resolution for one material, a simple but faster search method becomes feasible to reduce the computation. Multiple simultaneous impacts are also recognisable in certain cases. The TDOA method has also been evaluated with two conventional approaches. Taking into account the dispersive properties of the vibration propagation in plates, time-frequency analysis, with continuous wavelet transformation, has been employed for the accurate localising of dispersive signals. In addition, a statistical estimation of maximum likelihood has been developed to improve the accuracy and reliability of acoustic localisation. A method to measure and verify the dispersive velocities has also been introduced. To enable the commonly required "drag & drop" function in the operation of graphical user interface (GUI) software, the tracking of a finger scratching on a surface needs to be implemented. To minimise the tracking error, a priori knowledge of previous measurements of source locations is needed to linearise the state model that enables prediction of the location of the contact point and the direction of movement. An adaptive Kalman filter has been used for this purpose

Sensor fusion for tangible acoustic interfaces for human computer intreraction

This thesis presents the development of tangible acoustic interfaces for human computer interaction. The method adopted was to position sensors on the surface of a solid object to detect acoustic waves generated during an interaction, process the sensor signals and estimate either the location of a discrete impact or the trajectory of a moving point of contact on the surface. Higher accuracy and reliability were achieved by employing sensor fusion to combine the information collected from redundant sensors electively positioned on the solid object. Two different localisation approaches are proposed in the thesis. The learning-based approach is employed to detect discrete impact positions. With this approach, a signature vector representation of time-series patterns from a single sensor is matched with database signatures for known impact locations. For improved reliability, a criterion is proposed to extract the location signature from two vectors. The other approach is based on the Time Difference of Arrival (TDOA) of a source signal captured by a spatially distributed array of sensors. Enhanced positioning algorithms that consider near-field scenario, dispersion, optimisation and filtration are proposed to tackle the problems of passive acoustic localisation in solid objects. A computationally efficient algorithm for tracking a continuously moving source is presented. Spatial filtering of the estimated trajectory has been performed using Kalman filtering with automated initialisation