3DTouch: A wearable 3D input device with an optical sensor and a 9-DOF inertial measurement unit

We present 3DTouch, a novel 3D wearable input device worn on the fingertip for 3D manipulation tasks. 3DTouch is designed to fill the missing gap of a 3D input device that is self-contained, mobile, and universally working across various 3D platforms. This paper presents a low-cost solution to designing and implementing such a device. Our approach relies on relative positioning technique using an optical laser sensor and a 9-DOF inertial measurement unit. 3DTouch is self-contained, and designed to universally work on various 3D platforms. The device employs touch input for the benefits of passive haptic feedback, and movement stability. On the other hand, with touch interaction, 3DTouch is conceptually less fatiguing to use over many hours than 3D spatial input devices. We propose a set of 3D interaction techniques including selection, translation, and rotation using 3DTouch. An evaluation also demonstrates the device's tracking accuracy of 1.10 mm and 2.33 degrees for subtle touch interaction in 3D space. Modular solutions like 3DTouch opens up a whole new design space for interaction techniques to further develop on.Comment: 8 pages, 7 figure

Tactile feedback display with spatial and temporal resolutions.

We report the electronic recording of the touch contact and pressure using an active matrix pressure sensor array made of transparent zinc oxide thin-film transistors and tactile feedback display using an array of diaphragm actuators made of an interpenetrating polymer elastomer network. Digital replay, editing and manipulation of the recorded touch events were demonstrated with both spatial and temporal resolutions. Analog reproduction of the force is also shown possible using the polymer actuators, despite of the high driving voltage. The ability to record, store, edit, and replay touch information adds an additional dimension to digital technologies and extends the capabilities of modern information exchange with the potential to revolutionize physical learning, social networking, e-commerce, robotics, gaming, medical and military applications

Surface MIMO: Using Conductive Surfaces For MIMO Between Small Devices

As connected devices continue to decrease in size, we explore the idea of leveraging everyday surfaces such as tabletops and walls to augment the wireless capabilities of devices. Specifically, we introduce Surface MIMO, a technique that enables MIMO communication between small devices via surfaces coated with conductive paint or covered with conductive cloth. These surfaces act as an additional spatial path that enables MIMO capabilities without increasing the physical size of the devices themselves. We provide an extensive characterization of these surfaces that reveal their effect on the propagation of EM waves. Our evaluation shows that we can enable additional spatial streams using the conductive surface and achieve average throughput gains of 2.6-3x for small devices. Finally, we also leverage the wideband characteristics of these conductive surfaces to demonstrate the first Gbps surface communication system that can directly transfer bits through the surface at up to 1.3 Gbps.Comment: MobiCom '1

Deep Thermal Imaging: Proximate Material Type Recognition in the Wild through Deep Learning of Spatial Surface Temperature Patterns

We introduce Deep Thermal Imaging, a new approach for close-range automatic recognition of materials to enhance the understanding of people and ubiquitous technologies of their proximal environment. Our approach uses a low-cost mobile thermal camera integrated into a smartphone to capture thermal textures. A deep neural network classifies these textures into material types. This approach works effectively without the need for ambient light sources or direct contact with materials. Furthermore, the use of a deep learning network removes the need to handcraft the set of features for different materials. We evaluated the performance of the system by training it to recognise 32 material types in both indoor and outdoor environments. Our approach produced recognition accuracies above 98% in 14,860 images of 15 indoor materials and above 89% in 26,584 images of 17 outdoor materials. We conclude by discussing its potentials for real-time use in HCI applications and future directions.Comment: Proceedings of the 2018 CHI Conference on Human Factors in Computing System

Anticipatory Mobile Computing: A Survey of the State of the Art and Research Challenges

Today's mobile phones are far from mere communication devices they were ten years ago. Equipped with sophisticated sensors and advanced computing hardware, phones can be used to infer users' location, activity, social setting and more. As devices become increasingly intelligent, their capabilities evolve beyond inferring context to predicting it, and then reasoning and acting upon the predicted context. This article provides an overview of the current state of the art in mobile sensing and context prediction paving the way for full-fledged anticipatory mobile computing. We present a survey of phenomena that mobile phones can infer and predict, and offer a description of machine learning techniques used for such predictions. We then discuss proactive decision making and decision delivery via the user-device feedback loop. Finally, we discuss the challenges and opportunities of anticipatory mobile computing.Comment: 29 pages, 5 figure