VIDEO HANDOVER FOR RETRIEVAL IN A UBIQUITOUS ENVIRONMENT USING FLOOR SENSOR DATA

ABSTRACT A system for retrieving video captured in a ubiquitous environment is presented. Data from pressure-based floor sensors are obtained as a supplementary input together with video from multiple stationary cameras. Unsupervised data mining techniques are used to reduce noise present in floor sensor data. An algorithm based on agglomerative hierarchical clustering is used to segment footpaths of individual persons. Video handover is proposed and two methods are implemented to retrieve video and key frame sequences showing a person moving in the house. Users can query the system based on time and retrieve video or key frames using either of the handover techniques. We compare the results of retrieval using different techniques subjectively. We conclude with suggestions for improvements, and future directions. INTRODUCTION Video retrieval has been a fast growing research area in the recent few years. Despite the substantial amount of research, video retrieval is still a difficult task other than for highly structured video. Accessing and retrieving relevant video segments from unstructured video becomes especially important for electronic chronicles Video retrieval from ubiquitous environments poses additional challenges. Larger and more real-life environments with a large number of cameras are being built. The amount of video is large, and increasing with time. The content is much less structured compared to a single video from a specific category. Retrieval is required at multiple levels of granularity, not merely as a summary. One difficult task in video retrieval from ubiquitous environments is to retrieve video that corresponds to a particular person, or event. Switching between videos from multiple cameras to show a particular person, we call video handover, is challenging. Given the large amount of image data and the current state of the art of image processing algorithms, it is evident that video retrieval based solely on image data is a difficult task. Therefore it is desirable to make use of supplementary data from other sensors for easier retrieval. This paper presents our work on video retrieval using video and sensory data from a ubiquitous environment. Unsupervised data mining algorithms have been used to reduce noise in data and retrieve video corresponding to people in the environment. The results are used to create a video chronicle that can be queried interactively. RELATED WORK A fair number of smart and ubiquitous environments have been built during the last decade. The Ubiquitous Sensor Room Although there exists a fair amount of research on video retrieval, most of the work deals with specific content. Examples are sports video summarization ENVIRONMENT AND SENSORS The environment selected for this work is the Ubiquitous Home [8] at Keihanna Human Info-Communications Research Center, Kyoto, Japan. This is a two-bedroom house equipped with cameras and pressure-based floor sensors. We use the data from floor sensors for summarization and retrieval of video from the cameras. Ceiling-mounted stationary cameras record images at the rate of 5 frames/second. Point-based floor sensors are spaced by 180mm on a square grid. Their coordinates are specified in millimeters, starting from the bottom left corner of VIDEO HANDOVER FOR RETRIEVAL IN A UBIQUITOUS ENVIRONMENT USING FLOOR SENSOR DAT

Video Handover for Retrieval in a Ubiquitous Environment Using Floor Sensor Data

A system for retrieving video captured in a ubiquitous environment is presented. Data from pressure-based floor sensors are obtained as a supplementary input together with video from multiple stationary cameras. Unsupervised data mining techniques are used to reduce noise present in floor sensor data. An algorithm based on agglomerative hierarchical clustering is used to segment footpaths of individual persons. Video handover is proposed and two methods are implemented to retrieve video and key frame sequences showing a person moving in the house. Users can query the system based on time and retrieve video or key frames using either of the handover techniques. We compare the results of retrieval using different techniques subjectively. We conclude with suggestions for improvements, and future directions. 1