Knowledge assisted data management and retrieval in multimedia database sistems

With the proliferation of multimedia data and ever-growing requests for multimedia applications, there is an increasing need for efficient and effective indexing, storage and retrieval of multimedia data, such as graphics, images, animation, video, audio and text. Due to the special characteristics of the multimedia data, the Multimedia Database management Systems (MMDBMSs) have emerged and attracted great research attention in recent years. Though much research effort has been devoted to this area, it is still far from maturity and there exist many open issues. In this dissertation, with the focus of addressing three of the essential challenges in developing the MMDBMS, namely, semantic gap, perception subjectivity and data organization, a systematic and integrated framework is proposed with video database and image database serving as the testbed. In particular, the framework addresses these challenges separately yet coherently from three main aspects of a MMDBMS: multimedia data representation, indexing and retrieval. In terms of multimedia data representation, the key to address the semantic gap issue is to intelligently and automatically model the mid-level representation and/or semi-semantic descriptors besides the extraction of the low-level media features. The data organization challenge is mainly addressed by the aspect of media indexing where various levels of indexing are required to support the diverse query requirements. In particular, the focus of this study is to facilitate the high-level video indexing by proposing a multimodal event mining framework associated with temporal knowledge discovery approaches. With respect to the perception subjectivity issue, advanced techniques are proposed to support users’ interaction and to effectively model users’ perception from the feedback at both the image-level and object-level

Advances in Human-Robot Interaction

Rapid advances in the field of robotics have made it possible to use robots not just in industrial automation but also in entertainment, rehabilitation, and home service. Since robots will likely affect many aspects of human existence, fundamental questions of human-robot interaction must be formulated and, if at all possible, resolved. Some of these questions are addressed in this collection of papers by leading HRI researchers

Sensor based systems for quantification of sensorimotor function and rehabilitation of the upper limb

The thesis presents targeted sensor-based devices and methods for the training and assessment of upper extremity. These systems are all passive (non-actuated) thus intrinsically safe for (semi) independent use. An isometric assessment system is first presented, which uses a handle fixed on a force/torque sensor to investigate the force signal parameters and their relation to functional disability scales. The results from multiple sclerosis and healthy populations establish relation of isometric control and strength measures, its dependence on direction and how they are related to functional scales. The dissertation then introduces the novel platform MIMATE, Multimodal Interactive Motor Assessment and Training Environment, which is a wireless embedded platform for designing systems for training and assessing sensorimotor behaviour. MIMATE’s potential for designing clinically useful neurorehabilitation systems was demonstrated in a rehabilitation technology course. Based on MIMATE, intelligent objects (IObjects) are presented, which can measure position and force during training and assessing of manipulation tasks relevant to activities of daily living. A preliminary study with an IObject exhibits potential metrics and techniques that can be used to assess motor performance during fine manipulation tasks. The IObjects are part of the SITAR system, which is a novel sensor-based platform based on a force sensitive touchscreen and IObjects. It is used for training and assessment of sensorimotor deficits by focusing on meaningful functional tasks. Pilot assessment study with SITAR indicated a significant difference in performance of stroke and healthy populations during different sensorimotor tasks. Finally the thesis presents LOBSTER, a low cost, portable, bimanual self-trainer for exercising hand opening/closing, wrist flexion/extension or pronation/supination. The major novelty of the system relies on exploiting the movement of the unaffected limb to train the affected limb, making it safe for independent use. Study with LOBSTER will determine its usability for home based use.Open Acces

The Grid Sketcher: An AutoCad-based tool for conceptual design processes

Sketching with pencil and paper is reminiscent of the varied, rich, and loosely defined formal processes associated with conceptual design. Architects actively engage such creative paradigms in their exploration and development of conceptual design solutions. The Grid Sketcher, as a conceptual sketching tool, presents one possible computer implementation for enhancing and supporting these processes. It effectively demonstrates the facility with which current technology and the computing environment can enhance and simulate sketching intents and expectations; Typically with respect to design, the position taken is that the two are virtually void of any fundamental commonality. A designer\u27s thoughts are intuitive, at times irrational, and rarely follow consistently identifiable patterns. Conversely, computing requires predictability in just these endeavors. The computing environment, as commonly defined, can not reasonably expect to mimic the typically human domain of creative design. In this context, this thesis accentuates the computer\u27s role as a form generator as opposed to a form evaluator. The computer, under the influence of certain contextual parameters can, however, provide the designer with a rich and elegant set of forms that respond through algorithmics to the designer\u27s creative intents. (Abstract shortened by UMI.)

Sound Object Recognition

Humans are constantly exposed to a variety of acoustic stimuli ranging from music and speech to more complex acoustic scenes like a noisy marketplace. The human auditory perception mechanism is able to analyze these different kinds of sounds and extract meaningful information suggesting that the same processing mechanism is capable of representing different sound classes. In this thesis, we test this hypothesis by proposing a high dimensional sound object representation framework, that captures the various modulations of sound by performing a multi-resolution mapping. We then show that this model is able to capture a wide variety of sound classes (speech, music, soundscapes) by applying it to the tasks of speech recognition, speaker verification, musical instrument recognition and acoustic soundscape recognition. We propose a multi-resolution analysis approach that captures the detailed variations in the spectral characterists as a basis for recognizing sound objects. We then show how such a system can be fine tuned to capture both the message information (speech content) and the messenger information (speaker identity). This system is shown to outperform state-of-art system for noise robustness at both automatic speech recognition and speaker verification tasks. The proposed analysis scheme with the included ability to analyze temporal modulations was used to capture musical sound objects. We showed that using a model of cortical processing, we were able to accurately replicate the human perceptual similarity judgments and also were able to get a good classification performance on a large set of musical instruments. We also show that neither just the spectral feature or the marginals of the proposed model are sufficient to capture human perception. Moreover, we were able to extend this model to continuous musical recordings by proposing a new method to extract notes from the recordings. Complex acoustic scenes like a sports stadium have multiple sources producing sounds at the same time. We show that the proposed representation scheme can not only capture these complex acoustic scenes, but provides a flexible mechanism to adapt to target sources of interest. The human auditory perception system is known to be a complex system where there are both bottom-up analysis pathways and top-down feedback mechanisms. The top-down feedback enhances the output of the bottom-up system to better realize the target sounds. In this thesis we propose an implementation of top-down attention module which is complimentary to the high dimensional acoustic feature extraction mechanism. This attention module is a distributed system operating at multiple stages of representation, effectively acting as a retuning mechanism, that adapts the same system to different tasks. We showed that such an adaptation mechanism is able to tremendously improve the performance of the system at detecting the target source in the presence of various distracting background sources