Binding and unbinding the auditory and visual streams in the McGurk effect

International audienceSubjects presented with coherent auditory and visual streams generally fuse them into a single per- cept. This results in enhanced intelligibility in noise, or in visual modification of the auditory per- cept in the McGurk effect. It is classically considered that processing is done independently in the auditory and visual systems before interaction occurs at a certain representational stage, resulting in an integrated percept. However, some behavioral and neurophysiological data suggest the existence of a two-stage process. A first stage would involve binding together the appropriate pieces of audio and video information before fusion per se in a second stage. Then it should be possible to design experiments leading to unbinding . It is shown here that if a given McGurk stimulus is preceded by an incoherent audiovisual context, the amount of McGurk effect is largely reduced. Various kinds of incoherent contexts (acoustic syllables dubbed on video sentences or phonetic or temporal modi- fications of the acoustic content of a regular sequence of audiovisual syllables) can significantly reduce the McGurk effect even when they are short (less than 4s). The data are interpreted in the framework of a two-stage "binding and fusion" model for audiovisual speech perception

NASA Thesaurus supplement: A four part cumulative supplement to the 1988 edition of the NASA Thesaurus (supplement 3)

The four-part cumulative supplement to the 1988 edition of the NASA Thesaurus includes the Hierarchical Listing (Part 1), Access Vocabulary (Part 2), Definitions (Part 3), and Changes (Part 4). The semiannual supplement gives complete hierarchies and accepted upper/lowercase forms for new terms

Egocentric Video Summarization of Cultural Tour based on User Preferences

In this paper, we propose a new method to obtain customized video summarization according to specific user preferences. Our approach is tailored on Cultural Heritage scenario and is designed on identifying candidate shots, selecting from the original streams only the scenes with behavior patterns related to the presence of relevant experiences, and further filtering them in order to obtain a summary matching the requested user's preferences. Our preliminary results show that the proposed approach is able to leverage user's preferences in order to obtain a customized summary, so that different users may extract from the same stream different summaries

Data compression techniques applied to high resolution high frame rate video technology

An investigation is presented of video data compression applied to microgravity space experiments using High Resolution High Frame Rate Video Technology (HHVT). An extensive survey of methods of video data compression, described in the open literature, was conducted. The survey examines compression methods employing digital computing. The results of the survey are presented. They include a description of each method and assessment of image degradation and video data parameters. An assessment is made of present and near term future technology for implementation of video data compression in high speed imaging system. Results of the assessment are discussed and summarized. The results of a study of a baseline HHVT video system, and approaches for implementation of video data compression, are presented. Case studies of three microgravity experiments are presented and specific compression techniques and implementations are recommended

Learning to Extract Motion from Videos in Convolutional Neural Networks

This paper shows how to extract dense optical flow from videos with a convolutional neural network (CNN). The proposed model constitutes a potential building block for deeper architectures to allow using motion without resorting to an external algorithm, \eg for recognition in videos. We derive our network architecture from signal processing principles to provide desired invariances to image contrast, phase and texture. We constrain weights within the network to enforce strict rotation invariance and substantially reduce the number of parameters to learn. We demonstrate end-to-end training on only 8 sequences of the Middlebury dataset, orders of magnitude less than competing CNN-based motion estimation methods, and obtain comparable performance to classical methods on the Middlebury benchmark. Importantly, our method outputs a distributed representation of motion that allows representing multiple, transparent motions, and dynamic textures. Our contributions on network design and rotation invariance offer insights nonspecific to motion estimation

The Development of Visual Motion Processing in Human Infants

The experiments of this thesis have used apparent motion in random-dot patterns to explore the development of motion processing in infants. Most of the experiments involved discrimination of a segregated pattern, in which different regions moved in different ways (eg opposite directions), from a uniform pattern containing just one kind of motion. Maximum displacement limits (dmax) for discrimination of coherent from incoherent motion, and for discrimination of opposite directions of coherent motion, increased between 8 and 15 weeks. The higher threshold of adults indicated that this increase continues beyond 15 weeks. The effect of changing the interval between displacements indicated two processes underlying the increase in direction discrimination dmax: a maturation of the temporal properties of motion detectors (eg improving sensitivity to high temporal frequencies), which is largely complete by about 12 weeks; and a more prolonged development of their spatial properties which dominates the change in dmax after 12 weeks, and may also be involved before this. Measurements of coherence thresholds for direction discrimination showed that, in addition to the rise in dmax with age, there is a substantial improvement in motion sensitivity at displacements below dmax. Hence a uniform increase in sensitivity across all displacements is likely to be an important factor behind the development of dmax. However there may be additional specific improvements in sensitivity to large displacements, perhaps reflecting the emergence of low spatial frequency channels. A series of habituation and preferential looking experiments failed to find evidence for direction discrimination before 6 weeks, though positive evidence was obtained at 6-8 weeks. The results suggest that directionality emerges at about 7 weeks of age. Interestingly, despite their success at discriminating direction in a segregated stimulus, 6-8-week-olds were insensitive to the absolute direction of uniform motion. This suggests that they have not yet learnt to combine measurements of retinal image motion with information about eye movements

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions