Investigating human audio-visual object perception with a combination of hypothesis-generating and hypothesis-testing fMRI analysis tools

Primate multisensory object perception involves distributed brain regions. To investigate the network character of these regions of the human brain, we applied data-driven group spatial independent component analysis (ICA) to a functional magnetic resonance imaging (fMRI) data set acquired during a passive audio-visual (AV) experiment with common object stimuli. We labeled three group-level independent component (IC) maps as auditory (A), visual (V), and AV, based on their spatial layouts and activation time courses. The overlap between these IC maps served as definition of a distributed network of multisensory candidate regions including superior temporal, ventral occipito-temporal, posterior parietal and prefrontal regions. During an independent second fMRI experiment, we explicitly tested their involvement in AV integration. Activations in nine out of these twelve regions met the max-criterion (A < AV > V) for multisensory integration. Comparison of this approach with a general linear model-based region-of-interest definition revealed its complementary value for multisensory neuroimaging. In conclusion, we estimated functional networks of uni- and multisensory functional connectivity from one dataset and validated their functional roles in an independent dataset. These findings demonstrate the particular value of ICA for multisensory neuroimaging research and using independent datasets to test hypotheses generated from a data-driven analysis

An audio-visual system for object-based audio : from recording to listening

Object-based audio is an emerging representation for audio content, where content is represented in a reproduction format-agnostic way and, thus, produced once for consumption on many different kinds of devices. This affords new opportunities for immersive, personalized, and interactive listening experiences. This paper introduces an end-to-end object-based spatial audio pipeline, from sound recording to listening. A high-level system architecture is proposed, which includes novel audiovisual interfaces to support object-based capture and listenertracked rendering, and incorporates a proposed component for objectification, that is, recording content directly into an object-based form. Text-based and extensible metadata enable communication between the system components. An open architecture for object rendering is also proposed. The system’s capabilities are evaluated in two parts. First, listener-tracked reproduction of metadata automatically estimated from two moving talkers is evaluated using an objective binaural localization model. Second, object-based scene capture with audio extracted using blind source separation (to remix between two talkers) and beamforming (to remix a recording of a jazz group) is evaluate

Frameworks for Component-based Simulation

AbstractThe need to reduce development costs of simulation models has led to recent efforts for setting simulation standards that foster model reuse and interoperability. Specifically, the High Level Architecture (HLA) is a new simulation standard supported by the US Defense Modeling and Simulation Office (DMSO). It has been adopted as the standard technical architecture for all US Department of Defense simulations. In the meantime, the commercial sector has had successful efforts in developing enabling technologies for distributed computing; namely, the Common Object Request Broker Architecture (CORBA) by the Object Management Group (OMG). CORBA is a large and complex set of specifications and protocols that utilizes the objectoriented paradigm to achieve distributed object-oriented computing environments that allow object interoperability and reuse. When used as an infrastructure for simulation model reuse and interoperability, both HLA and CORBA exhibit merits and limitations. Since HLA and CORBA were developed independently, need exists for a comparative evaluation of the two architectures as a basis for component-based simulation. In this paper, both HLA and CORBA are presented in the context of component-based simulation model development and interoperability. The two architectures are compared against four comparison criteria that are related to their conceptual foundation and design

Volumetric Procedural Models for Shape Representation

This article describes a volumetric approach for procedural shape modeling and a new Procedural Shape Modeling Language (PSML) that facilitates the specification of these models. PSML provides programmers the ability to describe shapes in terms of their 3D elements where each element may be a semantic group of 3D objects, e.g., a brick wall, or an indivisible object, e.g., an individual brick. Modeling shapes in this manner facilitates the creation of models that more closely approximate the organization and structure of their real-world counterparts. As such, users may query these models for volumetric information such as the number, position, orientation and volume of 3D elements which cannot be provided using surface based model-building techniques. PSML also provides a number of new language-specific capabilities that allow for a rich variety of context-sensitive behaviors and post-processing functions. These capabilities include an object-oriented approach for model design, methods for querying the model for component-based information and the ability to access model elements and components to perform Boolean operations on the model parts. PSML is open-source and includes freely available tutorial videos, demonstration code and an integrated development environment to support writing PSML programs

Using VCR to Support Different Styles and Types of Group Collaborations in Virtual Universities

[[abstract]]To support many different group activities such as computer and network based meeting, seminar, conference, tele-lecturing, group project, team work, student instruction and club in a virtual university, a general framework based on a room metaphor or model is used to develop a desktop groupware system, called virtual collaboration room (VCR) that enables teachers and students to flexibly and naturally conduct their collaborative teaching/learning/working over the Internet without constraints on collaboration type, working style, group scale and system platform. As important component in our integrated educational system, University21, the VCR is discussed how to be employed and managed in a virtual university. With the use of Java applets for system implementations, users can enter the VCR from any standard Java enabled Web browser. VCR consists of user panel, object cabinet, object panel, a variety of objects in workspace, workspace panel. An object in VCR has a life cycle from birth, alive to death, and attributes such as owner, state, access mode, handler, and persistence. To further improve the quality of collaborations, natural object control and rich awareness function are fully supported.[[notice]]補正完

Object detection and activity recognition in digital image and video libraries

This thesis is a comprehensive study of object-based image and video retrieval, specifically for car and human detection and activity recognition purposes. The thesis focuses on the problem of connecting low level features to high level semantics by developing relational object and activity presentations. With the rapid growth of multimedia information in forms of digital image and video libraries, there is an increasing need for intelligent database management tools. The traditional text based query systems based on manual annotation process are impractical for today\u27s large libraries requiring an efficient information retrieval system. For this purpose, a hierarchical information retrieval system is proposed where shape, color and motion characteristics of objects of interest are captured in compressed and uncompressed domains. The proposed retrieval method provides object detection and activity recognition at different resolution levels from low complexity to low false rates. The thesis first examines extraction of low level features from images and videos using intensity, color and motion of pixels and blocks. Local consistency based on these features and geometrical characteristics of the regions is used to group object parts. The problem of managing the segmentation process is solved by a new approach that uses object based knowledge in order to group the regions according to a global consistency. A new model-based segmentation algorithm is introduced that uses a feedback from relational representation of the object. The selected unary and binary attributes are further extended for application specific algorithms. Object detection is achieved by matching the relational graphs of objects with the reference model. The major advantages of the algorithm can be summarized as improving the object extraction by reducing the dependence on the low level segmentation process and combining the boundary and region properties. The thesis then addresses the problem of object detection and activity recognition in compressed domain in order to reduce computational complexity. New algorithms for object detection and activity recognition in JPEG images and MPEG videos are developed. It is shown that significant information can be obtained from the compressed domain in order to connect to high level semantics. Since our aim is to retrieve information from images and videos compressed using standard algorithms such as JPEG and MPEG, our approach differentiates from previous compressed domain object detection techniques where the compression algorithms are governed by characteristics of object of interest to be retrieved. An algorithm is developed using the principal component analysis of MPEG motion vectors to detect the human activities; namely, walking, running, and kicking. Object detection in JPEG compressed still images and MPEG I frames is achieved by using DC-DCT coefficients of the luminance and chrominance values in the graph based object detection algorithm. The thesis finally addresses the problem of object detection in lower resolution and monochrome images. Specifically, it is demonstrated that the structural information of human silhouettes can be captured from AC-DCT coefficients

A Study on Variational Component Splitting approach for Mixture Models

Increase in use of mobile devices and the introduction of cloud-based services have resulted in the generation of enormous amount of data every day. This calls for the need to group these data appropriately into proper categories. Various clustering techniques have been introduced over the years to learn the patterns in data that might better facilitate the classification process. Finite mixture model is one of the crucial methods used for this task. The basic idea of mixture models is to fit the data at hand to an appropriate distribution. The design of mixture models hence involves finding the appropriate parameters of the distribution and estimating the number of clusters in the data. We use a variational component splitting framework to do this which could simultaneously learn the parameters of the model and estimate the number of components in the model. The variational algorithm helps to overcome the computational complexity of purely Bayesian approaches and the over fitting problems experienced with Maximum Likelihood approaches guaranteeing convergence. The choice of distribution remains the core concern of mixture models in recent research. The efficiency of Dirichlet family of distributions for this purpose has been proved in latest studies especially for non-Gaussian data. This led us to study the impact of variational component splitting approach on mixture models based on several distributions. Hence, our contribution is the application of variational component splitting approach to design finite mixture models based on inverted Dirichlet, generalized inverted Dirichlet and inverted Beta-Liouville distributions. In addition, we also incorporate a simultaneous feature selection approach for generalized inverted Dirichlet mixture model along with component splitting as another experimental contribution. We evaluate the performance of our models with various real-life applications such as object, scene, texture, speech and video categorization

Decentralised Control Flow: A Computational Model for Distributed Systems

PhD ThesisThis thesis presents two sets of principles for the organisation of distributed computing systems. Details of models of computation based on these principles are together given, with proposals for programming languages based on each model of computation. The recursive control flow principles are based on the concept of recursive control flow computing system structuring. A recursive comprises a group of subordinate computing systems connected together by Each subordinate computing system may either be a communications medium. which a a computing system consists of a processing unit, memory some is itself a recursive component, and input/output devices, or computing components control flow system. The memory of all the computing systems within a recursive control flow computing subordinate system are arranged in a hierarchy. Using suitable addresses, any part of the hierarchy is accessible to any sequence of instructions which may be executed by the processing unit of a subordinate computing system. This rise to serious difficulties in the global accessibility gives understanding of programs written the meaning of in a programming language recursive control flow on the model of computation. based Reasoning about a particular program in isolation is difficult because of the potential interference between the execution different programs cannot be ignored . alternative principles, decentralised control flow, restrict the The accessibility of subordinate global the memory components of the computing The basis of the concept of objects forms the systems. principles. Information channels may flow along unnamed between instances of these objects, this being the only way in which one instance of an object may communicate with some other instance of an object. Reasoning particular program written in a programming language about a based on the decentralised control flow model of computation is easier since it is that there will be no interference between the guaranteed execution of different programs.Science and Engineering Research Council of Great Britain, International Computers Limite

Combining mathematical programming and SysML for component sizing as applied to hydraulic systems

In this research, the focus is on improving a designer's capability to determine near-optimal sizes of components for a given system architecture. Component sizing is a hard problem to solve because of the presence of competing objectives, requirements from multiple disciplines, and the need for finding a solution quickly for the architecture being considered. In current approaches, designers rely on heuristics and iterate over the multiple objectives and requirements until a satisfactory solution is found. To improve on this state of practice, this research introduces advances in the following two areas: a.) Formulating a component sizing problem in a manner that is convenient to designers and b.) Solving the component sizing problem in an efficient manner so that all of the imposed requirements are satisfied simultaneously and the solution obtained is mathematically optimal. In particular, an acausal, algebraic, equation-based, declarative modeling approach is taken to solve component sizing problems efficiently. This is because global optimization algorithms exist for algebraic models and the computation time is considerably less as compared to the optimization of dynamic simulations. In this thesis, the mathematical programming language known as GAMS (General Algebraic Modeling System) and its associated global optimization solvers are used to solve component sizing problems efficiently. Mathematical programming languages such as GAMS are not convenient for formulating component sizing problems and therefore the Systems Modeling Language developed by the Object Management Group (OMG SysML ) is used to formally capture and organize models related to component sizing into libraries that can be reused to compose new models quickly by connecting them together. Model-transformations are then used to generate low-level mathematical programming models in GAMS that can be solved using commercial off-the-shelf solvers such as BARON (Branch and Reduce Optimization Navigator) to determine the component sizes that satisfy the requirements and objectives imposed on the system. This framework is illustrated by applying it to an example application for sizing a hydraulic log splitter.M.S.Committee Co-Chair: Paredis, Chris ; Committee Co-Chair: Schaefer, Dirk; Committee Member: Goel, Asho