The MoCA Workbench: Support for Creativity in Movie Content Analysis

Semantic access to the content of a video is highly desirable for multimedia content retrieval. Automatic extraction of semantics requires content analysis algorithms. Our MoCA (Movie Content Analysis) project provides an interactive workbench supporting the researcher in the development of new movie content analysis algorithms. The workbench offers data management facilities for large amounts of video/audio data and derived parameters. It also provides an easy-to-use interface for a free combination of basic operators into more sophisticated operators. We can combine results from video track and audio track analysis. The MoCA Workbench shields the researcher from technical details and provides advanced visualization capabilities, allowing attention to focus on the development of new algorithms. The paper presents the design and implementation of the MoCA Workbench and reports practical experience

Halide: a language and compiler for optimizing parallelism, locality, and recomputation in image processing pipelines

Image processing pipelines combine the challenges of stencil computations and stream programs. They are composed of large graphs of different stencil stages, as well as complex reductions, and stages with global or data-dependent access patterns. Because of their complex structure, the performance difference between a naive implementation of a pipeline and an optimized one is often an order of magnitude. Efficient implementations require optimization of both parallelism and locality, but due to the nature of stencils, there is a fundamental tension between parallelism, locality, and introducing redundant recomputation of shared values. We present a systematic model of the tradeoff space fundamental to stencil pipelines, a schedule representation which describes concrete points in this space for each stage in an image processing pipeline, and an optimizing compiler for the Halide image processing language that synthesizes high performance implementations from a Halide algorithm and a schedule. Combining this compiler with stochastic search over the space of schedules enables terse, composable programs to achieve state-of-the-art performance on a wide range of real image processing pipelines, and across different hardware architectures, including multicores with SIMD, and heterogeneous CPU+GPU execution. From simple Halide programs written in a few hours, we demonstrate performance up to 5x faster than hand-tuned C, intrinsics, and CUDA implementations optimized by experts over weeks or months, for image processing applications beyond the reach of past automatic compilers.United States. Dept. of Energy (Award DE-SC0005288)National Science Foundation (U.S.) (Grant 0964004)Intel CorporationCognex CorporationAdobe System

Halide: a language and compiler for optimizing parallelism, locality, and recomputation in image processing pipelines

Image processing pipelines combine the challenges of stencil computations and stream programs. They are composed of large graphs of different stencil stages, as well as complex reductions, and stages with global or data-dependent access patterns. Because of their complex structure, the performance difference between a naive implementation of a pipeline and an optimized one is often an order of magnitude. Efficient implementations require optimization of both parallelism and locality, but due to the nature of stencils, there is a fundamental tension between parallelism, locality, and introducing redundant recomputation of shared values. We present a systematic model of the tradeoff space fundamental to stencil pipelines, a schedule representation which describes concrete points in this space for each stage in an image processing pipeline, and an optimizing compiler for the Halide image processing language that synthesizes high performance implementations from a Halide algorithm and a schedule. Combining this compiler with stochastic search over the space of schedules enables terse, composable programs to achieve state-of-the-art performance on a wide range of real image processing pipelines, and across different hardware architectures, including multicores with SIMD, and heterogeneous CPU+GPU execution. From simple Halide programs written in a few hours, we demonstrate performance up to 5x faster than hand-tuned C, intrinsics, and CUDA implementations optimized by experts over weeks or months, for image processing applications beyond the reach of past automatic compilers.United States. Dept. of Energy (Award DE-SC0005288)National Science Foundation (U.S.) (Grant 0964004)Intel CorporationCognex CorporationAdobe System

Scale-invariant image editing

We introduce a system architecture for image editing which decouples image filtering from the image size, resulting in a system which allows interactive editing with constant response times, independent of image sizes. Scale invariance means filters are designed to allow scaled rendering from a pre-computed image pyramid, approximating the result of the filter when rendered at full resolution and scaled afterwards. Our implementation of the proposed architecture allows interactive editing on all image sizes with minimal hardware requirements, the least powerful device tested with a 300 megapixel image was based on a dual core ARM Cortex A9 clocked at 1Ghz. The architecture is based on a graph based image editing approach, extended by scaled rendering for all filters. The filter graph is exploited to allow automatic configuration of filter properties and conversion between color spaces, which simplifies filter implementation and increases performance. The handling of image data is based on tiles and a tile cache allows to manage memory requirements and increase interactive performance. The implementation is provided as a portable library written in c and can provides interactive editing on device as slow as last generation smartphones, while at the same time exploiting the performance available to current multi core processors, using effective multithreading. In this work we explore both the architectural details that make this possible as well as the properties of common image editing filters, regarding the required scale invariance. We also examine possible approaches that can be followed to implement practical filters for such as system. Finally, the implemented architecture and filters are extensively tested for performance and accuracy and the results are examined

Coordination implicite d'interactions sensorimotrices comme fondement de la cognition

This thesis promotes a cognitive infrastructure able to model sensorimotor behaviors in animals and humans. The theoretical developments upon which this infrastructure is drawn is inspired by the philosophical interactivist framework and the enaction paradigm: any system is represented by a set of active processes, in constant interaction with their subjective environment, which includes influences between these processes. Any living organism or cognitive system is therefore fractal, decomposed in different levels of emergence based on the same principles. These principles are now widely spread but appeared progressively during species evolution. Assimilation, regulation, anticipation or coordination made it possible for concurrent processes fighting for limited resources to cooperate, develop and maintain through ages. This joint evolution of environmental conditions and internal structures led to nowadays organisms, able to adapt to a genetically unpredictable environment of growing complexity.A mathematical model using the formalism of complex systems is detailed, as well as its computer implementation. The agent's dynamics is modeled by an activity field under the continuous influence of internal anticipations and external sensations. The global behavior of the agent then results from the implicit and stable coordination of localized interactive processes. On this aspect, the model extends artificial neural networks and classical probabilistic models. This essential characteristic enables applications in various domains and a unification of all levels of cognition. A set of applications validates the model, extending from physiological need satisfaction to mechanical systems handling, through auditory and visual perception. Finally and in order to extend the model to more complex behaviors in the future, technical contributions on algorithm optimization and parallel implementations are developed.Cette thèse propose une infrastructure cognitive permettant de modéliser les comportements sensori-moteurs animaux et humains. La réflexion théorique ayant conduit à cette infrastructure s'inspire du cadre philosophique interactiviste et du paradigme de l'énaction : tout système est représenté par un ensemble de processus actifs, en interaction permanente avec leur environnement propre, ce qui inclut leur influence mutuelle. Tout organisme vivant ou système cognitif peut ainsi être décomposé de manière fractale, chaque niveau d'émergence reposant sur les mêmes principes. Ces principes aujourd'hui largement répandus sont apparus durant l'évolution des espèces vivantes. L'assimilation, la régulation, l'anticipation ou encore la coordination ont ainsi permis à des processus en concurrence pour des ressources limitées de coopérer, se développer et se maintenir à travers les âges. Cette évolution conjointe des conditions environnementales et des structures internes a conduit aux organismes modernes, capables de s'adapter à un environnement génétiquement imprévisible et d'une complexité croissante.Un modèle mathématique utilisant le formalisme des systèmes complexes est détaillé, ainsi que son implémentation informatique. La dynamique de l'agent y est modélisée par un champ d'activité sous l'influence permanente d'anticipations internes et de sensations externes. Le comportement global de l'agent résulte alors de la coordination implicite et stable de processus interactifs localisés. A ce niveau, le modèle étend et complète les réseaux de neurones artificiels et les modèles probabilistes classiques. Cette caractéristique essentielle permet d'appliquer le modèle à des domaines variés et d'unifier tous les niveaux de la cognition. Le modèle est validé par un ensemble d'applications s'étendant de la satisfaction de besoins physiologiques à la manipulation de systèmes mécaniques, en passant par la perception auditive et visuelle. Enfin, et afin de pouvoir étendre ce type de modèles à des problèmes plus complexes dans le futur, des contributions techniques touchant à l'optimisation et à la parallélisation des algorithmes sont développées