Learning to detect video events from zero or very few video examples

In this work we deal with the problem of high-level event detection in video. Specifically, we study the challenging problems of i) learning to detect video events from solely a textual description of the event, without using any positive video examples, and ii) additionally exploiting very few positive training samples together with a small number of ``related'' videos. For learning only from an event's textual description, we first identify a general learning framework and then study the impact of different design choices for various stages of this framework. For additionally learning from example videos, when true positive training samples are scarce, we employ an extension of the Support Vector Machine that allows us to exploit ``related'' event videos by automatically introducing different weights for subsets of the videos in the overall training set. Experimental evaluations performed on the large-scale TRECVID MED 2014 video dataset provide insight on the effectiveness of the proposed methods.Comment: Image and Vision Computing Journal, Elsevier, 2015, accepted for publicatio

Deliverable D1.2 Visual, text and audio information analysis for hypervideo, first release

Enriching videos by offering continuative and related information via, e.g., audiostreams, web pages, as well as other videos, is typically hampered by its demand for massive editorial work. While there exist several automatic and semi-automatic methods that analyze audio/video content, one needs to decide which method offers appropriate information for our intended use-case scenarios. We review the technology options for video analysis that we have access to, and describe which training material we opted for to feed our algorithms. For all methods, we offer extensive qualitative and quantitative results, and give an outlook on the next steps within the project

Deliverable D7.5 LinkedTV Dissemination and Standardisation Report v2

This deliverable presents the LinkedTV dissemination and standardisation report for the project period of months 19 to 30 (April 2013 to March 2014)

Deliverable D1.4 Visual, text and audio information analysis for hypervideo, final release

Having extensively evaluated the performance of the technologies included in the first release of WP1 multimedia analysis tools, using content from the LinkedTV scenarios and by participating in international benchmarking activities, concrete decisions regarding the appropriateness and the importance of each individual method or combination of methods were made, which, combined with an updated list of information needs for each scenario, led to a new set of analysis requirements that had to be addressed through the release of the final set of analysis techniques of WP1. To this end, coordinated efforts on three directions, including (a) the improvement of a number of methods in terms of accuracy and time efficiency, (b) the development of new technologies and (c) the definition of synergies between methods for obtaining new types of information via multimodal processing, resulted in the final bunch of multimedia analysis methods for video hyperlinking. Moreover, the different developed analysis modules have been integrated into a web-based infrastructure, allowing the fully automatic linking of the multitude of WP1 technologies and the overall LinkedTV platform

Deliverable D9.3 Final Project Report

This document comprises the final report of LinkedTV. It includes a publishable summary, a plan for use and dissemination of foreground and a report covering the wider societal implications of the project in the form of a questionnaire

Computational methods for data discovery, harmonization and integration:Using lexical and semantic matching with an application to biobanking phenotypes

Grote gegevensverzamelingen rondom menselijke proefpersonen/patiënten, zoals biobanken en patiënten registraties, zijn onmisbaar geworden voor onderzoek naar ziekte en gezondheid, en de vertaling van dit onderzoek naar zorg en preventie. De afgelopen jaren heeft dit soort onderzoek een enorme vlucht genomen, van beperkte studies in context van specifieke ziektebeelden tot nu grootschalig bestuderen van ziekten en het complexe samenspel van genetische en omgevingsfactoren. Succesvolle uitvoering van dit soort studies vereist enorme datasets. Doordat de data in biobanken typisch is verzameld voor verschillende doelen, en daardoor dus ook qua structuur en samenstelling verschillen, is data integratie een moeizaam en tijdsintensief proces waarbij vele methodologische, technische en ethisch/juridische horden moeten worden genomen. Dit proefschrift beschrijft het onderzoek naar de uitdagingen rondom het ‘poolen’ van phenotypische gegevens over meerdere biobanken. In het bijzonder hebben we ons bezig gehouden met de vraagstukken rondom (i) het effectief in kaart brengen en vindbaar maken van relevante datasets en de bijbehorende data items, (ii) het kunnen vaststellen welke van de data items vanuit elke bron dataset potentieel gecombineerd kunnen worden als basis voor analyseen (iii) op welke wijze deze data efficiënt kunnen worden getransformeerd naar een gestandaardiseerde dataset om daadwerkelijk geïntegreerde analyse mogelijk te maken. Het resultaat is een collectie nieuwe computationele methoden, inclusief bruikbare software, waarmee (semi)automatisch en efficiënt verschillen in data verzameling en beschrijving kunnen worden overbrugd zodat onderzoekers veel sneller dan hiervoor data kunnen vinden, harmoniseren en integreren

Design of a modular digital computer system, DRL 4

The design is reported of an advanced modular computer system designated the Automatically Reconfigurable Modular Multiprocessor System, which anticipates requirements for higher computing capacity and reliability for future spaceborne computers. Subjects discussed include: an overview of the architecture, mission analysis, synchronous and nonsynchronous scheduling control, reliability, and data transmission

Dynamics of serotonergic neurons revealed by fiber photometry

This work was developed in the context of the MIT Portugal Program, area of Bioengineering Systems, in collaboration with the Champalimaud Research Programme, Champalimaud Center for the Unknown, Lisbon, Portugal. The project entitled Dynamics of serotonergic neurons revealed by fiber photometry was carried out at Instituto Gulbenkian de Ciência, Oeiras, Portugal and at the Champalimaud Research Programme, Champalimaud Center for the Unknown, Lisbon, PortugalSerotonin is an important neuromodulator implicated in the regulation of many physiological and cognitive processes. It is one of the most studied neuromodulators and one of the main targets of psychoactive drugs, since its dysregulation can contribute to altered perception and pathological conditions such as depression and obsessive-compulsive disorder. However, it is still one of the most mysterious and least understood neuromodulatory systems of the brain. In order to study the activity of serotonergic neurons in behaving mice, we used genetically encoded calcium indicators and developed a fiber photometry system to monitor neural activity from genetically defined populations of neurons. This approach was developed to study serotonin neurons but it can be used in any genetically defined neuronal population. To validate our approach, we first confirmed that increased neural activity, induced by electrical microstimulation, indeed produced increases in fluorescence detected by the system. We then used it to monitor activity in the dorsal striatum of freely behaving mice. We show that the two projection pathways of the basal ganglia are both active during spontaneous contraversive turns. Additionally, we show that this balanced activity in the two pathways is needed for such contraversive movements. Finally, we used the fiber photometry system to study the role of serotonin in learning and behavioral control and to compare it to that of dopamine, another important neuromodulator. Dopamine and serotonin are thought to act jointly to orchestrate learning and behavioral control. While dopamine is thought to invigorate behavior and drive learning by signaling reward prediction errors, i.e. better-than-expected outcomes, serotonin has been implicated in behavioral inhibition and aversive processing. More specifically, serotonin has been implicated in preventing perseverative responses in changing environments. However, whether or how serotonin neurons signal such changes is not clear. To investigate these issues, we used a reversal learning task in which mice first learned to associate different odor cues with specific outcomes and then we unexpectedly reversed these associations. We show that dorsal raphe serotonin neurons, like midbrain dopamine neurons, are specifically recruited following prediction errors that occur after reversal. Yet, unlike dopamine neurons, serotonin neurons are similarly activated by surprising events that are both better and worse than expected. Dopamine and serotonin responses both track learned cue-reward associations, but serotonin neurons are slower to adapt to the changes that occur at reversal. The different dynamics of these neurons following reversal creates an imbalance that favors dopamine activity when invigoration is needed to obtain rewards and serotonin activity when behavior should be inhibited. Our data supports a model in which serotonin acts by rapidly reporting erroneous associations, expectations or priors in order to suppress behaviors driven by such errors and enhance plasticity to facilitate error correction. Contrary to prevailing views, it supports a concept of serotonin based on primary functions in prediction, control and learning rather than affect and mood