Automating the construction of scene classifiers for content-based video retrieval

This paper introduces a real time automatic scene classifier within content-based video retrieval. In our envisioned approach end users like documentalists, not image processing experts, build classifiers interactively, by simply indicating positive examples of a scene. Classification consists of a two stage procedure. First, small image fragments called patches are classified. Second, frequency vectors of these patch classifications are fed into a second classifier for global scene classification (e.g., city, portraits, or countryside). The first stage classifiers can be seen as a set of highly specialized, learned feature detectors, as an alternative to letting an image processing expert determine features a priori. We present results for experiments on a variety of patch and image classes. The scene classifier has been used successfully within television archives and for Internet porn filtering

Real time automatic scene classification

This work has been done as part of the EU VICAR (IST) project and the EU SCOFI project (IAP). The aim of the first project was to develop a real time video indexing classification annotation and retrieval system. For our systems, we have adapted the approach of Picard and Minka [3], who categorized elements of a scene automatically with so-called ’stuff’ categories (e.g., grass, sky, sand, stone). Campbell et al. [1] use similar concepts to describe certain parts of an image, which they named “labeled image regions”. However, they did not use these elements to classify the topic of the scene. Subsequently, we developed a generic approach for the recognition of visual scenes, where an alphabet of basic visual elements (or “typed patches”) is used to classify the topic of a scene. We define a new image element: a patch, which is a group of adjacent pixels within an image, described by a specific local pixel distribution, brightness, and color. In contrast with pixels, a patch as a whole can incorporate semantics. A patch is described by a HSI color histogram with 16 bins and by three texture features (i.e., the variance and two values based on the two eigen values of the covariance matrix of the Intensity values of a mask ran over the image. For more details on the features used we refer to Israel et al. [2]. We aimed at describing each image as a vector with a fixed size and with information about the position of patches that is not strict (strict position would limit generalization). Therefore, a fixed grid is placed over the image and each grid cell is segmented into patches, which are then categorized by a patch classifier. For each grid cell a frequency vector of its classified patches is calculated. These vectors are concate- nated. The resulting vector describes the complete image. Several grids were applied and several patch sizes with the grid cells were tested. Grid size of 3x2 combined with patches of size 16x16 provided the best system performance. For the two classification phases of our system, back-propagation networks were trained: (i) classification of the patches and (ii) classification of the image vector, as a whole. The system was tested on the classification of eight categories of scenes from the Corel database: interiors, city/street, forest, agriculture/countryside, desert, sea, portrait, and crowds. Each of these categories were relevant for the VICAR project. Based upon their relevance for these eight categories of scenes, we choose nine categories for the classification of the patches: building, crowd, grass, road, sand, skin, sky, tree, and water. This approach was found to be successful (for classification of the patches 87.5% correct, and classification of the scenes 73.8% correct). An advantage of our method is its low computational complexity. Moreover, the classified patches themselves are intermediate image representations and can be used for image classification, image segmentation as well as for image matching. A disadvantage is that the patches with which the classifiers were trained had to be manually classified. To solve this drawback, we currently develop algorithms for automatic extraction of relevant patch types. Within the IST project VICAR, a video indexing system was built for the Netherlands Institute for Sound and Vision1, consisting of four independent mod- ules: car recognition, face recognition, movement recognition (of people) and scene recognition. The latter module was based upon the afore mentioned approach. Within the IAP project SCOFI, a real time Internet pornography filter was built, based upon this approach. The system is currently running on several schools in Europe. Within the SCOFI filtering system, our image classification system (with a performance of 92% correct) works together with a text classi- fication system that includes a proxy server (FilterX, developed by Demokritos, Greece) to classify web-pages. Its total performance is 0% overblocking and 1% underblocking

Multi-Level Visual Alphabets

A central debate in visual perception theory is the argument for indirect versus direct perception; i.e., the use of intermediate, abstract, and hierarchical representations versus direct semantic interpretation of images through interaction with the outside world. We present a content-based representation that combines both approaches. The previously developed Visual Alphabet method is extended with a hierarchy of representations, each level feeding into the next one, but based on features that are not abstract but directly relevant to the task at hand. Explorative benchmark experiments are carried out on face images to investigate and explain the impact of the key parameters such as pattern size, number of prototypes, and distance measures used. Results show that adding an additional middle layer improves results, by encoding the spatial co-occurrence of lower-level pattern prototypes

How to improve the process of forming biobased R&D collaborations

publishedVersionPaid Open Acces

D-branes at multicritical points

The moduli space of c=1 conformal field theories in two dimensions has a multicritical point, where a circle theory is equivalent to an orbifold theory. We analyse all the conformal branes in both descriptions of this theory, and find convincing evidence that the full brane spectrum coincides. This shows that the equivalence of the two descriptions at this multicritical point extends to the boundary sector. We also perform the analogous analysis for one of the multicritical points of the N=1 superconformal field theories at c=3/2. Again the brane spectra are identical for both descriptions, however the identification is more subtle.Comment: 32 pages, 2 figure

D-branes in a Big Bang/Big Crunch Universe: Nappi-Witten Gauged WZW Model

We study D-branes in the Nappi-Witten model, which is a gauged WZW model based on (SL(2,R) x SU(2)) / (U(1) x U(1)). The model describes a four dimensional space-time consisting of cosmological regions with big bang/big crunch singularities and static regions with closed time-like curves. The aim of this paper is to investigate by D-brane probes whether there are pathologies associated with the cosmological singularities and the closed time-like curves. We first classify D-branes in a group theoretical way, and then examine DBI actions for effective theories on the D-branes. In particular, we show that D-brane metric from the DBI action does not include singularities, and wave functions on the D-branes are well behaved even in the presence of closed time-like curves.Comment: 50 pages, 2 figures, minor change

Two Antagonistic MALT1 Auto-Cleavage Mechanisms Reveal a Role for TRAF6 to Unleash MALT1 Activation.

The paracaspase MALT1 has arginine-directed proteolytic activity triggered by engagement of immune receptors. Recruitment of MALT1 into activation complexes is required for MALT1 proteolytic function. Here, co-expression of MALT1 in HEK293 cells, either with activated CARD11 and BCL10 or with TRAF6, was used to explore the mechanism of MALT1 activation at the molecular level. This work identified a prominent self-cleavage site of MALT1 isoform A (MALT1A) at R781 (R770 in MALT1B) and revealed that TRAF6 can activate MALT1 independently of the CBM. Intramolecular cleavage at R781/R770 removes a C-terminal TRAF6-binding site in both MALT1 isoforms, leaving MALT1B devoid of the two key interaction sites with TRAF6. A previously identified auto-proteolysis site of MALT1 at R149 leads to deletion of the death-domain, thereby abolishing interaction with BCL10. By using MALT1 isoforms and cleaved fragments thereof, as well as TRAF6 WT and mutant forms, this work shows that TRAF6 induces N-terminal auto-proteolytic cleavage of MALT1 at R149 and accelerates MALT1 protein turnover. The MALT1 fragment generated by N-terminal self-cleavage at R149 was labile and displayed enhanced signaling properties that required an intact K644 residue, previously shown to be a site for mono-ubiquitination of MALT1. Conversely, C-terminal self-cleavage at R781/R770 hampered the ability for self-cleavage at R149 and stabilized MALT1 by hindering interaction with TRAF6. C-terminal self-cleavage had limited impact on MALT1A but severely reduced MALT1B proteolytic and signaling functions. It also abrogated NF-κB activation by N-terminally cleaved MALT1A. Altogether, this study provides further insights into mechanisms that regulate the scaffolding and activation cycle of MALT1. It also emphasizes the reduced functional capacity of MALT1B as compared to MALT1A

Double-Scaling Limit of Heterotic Bundles and Dynamical Deformation in CFT

We consider heterotic string theory on Eguchi-Hanson space, as a local model of a resolved A_1 singularity in a six-dimensional flux compactification, with an Abelian gauge bundle turned on and non-zero torsion. We show that in a suitable double scaling limit, that isolates the physics near the non-vanishing two-cycle, a worldsheet conformal field theory description can be found. It contains a heterotic coset whose target space is conformal to Eguchi-Hanson. Starting from the blow-down limit of the singularity, it can be viewed as a dynamical deformation of the near-horizon fivebrane background. We analyze in detail the spectrum of the theory in particular examples, as well as the important role of worldsheet non-perturbative effects.Comment: 45 pages, no figures; ver2: typos corrected, references added, an extra tadpole-free model covere

Rolling tachyon in anti-de Sitter space-time

We study the decay of the unstable D-particle in three-dimensional anti-de Sitter space-time using worldsheet boundary conformal field theory methods. We test the open string completeness conjecture in a background for which the phase space available is only field-theoretic. This could present a serious challenge to the claim. We compute the emission of closed strings in the AdS(3) x S^3 x T^4 background from the knowledge of the exact corresponding boundary state we construct. We show that the energy stored in the brane is mainly converted into very excited long strings. The energy stored in short strings and in open string pair production is much smaller and finite for any value of the string coupling. We find no "missing energy" problem. We compare our results to those obtained for a decay in flat space-time and to a background in the presence of a linear dilaton. Some remarks on holographic aspects of the problem are made.Comment: JHEP style, 45 pages, one figure; v2: typos corrected, references added, version to appear in JHE

Lattice models and Landau theory for type II incommensurate crystals

Ground state properties and phonon dispersion curves of a classical linear chain model describing a crystal with an incommensurate phase are studied. This model is the DIFFOUR (discrete frustrated phi4) model with an extra fourth-order term added to it. The incommensurability in these models may arise if there is frustration between nearest-neighbor and next-nearest-neighbor interactions. We discuss the effect of the additional term on the phonon branches and phase diagram of the DIFFOUR model. We find some features not present in the DIFFOUR model such as the renormalization of the nearest-neighbor coupling. Furthermore the ratio between the slopes of the soft phonon mode in the ferroelectric and paraelectric phase can take on values different from -2. Temperature dependences of the parameters in the model are different above and below the paraelectric transition, in contrast with the assumptions made in Landau theory. In the continuum limit this model reduces to the Landau free energy expansion for type II incommensurate crystals and it can be seen as the lowest-order generalization of the simplest Lifshitz-point model. Part of the numerical calculations have been done by an adaption of the Effective Potential Method, orginally used for models with nearest-neighbor interaction, to models with also next-nearest-neighbor interactions.Comment: 33 pages, 7 figures, RevTex, submitted to Phys. Rev.