Browse-to-search

This demonstration presents a novel interactive online shopping application based on visual search technologies. When users want to buy something on a shopping site, they usually have the requirement of looking for related information from other web sites. Therefore users need to switch between the web page being browsed and other websites that provide search results. The proposed application enables users to naturally search products of interest when they browse a web page, and make their even causal purchase intent easily satisfied. The interactive shopping experience is characterized by: 1) in session - it allows users to specify the purchase intent in the browsing session, instead of leaving the current page and navigating to other websites; 2) in context - -the browsed web page provides implicit context information which helps infer user purchase preferences; 3) in focus - users easily specify their search interest using gesture on touch devices and do not need to formulate queries in search box; 4) natural-gesture inputs and visual-based search provides users a natural shopping experience. The system is evaluated against a data set consisting of several millions commercial product images. © 2012 Authors

Image-set, Temporal and Spatiotemporal Representations of Videos for Recognizing, Localizing and Quantifying Actions

This dissertation addresses the problem of learning video representations, which is defined here as transforming the video so that its essential structure is made more visible or accessible for action recognition and quantification. In the literature, a video can be represented by a set of images, by modeling motion or temporal dynamics, and by a 3D graph with pixels as nodes. This dissertation contributes in proposing a set of models to localize, track, segment, recognize and assess actions such as (1) image-set models via aggregating subset features given by regularizing normalized CNNs, (2) image-set models via inter-frame principal recovery and sparsely coding residual actions, (3) temporally local models with spatially global motion estimated by robust feature matching and local motion estimated by action detection with motion model added, (4) spatiotemporal models 3D graph and 3D CNN to model time as a space dimension, (5) supervised hashing by jointly learning embedding and quantization, respectively. State-of-the-art performances are achieved for tasks such as quantifying facial pain and human diving. Primary conclusions of this dissertation are categorized as follows: (i) Image set can capture facial actions that are about collective representation; (ii) Sparse and low-rank representations can have the expression, identity and pose cues untangled and can be learned via an image-set model and also a linear model; (iii) Norm is related with recognizability; similarity metrics and loss functions matter; (v) Combining the MIL based boosting tracker with the Particle Filter motion model induces a good trade-off between the appearance similarity and motion consistence; (iv) Segmenting object locally makes it amenable to assign shape priors; it is feasible to learn knowledge such as shape priors online from Web data with weak supervision; (v) It works locally in both space and time to represent videos as 3D graphs; 3D CNNs work effectively when inputted with temporally meaningful clips; (vi) the rich labeled images or videos help to learn better hash functions after learning binary embedded codes than the random projections. In addition, models proposed for videos can be adapted to other sequential images such as volumetric medical images which are not included in this dissertation

Controlling and reshaping biological reaction-diffusion

Pattern formation by reaction-diffusion mechanisms is of crucial importance for the development and sustenance of all living beings. However, biological model systems so far lack the tools and versatility of the established chemical models. In this thesis, we set out to develop and expand the Min system of Escherichia coli towards becoming a universal model for biological reaction-diffusion in an in vitro setting. To this end, we firstly developed a strategy to control the Min reaction in situ. This was facilitated by incorporating a chemically synthesized azobenzene-moiety into a peptide derived from MinE. This MinE-peptide is capable of stimulating hydrolysis of ATP by MinD. Photoswitching the azobenzene crosslinker allows to also switch alpha-helicity of the peptide and therefore its activity. By periodically activating this peptide photoswitch we found resonance phenomena in the Min reaction. The photoswitch described here could thus be used in many synthetic biology scenarios, but also to learn about Min and biological reaction-diffusion systems. Secondly, we discovered that the Min system can form stationary patterns, which greatly expands the pattern diversity and therefore the phenomena which the Min model can help us understand. Especially when it comes to important decisions in development, such as cell fate or macroscopically visible effects such as fur patterns, stationary patterns are much more prominent than oscillations and waves. The discovery of these patterns also creates many opportunities for applications, especially when combined with the newly found ability of Min proteins to position arbitrary membrane-bound factors. Thirdly, this thesis shows that the Min system's complexity can be reduced even more by substituting MinE with small peptides. A combined theory-experiment approach outlines how pattern forming capabilities are restored in a small MinE-derived peptide either by adding membrane binding or by dimerizing it. This study further highlights how peptides and proteins excel as model morphogens due to their modularity and mutability. Lastly, protocols and resources are more easily available due to a combined method-paper and video that was published in open access. In conclusion, by adding tools and versatility, this thesis introduces great progress towards establishing the in vitro Min system as the ideal model for biological reaction-diffusion

Controlling and reshaping biological reaction-diffusion

Pattern formation by reaction-diffusion mechanisms is of crucial importance for the development and sustenance of all living beings. However, biological model systems so far lack the tools and versatility of the established chemical models. In this thesis, we set out to develop and expand the Min system of Escherichia coli towards becoming a universal model for biological reaction-diffusion in an in vitro setting. To this end, we firstly developed a strategy to control the Min reaction in situ. This was facilitated by incorporating a chemically synthesized azobenzene-moiety into a peptide derived from MinE. This MinE-peptide is capable of stimulating hydrolysis of ATP by MinD. Photoswitching the azobenzene crosslinker allows to also switch alpha-helicity of the peptide and therefore its activity. By periodically activating this peptide photoswitch we found resonance phenomena in the Min reaction. The photoswitch described here could thus be used in many synthetic biology scenarios, but also to learn about Min and biological reaction-diffusion systems. Secondly, we discovered that the Min system can form stationary patterns, which greatly expands the pattern diversity and therefore the phenomena which the Min model can help us understand. Especially when it comes to important decisions in development, such as cell fate or macroscopically visible effects such as fur patterns, stationary patterns are much more prominent than oscillations and waves. The discovery of these patterns also creates many opportunities for applications, especially when combined with the newly found ability of Min proteins to position arbitrary membrane-bound factors. Thirdly, this thesis shows that the Min system's complexity can be reduced even more by substituting MinE with small peptides. A combined theory-experiment approach outlines how pattern forming capabilities are restored in a small MinE-derived peptide either by adding membrane binding or by dimerizing it. This study further highlights how peptides and proteins excel as model morphogens due to their modularity and mutability. Lastly, protocols and resources are more easily available due to a combined method-paper and video that was published in open access. In conclusion, by adding tools and versatility, this thesis introduces great progress towards establishing the in vitro Min system as the ideal model for biological reaction-diffusion

Structural studies of eukaryotic ribosome biogenesis and the sec and Bcs1 protein translocation systems

Three publications of this cumulative dissertation use cryo-electron microscopy (cryo-EM) to dissect the assembly pathway of the eukaryotic large ribosomal subunit (LSU). This pathway commences with freshly transcribed and initially unfolded rRNA in the nucleolus, which folds and incorporates ribosomal proteins while traveling to the cytoplasm, ultimately culminating in the mature LSU. During this highly complex pathway, the yeast cell must assemble four rRNAs and 79 ribosomal proteins with the help of over 200 assembly factors (AFs). Using cryo-EM, structures of nucleo\-plasmic and cytoplasmic assembly intermediates of the LSU could be solved in recent years, thus shedding light on the later stages of LSU formation. Early assembly steps remain enigmatic, as nucleolar LSU assembly intermediates have been biochemically but not structurally characterized. Taken together, we solved the structure of seven nucleolar or early nucleoplasmic intermediates at resolutions ranging from 3.3 to 4.5 Å, showing a linear assembly sequence. The first five structures show how the rRNA of the LSU is incorporated stepwise, in a non-transcriptional sequence, first forming the solvent exposed back side, and later the peptide exit tunnel and parts of the intersubunit surface (ISS). At the late nucleolar stage, the L1-stalk rRNA of domain V blocks the site of central protuberance (CP) assembly and is stabilized in a premature conformation by a range of AFs associated with the meandering, long N-terminal tail of Erb1. Two further structures show progression from this stage after release of the Erb1-Ytm1 complex by the Rea1 remodeling machinery. These intermediates, purified via Nop53, show dissociation of many early AFs from the premature ISS and destabilization of the L1-stalk. After subsequent release of the Spb1 methyltransferase, the L1-stalk rRNA can be accommodated in its mature conformation. This allows the premature CP to form, leading to a previously characterized nucleoplasmic intermediate, with a formed but premature CP. This particle is the substrate for the second Rea1 mediated structural remodeling, an intermediate of which we resolved to molecular resolution revealing Ipi1 as a central integrator for the Rix1-Ipi1-Ipi3 complex on this pre-60S particle. The binding of the Rix1-Rea1 remodeling machinery at this nucleoplasmic stage progresses maturation by inducing a 180$^{\circ}$ rotation of the 5S ribonucleoprotein particle (5S RNP) and CP. Using a combination of yeast genetics and cryo-EM we investigated the function of the AF Cgr1 in this maturation step. We showed that Cgr1 is required for CP rotation to take place, likely by stabilizing the rotated conformation. The Cgr1 function can be bypassed by introducing suppressor mutations in Rpf2 and Rrs1, two factors stabilizing the CP prior to rotation. Apart from ribosome biogenesis, two additional publications of this dissertation address protein translocation machinery, required for transport of proteins across or into membranes. The Sec translocon allows co- and posttranslational translocation of mostly unfolded substrates across the bacterial plasma and the eukaryotic endoplasmic reticulum (ER) membrane. We solved the structure of a stalled 70S ribosome-nascent chain complex (RNC) bound to the SecYEG translocon in a native like environment provided by a large lipid nanodisc. The structure shows all three subunits of the bacterial SecYEG complex and displays the lateral gate at a defined, early stage of opening or unzipping on the cytoplasmic side upon insertion of the signal anchor domain of the nascent chain. Specific pathways, such as the assembly of the mitochondrial bc1 respiratory chain complex, require folding of proteins in one compartment before translocation across a membrane to allow the protein to be active in another compartment. The bc1-complex component Rip1 folds in the mitochondrial matrix and assembles a 2Fe-2S cluster before being translocated across the inner mitochondrial membrane (IM) by the AAA-protein Bcs1. We solved the structure of Bcs1 in an ADP-bound state and two apo states, displaying a heptameric ring of Bcs1 protomers. Bcs1 forms two large aqueous vestibules separated by a seal forming middle domain. One vestibule is accessible from the matrix side and one lies within the inner mitochondrial membrane. The architecture and structural dynamics between the three states suggest an airlock like mechanism, allowing transport of folded Rip1 while maintaining the permeability barrier of the membrane

Roadmap on label-free super-resolution imaging

Label-free super-resolution (LFSR) imaging relies on light-scattering processes in nanoscale objects without a need for fluorescent (FL) staining required in super-resolved FL microscopy. The objectives of this Roadmap are to present a comprehensive vision of the developments, the state-of-the-art in this field, and to discuss the resolution boundaries and hurdles that need to be overcome to break the classical diffraction limit of the label-free imaging. The scope of this Roadmap spans from the advanced interference detection techniques, where the diffraction-limited lateral resolution is combined with unsurpassed axial and temporal resolution, to techniques with true lateral super-resolution capability that are based on understanding resolution as an information science problem, on using novel structured illumination, near-field scanning, and nonlinear optics approaches, and on designing superlenses based on nanoplasmonics, metamaterials, transformation optics, and microsphere-assisted approaches. To this end, this Roadmap brings under the same umbrella researchers from the physics and biomedical optics communities in which such studies have often been developing separately. The ultimate intent of this paper is to create a vision for the current and future developments of LFSR imaging based on its physical mechanisms and to create a great opening for the series of articles in this field

Traitement d'images de radiographie à faible dose : Débruitage et rehaussement de contraste conjoints et détection automatique de points de repère anatomiques pour l'estimation de la qualité des images

We aim at reducing the ALARA (As Low As Reasonably Achievable) dose limits for images acquired with EOS full-body system by means of image processing techniques. Two complementary approaches are studied. First, we define a post-processing method that optimizes the trade-off between acquired image quality and X-ray dose. The Non-Local means filter is extended to restore EOS images. We then study how to combine it with a multi-scale contrast enhancement technique. The image quality for the diagnosis is optimized by defining non-parametric noise containment maps that limit the increase of noise depending on the amount of local redundant information captured by the filter. Secondly, we estimate exposure index (EI) values on EOS images which give an immediate feedback on image quality to help radiographers to verify the correct exposure level of the X-ray examination. We propose a landmark detection based approach that is more robust to potential outliers than existing methods as it exploits the redundancy of local estimates. Finally, the proposed joint denoising and contrast enhancement technique significantly increases the image quality with respect to an algorithm used in clinical routine. Robust image quality indicators can be automatically associated with clinical EOS images. Given the consistency of the measures assessed on preview images, these indices could be used to drive an exposure management system in charge of defining the optimal radiation exposure.Nos travaux portent sur la réduction de la dose de rayonnement lors d'examens réalisés avec le Système de radiologie EOS. Deux approches complémentaires sont étudiées. Dans un premier temps, nous proposons une méthode de débruitage et de rehaussement de contraste conjoints pour optimiser le compromis entre la qualité des images et la dose de rayons X. Nous étendons le filtre à moyennes non locales pour restaurer les images EOS. Nous étudions ensuite comment combiner ce filtre à une méthode de rehaussement de contraste multi-échelles. La qualité des images cliniques est optimisée grâce à des fonctions limitant l'augmentation du bruit selon la quantité d’information locale redondante captée par le filtre. Dans un deuxième temps, nous estimons des indices d’exposition (EI) sur les images EOS afin de donner aux utilisateurs un retour immédiat sur la qualité de l'image acquise. Nous proposons ainsi une méthode reposant sur la détection de points de repère qui, grâce à l'exploitation de la redondance de mesures locales, est plus robuste à la présence de données aberrantes que les méthodes existantes. En conclusion, la méthode de débruitage et de rehaussement de contraste conjoints donne des meilleurs résultats que ceux obtenus par un algorithme exploité en routine clinique. La qualité des images EOS peut être quantifiée de manière robuste par des indices calculés automatiquement. Étant donnée la cohérence des mesures sur des images de pré-affichage, ces indices pourraient être utilisés en entrée d'un système de gestion automatique des expositions

Alfaviiruse nsP2 valk biokeemilisest vaatekohast: lugu mitmedomäänse valgu ensümaatilisest analüüsist

Väitekirja elektrooniline versioon ei sisalda publikatsioone.Chikungunya viirus (CHIKV) on suure meditsiinilise tähtsusega viirus mis kuulub alfaviiruste perekonda sugukonnas Togaviridae. Pregusel ajal ei ole CHIKV vastast vaktsiini ega spetsiifilist ravi. CHIKV paljunemine nakatatud rakkudes sõltub tema RNA genoomilt sünteesitavatest replikaasi valkudest. NsP2 on nelja replikaasi valgu seas suurim ja tal on palju teadaolevaid või eeldatavaid aktiivsuseid. Käesolevas töös näidati eksperimentaalselt, et CHIKV nsP2 omab kaheahelalist RNAd lahtiharutavat aktiivsust ning võib läbi viia ka vastupidist protsessi – soodustada kaheahelalise RNA moodustamist. Mõlemad need funktsioonid on olemas ainult täispikal nsP2 valgul. Peale selle näidati, et nsP2 omab ka NTPaset ja proteaaset aktiivsust. Kuna ka need aktiivsused on kõige tugevamad täispikal valgul siis saab saadud tulemustest järeldada, et CHIKV nsP2 toimib kui üks tervik: valgu erinevad osad seonduvad omavahel ja mõjutavad vastastikku üksteise aktiivsusi. Lisaks sellele leiti, et nsP2 aktiivsuseid mõjutavad ka mutatsioonid, mis on seotud mitte-tsütotoksilise fenotüübiga st. viiruse võimetusega maha suruda raku biosünteese ja põhjustada raku surma. Siiski ei saa kogutud andmeist teha järeldust milline või millised defektid nsP2 funktsioonides seda fenotüüpi põhjustavad. Lisaks nsP2 funktsionaalsele analüüsile viidi läbi ka muude CHIKV replikaasi valkude ekspresserimine ja puhastamine. Saadud kõrge kvaliteediga valke kasutati efektiivsete polüklonaalsete antiseerumite saamiseks. Nüüdseks on need töövahedid kasutusel paljudes laborites üle maailma ja on võimaldanud välja selgitada palju uusi fakte CHIKV (ja alfaviiruste üldse) molekulaarbioloogia kohta.Chikungunya virus (CHIKV, genus Alphavirus, family Togaviridae) has a positive sense RNA genome with length approximately 12 kb. It codes for four nonstructural (ns) proteins designated as nsP1, nsP2, nsP3 and nsP4 and for five or six structural proteins. Ns-proteins are involved in replication of virus RNA, in addition they also have functions unrelated to RNA replication. Out of all the nsPs, nsP2 plays a pivotal role towards regulation of CHIKV RNA replication. The protein was known to have NTPase, RTPase and protease activity and its N-terminal region was presumed to have helicase activity. Out of the enzymatic activities of nsP2 the helicase related functionalities were most insufficiently studied. Further, it was not known why two different and seemingly disconnected functional entities such as protease and helicase are present on a single polypeptide and what could be the importance of N terminal most part of nsP2 on the helicase activity. The bioinformatical, biochemical and biophysical approaches were employed to characterize the helicase related activities and to reveal the apparent minimal requirements for these activities. The bioinformatics platform suggests that the 3D-structure of the first 470 aa of nsP2 resembles the fold pattern of ToMV helicase which is a superfamily 1 of helicase. In particular, this fragment was predicted to consist from three domains. From these the extreme N terminal domain appears to be disordered while the other two domains possess RecA-like fold which is commonly found in NTPases. The biochemical analysis, carried out with purified full length and manipulated versions of nsP2, revealed that the C-terminal part of nsP2, which was known to have protease activity, is also essential for RNA helicase activity. Thus, the presence of protease region in nsP2 is clearly not accidental and these different functional domains are co-evolved to accomplish more significant tasks. The use of biophysical method (CD spectroscopy) confirmed that secondary structures of wt and manipulated versions of nsP2 are comparable; this indicates that functional defects detected in various enzymatic activities did not result from misfolding of mutant proteins. This also applies to forms of nsP2 which were engineered to contain mutations associated with noncytotoxic (NCT) phenotype of CHIKV replicons. It was found, all analyzed nsP2 enzymatic activities (protease, NTPase and helicase activities) were invariably affected by the NCT related mutations. In general, however, there was no significant correlation observed between extent of enzymatic defect(s) of nsP2 and phenotype of corresponding replicon. Thus, the development of NCT phenotype is apparently more complicated and could involve a number of underneath viral replication related functionalities. Finally, a number of ns-proteins from different alphaviruses were expressed, purified to raise polyclonal sera. These represent tools for detection of viral proteins using different immunological, such as western blot and immunofluorescence, methods. Similarly, the standardized enzymatic assays of nsP2 represent platform for screening and analysis of potential inhibitors of CHIKV infection. Taken together, these works elevated general understanding of nsP2 from a biochemical perspective and provided useful tools for studies aiming to understand molecular biology of alphaviruses

Roadmap on Label-Free Super-resolution Imaging

Label-free super-resolution (LFSR) imaging relies on light-scattering processes in nanoscale objects without a need for fluorescent (FL) staining required in super-resolved FL microscopy. The objectives of this Roadmap are to present a comprehensive vision of the developments, the state-of-the-art in this field, and to discuss the resolution boundaries and hurdles that need to be overcome to break the classical diffraction limit of the label-free imaging. The scope of this Roadmap spans from the advanced interference detection techniques, where the diffraction-limited lateral resolution is combined with unsurpassed axial and temporal resolution, to techniques with true lateral super-resolution capability that are based on understanding resolution as an information science problem, on using novel structured illumination, near-field scanning, and nonlinear optics approaches, and on designing superlenses based on nanoplasmonics, metamaterials, transformation optics, and microsphere-assisted approaches. To this end, this Roadmap brings under the same umbrella researchers from the physics and biomedical optics communities in which such studies have often been developing separately. The ultimate intent of this paper is to create a vision for the current and future developments of LFSR imaging based on its physical mechanisms and to create a great opening for the series of articles in this field.Peer reviewe