Hierarchical structure-and-motion recovery from uncalibrated images

This paper addresses the structure-and-motion problem, that requires to find camera motion and 3D struc- ture from point matches. A new pipeline, dubbed Samantha, is presented, that departs from the prevailing sequential paradigm and embraces instead a hierarchical approach. This method has several advantages, like a provably lower computational complexity, which is necessary to achieve true scalability, and better error containment, leading to more stability and less drift. Moreover, a practical autocalibration procedure allows to process images without ancillary information. Experiments with real data assess the accuracy and the computational efficiency of the method.Comment: Accepted for publication in CVI

Histopathological image analysis : a review

Over the past decade, dramatic increases in computational power and improvement in image analysis algorithms have allowed the development of powerful computer-assisted analytical approaches to radiological data. With the recent advent of whole slide digital scanners, tissue histopathology slides can now be digitized and stored in digital image form. Consequently, digitized tissue histopathology has now become amenable to the application of computerized image analysis and machine learning techniques. Analogous to the role of computer-assisted diagnosis (CAD) algorithms in medical imaging to complement the opinion of a radiologist, CAD algorithms have begun to be developed for disease detection, diagnosis, and prognosis prediction to complement the opinion of the pathologist. In this paper, we review the recent state of the art CAD technology for digitized histopathology. This paper also briefly describes the development and application of novel image analysis technology for a few specific histopathology related problems being pursued in the United States and Europe

Text–to–Video: Image Semantics and NLP

When aiming at automatically translating an arbitrary text into a visual story, the main challenge consists in finding a semantically close visual representation whereby the displayed meaning should remain the same as in the given text. Besides, the appearance of an image itself largely influences how its meaningful information is transported towards an observer. This thesis now demonstrates that investigating in both, image semantics as well as the semantic relatedness between visual and textual sources enables us to tackle the challenging semantic gap and to find a semantically close translation from natural language to a corresponding visual representation. Within the last years, social networking became of high interest leading to an enormous and still increasing amount of online available data. Photo sharing sites like Flickr allow users to associate textual information with their uploaded imagery. Thus, this thesis exploits this huge knowledge source of user generated data providing initial links between images and words, and other meaningful data. In order to approach visual semantics, this work presents various methods to analyze the visual structure as well as the appearance of images in terms of meaningful similarities, aesthetic appeal, and emotional effect towards an observer. In detail, our GPU-based approach efficiently finds visual similarities between images in large datasets across visual domains and identifies various meanings for ambiguous words exploring similarity in online search results. Further, we investigate in the highly subjective aesthetic appeal of images and make use of deep learning to directly learn aesthetic rankings from a broad diversity of user reactions in social online behavior. To gain even deeper insights into the influence of visual appearance towards an observer, we explore how simple image processing is capable of actually changing the emotional perception and derive a simple but effective image filter. To identify meaningful connections between written text and visual representations, we employ methods from Natural Language Processing (NLP). Extensive textual processing allows us to create semantically relevant illustrations for simple text elements as well as complete storylines. More precisely, we present an approach that resolves dependencies in textual descriptions to arrange 3D models correctly. Further, we develop a method that finds semantically relevant illustrations to texts of different types based on a novel hierarchical querying algorithm. Finally, we present an optimization based framework that is capable of not only generating semantically relevant but also visually coherent picture stories in different styles.Bei der automatischen Umwandlung eines beliebigen Textes in eine visuelle Geschichte, besteht die größte Herausforderung darin eine semantisch passende visuelle Darstellung zu finden. Dabei sollte die Bedeutung der Darstellung dem vorgegebenen Text entsprechen. Darüber hinaus hat die Erscheinung eines Bildes einen großen Einfluß darauf, wie seine bedeutungsvollen Inhalte auf einen Betrachter übertragen werden. Diese Dissertation zeigt, dass die Erforschung sowohl der Bildsemantik als auch der semantischen Verbindung zwischen visuellen und textuellen Quellen es ermöglicht, die anspruchsvolle semantische Lücke zu schließen und eine semantisch nahe Übersetzung von natürlicher Sprache in eine entsprechend sinngemäße visuelle Darstellung zu finden. Des Weiteren gewann die soziale Vernetzung in den letzten Jahren zunehmend an Bedeutung, was zu einer enormen und immer noch wachsenden Menge an online verfügbaren Daten geführt hat. Foto-Sharing-Websites wie Flickr ermöglichen es Benutzern, Textinformationen mit ihren hochgeladenen Bildern zu verknüpfen. Die vorliegende Arbeit nutzt die enorme Wissensquelle von benutzergenerierten Daten welche erste Verbindungen zwischen Bildern und Wörtern sowie anderen aussagekräftigen Daten zur Verfügung stellt. Zur Erforschung der visuellen Semantik stellt diese Arbeit unterschiedliche Methoden vor, um die visuelle Struktur sowie die Wirkung von Bildern in Bezug auf bedeutungsvolle Ähnlichkeiten, ästhetische Erscheinung und emotionalem Einfluss auf einen Beobachter zu analysieren. Genauer gesagt, findet unser GPU-basierter Ansatz effizient visuelle Ähnlichkeiten zwischen Bildern in großen Datenmengen quer über visuelle Domänen hinweg und identifiziert verschiedene Bedeutungen für mehrdeutige Wörter durch die Erforschung von Ähnlichkeiten in Online-Suchergebnissen. Des Weiteren wird die höchst subjektive ästhetische Anziehungskraft von Bildern untersucht und "deep learning" genutzt, um direkt ästhetische Einordnungen aus einer breiten Vielfalt von Benutzerreaktionen im sozialen Online-Verhalten zu lernen. Um noch tiefere Erkenntnisse über den Einfluss des visuellen Erscheinungsbildes auf einen Betrachter zu gewinnen, wird erforscht, wie alleinig einfache Bildverarbeitung in der Lage ist, tatsächlich die emotionale Wahrnehmung zu verändern und ein einfacher aber wirkungsvoller Bildfilter davon abgeleitet werden kann. Um bedeutungserhaltende Verbindungen zwischen geschriebenem Text und visueller Darstellung zu ermitteln, werden Methoden des "Natural Language Processing (NLP)" verwendet, die der Verarbeitung natürlicher Sprache dienen. Der Einsatz umfangreicher Textverarbeitung ermöglicht es, semantisch relevante Illustrationen für einfache Textteile sowie für komplette Handlungsstränge zu erzeugen. Im Detail wird ein Ansatz vorgestellt, der Abhängigkeiten in Textbeschreibungen auflöst, um 3D-Modelle korrekt anzuordnen. Des Weiteren wird eine Methode entwickelt die, basierend auf einem neuen hierarchischen Such-Anfrage Algorithmus, semantisch relevante Illustrationen zu Texten verschiedener Art findet. Schließlich wird ein optimierungsbasiertes Framework vorgestellt, das nicht nur semantisch relevante, sondern auch visuell kohärente Bildgeschichten in verschiedenen Bildstilen erzeugen kann

Developing Algorithms for Quantifying the Super Resolution Microscopic Data: Applications to the Quantification of Protein-Reorganization in Bacteria Responding to Treatment by Silver Ions

Histone-like nucleoid structuring proteins (HNS) play significant roles in shaping the chromosomal DNA, regulation of transcriptional networks in microbes, as well as bacterial responses to environmental changes such as temperature fluctuations. In this work, the intracellular organization of HNS proteins in E. coli bacteria was investigated utilizing super-resolution fluorescence microscopy, which surpasses conventional microscopy by 10–20 fold in spatial resolution. More importantly, the changes of the spatial distribution of HNS proteins in E. coli, by addition of silver ions into the growth medium were explored. To quantify the spatial distribution of HNS in bacteria and its changes, an automatic method based on Voronoi diagram was implemented. The HNS proteins localized in super-resolution fluorescence microscopy were segmented and clustered based on several quantitative parameters, such as molecular areas, molecular densities, and mean inter-molecular distances of the k-th rank, all of which were computed from the Voronoi diagrams. These parameters, as well as the associated clustering analysis, allowed us to quantify how the spatial organization of HNS proteins responds to silver, and provided insight into understanding how microbes adapt to new environments

Hierarchical autoclassification of cryo-EM samples and macromolecular energy landscape determination

Background and objective: Cryo-electron microscopy using single particle analysis is a powerful technique for obtaining 3D reconstructions of macromolecules in near native conditions. One of its major advances is its capacity to reveal conformations of dynamic molecular complexes. Most popular and successful current approaches to analyzing heterogeneous complexes are founded on Bayesian inference. However, these 3D classification methods require the tuning of specific parameters by the user and the use of complicated 3D re-classification procedures for samples affected by extensive heterogeneity. Thus, the success of these approaches highly depends on the user experience. We introduce a robust approach to identify many different conformations presented in a cryo-EM dataset based on Bayesian inference through Relion classification methods that does not require tuning of parameters and reclassification strategies. Methods: The algorithm allows both 2D and 3D classification and is based on a hierarchical clustering approach that runs automatically without requiring typical inputs, such as the number of conformations present in the dataset or the required classification iterations. This approach is applied to robustly determine the energy landscapes of macromolecules. Results: We tested the performance of the methods proposed here using four different datasets, comprising structurally homogeneous and highly heterogeneous cases. In all cases, the approach provided excellent results. The routines are publicly available as part of the CryoMethods plugin included in the Scipion package. Conclusions: Our results show that the proposed method can be used to align and classify homogeneous and heterogeneous datasets without requiring previous alignment information or any prior knowledge about the number of co-existing conformations. The approach can be used for both 2D and 3D autoclassification and only requires an initial volume. In addition, the approach is robust to the "attractor" problem providing many different conformations/views for samples affected by extensive heterogeneity. The obtained 3D classes can render high resolution 3D structures, while the obtained energy landscapes can be used to determine structural trajectories

Principled control of approximate programs

In conventional computing, most programs are treated as implementations of mathematical functions for which there is an exact output that must computed from a given input. However, in many problem domains, it is sufficient to produce some approximation of this output. For example, when rendering a scene in graphics, it is acceptable to take computational short-cuts if human beings cannot tell the difference in the rendered scene. In other problem domains like machine learning, programs are often implementations of heuristic approaches to solving problems and therefore already compute approximate solutions to the original problem. This is the key insight for the new research area, approximate computing, which attempts to trade-off such approximations against the cost of computational resources such as program execution time, energy consumption, and memory usage. We believe that approximate computing is an important step towards a more fundamental and comprehensive goal that we call information-efficiency. Current applications compute more information (bits) than are needed to produce their outputs, and since producing and transporting bits of information inside a computer requires energy/computation time/memory usage, information-inefficient computing leads directly to resources inefficiency. Although there is now a fairly large literature on approximate computing, system researchers have focused mostly on what we can call the forward problem; that is, they have explored different ways in both hardware and software to introduce approximations in a program and have demonstrated that these approximations can enable significant execution speedups and energy savings with some quality degradation of the result. However, these efforts do not provide any guarantee on the amount of the quality degradation. Since the acceptable amount of degradation usually depends on the scenario in which the application is deployed, it is very important to be able to control the degree of approximation. In this dissertation, we refer to this problem as the inverse problem. Relatively little is known about how to solve the inverse problem in a disciplined way. This dissertation makes two contributions towards solving the inverse problem. First, we investigate a large set of approximate algorithms from a variety of domains in order to understand how approximation is used in real-world applications. From this investigation, we determine that many approximate programs are tunable approximate programs. Tunable approximate programs have one or more parameters called knobs that can be changed to vary the quality of the output of the approximate computation as well as the corresponding cost. For example, an iterative linear equation solver can vary the number of iterations to trade quality of the solution versus the execution time, a Monte Carlo path tracer can change the number of sampling light paths to trade the quality of the resulting image against execution time, etc. Tunable approximate programs provide many opportunities for trading accuracy versus cost. By carefully analyzing these algorithms, we have found a set of patterns for how approximation is applied in tunable programs. Our classification can be used to identify new approximation opportunities in programs. A second contribution of this dissertation is an approach to solving the inverse problem for tunable approximate programs. Concretely, the problem is to determine knob settings to minimize the cost while keeping the quality degradation within a given bound. There are four challenges: i) for real-world applications, the quality and cost are usually complex non-linear functions of the knobs and these functions are usually hard to express analytically; ii) the quality and the cost for an application vary greatly for different inputs; iii) when an acceptable quality degradation bound is presented, determining the knob setting has to be very efficient so that the extra overhead incurred by the identification will not exceed the cost saved by the approximation; and iv) the approach should be general so that it can be applied to many applications. To meet these requirements, we formulate the inverse problem as a constrained optimization problem and solve it using a machine learning based approach. We build a system which uses machine learning techniques to learn cost and quality models for the program by profiling the program with a set of representative inputs. Then, when a quality degradation bound is presented, the system searches these error and cost models to identify the knob settings which can achieve the best cost savings while simultaneously guaranteeing the quality degradation bound statistically. We evaluate the system with a set of real world applications, including a social network graph partitioner, an image search engine, a 2-D graph layout engine, a 3-D game physics engine, a SVM solver and a radar signal processing engine. The experiments showed great savings in execution time and energy savings for a variety of quality bounds.Computer Science

Deep Machine Learning with Spatio-Temporal Inference

Deep Machine Learning (DML) refers to methods which utilize hierarchies of more than one or two layers of computational elements to achieve learning. DML may draw upon biomemetic models, or may be simply biologically-inspired. Regardless, these architectures seek to employ hierarchical processing as means of mimicking the ability of the human brain to process a myriad of sensory data and make meaningful decisions based on this data. In this dissertation we present a novel DML architecture which is biologically-inspired in that (1) all processing is performed hierarchically; (2) all processing units are identical; and (3) processing captures both spatial and temporal dependencies in the observations to organize and extract features suitable for supervised learning. We call this architecture Deep Spatio-Temporal Inference Network (DeSTIN). In this framework, patterns observed in pixel data at the lowest layer of the hierarchy are organized and fit to generalizations using decomposition algorithms. Subsequent spatial layers draw upon previous layers, their own temporal observations and beliefs, and the observations and beliefs of parent nodes to extract features suitable for supervised learning using standard classifiers such as feedforward neural networks. Hence, DeSTIN is viewed as an unsupervised feature extraction scheme in the sense that rather than relying on human engineering to determine features for a particular problem, DeSTIN naturally constructs features of interest by representing salient regularities in the patterns observed. Detailed discussion and analysis of the DeSTIN framework is provided, including focus on its key components of generalization through online clustering and temporal inference. We present a variety of implementation details, including static and dynamic learning formulations, and function approximation methods. Results on standardized datasets of handwritten digits as well as face and optic nerve detection are presented, illustrating the efficacy of the proposed approach

A data science approach to portuguese road accidents’ data

Dissertação de mestrado integrado em Informatics EngineeringWe frequently hear about accidents and traffic news on television, radio and even social networks. Even though we have witnessed a decrease in mortality rate in Portuguese roads, the number of road victims have been increasing recently so we should be more aware of this problem, study it and come up with solutions to decrease the mortality rate and the number of victims in Portuguese roads. One possible solution to this problem is the identification of blackspots (areas with a high number of accidents or an abnormal number of fatalities) associated with temporal and spatial analysis, and relations between them. By doing this, we will be closer to decreasing accidents as well as the mortality rate on Portuguese roads. This dissertation is going to focus on these concerns using the information present on ANSR (Autoridade Nacional de Segurança Rodoviária) reports as well as other data gathered by the research team regarding road traffic incidents in Portuguese cities. After researching about the state of the art, we realize that, on one hand, there’s a big problem which is traffic accidents and resultant victims that are still to this day very concerning to society, on the other hand, many techniques and methods have been developed and improved to help mitigate this problem. The data have shown that Portugal still has work to do on decreasing the number of accidents and victims according to those evolution curves, data collected in ANSR reports and the comparison between traffic numbers in EU countries. This dissertation focused on understanding, processing and exploring data in-depth, developing models to analyze data, preventing accidents and enhancing road safety and coming up with useful insights about the road network and publishing them in a dashboard platform open to the community.Frequentemente, ouvimos falar de acidentes e notícias sobre trânsito na televisão, rádio e redes sociais. Apesar de estarmos a testemunhar um decréscimo da taxa de mortalidade em estradas portuguesas, o número de vítimas resultantes de acidentes têm vindo a aumentar recentemente, por isso, devemos estar mais atentos a este problema, estudá-lo e arranjar soluções para diminuir a taxa de mortalidade e o número de pessoas vítimas de acidentes em estradas portuguesas. Uma possível solução para este problema é a identificação de zonas negras (zonas com um número elevado de acidentes ou um número anormal de óbitos) associado a uma análise temporal e espacial, juntamente com as relações entre eles. Ao fazer isto, estaremos mais perto de diminuir o número de acidentes, bem como a taxa de mortalidade nas estradas portuguesas. Esta dissertação irá focar-se nestes aspetos, utilizando a informação presente no relatórios da ANSR (Autoridade Nacional de Segurança Rodoviária) e também outros dados recolhidos pela equipa de investigação relativamente a incidentes rodoviários em estradas portuguesas. Depois de recolher dados sobre o estado de arte, percebemos que, por um lado, existe um grande problema com os acidentes rodoviários e vítimas dos mesmos que são até ao dia de hoje muito preocupantes para a sociedade, por outro lado, muitas técnicas e métodos que têm vindo a ser desenvolvidos e melhorados para ajudar a mitigar este problema. Os dados mostram que Portugal ainda tem trabalho a fazer para diminuir os números de acidentes e de vítimas tendo em consideração as curvas de evolução destes indicadores, dados recolhidos em relatórios da ANSR e a comparação entre dados rodoviários entre países da UE. Esta dissertação focou-se em perceber, processar e explorar os dados a fundo, desenvolver modelos para analisar os dados, prevenir acidentes e aumentar a segurança rodoviária e encontrar perceções sobre a rede rodoviária e publicá-las numa plataforma com painéis de informação disponíveis para a comunidade

The robot's vista space : a computational 3D scene analysis

Swadzba A. The robot's vista space : a computational 3D scene analysis. Bielefeld (Germany): Bielefeld University; 2011.The space that can be explored quickly from a fixed view point without locomotion is known as the vista space. In indoor environments single rooms and room parts follow this definition. The vista space plays an important role in situations with agent-agent interaction as it is the directly surrounding environment in which the interaction takes place. A collaborative interaction of the partners in and with the environment requires that both partners know where they are, what spatial structures they are talking about, and what scene elements they are going to manipulate. This thesis focuses on the analysis of a robot's vista space. Mechanisms for extracting relevant spatial information are developed which enable the robot to recognize in which place it is, to detect the scene elements the human partner is talking about, and to segment scene structures the human is changing. These abilities are addressed by the proposed holistic, aligned, and articulated modeling approach. For a smooth human-robot interaction, the computed models should be aligned to the partner's representations. Therefore, the design of the computational models is based on the combination of psychological results from studies on human scene perception with basic physical properties of the perceived scene and the perception itself. The holistic modeling realizes a categorization of room percepts based on the observed 3D spatial layout. Room layouts have room type specific features and fMRI studies have shown that some of the human brain areas being active in scene recognition are sensitive to the 3D geometry of a room. With the aligned modeling, the robot is able to extract the hierarchical scene representation underlying a scene description given by a human tutor. Furthermore, it is able to ground the inferred scene elements in its own visual perception of the scene. This modeling follows the assumption that cognition and language schematize the world in the same way. This is visible in the fact that a scene depiction mainly consists of relations between an object and its supporting structure or between objects located on the same supporting structure. Last, the articulated modeling equips the robot with a methodology for articulated scene part extraction and fast background learning under short and disturbed observation conditions typical for human-robot interaction scenarios. Articulated scene parts are detected model-less by observing scene changes caused by their manipulation. Change detection and background learning are closely coupled because change is defined phenomenologically as variation of structure. This means that change detection involves a comparison of currently visible structures with a representation in memory. In range sensing this comparison can be nicely implement as subtraction of these two representations. The three modeling approaches enable the robot to enrich its visual perceptions of the surrounding environment, the vista space, with semantic information about meaningful spatial structures useful for further interaction with the environment and the human partner