Modelling 3D scanned data to visualise and analyse the Built Environment for regeneration

The renovation and refurbishment market is rapidly expanding in the construction industry. The regeneration and transformation of cities from the industrial age (unsustainable) to the knowledge age (sustainable) is essentially a "whole life cycle" process consisting of: planning, development, operation, reuse and renewal. Advanced digital mapping technologies are enablers for effective eplanning, consultation and communication of users' views during the planning, design, construction and lifecycle process of the built environment. Those technologies can be used to drive the productivity gains by promoting a free-flow of information between departments, divisions, offices, and sites; and between themselves, their contractors and partners. Such is the case of the 3D laser scanner which enables digital documentation of buildings, sites and physical objects for reconstruction and restoration. It also facilitates the creation of educational resources within the built environment, as well as the reconstruction of the built environment. The use of the 3D scanner in combination with the 3D printer provides the transformation of digital data from the captured CAD model back to a physical model at an appropriate scale - reverse prototyping. The use of these technologies is key enablers to the creation of new approaches to the ¿Whole Life Cycle¿ process within the built and human environment for the 21st century. The paper describes the research of a building data integration in the INTELCITIES project undertaken by a European consortium of researchers and practitioners under the Framework 6 research programme to develop a prototype system of the e-City Platform in order to pool advanced knowledge and experience of electronic government, planning systems and citizen participation from across Europe (www.intelcitiesproject.com). The scope includes capturing digital data of existing buildings using 3D laser scanning equipment and illustration of how digitised building data can be integrated with other types of city data, using nD modelling, to support integrated intelligent city systems for enhancing the refurbishment process in the built environment

Incorporating Background Invariance into Feature-Based Object Recognition

Current feature-based object recognition methods use information derived from local image patches. For robustness, features are engineered for invariance to various transformations, such as rotation, scaling, or affine warping. When patches overlap object boundaries, however, errors in both detection and matching will almost certainly occur due to inclusion of unwanted background pixels. This is common in real images, which often contain significant background clutter, objects which are not heavily textured, or objects which occupy a relatively small portion of the image. We suggest improvements to the popular Scale Invariant Feature Transform (SIFT) which incorporate local object boundary information. The resulting feature detection and descriptor creation processes are invariant to changes in background.We call this method the Background and Scale Invariant Feature Transform (BSIFT).We demonstrate BSIFT’s superior performance in feature detection and matching on synthetic and natural imag

Incorporating Background Invariance into Feature-Based Object Recognition

Current feature-based object recognition methods use information derived from local image patches. For robustness, features are engineered for invariance to various transformations, such as rotation, scaling, or affine warping. When patches overlap object boundaries, however, errors in both detection and matching will almost certainly occur due to inclusion of unwanted background pixels. This is common in real images, which often contain significant background clutter, objects which are not heavily textured, or objects which occupy a relatively small portion of the image. We suggest improvements to the popular Scale Invariant Feature Transform (SIFT) which incorporate local object boundary information. The resulting feature detection and descriptor creation processes are invariant to changes in background. We call this method the Background and Scale Invariant Feature Transform (BSIFT). We demonstrate BSIFT’s superior performance in feature detection and matching on synthetic and natural images. 1

Curve-Based Shape Matching Methods and Applications

One of the main cues we use in our everyday life when interacting with the environment is shape. For example, we use shape information to recognise a chair, grasp a cup, perceive traffic signs and solve jigsaw puzzles. We also use shape when dealing with more sophisticated tasks, such as the medical diagnosis of radiographs or the restoration of archaeological artifacts. While the perception of shape and its use is a natural ability of human beings, endowing machines with such skills is not straightforward. However, the exploitation of shape cues is important for the development of competent computer methods that will automatically perform tasks such as those just mentioned. With this aim, the present work proposes computer methods which use shape to tackle two important tasks, namely packing and object recognition. The packing problem arises in a variety of applications in industry, where the placement of a set of two-dimensional shapes on a surface such that no shapes overlap and the uncovered surface area is minimised is important. Given that this problem is NP-complete, we propose a heuristic method which searches for a solution of good quality, though not necessarily the optimal one, within a reasonable computation time. The proposed method adopts a pictorial representation and employs a greedy algorithm which uses a shape matching module in order to dynamically select the order and the pose of the parts to be placed based on the “gaps” appearing in the layout during the execution. This thesis further investigates shape matching in the context of object recognition and first considers the case where the target object and the input scene are represented by their silhouettes. Two distinct methods are proposed; the first method follows a local string matching approach, while the second one adopts a global optimisation approach using dynamic programming. Their use of silhouettes, however, rules out the consideration of any internal contours that might appear in the input scene, and in order to address this limitation, we later propose a graph-based scheme that performs shape matching incorporating information from both internal and external contours. Finally, we lift the assumption made that input data are available in the form of closed curves, and present a method which can robustly perform object recognition using curve fragments (edges) as input evidence. Experiments conducted with synthetic and real images, involving rigid and deformable objects, show the robustness of the proposed methods with respect to geometrical transformations, heavy clutter and substantial occlusion

Enhancing low-level features with mid-level cues

Local features have become an essential tool in visual recognition. Much of the progress in computer vision over the past decade has built on simple, local representations such as SIFT or HOG. SIFT in particular shifted the paradigm in feature representation. Subsequent works have often focused on improving either computational efficiency, or invariance properties. This thesis belongs to the latter group. Invariance is a particularly relevant aspect if we intend to work with dense features. The traditional approach to sparse matching is to rely on stable interest points, such as corners, where scale and orientation can be reliably estimated, enforcing invariance; dense features need to be computed on arbitrary points. Dense features have been shown to outperform sparse matching techniques in many recognition problems, and form the bulk of our work. In this thesis we present strategies to enhance low-level, local features with mid-level, global cues. We devise techniques to construct better features, and use them to handle complex ambiguities, occlusions and background changes. To deal with ambiguities, we explore the use of motion to enforce temporal consistency with optical flow priors. We also introduce a novel technique to exploit segmentation cues, and use it to extract features invariant to background variability. For this, we downplay image measurements most likely to belong to a region different from that where the descriptor is computed. In both cases we follow the same strategy: we incorporate mid-level, "big picture" information into the construction of local features, and proceed to use them in the same manner as we would the baseline features. We apply these techniques to different feature representations, including SIFT and HOG, and use them to address canonical vision problems such as stereo and object detection, demonstrating that the introduction of global cues yields consistent improvements. We prioritize solutions that are simple, general, and efficient. Our main contributions are as follows: (a) An approach to dense stereo reconstruction with spatiotemporal features, which unlike existing works remains applicable to wide baselines. (b) A technique to exploit segmentation cues to construct dense descriptors invariant to background variability, such as occlusions or background motion. (c) A technique to integrate bottom-up segmentation with recognition efficiently, amenable to sliding window detectors.Les "features" locals s'han convertit en una eina fonamental en el camp del reconeixement visual. Gran part del progrés experimentat en el camp de la visió per computador al llarg de l'última decada es basa en representacions locals de baixa complexitat, com SIFT o HOG. SIFT, en concret, ha canviat el paradigma en representació de característiques visuals. Els treballs que l'han succeït s'acostumen a centrar o bé a millorar la seva eficiencia computacional, o bé propietats d'invariança. El treball presentat en aquesta tesi pertany al segon grup. L'invariança es un aspecte especialment rellevant quan volem treballab amb "features" denses, és a dir per a cada pixel. La manera tradicional d'atacar el problema amb "features" de baixa densitat consisteix en seleccionar punts d'interés estables, com per exemple cantonades, on l'escala i l'orientació poden ser estimades de manera robusta. Les "features" denses, per definició, han de ser calculades en punts arbitraris de la imatge. S'ha demostrat que les "features" denses obtenen millors resultats en tècniques de correspondència per a molts problemes en reconeixement, i formen la major part del nostre treball. En aquesta tesi presentem estratègies per a enriquir "features" locals de baix nivell amb "cues" o dades globals, de mitja complexitat. Dissenyem tècniques per a construïr millors "features", que usem per a atacar problemes tals com correspondències amb un grau elevat d'ambigüetat, oclusions, i canvis del fons de la imatge. Per a atacar ambigüetats, explorem l'ús del moviment per a imposar consistència espai-temporal mitjançant informació d'"optical flow". També presentem una tècnica per explotar dades de segmentació que fem servir per a extreure "features" invariants a canvis en el fons de la imatge. Aquest mètode consisteix en atenuar els components de la imatge (i per tant les "features") que probablement corresponguin a regions diferents a la del descriptor que estem calculant. En ambdós casos seguim la mateixa estratègia: la nostra voluntat és incorporar dades globals d'un nivell de complexitat mitja a la construcció de "features" locals, que procedim a utilitzar de la mateixa manera que les "features" originals. Aquestes tècniques són aplicades a diferents tipus de representacions, incloent SIFT i HOG, i mostrem com utilitzar-les per a atacar problemes fonamentals en visió per computador tals com l'estèreo i la detecció d'objectes. En aquest treball demostrem que introduïnt informació global en la construcció de "features" locals podem obtenir millores consistentment. Donem prioritat a solucions senzilles, generals i eficients. Aquestes són les principals contribucions de la tesi: (a) Una tècnica per a reconstrucció estèreo densa mitjançant "features" espai-temporals, amb l'avantatge respecte a treballs existents que podem aplicar-la a càmeres en qualsevol configuració geomètrica ("wide-baseline"). (b) Una tècnica per a explotar dades de segmentació dins la construcció de descriptors densos, fent-los invariants a canvis al fons de la imatge, i per tant a problemes com les oclusions en estèreo o objectes en moviment. (c) Una tècnica per a integrar segmentació de manera ascendent ("bottom-up") en problemes de reconeixement d'una manera eficient, dissenyada per a detectors de tipus "sliding window"