Improving Spatial Codification in Semantic Segmentation

This paper explores novel approaches for improving the spatial codification for the pooling of local descriptors to solve the semantic segmentation problem. We propose to partition the image into three regions for each object to be described: Figure, Border and Ground. This partition aims at minimizing the influence of the image context on the object description and vice versa by introducing an intermediate zone around the object contour. Furthermore, we also propose a richer visual descriptor of the object by applying a Spatial Pyramid over the Figure region. Two novel Spatial Pyramid configurations are explored: Cartesian-based and crown-based Spatial Pyramids. We test these approaches with state-of-the-art techniques and show that they improve the Figure-Ground based pooling in the Pascal VOC 2011 and 2012 semantic segmentation challenges.Comment: Paper accepted at the IEEE International Conference on Image Processing, ICIP 2015. Quebec City, 27-30 September. Project page: https://imatge.upc.edu/web/publications/improving-spatial-codification-semantic-segmentatio

From point cloud to BIM: a survey of existing approaches

International audienceIn order to handle more efficiently projects of restoration, documentation and maintenance of historical buildings, it is essentialto rely on a 3D enriched model for the building. Today, the concept of Building Information Modelling (BIM) is widely adoptedfor the semantization of digital mockups and few research focused on the value of this concept in the field of cultural heritage.In addition historical buildings are already built, so it is necessary to develop a performing approach, based on a first step ofbuilding survey, to develop a semantically enriched digital model. For these reasons, this paper focuses on this chain startingwith a point cloud and leading to the well-structured final BIM; and proposes an analysis and a survey of existing approacheson the topics of: acquisition, segmentation and BIM creation. It also, presents a critical analysis on the application of this chainin the field of cultural heritag

Review of the “ as-buit BIM ” approaches

International audienceToday, we need 3D models of heritage buildings in order to handle more efficiently projects of restoration, documentation and maintenance. In this context, developing a performing approach, based on a first phase of building survey, is a necessary step in order to build a semantically enriched digital model. For this purpose, the Building Information Modeling is an efficient tool for storing and exchanging knowledge about buildings. In order to create such a model, there are three fundamental steps: acquisition, segmentation and modeling. For these reasons, it is essential to understand and analyze this entire chain that leads to a well- structured and enriched 3D digital model. This paper proposes a survey and an analysis of the existing approaches on these topics and tries to define a new approach of semantic structuring taking into account the complexity of this chain

A PhD Dissertation on Road Topology Classification for Autonomous Driving

La clasificaci´on de la topolog´ıa de la carretera es un punto clave si queremos desarrollar sistemas de conducci´on aut´onoma completos y seguros. Es l´ogico pensar que la comprensi ´on de forma exhaustiva del entorno que rodea al vehiculo, tal como sucede cuando es un ser humano el que toma las decisiones al volante, es una condici´on indispensable si se quiere avanzar en la consecuci´on de veh´ıculos aut´onomos de nivel 4 o 5. Si el conductor, ya sea un sistema aut´onomo, como un ser humano, no tiene acceso a la informaci´on del entorno la disminuci´on de la seguridad es cr´ıtica y el accidente es casi instant´aneo i.e., cuando un conductor se duerme al volante. A lo largo de esta tesis doctoral se presentan sendos sistemas basados en deep leaning que ayudan al sistema de conducci´on aut´onoma a comprender el entorno en el que se encuentra en ese instante. El primero de ellos 3D-Deep y su optimizaci´on 3D-Deepest, es una nueva arquitectura de red para la segmentaci´on sem´antica de carretera en el que se integran fuentes de datos de diferente tipolog´ıa. La segmentaci´on de carretera es clave en un veh´ıculo aut´onomo, ya que es el medio por el que deber´ıa circular en el 99,9% de los casos. El segundo es un sistema de clasificaci´on de intersecciones urbanas mediante diferentes enfoques comprendidos dentro del metric-learning, la integraci´on temporal y la generaci´on de im´agenes sint´eticas. La seguridad es un punto clave en cualquier sistema aut´onomo, y si es de conducci´on a´un m´as. Las intersecciones son uno de los lugares dentro de las ciudades donde la seguridad es cr´ıtica. Los coches siguen trayectorias secantes y por tanto pueden colisionar, la mayor´ıa de ellas son usadas por los peatones para atravesar la v´ıa independientemente de si existen pasos de cebra o no, lo que incrementa de forma alarmante los riesgos de atropello y colisi´on. La implementaci´on de la combinaci´on de ambos sistemas mejora substancialmente la comprensi´on del entorno, y puede considerarse que incrementa la seguridad, allanando el camino en la investigaci´on hacia un veh´ıculo completamente aut´onomo.Road topology classification is a crucial point if we want to develop complete and safe autonomous driving systems. It is logical to think that a thorough understanding of the environment surrounding the ego-vehicle, as it happens when a human being is a decision-maker at the wheel, is an indispensable condition if we want to advance in the achievement of level 4 or 5 autonomous vehicles. If the driver, either an autonomous system or a human being, does not have access to the information of the environment, the decrease in safety is critical, and the accident is almost instantaneous, i.e., when a driver falls asleep at the wheel. Throughout this doctoral thesis, we present two deep learning systems that will help an autonomous driving system understand the environment in which it is at that instant. The first one, 3D-Deep and its optimization 3D-Deepest, is a new network architecture for semantic road segmentation in which data sources of different types are integrated. Road segmentation is vital in an autonomous vehicle since it is the medium on which it should drive in 99.9% of the cases. The second is an urban intersection classification system using different approaches comprised of metric-learning, temporal integration, and synthetic image generation. Safety is a crucial point in any autonomous system, and if it is a driving system, even more so. Intersections are one of the places within cities where safety is critical. Cars follow secant trajectories and therefore can collide; most of them are used by pedestrians to cross the road regardless of whether there are crosswalks or not, which alarmingly increases the risks of being hit and collision. The implementation of the combination of both systems substantially improves the understanding of the environment and can be considered to increase safety, paving the way in the research towards a fully autonomous vehicle

Digitalization and Spatial Documentation of Post-Earthquake Temporary Housing in Central Italy: An Integrated Geomatic Approach Involving UAV and a GIS-Based System

Geoinformation and aerial data collection are essential during post-earthquake emergency response. This research focuses on the long-lasting spatial impacts of temporary solutions, which have persisted in regions of Central Italy affected by catastrophic seismic events over the past 25 years, significantly and permanently altering their landscapes. The paper analyses the role of geomatic and photogrammetric tools in documenting the emergency process and projects in post-disaster phases. An Atlas of Temporary Architectures is proposed, which defines a common semantic and geometric codification for mapping temporary housing from territorial to urban and building scales. The paper presents an implementation of attribute specification in existing official cartographic data, including geometric entities in a 3D GIS data model platform for documenting and digitalising these provisional contexts. To achieve this platform, UAV point clouds are integrated with non-metric data to ensure a complete description in a multiscalar approach. Accurate topographic modifications can be captured by extracting very high-resolution orthophotos and elevation models (DSM and DTM). The results have been validated in Visso (Macerata), a small historical mountain village in Central Italy which was heavily damaged by the seismic events of 2016/2017. The integrated approach overcomes the existing gaps and emphasizes the importance of managing heterogeneous geospatial emergency data for classification purposes. It also highlights the need to enhance an interoperable knowledge base method for post-disaster temporary responses. By combining geomatic tools with architectural studies, these visualization techniques can support national and local organizations responsible for post-earthquake management through a 3D modelling method to aid future transformations or interventions following other natural disasters

Probabilistic techniques in semantic mapping for mobile robotics

Los mapas semánticos son representaciones del mundo que permiten a un robot entender no sólo los aspectos espaciales de su lugar de trabajo, sino también el significado de sus elementos (objetos, habitaciones, etc.) y como los humanos interactúan con ellos (e.g. funcionalidades, eventos y relaciones). Para conseguirlo, un mapa semántico añade a las representaciones puramente espaciales, tales como mapas geométricos o topológicos, meta-información sobre los tipos de elementos y relaciones que pueden encontrarse en el entorno de trabajo. Esta meta-información, denominada conocimiento semántico o de sentido común, se codifica típicamente en Bases de Conocimiento. Un ejemplo de este tipo de información podría ser: "los frigoríficos son objetos grandes, con forma rectangular, colocados normalmente en las cocinas, y que pueden contener comida perecedera y medicación". Codificar y manejar este conocimiento semántico permite al robot razonar acerca de la información obtenida de un cierto lugar de trabajo, así como inferir nueva información con el fin de ejecutar eficientemente tareas de alto nivel como "¡hola robot! llévale la medicación a la abuela, por favor". La presente tesis propone la utilización de técnicas probabilísticas para construir y mantener mapas semánticos, lo cual presenta tres ventajas principales en comparación con los enfoques tradicionales: i) permite manejar incertidumbre (proveniente de los sensores imprecisos del robot y de los modelos empleados), ii) provee representaciones del entorno coherentes por medio del aprovechamiento de las relaciones contextuales entre los elementos observados (e.g. los frigoríficos usualmente se encuentran en las cocinas) desde un punto de vista holístico, y iii) produce valores de certidumbre que reflejan el grado de exactitud de la comprensión del robot acerca de su entorno. Específicamente, las contribuciones presentadas pueden agruparse en dos temas principales. El primer conjunto de contribuciones se basa en el problema del reconocimiento de objetos y/o habitaciones, ya que los sistemas de mapeo semántico deben contar con algoritmos de reconocimiento fiables para la construcción de representaciones válidas. Para ello se ha explorado la utilización de Modelos Gráficos Probabilísticos (Probabilistic Graphical Models o PGMs en inglés) con el fin de aprovechar las relaciones de contexto entre objetos y/o habitaciones a la vez que se maneja la incertidumbre inherente al problema de reconocimiento, y el empleo de Bases de Conocimiento para mejorar su desempeño de distintos modos, e.g., detectando resultados incoherentes, proveyendo información a priori, reduciendo la complejidad de los algoritmos de inferencia probabilística, generando ejemplos de entrenamiento sintéticos, habilitando el aprendizaje a partir de experiencias pasadas, etc. El segundo grupo de contribuciones acomoda los resultados probabilísticos provenientes de los algoritmos de reconocimiento desarrollados en una nueva representación semántica, denominada Multiversal Semantic Map (MvSmap). Este mapa gestiona múltiples interpretaciones del espacio de trabajo del robot, llamadas universos, los cuales son anotados con la probabilidad de ser los correctos de acuerdo con el conocimiento actual del robot. Así, este enfoque proporciona una creencia fundamentada sobre la exactitud de la comprensión del robot sobre su entorno, lo que le permite operar de una manera más eficiente y coherente. Los algoritmos probabilísticos propuestos han sido testeados concienzudamente y comparados con otros enfoques actuales e innovadores empleando conjuntos de datos del estado del arte. De manera adicional, esta tesis también contribuye con dos conjuntos de datos, UMA-Offices and Robot@Home, los cuales contienen información sensorial capturada en distintos entornos de oficinas y casas, así como dos herramientas software, la librería Undirected Probabilistic Graphical Models in C++ (UPGMpp), y el conjunto de herramientas Object Labeling Toolkit (OLT), para el trabajo con Modelos Gráficos Probabilísticos y el procesamiento de conjuntos de datos respectivamente

Review of the “ as-buit BIM ” approaches

Today, we need 3D models of heritage buildings in order to handle more efficiently projects of restoration, documentation and maintenance. In this context, developing a performing approach, based on a first phase of building survey, is a necessary step in order to build a semantically enriched digital model. For this purpose, the Building Information Modeling is an efficient tool for storing and exchanging knowledge about buildings. In order to create such a model, there are three fundamental steps: acquisition, segmentation and modeling. For these reasons, it is essential to understand and analyze this entire chain that leads to a well- structured and enriched 3D digital model. This paper proposes a survey and an analysis of the existing approaches on these topics and tries to define a new approach of semantic structuring taking into account the complexity of this chain

Methodology for the generation of 3D city models and integration of HBIM models in GIS: Case studies

[EN] The Architecture, Engineering and Construction (AEC) industry increasingly demands the availability of semantic and interactive digital models with the environment, capable of simulating decision-making during its life cycle and representing the results achieved. This motivates the need to develop models that integrate spatial information (GIS) and construction information (HBIM), favouring the achievement of the Smart City and Digital Twin concepts. GIS & HBIM platform is a useful tool, with potential applications in the world of built heritage; but it still has certain inefficiencies related to interoperability, the semantics of the formats and the geometry of the models. The objective of this contribution is to suggest a procedure for the generation of 3D visualization models of existing cities by integrating HBIM models in GIS environments. For this, three software and two types of data sources (existing plans and point cloud) are used. The methodology is tested in four locations of different dimensions, managing to identify the advantages/disadvantages of each application.Carrasco, CA.; Lombillo, I.; Sánchez-Espeso, J. (2022). Methodology for the generation of 3D city models and integration of HBIM models in GIS: Case studies. VITRUVIO - International Journal of Architectural Technology and Sustainability. 7(2):74-87. https://doi.org/10.4995/vitruvioijats.2022.1880874877