Ten years of active learning techniques and object detection: a systematic review

Object detection (OD) coupled with active learning (AL) has emerged as a powerful synergy in the field of computer vision, harnessing the capabilities of machine learning (ML) to automatically identify and perform image-based objects localisation while actively engaging human expertise to iteratively enhance model performance and foster machine-based knowledge expansion. Their prior success, demonstrated in a wide range of fields (e.g., industry and medicine), motivated this work, in which a comprehensive and systematic review of OD and AL techniques was carried out, considering reputed technical/scientific publication databases—such as ScienceDirect, IEEE, PubMed, and arXiv—and a temporal range between 2010 and December 2022. The primary inclusion criterion for papers in this review was the application of AL techniques for OD tasks, regardless of the field of application. A total of 852 articles were analysed, and 60 articles were included after full screening. Among the remaining ones, relevant topics such as AL sampling strategies used for OD tasks and groups categorisation can be found, along with details regarding the deep neural network architectures employed, application domains, and approaches used to blend learning techniques with those sampling strategies. Furthermore, an analysis of the geographical distribution of OD researchers across the globe and their affiliated organisations was conducted, providing a comprehensive overview of the research landscape in this field. Finally, promising research opportunities to enhance the AL process were identified, including the development of novel sampling strategies and their integration with different learning techniques.This research was funded by the RRP- Recovery and Resilience Plan and the European NextGeneration EU Funds, within the scope of the Mobilizing Agendas for Business Innovation, under reference C644937233-00000047 and by the Vine&Wine Portugal Project, co-financed by the RRP- Recovery and Resilience Plan and the European NextGeneration EU Funds, within the scope of the Mobilizing Agendas for Reindustrialization, under reference C644866286-00000011

Defect detection in the textile industry using image-based machine learning methods: A brief review

Traditionally, computer vision solutions for detecting elements of interest (e.g., defects) are based on strict context-sensitive implementations to address contained problems with a set of well-defined conditions. On the other hand, several machine learning approaches have proven their generalization capacity, not only to improve classification continuously, but also to learn from new examples, based on a fundamental aspect: the separation of data from the algorithmic setup. The findings regarding backward-propagation and the progresses built upon graphical cards technologies boost the advances in machine learning towards a subfield known as deep learning that is becoming very popular among many industrial areas, due to its even greater robustness and flexibility to map and deal knowledge that is typically handled by humans, with, also, incredible scalability proneness. Fabric defect detection is one of the manual processes that has been progressively automatized resorting to the aforementioned approaches, as it is an essential process for quality control. The goal is manifold: reduce human error, fatigue, ergonomic issues and associated costs, while simultaneously improving the expeditiousness and preciseness of the involved tasks, with a direct impact on profit. Following such research line with a specific focus in the textile industry, this work aims to constitute a brief review of both defect types and Automated Optical Inspection (AOI) mostly based on machine learning techniques, which have been proving their effectiveness in identifying anomalies within the context of textile material analysis. The inclusion of Convolutional Neural Network (CNN) based on known architectures such as AlexNet or Visual Geometry Group (VGG16) on computerized defect analysis allowed to reach accuracies over 98%. A short discussion is also provided along with an analysis of the current state characterizing this field of intervention, as well as some future challenges.ERDF - European Regional Development Fund(undefined

Using deep learning to detect the presence/absence of defects on leather: On the way to build an industry-driven approach

In textile/leather manufacturing environments, as in many other industrial contexts, quality inspection is an essential activity that is commonly performed by human operators. Error, fatigue, ergonomic issues, and related costs associated to this fashion of carrying out fabric validation are aspects concerning companies' strategists, whose mission includes to watch over the physical integrity of their employees, while aiming at enhanced quality control methods implementation towards profit maximization. Considering these challenges from a technical/scientific perspective, machine/deep learning approaches have been showing great skills in adapting a wide range of contexts and, in particular, industrial environments, complementing traditional computer vision methods with characteristics such as increased accuracy while dealing with image classification and segmentation problems, capacity for continuous learning from experts input and feedback, flexibility to easily scale training for new contextualization classes – unknown types of occurrences relevant to characterize a given problem –, among other advantages. The goal of crossing deep learning strategies with fabric inspection processes is pursued in this paper. After providing a brief but representative characterization of the targeted industrial context, in which, typically, fabric rolls of rawmaterial mats must be processed at a relatively low latency, an Automatic Optical Inspection (AOI) system architecture designed for such environments is revisited [1], for contextualization purposes. Afterwards, a set of deep learning-oriented training methods/processes is proposed in combination with neural networks built based on Xception architecture, towards the implementation of one of the components that integrate the aforementioned system, from which is expected the identification of presence/absence of defective textile/leather raw material at a low-latency. Several models powered by Xception were trained with different tunning parameters, resorting to datasets variations that were set up from raw images of leather, following different annotation strategies (meticulous and rough). The model that performed better reached 96% of accuracy.ERDF - European Regional Development Fund(undefined

Multi-temporal analysis of forestry and coastal environments using UASs

Due to strong improvements and developments achieved in the last decade, it is clear that applied research using remote sensing technology such as unmanned aerial vehicles (UAVs) can provide a flexible, efficient, non-destructive, and non-invasive means of acquiring geoscientific data, especially aerial imagery. Simultaneously, there has been an exponential increase in the development of sensors and instruments that can be installed in UAV platforms. By combining the aforementioned factors, unmanned aerial system (UAS) setups composed of UAVs, sensors, and ground control stations, have been increasingly used for remote sensing applications, with growing potential and abilities. This paper's overall goal is to identify advantages and challenges related to the use of UAVs for aerial imagery acquisition in forestry and coastal environments for preservation/prevention contexts. Moreover, the importance of monitoring these environments over time will be demonstrated. To achieve these goals, two case studies using UASs were conducted. The first focuses on phytosanitary problem detection and monitoring of chestnut tree health (Padrela region, Valpaços, Portugal). The acquired high-resolution imagery allowed for the identification of tree canopy cover decline by means of multi-temporal analysis. The second case study enabled the rigorous and non-evasive registry process of topographic changes that occurred in the sandspit of Cabedelo (Douro estuary, Porto, Portugal) in different time periods. The obtained results allow us to conclude that the UAS constitutes a low-cost, rigorous, and fairly autonomous form of remote sensing technology, capable of covering large geographical areas and acquiring high precision data to aid decision support systems in forestry preservation and coastal monitoring applications. Its swift evolution makes it a potential big player in remote sensing technologies today and in the near future.info:eu-repo/semantics/publishedVersio

Polymerase chain reaction for soybean detection in heat processed meat products.

Since vegetable proteins are considerably cheaper than muscle proteins, they are frequently used as meat extenders in order to reduce the cost of the final product. Due to several interesting characteristics, soybean is reported to be the most widely used vegetable protein in the meat industry. Nevertheless, soybean is included in the group of 12 ingredients potentially allergenic, which should therefore be labelled according to the Codex Alimentarius FAO/WHO and the European Commission (Directive 2003/89/EC). In fact, it has been described that amounts of soy bellow 0.1% and 1% (w/w) can lead to allergic reactions in sensitive consumers (1)

A ubiquitous service-oriented automatic optical inspection platform for textile industry

Within a highly competitive market context, quality standards are vital for the textile industry, in which related procedures to assess respective manufacture still mainly rely on human-based visual inspection. Thereby, factors such as ergonomics, analytical subjectivity, tiredness and error susceptibility affect the employee's performance and comfort in particular and impact the economic healthiness of each company operating in this industry, generally. In this paper, a defect detection-oriented platform for quality control in the textile industry is proposed to tackle these issues and respective impacts, combining computer vision, deep learning, geolocation and communication technologies. The system under development can integrate and improve the production ecosystem of a textile company through a properly adapted information technology setup and associated functionalities such as automatic defect detection and classification, real-time monitoring of operators, among others.This work was financed by the project “Smart Production Process” (No. POCI-01-0247-FEDER-045366), supported under the Incentive System for Research and Technological Development - Business R&DT (Individual Projects)

Ontology-based procedural modelling of traversable buildings composed by arbitrary shapes

Tese de Doutoramento em InformáticaModelos virtuais 3D de edifícios são usualmente utilizados em áreas como a arquitetura e videojogos para fins de visualização de projetos de casas e povoamento de cenários virtuais, respetivamente. Tradicionalmente, a produção destes modelos requer mão-de-obra altamente especializada e consideráveis quantidades de tempo. Para abordar esta questão, muitos investigadores desenvolveram técnicas semiautomáticas para produzir modelos virtuais de forma expedita. Estas técnicas procedimentais providenciam diferentes formas de geração de edifícios, incluindo interiores e fachadas exteriores, que servem vários propósitos (por exemplo, geração de conteúdo para videojogos ou reconstruções arqueológicas). No entanto, as técnicas existentes com foco na construção de interiores normalmente só suportam a geração de plantas restritas por formas regulares ou polígonos de contorno obtidos a partir de conjuntos de retângulos. Ao mesmo tempo, a possibilidade de modelar quartos interiores através da especificação das suas paredes de restrição permanece pouco explorada. Além disso, a maioria das soluções de geração procedimental existentes recorrem a gramáticas complexas referentes aos aspetos geométricos, ou então, estruturas semânticas desenvolvidas para projetos com requisitos específicos, desconsiderando os standards desenvolvidos para ambientes urbanos virtuais, concretamente, CityGML. No sentido de abordar as questões indicadas, uma nova metodologia de modelação procedimental é proposta nesta tese, tendo como foco a produção de modelos virtuais de edifícios, incluindo exteriores circunscritos por formas arbitrárias e interiores formados por polígonos convexos. A regulação da metodologia é fornecida por uma ontologia para edifícios - que pode ser vista como um conjunto relacional de entidades baseadas em CityGML, extensíveis a estilos arquitetónicos específicos - através de várias estruturas de dados, tais como XML estruturado e gramática baseada na ontologia. Relativamente ao processo de suporte da repartição de espaço, uma abordagem treemap é usada para subdividir o layout representativo de uma dada base de edifício em subáreas inerentes a contentores e quartos interiores. Durante o desenvolvimento deste trabalho, diversas melhorias foram feitas ao treemap de forma progressiva, com o objetivo de permitir a subdivisão de diferentes tipos de polígonos de restrição que variam entre retângulos a formas arbitrárias. Além disso, na fase final deste trabalho, foi incorporado um método para a adaptação de paredes de quartos. Na sequência da subdivisão, vem um conjunto de operações que vai desde a marcação das transições até à extrusão das paredes que fornece o aspeto 3D. Também, uma abordagem estocástica experimental é proposta para automatizar a geração aleatória de edifícios, utilizando esta metodologia de modelação procedimental. Um conjunto de testes foi feito para demonstrar as capacidades da metodologia proposta na produção de edifícios com formatos distintos (edifícios limitados por formas convexas e não-convexas e quartos com um número específico de paredes de restrição) e diferentes estruturas arquitetónicas (casas de propósito geral, domus romanas) em curtos períodos de tempo. Além disso, a eficácia da abordagem treemap na subdivisão de layouts é mostrada, juntamente com um processo estocástico experimental para a geração automática de edifícios e também algumas medições de desempenho computacional.3D virtual models of buildings are commonly used in areas such as architecture and video games to preview a house project and to populate a virtual scenario, respectively. Traditionally, the production of these models requires highly skilled manpower and a considerable amount of time. To address this issue, many researchers have developed semi-automatic techniques to produce virtual models expeditiously. These procedural techniques provide different ways of generating buildings, including interiors and outer facades, to serve several purposes (e.g., content generation for video games or archaeological reconstruction). However, the existing techniques focusing on building interiors usually only support the generation of floor plans constrained by regular shapes or contour polygons obtained from rectangles sets. At the same time, the possibility of modelling interior rooms through the specification of its constraint walls remains poorly explored. Moreover, most of the existing procedural generation solutions are guided by complex grammars concerned with geometrical aspects or semantic structures that fit specific project requirements, apparently disregarding the established standards for virtual urban environments, specifically, CityGML. To overcome the noted issues, a novel procedural modelling methodology is proposed in this thesis, one that produces virtual models of buildings, including exteriors outlined by arbitrary shapes and interiors formed by convex polygons. Methodology’s regulation is provided by a building ontology - a CityGML-based knowledge structure, planned to be extensible to specific architecture styles - through several guiding data structures such as structured XML and ontology-based grammar. Regarding the supporting process, a treemap approach is used to subdivide the building layout into floor plan areas. During the development of this work, several improvements were progressively made to the treemap in order to enable the subdivision of different constraint polygon types which range from rectangles to arbitrary shapes. Moreover, in the most mature work stage, a method concerning inner room walls adaptation is addressed. Next, a set of operations is performed, from the marking transitions step to the extrusion process that provides the 3D aspect. In addition, an experimental stochastic approach is proposed to automate the production of random buildings using this procedural modelling methodology. A set of tests was made to demonstrate the capabilities of the proposed methodology in producing distinct building formats (buildings constrained by convex and nonconvex shapes, houses with specific room constraint walls) and different architectonic structures (general purpose houses, roman domus) in short time periods. Moreover, the effectiveness of the treemap approach in subdividing random layouts is shown, along with a generic stochastic process for automatic building generation and also some computational performance measurements

Reconstructing the past: providing an enhanced perceptual experience

Accurate modeling/reconstruction and visualization of real environments, particularly archaeological sites, is both a major challenge and a crucial task. This work will address the entire process of the virtual reconstruction of archaeological sites, since the construction of the virtual model until its visualization. The chapter begins with an introduction to the process of virtual reconstruction of archaeological sites, where the several stages that should take place to obtain a faithful virtual representation of an archaeological site and its artifacts are identified. Moreover, each stage is characterized and its main methods and techniques are identified, in dedicated sections. The authors' contribution for the state of the art will be highlighted in each stage. The chapter ends with the authors' vision about future trends for this field and unveils what could be their contributions to this vision.info:eu-repo/semantics/publishedVersio

Ontology-based procedural modelling of traversable buildings composed by arbitrary shapes

This book presents a new procedural modelling methodology capable of producing traversable buildings constrained by arbitrary convex shapes, based on a pure treemap approach. The authors establish a process to change the format of interior rooms, through wall number modification and offer an adaptation of a “fake-concave” technique to support non-convex building layouts. It will also include: • A proposal for an extensible building ontology to guide the methodology process and support the generation of other architectural style buildings (e.g. roman houses); • A presentation of an ontology-based grammar to provide the procedural modelling methodology with production rules; • Experimental computer managed processes for the stochastic generation of buildings. Most of the existing solutions regarding building interiors only focus on the generation of floor plans mainly composed of rectangular shapes. Yet there are a wide variety of ancient and contemporary buildings that are composed of shapes other than rectangles, both internally and externally. Ontology-based Procedural Modelling of Traversable Buildings Composed by Arbitrary Shapes will address this by providing the Procedural Modelling field with processes and techniques capable of properly supporting for example, digital preservation of cultural heritage or extensive virtual urban environment productions, specifically ones involving the generation/reconstruction of virtual buildings with such geometric requirements.info:eu-repo/semantics/publishedVersio

Individual Grapevine Analysis in a Multi-Temporal Context Using UAV-Based Multi-Sensor Imagery

The use of unmanned aerial vehicles (UAVs) for remote sensing applications in precision viticulture significantly increased in the last years. UAVs’ capability to acquire high spatiotemporal resolution and georeferenced imagery from different sensors make them a powerful tool for a better understanding of vineyard spatial and multitemporal heterogeneity, allowing the estimation of parameters directly impacting plants’ health status. In this way, the decision support process in precision viticulture can be greatly improved. However, despite the proliferation of these innovative technologies in viticulture, most of the published studies rely only on data from a single sensor in order to achieve a specific goal and/or in a single/small period of the vineyard development. In order to address these limitations and fully exploit the advantages offered by the use of UAVs, this study explores the multi-temporal analysis of vineyard plots at a grapevine scale using different imagery sensors. Individual grapevine detection enables the estimation of biophysical and geometrical parameters, as well as missing grapevine plants. A validation procedure was carried out in six vineyard plots focusing on the detected number of grapevines and missing grapevines. A high overall agreement was obtained concerning the number of grapevines present in each row (99.8%), as well as in the individual grapevine identification (mean overall accuracy of 97.5%). Aerial surveys were conducted in two vineyard plots at different growth stages, being acquired for RGB, multispectral and thermal imagery. Moreover, the extracted individual grapevine parameters enabled us to assess the vineyard variability in a given epoch and to monitor its multi-temporal evolution. This type of analysis is critical for precision viticulture, constituting as a tool to significantly support the decision-making process