Pseudo-Labeling Approach for Land Cover Classification Through Remote Sensing Observations With Noisy Labels

Satellite data allows us to solve a wide range of challenging tasks remotely, including monitoring changing environmental conditions, assessing resources, and evaluating hazards. Computer vision algorithms such as convolutional neural networks have proven to be powerful tools for handling huge visual datasets. Although the number of satellite imagery is constantly growing and artificial intelligence is advancing, the present sticking point in remote sensing studies is the quality and amount of annotated datasets. Typically, manual labels have particular uncertainties and mismatches. Also, a lot of annotated datasets available in low resolution. Available visual representation of the observed objects can be more detailed than annotation. This causes the need for markup adjustment, which can be referred to as a pseudo-labeling task. The main contribution of this research is that we propose a pipeline for pseudo-labeling to address the problem of inaccurate and low-resolution markup improvement for solving land-cover and land-use segmentation task based on the data from the Sentinel-2 satellite. Our methodology takes advantages both of classical machine learning (ML) and deep learning (DL) algorithms. We examine random sampling, uniform sampling, and K-Means sampling and compare it with the full dataset usage. U-Net, DeepLab, and FPN models are trained on the adjusted dataset. The achieved findings show that a simple yet effective approach of data preliminary sampling and further markup refinement leads to significantly higher results than just using raw inaccurate data in a deep neural network pipeline. Moreover, the considered sampling technique allows to use less data for ML model training. The experiments involve markup adjustment and up-scaling from 30m to 10m. We verify the proposed approach in precise test area with manual annotation and show the improvement in F1-score from 0.792 to 0.816

Generation of the NIR Spectral Band for Satellite Images with Convolutional Neural Networks

The near-infrared (NIR) spectral range (from 780 to 2500 nm) of the multispectral remote sensing imagery provides vital information for landcover classification, especially concerning vegetation assessment. Despite the usefulness of NIR, it does not always accomplish common RGB. Modern achievements in image processing via deep neural networks make it possible to generate artificial spectral information, for example, to solve the image colorization problem. In this research, we aim to investigate whether this approach can produce not only visually similar images but also an artificial spectral band that can improve the performance of computer vision algorithms for solving remote sensing tasks. We study the use of a generative adversarial network (GAN) approach in the task of the NIR band generation using only RGB channels of high-resolution satellite imagery. We evaluate the impact of a generated channel on the model performance to solve the forest segmentation task. Our results show an increase in model accuracy when using generated NIR compared to the baseline model, which uses only RGB (0.947 and 0.914 F1-scores, respectively). The presented study shows the advantages of generating the extra band such as the opportunity to reduce the required amount of labeled data

Influence of k-casein genotype on heat stability and cheese making properties of milk powder

U ovom istraživanju je ispitivan učinak polimorfizma gena za k-kazein na tehnološke karakteristike mlijeka u prahu dobivenog sušenjem raspršivanjem (AA1: BB1) i liofilizacijom (AA2: BB2). Korištene su standardne i općeprihvaćene metode fizikalno-kemijskih analiza mliječnih proizvoda, kao i metode za procjenu toplinske stabilnosti i prikladnosti mlijeka za proizvodnju sira. Najveću stabilnost naspram toplinskih tretmana pokazali su uzorci mlijeka u kojima je prevladavalo mlijeko krava s genotipom AA CSN3, pri pH od 6,4 do 7,0 (36-91 minuta za AA1: BB1 i 37-101 minuta za AA2: BB2). U modelnim sustavima s 25 % do 100 % mlijeka krava s genotipom AA CSN3 dobivenim liofilizacijom utvrđena je veća (za 3-10 %) termostabilnost proteina u rasponu pH od 6,4 do 7,0 u usporedbi s uzorkom dobivenim sušenjem raspršivanjem. Analiza rezultata vezanih uz prikladnost mlijeka za proizvodnju sira pokazala je da se s porastom udjela mlijeka krava s genotipom BB CSN3 od 0 % do 100 % u modelnim sustavima, trajanje koagulacije sirilom smanjuje za sve uzorke, bez obzira na metodu sušenja mlijeka. Utvrđeno je da su korištenjem liofilizacije koagulumi svih uzoraka svrstani u najvišu kvalitetu mlijeka u pogledu prikladnosti za proizvodnju sira, dok su kod uzoraka dobivenih sušenja raspršivanjem u ovu kategoriju bili svrstani samo oni koji se sastoje od min. 75 % mlijeka dobivenog od krava s genotipom BB CSN3.In this study the effect of k-casein gene polymorphism on the technological characteristics of milk powder obtained by spray (AA1:BB1) and freeze drying (AA2:BB2) was investigated. Standardized and generally accepted methods were used in the field of physical and chemical control of dairy products, as well as methods for assessing the heat stability and cheese making properties of milk. The most heat-resistant were the samples with a predominance of milk obtained from cows with the AA CSN3 genotype, in the pH range from 6.4 to 7.0 (36-91 minutes for AA1:BB1 and 37-101 minutes for AA2:BB2). In systems with a fraction of 25% to 100% of milk from cows with the AA CSN3 genotype obtained by freeze drying, higher (by 3-10%) stabilization qualities of protein were revealed when heated in the pH range from 6.4 to 7.0 compared to spray drying. The analysis of the results regarding the cheese making properties showed that with an increase in the proportion of milk from cows with the BB CSN3 genotype from 0% to 100% in model systems, the rennet coagulation time decreases for all samples, regardless of the drying method. It was also found that when using freeze drying, coagulum of all samples were assigned to the highest class of milk quality in terms of cheese making properties, while during spray drying only the samples consisting of min. 75% or completely of milk obtained from cows with genotype BB CSN3 corresponded to this category

Augmentation-Based Methodology for Enhancement of Trees Map Detalization on a Large Scale

Remote sensing tasks play a very important role in the domain of sensing and measuring, and can be very specific. Advances in computer vision techniques allow for the extraction of various information from remote sensing satellite imagery. This information is crucial in making quantitative and qualitative assessments for monitoring of forest clearing in protected areas for power lines, as well as for environmental analysis, in particular for making assessments of carbon footprint, which is a highly relevant task. Solving these problems requires precise segmentation of the forest mask. Although forest mask extraction from satellite data has been considered previously, no open-access applications are able to provide the high-detailed forest mask. Detailed forest masks are usually obtained using unmanned aerial vehicles (UAV) that set particular limitations such as cost and inapplicability for vast territories. In this study, we propose a novel neural network-based approach for high-detailed forest mask creation. We implement an object-based augmentation technique for a minimum amount of labeled high-detailed data. Using this augmented data we fine-tune the models, which are trained on a large forest dataset with less precise labeled masks. The provided algorithm is tested for multiple territories in Russia. The F1-score, for small details (such as individual trees) was improved to 0.929 compared to the baseline score of 0.856. The developed model is available in an SAAS platform. The developed model allows a detailed and precise forest mask to be easily created, which then be used for solving various applied problems

Augmentation-Based Methodology for Enhancement of Trees Map Detalization on a Large Scale

Remote sensing tasks play a very important role in the domain of sensing and measuring, and can be very specific. Advances in computer vision techniques allow for the extraction of various information from remote sensing satellite imagery. This information is crucial in making quantitative and qualitative assessments for monitoring of forest clearing in protected areas for power lines, as well as for environmental analysis, in particular for making assessments of carbon footprint, which is a highly relevant task. Solving these problems requires precise segmentation of the forest mask. Although forest mask extraction from satellite data has been considered previously, no open-access applications are able to provide the high-detailed forest mask. Detailed forest masks are usually obtained using unmanned aerial vehicles (UAV) that set particular limitations such as cost and inapplicability for vast territories. In this study, we propose a novel neural network-based approach for high-detailed forest mask creation. We implement an object-based augmentation technique for a minimum amount of labeled high-detailed data. Using this augmented data we fine-tune the models, which are trained on a large forest dataset with less precise labeled masks. The provided algorithm is tested for multiple territories in Russia. The F1-score, for small details (such as individual trees) was improved to 0.929 compared to the baseline score of 0.856. The developed model is available in an SAAS platform. The developed model allows a detailed and precise forest mask to be easily created, which then be used for solving various applied problems

PseudoAugment: Enabling Smart Checkout Adoption for New Classes Without Human Annotation

Increasingly, automation helps to minimize human involvement in many mundane aspects of life, especially retail. During the pandemic it became clear that shop automation helps not only to reduce labor and speedup service but also to reduce the spread of disease. The recognition of produce that has no barcode remains among the processes that are complicated to automate. The ability to distinguish weighted goods is necessary to correctly bill a customer at a self checkout station. A computer vision system can be deployed on either smart scales or smart cash registers. Such a system needs to recognize all the varieties of fruits, vegetables, groats and other commodities which are available for purchase unpacked. The difficulty of this problem is in the diversity of goods and visual variability of items within the same category. Furthermore, the produce at a shop frequently changes between seasons as different varieties are introduced. In this work, we present a computer vision approach that allows efficient scaling for new goods classes without any manual image labelling. To the best of our knowledge, this is the first approach that allows a smart checkout system to recognize new items without manual labelling. We provide open access to the collected dataset in conjunction with our methods. The proposed method uses top-view images of a new class, applies a pseudo-labelling algorithm to crop the samples, and uses object-based augmentation to create training data for neural networks. We test this approach to classify five fruits varieties, and show that when the number of natural training images is below 50, the baseline pipeline result is almost random guess (20% for 5 classes). PseudoAugment can achieve over 92% accuracy with only top-view images that have no pixel-level annotations. The substantial advantage of our approach remains when the number of original training images is below 250. In practice, it means that when a new fruit is introduced in a shop, we need just a handful of top-view images of containers filled with a new class for the system to start operating. The PseudoAugment method is well-suited for continual learning as it can effectively handle an ever-expanding set of classes. Other computer vision problems can be also addressed using the suggested approach

Benchmark for Building Segmentation on Up-Scaled Sentinel-2 Imagery

Currently, we can solve a wide range of tasks using computer vision algorithms, which reduce manual labor and enable rapid analysis of the environment. The remote sensing domain provides vast amounts of satellite data, but it also poses challenges associated with processing this data. Baseline solutions with intermediate results are available for various tasks, such as forest species classification, infrastructure recognition, and emergency situation analysis using satellite data. Despite these advances, two major issues with high-performing artificial intelligence algorithms remain in the current decade. The first issue relates to the availability of data. To train a robust algorithm, a reasonable amount of well-annotated training data is required. The second issue is the availability of satellite data, which is another concern. Even though there are a number of data providers, high-resolution and up-to-date imagery is extremely expensive. This paper aims to address these challenges by proposing an effective pipeline for building segmentation that utilizes freely available Sentinel-2 data with 10 m spatial resolution. The approach we use combines a super-resolution (SR) component with a semantic segmentation component. As a result, we simultaneously consider and analyze SR and building segmentation tasks to improve the quality of the infrastructure analysis through medium-resolution satellite data. Additionally, we collected and made available a unique dataset for the Russian Federation covering area of 1091.2 square kilometers. The dataset provides Sentinel-2 imagery adjusted to the spatial resolution of 2.5 m and is accompanied by semantic segmentation masks. The building footprints were created using OpenStreetMap data that was manually checked and verified. Several experiments were conducted for the SR task, using advanced image SR methods such as the diffusion-based SR3 model, RCAN, SRGAN, and MCGR. The MCGR network produced the best result, with a PSNR of 27.54 and SSIM of 0.79. The obtained SR images were then used to tackle the building segmentation task with different neural network models, including DeepLabV3 with different encoders, SWIN, and Twins transformers. The SWIN transformer achieved the best results, with an F1-score of 79.60

Wildfire spreading prediction using multimodal data and deep neural network approach

Abstract Predicting wildfire spread behavior is an extremely important task for many countries. On a small scale, it is possible to ensure constant monitoring of the natural landscape through ground means. However, on the scale of large countries, this becomes practically impossible due to remote and vast forest territories. The most promising source of data in this case that can provide global monitoring is remote sensing data. Currently, the main challenge is the development of an effective pipeline that combines geospatial data collection and the application of advanced machine learning algorithms. Most approaches focus on short-term fire spreading prediction and utilize data from unmanned aerial vehicles (UAVs) for this purpose. In this study, we address the challenge of predicting fire spread on a large scale and consider a forecasting horizon ranging from 1 to 5 days. We train a neural network model based on the MA-Net architecture to predict wildfire spread based on environmental and climate data, taking into account spatial distribution features. Estimating the importance of features is another critical issue in fire behavior prediction, so we analyze their contribution to the model’s results. According to the experimental results, the most significant features are wind direction and land cover parameters. The F1-score for the predicted burned area varies from 0.64 to 0.68 depending on the day of prediction (from 1 to 5 days). The study was conducted in northern Russian regions and shows promise for further transfer and adaptation to other regions. This geospatial data-based artificial intelligence (AI) approach can be beneficial for supporting emergency systems and facilitating rapid decision-making