Individual Tree Detection in Large-Scale Urban Environments using High-Resolution Multispectral Imagery

We introduce a novel deep learning method for detection of individual trees in urban environments using high-resolution multispectral aerial imagery. We use a convolutional neural network to regress a confidence map indicating the locations of individual trees, which are localized using a peak finding algorithm. Our method provides complete spatial coverage by detecting trees in both public and private spaces, and can scale to very large areas. We performed a thorough evaluation of our method, supported by a new dataset of over 1,500 images and almost 100,000 tree annotations, covering eight cities, six climate zones, and three image capture years. We trained our model on data from Southern California, and achieved a precision of 73.6% and recall of 73.3% using test data from this region. We generally observed similar precision and slightly lower recall when extrapolating to other California climate zones and image capture dates. We used our method to produce a map of trees in the entire urban forest of California, and estimated the total number of urban trees in California to be about 43.5 million. Our study indicates the potential for deep learning methods to support future urban forestry studies at unprecedented scales

Automatic Large Scale Detection of Red Palm Weevil Infestation using Aerial and Street View Images

The spread of the Red Palm Weevil has dramatically affected date growers, homeowners and governments, forcing them to deal with a constant threat to their palm trees. Early detection of palm tree infestation has been proven to be critical in order to allow treatment that may save trees from irreversible damage, and is most commonly performed by local physical access for individual tree monitoring. Here, we present a novel method for surveillance of Red Palm Weevil infested palm trees utilizing state-of-the-art deep learning algorithms, with aerial and street-level imagery data. To detect infested palm trees we analyzed over 100,000 aerial and street-images, mapping the location of palm trees in urban areas. Using this procedure, we discovered and verified infested palm trees at various locations

Big data driven detection of trees in suburban scenes using visual spectrum eye level photography

The aim of the work described in this paper is to detect trees in eye level view images. Unlike previous work that universally considers highly constrained environments, such as natural parks and wooded areas, or simple scenes with little clutter and clear tree separation, our focus is on much more challenging suburban scenes, which are rich in clutter and highly variable in type and appearance (houses, falls, shrubs, cars, bicycles, pedestrians, hydrants, lamp posts, etc.). Thus, we motivate and introduce three different approaches: (i) a conventional computer vision based approach, employing manually engineered steps and making use of explicit human knowledge of the application domain, (ii) a more machine learning oriented approach, which learns from densely extracted local features in the form of scale invariant features (SIFT), and (iii) a machine learning based approach, which employs both colour and appearance models as a means of making the most of available discriminative information. We also make a significant contribution in regards to the collection of training and evaluation data. In contrast to the existing work, which relies on manual data collection (thus risking unintended bias) or corpora constrained in variability and limited in size (thus not allowing for reliable generalisation inferences to be made), we show how large amounts of representative data can be collected automatically using freely available tools, such as Google’s Street View, and equally automatically processed to produce a large corpus of minimally biased imagery. Using a large data set collected in the manner and comprising tens of thousands of images, we confirm our theoretical arguments that motivated our machine learning based and colour-aware histograms of oriented gradients based method, which achieved a recall of 95% and precision of 97%.Publisher PDFPeer reviewe

Geocoding of trees from street addresses and street-level images

We introduce an approach for updating older tree inventories with geographic coordinates using street-level panorama images and a global optimization framework for tree instance matching. Geolocations of trees in inventories until the early 2000s where recorded using street addresses whereas newer inventories use GPS. Our method retrofits older inventories with geographic coordinates to allow connecting them with newer inventories to facilitate long-term studies on tree mortality etc. What makes this problem challenging is the different number of trees per street address, the heterogeneous appearance of different tree instances in the images, ambiguous tree positions if viewed from multiple images and occlusions. To solve this assignment problem, we (i) detect trees in Google street-view panoramas using deep learning, (ii) combine multi-view detections per tree into a single representation, (iii) and match detected trees with given trees per street address with a global optimization approach. Experiments for trees in 5 cities in California, USA, show that we are able to assign geographic coordinates to 38% of the street trees, which is a good starting point for long-term studies on the ecosystem services value of street trees at large scale

Cataloging Public Objects Using Aerial and Street-Level Images – Urban Trees

Each corner of the inhabited world is imaged from multiple viewpoints with increasing frequency. Online map services like Google Maps or Here Maps provide direct access to huge amounts of densely sampled, georeferenced images from street view and aerial perspective. There is an opportunity to design computer vision systems that will help us search, catalog and monitor public infrastructure, buildings and artifacts. We explore the architecture and feasibility of such a system. The main technical challenge is combining test time information from multiple views of each geographic location (e.g., aerial and street views). We implement two modules: det2geo, which detects the set of locations of objects belonging to a given category, and geo2cat, which computes the fine-grained category of the object at a given location. We introduce a solution that adapts state-of-the-art CNN-based object detectors and classifiers. We test our method on “Pasadena Urban Trees”, a new dataset of 80,000 trees with geographic and species annotations, and show that combining multiple views significantly improves both tree detection and tree species classification, rivaling human performance

Benchmarking Individual Tree Mapping with Sub-meter Imagery

There is a rising interest in mapping trees using satellite or aerial imagery, but there is no standardized evaluation protocol for comparing and enhancing methods. In dense canopy areas, the high variability of tree sizes and their spatial proximity makes it arduous to define the quality of the predictions. Concurrently, object-centric approaches such as bounding box detection usuallyperform poorly on small and dense objects. It thus remains unclear what is the ideal framework for individual tree mapping, in regards to detection and segmentation approaches, convolutional neural networks and transformers. In this paper, we introduce an evaluation framework suited for individual tree mapping in any physical environment, with annotation costs and applicative goals in mind. We review and compare different approaches and deep architectures, and introduce a new method that we experimentally prove to be a good compromise between segmentation and detection

Individual Building Rooftop and Tree Crown Segmentation from High-Resolution Urban Aerial Optical Images

We segment buildings and trees from aerial photographs by using superpixels, and we estimate the tree’s parameters by using a cost function proposed in this paper. A method based on image complexity is proposed to refine superpixels boundaries. In order to classify buildings from ground and classify trees from grass, the salient feature vectors that include colors, Features from Accelerated Segment Test (FAST) corners, and Gabor edges are extracted from refined superpixels. The vectors are used to train the classifier based on Naive Bayes classifier. The trained classifier is used to classify refined superpixels as object or nonobject. The properties of a tree, including its locations and radius, are estimated by minimizing the cost function. The shadow is used to calculate the tree height using sun angle and the time when the image was taken. Our segmentation algorithm is compared with other two state-of-the-art segmentation algorithms, and the tree parameters obtained in this paper are compared to the ground truth data. Experiments show that the proposed method can segment trees and buildings appropriately, yielding higher precision and better recall rates, and the tree parameters are in good agreement with the ground truth data

Inferring landscape preferences from social media using data science techniques

People and societies attribute different values to landscapes, which are often derived from their preferences. Such preferences are shaped by aesthetics, recreational benefits, safety, and other services provided by landscapes. Researchers have found that more appealing landscapes can promote human health and well-being. Existing methods used to study landscape preferences, such as social surveys, create high quality data but have high cost of time and effort and are poorly suited to capture dynamic landscape-scale changes across large geographic scales. With the rapid rise in social media, a huge amount of user-generated data is now available for researchers to study emotions or sentiments (i.e., preferences) towards particular topics of interest. This dissertation investigates how social media data can be used to indirectly measure (Zanten et al., 2016) and learn features relevant to landscape preferences, focusing primarily on a specific landscape called green infrastructure (GI). The first phase of the work introduces a first-ever benchmark GI location dataset within the US (GReen Infrastructure Dataset, or GRID) and develops a computer vision algorithm for identifying GI from aerial images using Google/Bing Map API. The data collected from this object detection method is then used to re-train a human preference model developed previously (Rai, 2013) and it improved the prediction accuracy significantly. I found that with the framework introduced here, we can collect the landscape data, which is comparable to the current methods in terms of quality with much less efforts. Second phase uses GI images and textual comments from Flickr, Instagram, and Twitter to train a lexicon-based sentiment model for predicting people's sentiments for GI. Since almost 70 percent of US adults are using some social media platform to connect with their friends, families or to follow recent news and topic of interest (Pew research, 2015), it is imperative to understand whether people share, post, or comment about the landscape settings they live in or prefer. And the results show that social media information can be really useful in predicting people’s sentiments about landscape they live or visit. The third phase builds on the second phase to identify specific features that are correlated with higher and lower preferences. The findings demonstrate that we can learn features that impacts people’s preference about the landscape. These features are very descriptive that a layperson can understand and can also be useful for designers, storm-water engineers, city planners to incorporate in their landscape designs such that it improves human health and well- being. Finally, I will conclude and describe some follow up research that I think would be potential in understanding landscape: work on speeding up the object detection algorithms using more advanced computer vision methods and harnessing the power of GPUs and extension of the findings to other types of GI and landscape designs