Deep learning-based graffiti detection: A study using Images from the streets of Lisbon

This research work comes from a real problem from Lisbon City Council that was interested in developing a system that automatically detects in real-time illegal graffiti present throughout the city of Lisbon by using cars equipped with cameras. This system would allow a more efficient and faster identification and clean-up of the illegal graffiti constantly being produced, with a georeferenced position. We contribute also a city graffiti database to share among the scientific community. Images were provided and collected from different sources that included illegal graffiti, images with graffiti considered street art, and images without graffiti. A pipeline was then developed that, first, classifies the image with one of the following labels: illegal graffiti, street art, or no graffiti. Then, if it is illegal graffiti, another model was trained to detect the coordinates of graffiti on an image. Pre-processing, data augmentation, and transfer learning techniques were used to train the models. Regarding the classification model, an overall accuracy of 81.4% and F1-scores of 86%, 81%, and 66% were obtained for the classes of street art, illegal graffiti, and image without graffiti, respectively. As for the graffiti detection model, an Intersection over Union (IoU) of 70.3% was obtained for the test set.info:eu-repo/semantics/publishedVersio

Miehittämättömien ilma-alusten soveltuvuus kunnallisen ympäristövalvonnan työkaluksi

Kuntien ympäristöviranomaiset ovat velvoitettuja suorittamaan ympäristövalvontaa. Miehittämättömät ilma-alukset (droonit) voivat helpottaa ympäristövalvontaa mutta niiden soveltuvuutta kunnallisen ympäristövalvonnan työkaluksi ei ole tutkittu. Tässä työssä tarkasteltiin, miten kunnat ovat käyttäneet drooneja, ja testattiin droonien soveltuvuutta ympäristövalvontaan ja tarkastustyöhön roskittumisen seurantaa esimerkkinä käyttäen. Tutkimuksen ensimmäisessä osassa Suomen kuntien ympäristöviranomaisille, Ruotsin kunnille ja Eurocities WG Waste -ryhmään kuuluville kunnille (n = 512) lähetettiin kysely, jossa kysyttiin droonien käyttösovelluksia, käytön tiheyttä, onnistumisastetta, epäonnistumisten syitä ja tulevaisuuden suunnitelmia. Kyselyn tulokset analysoitiin kuvailevan tilastoanalyysin avulla. Tutkimuksen toisessa osassa droonia käytettiin roskamonitorointitutkimuksessa neljässä kohteessa Helsingissä. Otetuista droonikuvista laskettiin visuaalisen havainnoinnin avulla roskat kategorioittain ja lehdet. Droonikuvahavainnoinnin tarkkuutta arvioitiin vertaamalla havaittujen roskien lukumäärää maastossa tehtyyn roskien laskentaan. Yhdessä kohteessa droonikuvahavainnointia teki myös kontrolliryhmä. Sen tarkoitus oli mitata tulosten vääristymää, joka syntyy, kun sama yksilö suorittaa sekä maastotutkimukset että laskennat kuvista. Tulosten tilastolliseen analysointiin käytettiin Wilcoxonin merkittyjen sijalukujen testiä ja Cronbachin α -reliabiliteettitestiä. Kyselyn osallistumisprosentti oli alhainen, 3,7 % (n = 19). Käytettyjen sovellusten kirjo oli laaja ja painottui sovelluksiin, joissa droonia oletettavasti ohjataan manuaalisesti. Käyttö oli erittäin onnistunutta. Tärkeimmät epäonnistumisen syyt olivat säätekijät sekä tietotaidon puute. Droonit olivat osa valtaosan tulevaisuudensuunnitelmia. Roskamonitorointitutkimuksessa suoritettujen droonikuvahavainnointien tarkkuus maastotutkimukseen verrattuna oli 90,5 % vain roskat ja 87,5 % myös lehdet huomioiden, eivätkä droonikuvahavainnoinnit ja maastotutkimukset erinneet toisistaan tilastollisella merkitsevyydellä. Etenkin lehdet osoittautuivat haastaviksi havaita kuvista. Kontrolliryhmän havainnointitarkkuus verrattuna maastotutkimukseen oli 67,9 % vain roskat ja 49,0 % myös lehdet huomioiden, jolloin kontrolliryhmän ja maastotutkimuksen tulokset erosivat tilastollisella merkitsevyydellä (p = 0,028). Kontrolliryhmän sisäinen reliabiliteetti oli suhteellisen korkea, α = 0,776 ilman lehtiä ja α = 0,805 lehtien kanssa. Tulosten perusteella droonit ovat tarpeeksi tarkkoja ja sovelluksiltaan monipuolisia sopiakseen kunnallisten ympäristöviranomaisten valvonta- ja tarkastustyökaluiksi. Drooneilla on kyky täydentää maastokäyntien havaintoja tai tietyin edellytyksin jopa korvata ne itsenäisenä havainnointimetodina. Sovellusten ja havainnonititapojen kehitystyölle sekä jatkotutkimukselle droonien käytöstä kunnissa on lisätarvetta.Municipal environmental authorities are required to conduct environmental monitoring. Unmanned aerial vehicles, UAVs, may be helpful in environmental monitoring but their applicability as a tool for municipal environmental monitoring has not been studied. In this thesis it was studied, how municipalities have been utilizing UAVs. Additionally, UAVs applicability for environmental monitoring and inspection work was tested using a litter monitoring experiment as an example. In the first part of the study, a questionnaire was sent to municipal environmental authorities in Finland, to municipalities in Sweden and to those participating in Eurocities WG Waste group (n = 512), covering the used applications, their utilization frequencies and successfulness, reasons for failures and future plans. The results were analyzed using descriptive statistics. In the second part of the study, a UAV was utilized in a litter monitoring experiment on four sites in Helsinki. Litter by category and leaves were counted based on visual observations from UAV imagery. The accuracy of UAV imagery detection was assessed by comparing its and ground assessment (GA) results. On one site, a control group also carried out UAV imagery detections in order to assess the magnitude of bias or offset occurring when both the GA and the litter detection from UAV imagery are conducted by a single individual. The Wilcoxon signed rank and Cronbach’s α reliability tests were used for statistical analysis of the results. Response rate of the questionnaire was low, 3.7% (n = 19). The pool of used applications was extensive and covered a variety of monitoring and inspecting targets with emphasis on the presumably manually piloted applications. Utilization was very successful. The most important reasons for failures were poor weather followed by lack of information and expertise. UAVs were included in the future plans of most participants for municipal environmental monitoring purposes. The UAV imagery detection accuracies of litter and leaves compared to the GA results were high, 90.5% for litter and 87.5% for litter and leaves, and no statistically significant differences existed between the assessment results. Especially leaves proved challenging to detect from UAV imagery. The control group’s detection accuracies were 67.9% without and 49.0% with leaves, and with leaves the results differed with statistical significance (p = 0.028). The internal reliability of the control group was relatively high, α = 0.776 without and α = 0.805 with leaves. UAVs are deemed sufficiently accurate and versatile as monitoring and inspecting tools for municipal environmental authorities. They have the capability to complement ground assessments or, with certain prerequisites, even function as an independent monitoring method. Further application and detection method development and research on municipal UAV utilization are needed

Multimodal segmentation of lifelog data

A personal lifelog of visual and audio information can be very helpful as a human memory augmentation tool. The SenseCam, a passive wearable camera, used in conjunction with an iRiver MP3 audio recorder, will capture over 20,000 images and 100 hours of audio per week. If used constantly, very soon this would build up to a substantial collection of personal data. To gain real value from this collection it is important to automatically segment the data into meaningful units or activities. This paper investigates the optimal combination of data sources to segment personal data into such activities. 5 data sources were logged and processed to segment a collection of personal data, namely: image processing on captured SenseCam images; audio processing on captured iRiver audio data; and processing of the temperature, white light level, and accelerometer sensors onboard the SenseCam device. The results indicate that a combination of the image, light and accelerometer sensor data segments our collection of personal data better than a combination of all 5 data sources. The accelerometer sensor is good for detecting when the user moves to a new location, while the image and light sensors are good for detecting changes in wearer activity within the same location, as well as detecting when the wearer socially interacts with others

NIR Imagery-based Grass Fire Detection and Metrics Measurement using Small UAS

This thesis focuses on the generation of a new grass fire aerial image dataset and development of novel methods for near-infrared (NIR) imagery-based fire front identification and fire depth estimation using small unmanned aircraft systems (sUAS). The procedure for collection and creation of the Grass Fire Front and near-Infrared (NIR) and Thermal Imagery (GRAFFITI) dataset is introduced first including two levels of data: synced raw thermal and red, green and near-infrared (RGNIR) image pairs and processed image pairs of the same overlapping field-of-view. A novel NIR imagery-based fire detection and fire front identification algorithm is then proposed and validated against manually labeled ground truth, using the GRAFFITI dataset. A comparative study is further performed on the problem of grass fire front location and flame depth estimation using thermal and NIR imagery. Finally, recommendations are made to future researchers who are interested in wildland fire sensing using thermal or NIR imagery

Sign Language Fingerspelling Classification from Depth and Color Images using a Deep Belief Network

Automatic sign language recognition is an open problem that has received a lot of attention recently, not only because of its usefulness to signers, but also due to the numerous applications a sign classifier can have. In this article, we present a new feature extraction technique for hand pose recognition using depth and intensity images captured from a Microsoft Kinect sensor. We applied our technique to American Sign Language fingerspelling classification using a Deep Belief Network, for which our feature extraction technique is tailored. We evaluated our results on a multi-user data set with two scenarios: one with all known users and one with an unseen user. We achieved 99% recall and precision on the first, and 77% recall and 79% precision on the second. Our method is also capable of real-time sign classification and is adaptive to any environment or lightning intensity.Comment: Published in 2014 Canadian Conference on Computer and Robot Visio

A Practitioner’s Guide to Small Unmanned Aerial Systems for Bridge Inspection

Small unmanned aerial system(s) (sUAS) are rapidly emerging as a practical means of performing bridge inspections. Under the right condition, sUAS assisted inspections can be safer, faster, and less costly than manned inspections. Many Departments of Transportation in the United States are in the early stages of adopting this emerging technology. However, definitive guidelines for the selection of equipment for various types of bridge inspections or for the possible challenges during sUAS assisted inspections are absent. Given the large investments of time and capital associated with deploying a sUAS assisted bridge inspection program, a synthesis of authors experiences will be useful for technology transfer between academics and practitioners. In this paper, the authors list the challenges associated with sUAS assisted bridge inspection, discuss equipment and technology options suitable for mitigating these challenges, and present case studies for the application of sUAS to several specific bridge inspection scenarios. The authors provide information to sUAS designers and manufacturers who may be unaware of the specific challenges associated with sUAS assisted bridge inspection. As such, the information presented here may reveal the demands in the design of purpose-built sUAS inspection platforms

Technology and Economics, Inc. Technology Application Team

Technology + Economics, Inc. (T+E), under contract to the NASA Headquarters Technology Transfer Division, operates a Technology Applications Team (TATeam) to assist in the transfer of NASA-developed aerospace technology. T+E's specific areas of interest are selected urban needs at the local, county, and state levels. T+E contacts users and user agencies at the local, state, and county levels to assist in identifying significant urban needs amenable to potential applications of aerospace technology. Once viable urban needs have been identified in this manner, or through independent research, T+E searches the NASA technology database for technology and/or expertise applicable to the problem. Activities currently under way concerning potential aerospace applications are discussed

Assessment of plastics in the National Trust: a case study at Mr Straw's House

The National Trust is a charity that cares for over 300 publically accessible historic buildings and their contents across England, Wales and Northern Ireland. There have been few previous studies on preservation of plastics within National Trust collections, which form a significant part of the more modern collections of objects. This paper describes the design of an assessment system which was successfully trialled at Mr Straws House, a National Trust property in Worksop, UK. This system can now be used for future plastic surveys at other National Trust properties. In addition, the survey gave valuable information about the state of the collection, demonstrating that the plastics that are deteriorating are those that are known to be vulnerable, namely cellulose nitrate/acetate, PVC and rubber. Verifying this knowledge of the most vulnerable plastics enables us to recommend to properties across National Trust that these types should be seen as a priority for correct storage and in-depth recording