Leaf Recognition with Deep Learning and Keras using GPU computing

Our work is about using Deep Learning for leaf recognition using Keras and GPU computation. We used 17 CNNs of "Kaggle" [1], a Machine Learning training webpage that using simple challenges with prizes help people to learn how to use Deep Learning. Kaggle made a challenge in August, 30th, 2016 that was about Leaf Recognition. In that challenge more than 1500 users participated in it. They made teams to participate and we grab 17 codes of them to see how their codes was working. We updated the codes, because they were written in 1.0 version of Keras, and we use 2.0.6 version and then we made a ranking to test their accuracy in leaf recognition using the dataset provided by kaggle. Also we downloaded two more datasets to make more tests with them. On the other hand, we found two papers in "the ImageClef Competition" [2], and we implemented them from the beggining to see how simple is to transform a paper into code. ECOCUAN team, from 2015, used a tunned AlexNet CNN and KDETUT, from 2017, used a ResNet50 modified CNN. Another paper found on the net was one that use ResNet26 CNN, so we think that was a good idea to make a ranking of the three of them to see which is better.El nostre treball es basa en utilitzar Deep Learning per al reconeixement de fulles d'arbres utilitzant la llibreria Keras i fent servir una GPU per als càlculs de la xarxa. Hem utilitzat 17 CNNs de la web "Kaggle" que va fer una competició amb alguns premis per a la gent que volia aprendre Deep Learning. Kaggle va fer el concurs el 30 d'Agost del 2016 i en aquest concurs van participar més de 1500 usuaris. Van fer equips per participar i nosaltres hem agafat 17 codis d'aquests grups per veure com funcionen. Vam actualitzar els codis ja que eren fets amb la versió 1.0 de Keras i nosaltres utilitzem la versió 2.0.6 i després vam fer un ranking per veure la seva precisió en el reconeixement de fulles utilitzant el dataset que Kaggle proporcionava. També vam utilitzar dos datasets més per ampliar resultats. Per altra banda, vam trobar dos papers en la competició ImageClef que vam implementar des de 0 per veure com de fàcil és implementar una xarxa des d'un paper. ECOCUAN team del 2015 utilitzava una xarxa Alexnet tunejada y KDETUT del 2017 feia dues modificacions a la xarxa tipus ResNet50. També vam trobar un paper que feia servir la xarxa ResNet26 i vam pensar que era una bona idea utilitzar-lo per comparar amb la de KDETUT.Nuestro trabajo se basa en utilizar Deep Learning para el reconocimiento de hojas de árboles usando la librería Keras y cálculos mediante GPU. Hemos usado 17 CNNs encontradas en la web Kaggle que hizo una competición de reconocimiento de hojas con algunos premios para la gente que quería aprender Deep Learning. Kaggle hizo el concurso el 30 de Agosto de 2016 y participaron más de 1500 usuarios. Hicieron equipos para concursar y nosotros cogimos 17 de estos códigos para ver cómo funcionaban. Actualizamos los códigos ya que estaban hechos en la versión 1.0 de keras y nosotros teníamos la 2.0.6 y después hicimos un ranking para ver su precisión en el reconocimiento de hojas utilizando el dataset proporcionado por Kaggle. También utilizamos dos datasets más para ampliar resultados. Por otra parte, encontramos dos papeles de la competición ImageClef que implementamos desde cero para ver cómo de fácil es implementar una red desde un papel. ECOCUAN team del 2015 utilizaba una red tipo AlexNet tuneada y KDETUT del 2017 hacía dos modificaciones a la red tipo ResNet50. También encontramos otro papel que utilizaba la red ResNet26 y pensamos que era una buena idea compararlo con la red de KDETUT

Deep Learning Methods for Remote Sensing

Remote sensing is a field where important physical characteristics of an area are exacted using emitted radiation generally captured by satellite cameras, sensors onboard aerial vehicles, etc. Captured data help researchers develop solutions to sense and detect various characteristics such as forest fires, flooding, changes in urban areas, crop diseases, soil moisture, etc. The recent impressive progress in artificial intelligence (AI) and deep learning has sparked innovations in technologies, algorithms, and approaches and led to results that were unachievable until recently in multiple areas, among them remote sensing. This book consists of sixteen peer-reviewed papers covering new advances in the use of AI for remote sensing

Learning-based crop management optimization using multi-stream convolutional neural networks

Improving crop management is an essential step towards solving the food security challenge. Despite the advances in precision agriculture, new methods are needed to create decision-support systems to help farmers increase productivity while accounting for environmental impacts and financial risks. This dissertation presents a class of learning-based optimization algorithms for spatial allocation of crop inputs, and a new framework for online coverage path planning with potential use in tasks such as planting and harvesting. The proposed algorithms use Multi-stream Convolutional Neural Networks (MSCNN) to learn relevant spatial features from the environment and use them to optimize the available control inputs. In the crop inputs optimization problem, an MSCNN combines five input variables as in a regression problem to better predict yield. The predictive model is then used as the base of a gradient-ascent algorithm to maximize a custom objective function. To leverage the applicability of this algorithm, a risk-aware version of this method is also proposed. The predictive uncertainty is measured and used as a constraint to comply with different levels of risk-aversion. Experiments with real crop fields demonstrate that this method significantly reduces the yield prediction errors when compared to the state of the art algorithms. Results from the optimization algorithm show an increase in the expected net revenue of up to 6.8% when compared with the status quo management while providing safety bounds. In the coverage path planning framework, an MSCNN agent learns a control policy from demonstrations of paths obtained offline through heuristic algorithms, by using imitation learning. The resulting control policy is further improved through policy-gradient reinforcement learning. Simulations show that the improved control policy outperforms the offline algorithms used during the imitation learning phase, and that the proposed framework can be easily adapted to different cost functions

Advanced Process Monitoring for Industry 4.0

This book reports recent advances on Process Monitoring (PM) to cope with the many challenges raised by the new production systems, sensors and “extreme data” conditions that emerged with Industry 4.0. Concepts such as digital-twins and deep learning are brought to the PM arena, pushing forward the capabilities of existing methodologies to handle more complex scenarios. The evolution of classical paradigms such as Latent Variable modeling, Six Sigma and FMEA are also covered. Applications span a wide range of domains such as microelectronics, semiconductors, chemicals, materials, agriculture, as well as the monitoring of rotating equipment, combustion systems and membrane separation processes

Machine learning supported forecasting of baseline energy consumption for industrial processes

Abstract. The purpose of the thesis was to study and evaluate machine learning supported methods in order to forecast a baseline energy consumption from time-series data of energy-intensive industry. In addition, time-series anomaly detection methods were studied and the anomaly detection accuracy of them was evaluated with hourly and daily average energy consumption data. In the experimental part of the thesis a simulation scenario was established for hourly average data of two factories. The energy baseline was identified dynamically with week-ahead time-series forecasting by utilizing previous 52 weeks of data in the model training. In addition, model adaptation was considered in the simulation scenario. Predictor variables of the models were designed to imitate natural calendar effect. The energy baseline data of factory A was used to evaluate five linear and non-linear model structures. An average ensemble model structure appeared to outperform other model structures resulting in mean absolute percentage error of 9.3% for validation data of Factory A. The generalization ability of the model structure was evaluated with the data of factory B. For factory B the average ensemble model resulted in mean absolute percentage error of 9.9% for validation data. Overall, the results seemed promising especially as the set of input variables remained relatively simple as more precise subject matter expertise was not available during variable design and selection phase.Koneoppimiseen perustuva ominaisenergian kulutuksen ennustaminen teollisissa prosesseissa . Tiivistelmä. Diplomityön tavoitteena oli tutkia ja evaluoida koneoppimiseen pohjautuvia menetelmiä energiaintensiivisen teollisuuden aikasarjamuotoisen energiankulutusdatan käsittelyssä energiankulutuksen perusuran ennustamiseksi. Lisäksi työssä tutkittiin aikasarjadatan anomaliantunnistusmenetelmiä ja evaluoitiin niiden kykyä tunnistaa poikkeamia tuntija päiväkeskiarvoresoluutioisessa energiankulutusdatassa. Työn kokeellisessa osiossa muodostettiin simulaatioskenaario kahden eri tehtaan vuosien 2020 sekä 2021 tuntikeskiarvoisten energiankulutusaineistojen mallinnukselle. Perusura muodostettiin dynaamisesti kerrallaan viikoksi eteenpäin aikasarjaennusteena edellisen 52 viikon aineistoa mallin opetuksessa hyödyntäen. Mallinnusskenaariossa huomioitiin lisäksi mallin suorituskyvylle olennainen adaptaatioproseduuri. Mallien selittävinä muuttujina käytettiin eksploratiivisen data-analyysin pohjalta luotuja luonnollista kalenterivaikutusta imitoivia muuttujia. Tehtaan A aineistolla evaluoitiin viittä eri lineaarista ja epälineaarista mallirakennetta. Parhaimmaksi mallirakenteeksi osoittautui keskiarvoyhdistelmämalli, jolle ennusteen keskimääräinen suhteellinen virhe oli 9,3 % validointiaineistolla. Mallirakenteen yleistyvyyttä testattiin toisen tehtaan (B) vastaavan ajanjakson aineistolla. Tehtaan B aineistolle keskiarvoyhdistelmämallin ennusteen keskimääräinen suhteellinen virhe oli 9,9 % validointiaineistolla. Tuloksia voidaan yleisesti ottaen pitää lupaavina etenkin, kun mallien tulomuuttujajoukko jäi verrattain yksinkertaiseksi, sillä tarkempaa aiheasiantuntemusta ei ollut saatavilla

Exploring deep learning for complex trait genomic prediction in polyploid outcrossing species

Genomic prediction (GP) is the procedure whereby the genetic merits of untested candidates are predicted using genome wide marker information. Although numerous examples of GP exist in plants and animals, applications to polyploid organisms are still scarce, partly due to limited genome resources and the complexity of this system. Deep learning (DL) techniques comprise a heterogeneous collection of machine learning algorithms that have excelled at many prediction tasks. A potential advantage of DL for GP over standard linear model methods is that DL can potentially take into account all genetic interactions, including dominance and epistasis, which are expected to be of special relevance in most polyploids. In this study, we evaluated the predictive accuracy of linear and DL techniques in two important small fruits or berries: strawberry and blueberry. The two datasets contained a total of 1,358 allopolyploid strawberry (2n=8x=112) and 1,802 autopolyploid blueberry (2n=4x=48) individuals, genotyped for 9,908 and 73,045 single nucleotide polymorphism (SNP) markers, respectively, and phenotyped for five agronomic traits each. DL depends on numerous parameters that influence performance and optimizing hyperparameter values can be a critical step. Here we show that interactions between hyperparameter combinations should be expected and that the number of convolutional filters and regularization in the first layers can have an important effect on model performance. In terms of genomic prediction, we did not find an advantage of DL over linear model methods, except when the epistasis component was important. Linear Bayesian models were better than convolutional neural networks for the full additive architecture, whereas the opposite was observed under strong epistasis. However, by using a parameterization capable of taking into account these non-linear effects, Bayesian linear models can match or exceed the predictive accuracy of DL. A semiautomatic implementation of the DL pipeline is available at https://github.com/lauzingaretti/deepGP/

Exploring deep learning for complex trait genomic prediction in polyploid outcrossing species

Genomic prediction (GP) is the procedure whereby the genetic merits of untested candidates are predicted using genome wide marker information. Although numerous examples of GP exist in plants and animals, applications to polyploid organisms are still scarce, partly due to limited genome resources and the complexity of this system. Deep learning (DL) techniques comprise a heterogeneous collection of machine learning algorithms that have excelled at many prediction tasks. A potential advantage of DL for GP over standard linear model methods is that DL can potentially take into account all genetic interactions, including dominance and epistasis, which are expected to be of special relevance in most polyploids. In this study, we evaluated the predictive accuracy of linear and DL techniques in two important small fruits or berries: strawberry and blueberry. The two datasets contained a total of 1,358 allopolyploid strawberry (2n=8x=112) and 1,802 autopolyploid blueberry (2n=4x=48) individuals, genotyped for 9,908 and 73,045 single nucleotide polymorphism (SNP) markers, respectively, and phenotyped for five agronomic traits each. DL depends on numerous parameters that influence performance and optimizing hyperparameter values can be a critical step. Here we show that interactions between hyperparameter combinations should be expected and that the number of convolutional filters and regularization in the first layers can have an important effect on model performance. In terms of genomic prediction, we did not find an advantage of DL over linear model methods, except when the epistasis component was important. Linear Bayesian models were better than convolutional neural networks for the full additive architecture, whereas the opposite was observed under strong epistasis. However, by using a parameterization capable of taking into account these non-linear effects, Bayesian linear models can match or exceed the predictive accuracy of DL. A semiautomatic implementation of the DL pipeline is available at https://github.com/lauzingaretti/deepGP/.info:eu-repo/semantics/publishedVersio

Convergence of Intelligent Data Acquisition and Advanced Computing Systems

This book is a collection of published articles from the Sensors Special Issue on "Convergence of Intelligent Data Acquisition and Advanced Computing Systems". It includes extended versions of the conference contributions from the 10th IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS’2019), Metz, France, as well as external contributions