177 research outputs found
Estimating general motion and intensity from event cameras
Robotic vision algorithms have become widely used in many consumer products which
enabled technologies such as autonomous vehicles, drones, augmented reality (AR) and
virtual reality (VR) devices to name a few. These applications require vision algorithms
to work in real-world environments with extreme lighting variations and fast moving
objects. However, robotic vision applications rely often on standard video cameras which
face severe limitations in fast-moving scenes or by bright light sources which diminish
the image quality with artefacts like motion blur or over-saturation.
To address these limitations, the body of work presented here investigates the use of
alternative sensor devices which mimic the superior perception properties of human
vision. Such silicon retinas were proposed by neuromorphic engineering, and we focus
here on one such biologically inspired sensor called the event camera which offers a new
camera paradigm for real-time robotic vision. The camera provides a high measurement
rate, low latency, high dynamic range, and low data rate. The signal of the camera is
composed of a stream of asynchronous events at microsecond resolution. Each event
indicates when individual pixels registers a logarithmic intensity changes of a pre-set
threshold size. Using this novel signal has proven to be very challenging in most computer
vision problems since common vision methods require synchronous absolute intensity
information.
In this thesis, we present for the first time a method to reconstruct an image and es-
timation motion from an event stream without additional sensing or prior knowledge of
the scene. This method is based on coupled estimations of both motion and intensity
which enables our event-based analysis, which was previously only possible with severe
limitations. We also present the first machine learning algorithm for event-based unsu-
pervised intensity reconstruction which does not depend on an explicit motion estimation
and reveals finer image details. This learning approach does not rely on event-to-image
examples, but learns from standard camera image examples which are not coupled to the
event data. In experiments we show that the learned reconstruction improves upon our
handcrafted approach. Finally, we combine our learned approach with motion estima-
tion methods and show the improved intensity reconstruction also significantly improves
the motion estimation results. We hope our work in this thesis bridges the gap between
the event signal and images and that it opens event cameras to practical solutions to
overcome the current limitations of frame-based cameras in robotic vision.Open Acces
Mining a Small Medical Data Set by Integrating the Decision Tree and t-test
[[abstract]]Although several researchers have used statistical methods to prove that aspiration followed by the injection of 95% ethanol left in situ (retention) is an effective treatment for ovarian endometriomas, very few discuss the different conditions that could generate different recovery rates for the patients. Therefore, this study adopts the statistical method and decision tree techniques together to analyze the postoperative status of ovarian endometriosis patients under different conditions. Since our collected data set is small, containing only 212 records, we use all of these data as the training data. Therefore, instead of using a resultant tree to generate rules directly, we use the value of each node as a cut point to generate all possible rules from the tree first. Then, using t-test, we verify the rules to discover some useful description rules after all possible rules from the tree have been generated. Experimental results show that our approach can find some new interesting knowledge about recurrent ovarian endometriomas under different conditions.[[journaltype]]國外[[incitationindex]]EI[[booktype]]紙本[[countrycodes]]FI
Deep learning applied to computational mechanics: A comprehensive review, state of the art, and the classics
Three recent breakthroughs due to AI in arts and science serve as motivation:
An award winning digital image, protein folding, fast matrix multiplication.
Many recent developments in artificial neural networks, particularly deep
learning (DL), applied and relevant to computational mechanics (solid, fluids,
finite-element technology) are reviewed in detail. Both hybrid and pure machine
learning (ML) methods are discussed. Hybrid methods combine traditional PDE
discretizations with ML methods either (1) to help model complex nonlinear
constitutive relations, (2) to nonlinearly reduce the model order for efficient
simulation (turbulence), or (3) to accelerate the simulation by predicting
certain components in the traditional integration methods. Here, methods (1)
and (2) relied on Long-Short-Term Memory (LSTM) architecture, with method (3)
relying on convolutional neural networks. Pure ML methods to solve (nonlinear)
PDEs are represented by Physics-Informed Neural network (PINN) methods, which
could be combined with attention mechanism to address discontinuous solutions.
Both LSTM and attention architectures, together with modern and generalized
classic optimizers to include stochasticity for DL networks, are extensively
reviewed. Kernel machines, including Gaussian processes, are provided to
sufficient depth for more advanced works such as shallow networks with infinite
width. Not only addressing experts, readers are assumed familiar with
computational mechanics, but not with DL, whose concepts and applications are
built up from the basics, aiming at bringing first-time learners quickly to the
forefront of research. History and limitations of AI are recounted and
discussed, with particular attention at pointing out misstatements or
misconceptions of the classics, even in well-known references. Positioning and
pointing control of a large-deformable beam is given as an example.Comment: 275 pages, 158 figures. Appeared online on 2023.03.01 at
CMES-Computer Modeling in Engineering & Science
Application of Artificial Intelligence algorithms to support decision-making in agriculture activities
Deep Learning has been successfully applied to image recognition, speech recognition, and
natural language processing in recent years. Therefore, there has been an incentive to apply
it in other fields as well. The field of agriculture is one of the most important in which the
application of artificial intelligence algorithms, and particularly, of deep learning needs to
be explored, as it has a direct impact on human well-being. In particular, there is a need
to explore how deep learning models for decision-making can be used as a tool for optimal
planting, land use, yield improvement, production/disease/pest control, and other activities.
The vast amount of data received from sensors in smart farms makes it possible to use deep
learning as a model for decision-making in this field. In agriculture, no two environments are
exactly alike, which makes testing, validating, and successfully implementing such technologies
much more complex than in most other sectors. Recent scientific developments in the
field of deep learning, applied to agriculture, are reviewed and some challenges and potential
solutions using deep learning algorithms in agriculture are discussed. Higher performance
in terms of accuracy and lower inference time can be achieved, and the models can be made
useful in real-world applications. Finally, some opportunities for future research in this area
are suggested. The ability of artificial neural networks, specifically Long Short-Term Memory
(LSTM) and Bidirectional LSTM (BLSTM), to model daily reference evapotranspiration
and soil water content is investigated. The application of these techniques to predict these
parameters was tested for three sites in Portugal. A single-layer BLSTM with 512 nodes was
selected. Bayesian optimization was used to determine the hyperparameters, such as learning
rate, decay, batch size, and dropout size. The model achieved mean square error (MSE)
values ranging from 0.07 to 0.27 (mm d–1)² for ETo (Reference Evapotranspiration) and
0.014 to 0.056 (m³m–3)² for SWC (Soil Water Content), with R2 values ranging from 0.96
to 0.98. A Convolutional Neural Network (CNN) model was added to the LSTM to investigate
potential performance improvement. Performance dropped in all datasets due to the
complexity of the model. The performance of the models was also compared with CNN, traditional
machine learning algorithms Support Vector Regression, and Random Forest. LSTM
achieved the best performance. Finally, the impact of the loss function on the performance
of the proposed models was investigated. The model with the mean square error (MSE) as
loss function performed better than the model with other loss functions. Afterwards, the
capabilities of these models and their extension, BLSTM and Bidirectional Gated Recurrent
Units (BGRU) to predict end-of-season yields are investigated. The models use historical
data, including climate data, irrigation scheduling, and soil water content, to estimate endof-
season yield. The application of this technique was tested for tomato and potato yields at a
site in Portugal. The BLSTM network outperformed the GRU, the LSTM, and the BGRU networks
on the validation dataset. The model was able to capture the nonlinear relationship
between irrigation amount, climate data, and soil water content and predict yield with an
MSE of 0.017 to 0.039 kg/ha. The performance of the BLSTM in the test was compared with
the most commonly used deep learning method called CNN, and machine learning methods
including a Multi-Layer Perceptrons model and Random Forest regression. The BLSTM out-performed the other models with a R2-score between 0.97 and 0.99. The results show that
analyzing agricultural data with the LSTM model improves the performance of the model in
terms of accuracy. The CNN model achieved the second-best performance. Therefore, the
deep learning model has a remarkable ability to predict the yield at the end of the season. Additionally,
a Deep Q-Network was trained for irrigation scheduling. The agent was trained to
schedule irrigation for a tomato field in Portugal. Two LSTM models trained previously were
used as the agent environment. One predicts the total water in the soil profile on the next
day. The other one was employed to estimate the yield based on the environmental condition
during a season and then measure the net return. The agent uses this information to decide
the following irrigation amount. LSTM and CNN networks were used to estimate the Q-table
during training. Unlike the LSTM model, the ANN and the CNN could not estimate the Qtable,
and the agent’s reward decreased during training. The comparison of the performance
of the model was done with fixed-base irrigation and threshold-based irrigation. The trained
model increased productivity by 11% and decreased water consumption by 20% to 30% compared
to the fixed method. Also, an on-policy model, Advantage Actor–Critic (A2C), was
implemented to compare irrigation scheduling with Deep Q-Network for the same tomato
crop. The results show that the on-policy model A2C reduced water consumption by 20%
compared to Deep Q-Network with a slight change in the net reward. These models can be
developed to be applied to other cultures with high importance in Portugal, such as fruit,
cereals, and grapevines, which also have large water requirements. The models developed
along this thesis can be re-evaluated and trained with historical data from other cultures with
high production in Portugal, such as fruits, cereals, and grapes, which also have high water
demand, to create a decision support and recommendation system that tells farmers when
and how much to irrigate. This system helps farmers avoid wasting water without reducing
productivity. This thesis aims to contribute to the future steps in the development of precision
agriculture and agricultural robotics. The models developed in this thesis are relevant to
support decision-making in agricultural activities, aimed at optimizing resources, reducing
time and costs, and maximizing production.Nos últimos anos, a técnica de aprendizagem profunda (Deep Learning) foi aplicada com
sucesso ao reconhecimento de imagem, reconhecimento de fala e processamento de linguagem
natural. Assim, tem havido um incen tivo para aplicá-la também em outros sectores.
O sector agrícola é um dos mais importantes, em que a aplicação de algoritmos de inteligência
artificial e, em particular, de deep learning, precisa ser explorada, pois tem impacto direto
no bem-estar humano. Em particular, há uma necessidade de explorar como os modelos de
aprendizagem profunda para a tomada de decisão podem ser usados como uma ferramenta
para cultivo ou plantação ideal, uso da terra, melhoria da produtividade, controlo de produção,
de doenças, de pragas e outras atividades. A grande quantidade de dados recebidos
de sensores em explorações agrícolas inteligentes (smart farms) possibilita o uso de deep
learning como modelo para tomada de decisão nesse campo. Na agricultura, não há dois
ambientes iguais, o que torna o teste, a validação e a implementação bem-sucedida dessas
tecnologias muito mais complexas do que na maioria dos outros setores. Desenvolvimentos
científicos recentes no campo da aprendizagem profunda aplicada à agricultura, são revistos
e alguns desafios e potenciais soluções usando algoritmos de aprendizagem profunda na agricultura
são discutidos. Maior desempenho em termos de precisão e menor tempo de inferência
pode ser alcançado, e os modelos podem ser úteis em aplicações do mundo real. Por fim,
são sugeridas algumas oportunidades para futuras pesquisas nesta área. A capacidade de redes
neuronais artificiais, especificamente Long Short-Term Memory (LSTM) e LSTM Bidirecional
(BLSTM), para modelar a evapotranspiração de referência diária e o conteúdo de água
do solo é investigada. A aplicação destas técnicas para prever estes parâmetros foi testada em
três locais em Portugal. Um BLSTM de camada única com 512 nós foi selecionado. A otimização
bayesiana foi usada para determinar os hiperparâmetros, como taxa de aprendizagem,
decaimento, tamanho do lote e tamanho do ”dropout”. O modelo alcançou os valores de erro
quadrático médio na faixa de 0,014 a 0,056 e R2 variando de 0,96 a 0,98. Um modelo de
Rede Neural Convolucional (CNN – Convolutional Neural Network) foi adicionado ao LSTM
para investigar uma potencial melhoria de desempenho. O desempenho decresceu em todos
os conjuntos de dados devido à complexidade do modelo. O desempenho dos modelos
também foi comparado com CNN, algoritmos tradicionais de aprendizagem máquina Support
Vector Regression e Random Forest. O LSTM obteve o melhor desempenho. Por fim,
investigou-se o impacto da função de perda no desempenho dos modelos propostos. O modelo
com o erro quadrático médio (MSE) como função de perda teve um desempenho melhor
do que o modelo com outras funções de perda. Em seguida, são investigadas as capacidades
desses modelos e sua extensão, BLSTM e Bidirectional Gated Recurrent Units (BGRU) para
prever os rendimentos da produção no final da campanha agrícola. Os modelos usam dados
históricos, incluindo dados climáticos, calendário de rega e teor de água do solo, para estimar
a produtividade no final da campanha. A aplicação desta técnica foi testada para os rendimentos
de tomate e batata em um local em Portugal. A rede BLSTM superou as redes GRU,
LSTM e BGRU no conjunto de dados de validação. O modelo foi capaz de captar a relação não
linear entre dotação de rega, dados climáticos e teor de água do solo e prever a produtividade com um MSE variando de 0,07 a 0,27 (mm d–1)² para ETo (Evapotranspiração de Referência)
e de 0,014 a 0,056 (m³m–3)² para SWC (Conteúdo de Água do Solo), com valores de R2
variando de 0,96 a 0,98. O desempenho do BLSTM no teste foi comparado com o método de
aprendizagem profunda CNN, e métodos de aprendizagem máquina, incluindo um modelo
Multi-Layer Perceptrons e regressão Random Forest. O BLSTM superou os outros modelos
com um R2 entre 97% e 99%. Os resultados mostram que a análise de dados agrícolas
com o modelo LSTM melhora o desempenho do modelo em termos de precisão. O modelo
CNN obteve o segundo melhor desempenho. Portanto, o modelo de aprendizagem profunda
tem uma capacidade notável de prever a produtividade no final da campanha. Além disso,
uma Deep Q-Network foi treinada para programação de irrigação para a cultura do tomate.
O agente foi treinado para programar a irrigação de uma plantação de tomate em Portugal.
Dois modelos LSTM treinados anteriormente foram usados como ambiente de agente. Um
prevê a água total no perfil do solo no dia seguinte. O outro foi empregue para estimar a produtividade
com base nas condições ambientais durante uma o ciclo biológico e então medir
o retorno líquido. O agente usa essas informações para decidir a quantidade de irrigação.
As redes LSTM e CNN foram usadas para estimar a Q-table durante o treino. Ao contrário
do modelo LSTM, a RNA e a CNN não conseguiram estimar a tabela Q, e a recompensa do
agente diminuiu durante o treino. A comparação de desempenho do modelo foi realizada
entre a irrigação com base fixa e a irrigação com base em um limiar. A aplicação das doses
de rega preconizadas pelo modelo aumentou a produtividade em 11% e diminuiu o consumo
de água em 20% a 30% em relação ao método fixo. Além disso, um modelo dentro da táctica,
Advantage Actor–Critic (A2C), é foi implementado para comparar a programação de
irrigação com o Deep Q-Network para a mesma cultura de tomate. Os resultados mostram
que o modelo de táctica A2C reduziu o consumo de água consumo em 20% comparado ao
Deep Q-Network com uma pequena mudança na recompensa líquida. Estes modelos podem
ser desenvolvidos para serem aplicados a outras culturas com elevada produção em Portugal,
como a fruta, cereais e vinha, que também têm grandes necessidades hídricas. Os modelos
desenvolvidos ao longo desta tese podem ser reavaliados e treinados com dados históricos
de outras culturas com elevada importância em Portugal, tais como frutas, cereais e uvas,
que também têm elevados consumos de água. Assim, poderão ser desenvolvidos sistemas
de apoio à decisão e de recomendação aos agricultores de quando e quanto irrigar. Estes
sistemas poderão ajudar os agricultores a evitar o desperdício de água sem reduzir a produtividade.
Esta tese visa contribuir para os passos futuros na evolução da agricultura de
precisão e da robótica agrícola. Os modelos desenvolvidos ao longo desta tese são relevantes
para apoiar a tomada de decisões em atividades agrícolas, direcionadas à otimização de recursos,
redução de tempo e custos, e maximização da produção.Centro-01-0145-FEDER000017-EMaDeS-Energy,
Materials, and Sustainable Development, co-funded by the Portugal 2020 Program (PT 2020),
within the Regional Operational Program of the Center (CENTRO 2020) and the EU through
the European Regional Development Fund (ERDF). Fundação para a Ciência e a Tecnologia
(FCT—MCTES) also provided financial support via project UIDB/00151/2020 (C-MAST).
It was also supported by the R&D Project BioDAgro – Sistema operacional inteligente de
informação e suporte á decisão em AgroBiodiversidade, project PD20-00011, promoted by
Fundação La Caixa and Fundação para a Ciência e a Tecnologia, taking place at the C-MAST
- Centre for Mechanical and Aerospace Sciences and Technology, Department of Electromechanical
Engineering of the University of Beira Interior, Covilhã, Portugal
Sensors Fault Diagnosis Trends and Applications
Fault diagnosis has always been a concern for industry. In general, diagnosis in complex systems requires the acquisition of information from sensors and the processing and extracting of required features for the classification or identification of faults. Therefore, fault diagnosis of sensors is clearly important as faulty information from a sensor may lead to misleading conclusions about the whole system. As engineering systems grow in size and complexity, it becomes more and more important to diagnose faulty behavior before it can lead to total failure. In the light of above issues, this book is dedicated to trends and applications in modern-sensor fault diagnosis
Defect detection in infrared thermography by deep learning algorithms
L'évaluation non destructive (END) est un domaine permettant d'identifier tous les types de dommages structurels dans un objet d'intérêt sans appliquer de dommages et de modifications permanents. Ce domaine fait l'objet de recherches intensives depuis de nombreuses années. La thermographie infrarouge (IR) est l'une des technologies d'évaluation non destructive qui permet d'inspecter, de caractériser et d'analyser les défauts sur la base d'images infrarouges (séquences) provenant de l'enregistrement de l'émission et de la réflexion de la lumière infrarouge afin d'évaluer les objets non autochauffants pour le contrôle de la qualité et l'assurance de la sécurité. Ces dernières années, le domaine de l'apprentissage profond de l'intelligence artificielle a fait des progrès remarquables dans les applications de traitement d'images. Ce domaine a montré sa capacité à surmonter la plupart des inconvénients des autres approches existantes auparavant dans un grand nombre d'applications. Cependant, en raison de l'insuffisance des données d'entraînement, les algorithmes d'apprentissage profond restent encore inexplorés, et seules quelques publications font état de leur application à l'évaluation non destructive de la thermographie (TNDE). Les algorithmes d'apprentissage profond intelligents et hautement automatisés pourraient être couplés à la thermographie infrarouge pour identifier les défauts (dommages) dans les composites, l'acier, etc. avec une confiance et une précision élevée. Parmi les sujets du domaine de recherche TNDE, les techniques d'apprentissage automatique supervisées et non supervisées sont les tâches les plus innovantes et les plus difficiles pour l'analyse de la détection des défauts. Dans ce projet, nous construisons des cadres intégrés pour le traitement des données brutes de la thermographie infrarouge à l'aide d'algorithmes d'apprentissage profond et les points forts des méthodologies proposées sont les suivants: 1. Identification et segmentation automatique des défauts par des algorithmes d'apprentissage profond en thermographie infrarouge. Les réseaux neuronaux convolutifs (CNN) pré-entraînés sont introduits pour capturer les caractéristiques des défauts dans les images thermiques infrarouges afin de mettre en œuvre des modèles basés sur les CNN pour la détection des défauts structurels dans les échantillons composés de matériaux composites (diagnostic des défauts). Plusieurs alternatives de CNNs profonds pour la détection de défauts dans la thermographie infrarouge. Les comparaisons de performance de la détection et de la segmentation automatique des défauts dans la thermographie infrarouge en utilisant différentes méthodes de détection par apprentissage profond : (i) segmentation d'instance (Center-mask ; Mask-RCNN) ; (ii) détection d’objet (Yolo-v3 ; Faster-RCNN) ; (iii) segmentation sémantique (Unet ; Res-unet); 2. Technique d'augmentation des données par la génération de données synthétiques pour réduire le coût des dépenses élevées associées à la collecte de données infrarouges originales dans les composites (composants d'aéronefs.) afin d'enrichir les données de formation pour l'apprentissage des caractéristiques dans TNDE; 3. Le réseau antagoniste génératif (GAN convolutif profond et GAN de Wasserstein) est introduit dans la thermographie infrarouge associée à la thermographie partielle des moindres carrés (PLST) (réseau PLS-GANs) pour l'extraction des caractéristiques visibles des défauts et l'amélioration de la visibilité des défauts pour éliminer le bruit dans la thermographie pulsée; 4. Estimation automatique de la profondeur des défauts (question de la caractérisation) à partir de données infrarouges simulées en utilisant un réseau neuronal récurrent simplifié : Gate Recurrent Unit (GRU) à travers l'apprentissage supervisé par régression.Non-destructive evaluation (NDE) is a field to identify all types of structural damage in an object of interest without applying any permanent damage and modification. This field has been intensively investigated for many years. The infrared thermography (IR) is one of NDE technology through inspecting, characterize and analyzing defects based on the infrared images (sequences) from the recordation of infrared light emission and reflection to evaluate non-self-heating objects for quality control and safety assurance. In recent years, the deep learning field of artificial intelligence has made remarkable progress in image processing applications. This field has shown its ability to overcome most of the disadvantages in other approaches existing previously in a great number of applications. Whereas due to the insufficient training data, deep learning algorithms still remain unexplored, and only few publications involving the application of it for thermography nondestructive evaluation (TNDE). The intelligent and highly automated deep learning algorithms could be coupled with infrared thermography to identify the defect (damages) in composites, steel, etc. with high confidence and accuracy. Among the topics in the TNDE research field, the supervised and unsupervised machine learning techniques both are the most innovative and challenging tasks for defect detection analysis. In this project, we construct integrated frameworks for processing raw data from infrared thermography using deep learning algorithms and highlight of the methodologies proposed include the following: 1. Automatic defect identification and segmentation by deep learning algorithms in infrared thermography. The pre-trained convolutional neural networks (CNNs) are introduced to capture defect feature in infrared thermal images to implement CNNs based models for the detection of structural defects in samples made of composite materials (fault diagnosis). Several alternatives of deep CNNs for the detection of defects in the Infrared thermography. The comparisons of performance of the automatic defect detection and segmentation in infrared thermography using different deep learning detection methods: (i) instance segmentation (Center-mask; Mask-RCNN); (ii) objective location (Yolo-v3; Faster-RCNN); (iii) semantic segmentation (Unet; Res-unet); 2. Data augmentation technique through synthetic data generation to reduce the cost of high expense associated with the collection of original infrared data in the composites (aircraft components.) to enrich training data for feature learning in TNDE; 3. The generative adversarial network (Deep convolutional GAN and Wasserstein GAN) is introduced to the infrared thermography associated with partial least square thermography (PLST) (PLS-GANs network) for visible feature extraction of defects and enhancement of the visibility of defects to remove noise in Pulsed thermography; 4. Automatic defect depth estimation (Characterization issue) from simulated infrared data using a simplified recurrent neural network: Gate Recurrent Unit (GRU) through the regression supervised learning
Engineering Data Compendium. Human Perception and Performance, Volume 1
The concept underlying the Engineering Data Compendium was the product an R and D program (Integrated Perceptual Information for Designers project) aimed at facilitating the application of basic research findings in human performance to the design of military crew systems. The principal objective was to develop a workable strategy for: (1) identifying and distilling information of potential value to system design from existing research literature, and (2) presenting this technical information in a way that would aid its accessibility, interpretability, and applicability by system designers. The present four volumes of the Engineering Data Compendium represent the first implementation of this strategy. This is Volume 1, which contains sections on Visual Acquisition of Information, Auditory Acquisition of Information, and Acquisition of Information by Other Senses
Mobile Robots
The objective of this book is to cover advances of mobile robotics and related technologies applied for multi robot systems' design and development. Design of control system is a complex issue, requiring the application of information technologies to link the robots into a single network. Human robot interface becomes a demanding task, especially when we try to use sophisticated methods for brain signal processing. Generated electrophysiological signals can be used to command different devices, such as cars, wheelchair or even video games. A number of developments in navigation and path planning, including parallel programming, can be observed. Cooperative path planning, formation control of multi robotic agents, communication and distance measurement between agents are shown. Training of the mobile robot operators is very difficult task also because of several factors related to different task execution. The presented improvement is related to environment model generation based on autonomous mobile robot observations
- …