Selected aspects of complex, hypercomplex and fuzzy neural networks

This short report reviews the current state of the research and methodology on theoretical and practical aspects of Artificial Neural Networks (ANN). It was prepared to gather state-of-the-art knowledge needed to construct complex, hypercomplex and fuzzy neural networks. The report reflects the individual interests of the authors and, by now means, cannot be treated as a comprehensive review of the ANN discipline. Considering the fast development of this field, it is currently impossible to do a detailed review of a considerable number of pages. The report is an outcome of the Project 'The Strategic Research Partnership for the mathematical aspects of complex, hypercomplex and fuzzy neural networks' meeting at the University of Warmia and Mazury in Olsztyn, Poland, organized in September 2022.Comment: 46 page

Why and When Can Deep -- but Not Shallow -- Networks Avoid the Curse of Dimensionality: a Review

The paper characterizes classes of functions for which deep learning can be exponentially better than shallow learning. Deep convolutional networks are a special case of these conditions, though weight sharing is not the main reason for their exponential advantage

Inquisitive Pattern Recognition

The Department of Defense and the Department of the Air Force have funded automatic target recognition for several decades with varied success. The foundation of automatic target recognition is based upon pattern recognition. In this work, we present new pattern recognition concepts specifically in the area of classification and propose new techniques that will allow one to determine when a classifier is being arrogant. Clearly arrogance in classification is an undesirable attribute. A human is being arrogant when their expressed conviction in a decision overstates their actual experience in making similar decisions. Likewise given an input feature vector, we say a classifier is arrogant in its classification if its veracity is high yet its experience is low. Conversely a classifier is non-arrogant in its classification if there is a reasonable balance between its veracity and its experience. We quantify this balance and we discuss new techniques that will detect arrogance in a classifier. Inquisitiveness is in many ways the opposite of arrogance. In nature inquisitiveness is an eagerness for knowledge characterized by the drive to question to seek a deeper understanding and to challenge assumptions. The human capacity to doubt present beliefs allows us to acquire new experiences and to learn from our mistakes. Within the discrete world of computers, inquisitive pattern recognition is the constructive investigation and exploitation of conflict in information. This research defines inquisitiveness within the context of self-supervised machine learning and introduces mathematical theory and computational methods for quantifying incompleteness that is for isolating unstable, nonrepresentational regions in present information models

A Study on Physics Informed Neural Networks, with Applications for Compartment Models

The main goal of this thesis was to investigate the methodology of Physics Informed Neural Networks (PiNN), as a computational tool leveraging differential equations as a regularization for a learning task. PiNN is a new field of research and therefore particular concern was given to the task of obtaining an understanding of the method, gauging benefits, performance, and appropriateness in relation to established methods. In order to develop this knowledge, the methodology was implemented and applied through four case studies, three of which demonstrates achievements already supported by the literature. In addition case three incorporates a thorough testing scheme, scoping out PiNNs’ capabilities of parameter discovery and regularization. From this a larger framework is developed. In case four, the framework is utilized applying of the method of PiNN in a real world biomedical context, realized as a model of the circulatory system. The implementations were realized in a bottom up approach utilizing the neural network capabilities of PyTorch. Overall, the findings of the thesis support the established findings of previous literature in regards to performance and capabilities. Additionally, important details in regards to implementation and solution validity is highlighted, addressing the conditions necessary for the optimal use of PiNN as a methodology

Analysing and forecasting tourism demand in Vietnam with artificial neural networks

Mestrado APNORVietnam has experienced a tourism boom over the last decade with more than 18 million international tourists in 2019, compared to 1.5 million twenty-five years ago. Tourist spending has translated into rising employment and income for the tourism sector, making it the key driver to the socio-economic development of the country. Facing the COVID-19 pandemic, Vietnam´s tourism has suffered extreme economic losses. However, the number of international tourists is expected to reach the pre-pandemic levels in the next few years after the COVID-19 pandemic subsides. Forecasting tourism demand plays an essential role in predicting future economic development. Accurate predictions of tourism volume would facilitate decision-makers and managers to optimize resource allocation as well as to balance environmental and economic aspects. Various methods to predict tourism demand have been introduced over the years. One of the most prominent approaches is Artificial Neural Network (ANN) thanks to its capability to handle highly volatile and non-linear data. Given the significance of tourism to the economy, a precise forecast of tourism demand would help to foresee the potential economic growth of Vietnam. First, the research aims to analyse Vietnam´s tourism sector with a special focus on international tourists. Next, several ANN architectures are experimented with the datasets from 2008 to 2020, to predict the monthly number of international tourists traveling to Vietnam including COVID-19 lockdown periods. The results showed that with the correct selection of ANN architectures and data from the previous 12 months, the best ANN models can forecast the number of international tourists for next month with a MAPE between 7.9% and 9.2%. As the method proves its forecasting accuracy, it would serve as a valuable tool for Vietnam´s policymakers and firm managers to make better investment and strategic decisions to promote tourism after the COVID-19 situation.O Vietname conheceu um boom turístico na última década com mais de 18 milhões de turistas internacionais em 2019, em comparação com 1,5 milhões há vinte e cinco anos. As despesas turísticas traduziram-se num aumento do emprego e de receitas no sector do turismo, tornando-o no principal motor do desenvolvimento socioeconómico do país. Perante a pandemia da COVID-19, o turismo no Vietname sofreu perdas económicas extremas. Porém, espera-se que o número de turistas internacionais, pós pandemia da COVID-19, atinja os níveis pré-pandémicos nos próximos anos. A previsão da procura turística desempenha um papel essencial na previsão do desenvolvimento económico futuro. Previsões precisas facilitariam os decisores e gestores a otimizar a afetação de recursos, bem como o equilíbrio entre os aspetos ambientais e económicos. Vários métodos para prever a procura turística têm sido introduzidos ao longo dos anos. Uma das abordagens mais proeminentes assenta na metodologia das Redes Neuronais Artificiais (ANN) dada a sua capacidade de lidar com dados voláteis e não lineares. Dada a importância do turismo para a economia, uma previsão precisa da procura turística ajudaria a prever o crescimento económico potencial do Vietname. Em primeiro lugar, a investigação tem por objetivo analisar o sector turístico do Vietname com especial incidência nos turistas internacionais. Em seguida, várias arquiteturas de ANN são experimentadas com um conjunto de dados de 2008 a 2020, para prever o número mensal de turistas internacionais que se deslocam ao Vietname, incluindo os períodos de confinamento relacionados com a COVID-19. Os resultados mostraram, com a correta seleção de arquiteturas ANN e dados dos 12 meses anteriores, os melhores modelos ANN podem prever o número de turistas internacionais para o próximo mês com uma MAPE entre 7,9% e 9,2%. Como o método evidenciou a sua precisão de previsão, o mesmo pode servir como uma ferramenta valiosa para os decisores políticos e gestores de empresas do Vietname, pois irá permitir fazer melhores investimentos e tomarem decisões estratégicas para promover o turismo pós situação da COVID-19

Automatic control of a multirotor

Objective of this thesis is to describe the design and realisation phases of a multirotor to be used for low risk and cost aerial observation. Starting point of this activity was a wide literature study related to the technological evolution of multirotors design and to the state of the art. Firstly the most common multirotor configurations were defined and, according to a size and performance based evaluation, the most suitable one was chosen. A detailed computer aided design model was drawn as basis for the realisation of two prototypes. The realised multirotors were “X-shaped” octorotors with eight coaxially coupled motors. The mathematical model of the multirotor dynamics was studied. “Proportional Integral Derivative” and “Linear Quadratic” algorithms were chosen as techniques to regulate the attitude dynamics of the multirotor. These methods were tested with a nonlinear model simulation developed in the Matlab Simulink environment. In the meanwhile the Arduino board was selected as the best compromise between costs and performance and the above mentioned algorithms were implemented using this platform thanks to its main characteristic of being completely “open source”. Indeed the multirotor was conceived to be a serviceable tool for the public utility and, at the same time, to be an accessible device for research and studies. The behaviour of the physical multirotor was evaluated with a test bench designed to isolate the rotation about one single body axis at a time. The data of the experimental tests were gathered in real time using a custom Matlab code and several indoor tests allowed the “fine tuning” of the controllers gains. Afterwards a portable “ground station” was conceived and realised in adherence with the real scenarios users needs. Several outdoor experimental flights were executed with successful results and the data gathered during the outdoor tests were used to evaluate some key performance indicators as the endurance and the maximum allowable payload mass. Then the fault tolerance of the control system was evaluated simulating and experimenting the loss of one motor; even in this critical condition the system exhibited an acceptable behaviour. The reached project readiness allowed to meet some potential users as the “Turin Fire Department” and to cooperate with them in a simulated emergency. During this event the multirotor was used to gather and transmit real time aerial images for an improved “situation awareness”. Finally the study was extended to more innovative control techniques like the neural networks based ones. Simulations results demonstrated their effectiveness; nevertheless the inherent complexity and the unreliability outside the training ranges could have a catastrophic impact on the airworthiness. This is a factor that cannot be neglected especially in the applications related to flying platforms. Summarising, this research work was addressed mainly to the operating procedures for implementing automatic control algorithms to real platforms. All the design aspects, from the preliminary multirotor configuration choice to the tests in possible real scenarios, were covered obtaining performances comparable with other commercial of-the-shelf platforms

AHEAD: Automatic Holistic Energy-Aware Design Methodology for MLP Neural Network Hardware Generation in Proactive BMI Edge Devices

The prediction of a high-level cognitive function based on a proactive brain–machine interface (BMI) control edge device is an emerging technology for improving the quality of life for disabled people. However, maintaining the stability of multiunit neural recordings is made difficult by the nonstationary nature of neurons and can affect the overall performance of proactive BMI control. Thus, it requires regular recalibration to retrain a neural network decoder for proactive control. However, retraining may lead to changes in the network parameters, such as the network topology. In terms of the hardware implementation of the neural decoder for real-time and low-power processing, it takes time to modify or redesign the hardware accelerator. Consequently, handling the engineering change of the low-power hardware design requires substantial human resources and time. To address this design challenge, this work proposes AHEAD: an automatic holistic energy-aware design methodology for multilayer perceptron (MLP) neural network hardware generation in proactive BMI edge devices. By taking a holistic analysis of the proactive BMI design flow, the approach makes judicious use of the intelligent bit-width identification (BWID) and configurable hardware generation, which autonomously integrate to generate the low-power hardware decoder. The proposed AHEAD methodology begins with the trained MLP parameters and golden datasets and produces an efficient hardware design in terms of performance, power, and area (PPA) with the least loss of accuracy. The results show that the proposed methodology is up to a 4X faster in performance, 3X lower in terms of power consumption, and achieves a 5X reduction in area resources, with exact accuracy, compared to floating-point and half-floating-point design on a field-programmable gate array (FPGA), which makes it a promising design methodology for proactive BMI edge devices

A priori analysis on deep learning of subgrid-scale parameterizations for Kraichnan turbulence

In the present study, we investigate different data-driven parameterizations for large eddy simulation of two-dimensional turbulence in the \emph{a priori} settings. These models utilize resolved flow field variables on the coarser grid to estimate the subgrid-scale stresses. We use data-driven closure models based on localized learning that employs multilayer feedforward artificial neural network (ANN) with point-to-point mapping and neighboring stencil data mapping, and convolutional neural network (CNN) fed by data snapshots of the whole domain. The performance of these data-driven closure models is measured through a probability density function and is compared with the dynamic Smagorinksy model (DSM). The quantitative performance is evaluated using the cross-correlation coefficient between the true and predicted stresses. We analyze different frameworks in terms of the amount of training data, selection of input and output features, their characteristics in modeling with accuracy, and training and deployment computational time. We also demonstrate computational gain that can be achieved using the intelligent eddy viscosity model that learns eddy viscosity computed by the DSM instead of subgrid-scale stresses. We detail the hyperparameters optimization of these models using the grid search algorithm