PCA-based approximation of a class of distributed parameter systems: classical vs. neural network approach

In this article, an approximation of the spatiotemporal response of a distributed parameter system (DPS) with the use of the principal component analysis (PCA) is considered. Based on a data obtained by the numerical solution of a set of partial differential equations, a PCA-based approximation procedure is performed. It consists in the projection of the original data into the subspace spanned by the eigenvectors of the data covariance matrix, corresponding to its highest eigenvalues. The presented approach is carried out using both the classical PCA method as well as two different neural network structures: two-layer feed-forward network with supervised learning (FF-PCA) and single-layer network with unsupervised, generalized Hebbian learning rule (GHA-PCA). In each case considered, the effect of the approximation model structure represented by the number of eigenvectors (or, in the neural case, units in the network projection layer) on the mean square approximation error of the spatiotemporal response and on the data compression ratio is analysed. As shown in the paper, the best approximation quality is obtained for the classical PCA method as well as for the FF-PCA neural approach. On the other hand, an adaptive learning method for the GHA-PCA network allows to use it in e.g. an on-line identification scheme

Classical and neural network-based principal component analysis for image compression

W artykule omówiono zastosowanie analizy składników głównych (PCA) w zadaniu kompresji stratnej sygnału na przykładzie kompresji obrazu. Zadanie zrealizowano z wykorzystaniem klasycznej metody PCA oraz dwóch rodzajów sieci neuronowych: jednokierunkowej, dwuwarstwowej sieci z uczeniem nadzorowanym i jednowarstwowej sieci z uczeniem nienadzorowanym. W każdym z przypadków przeanalizowano wpływ struktury modelu PCA na wartości współczynnika kompresji oraz średniokwadratowego błędu kompresji.In the paper, lossy data compression techniques based on the principal component analysis (PCA) are considered on the example of image compression. The presented task is performed using the classical PCA method based on the eigen-decomposition of the image covari-ance matrix as well as two different kinds of artificial neural networks. The first neural structure used is a two-layer feed-forward network with supervised learning shown in Fig.1, while the second one is a single-layered network with unsupervised Hebbian learning. In each case considered, the effect of the PCA model structure on the data compression ratio and the mean square reconstruction error is analysed. The compression results for a Hebbian neural network with K=4 PCA units are presented in Figs. 2, 3 and 4. They show that only 4 eigenvectors are able to capture the main features of the processed image, giving as a result high value of the data compression ratio. However, the reconstructed image quality is not sufficient from a practical point of view. Therefore, selection of the appropriate value for K should take into account the tradeoff between a sufficiently high value for the compression ratio and a reasonably low value for the image reconstruction error. The summary results for both classical and neural PCA compression approaches obtained for different number of eigenvectors (neurons) are compared in Fig. 5. The author concludes that a positive aspect of using neural networks as a tool for extracting principal components from the image data is that they do not require calculating the correlation matrix explicitly, as in the case of the classical PCA-based approach

Transfer function–based impulse response analysis for a class of hyperbolic systems of balance laws

Results of the impulse response analysis for a class of dynamical systems, described by two weakly coupled linear partial differential equations of hyperbolic type, defined on a one-dimensional spatial domain are presented. For the case of two boundary inputs of the Dirichlet type, the analytical expressions for the impulse response functions are derived based on the inverse Laplace trans form of the 2×2 transfer function matrix of the system. The influence of the boundary inputs configuration on the impulse response functions is demonstrated. The considerations are illustrated with a practical example of a thin-walled double-pipe heat exchanger operating in parallel- and countercurrent-flow modes, which correspond to the analyzed boundary conditions

Some peculiarities of neural approximation on example of inverse kinematic problem

W artykule wskazano na pewne charakterystyczne aspekty związane z zastosowaniem jednokierunkowych sieci neuronowych jako uniwersalnych układów aproksymujących złożone zależności nieliniowe. Zaprezentowany przykład dotyczy klasycznego problemu z dziedziny robotyki -tzw. odwrotnego zadania kinematyki. Zademonstrowano wpływ właściwego doboru struktury sieci, jej algorytmu uczenia oraz wzorców uczących na jakość aproksymacji neuronowej.Characteristic features of feedforward artificial neural networks, acting as universal function approximators, are presented. The problem under consideration concerns inverse kinematics of a two-link planar manipulator (Fig. 1). As shown in this paper, a two-layer, feedforward neural network is able to learn the nonlinear mapping between the end effector position domain and the joint angle domain of the manipulator (Fig. 2). However, a necessary condition for achieving the required approximation quality is proper selection of the network structure, especially with respect to the number of nonlinear, sigmoidal units in its hidden layer. Using too few neurons in this layer results in underfitting (Fig. 3), while too many neurons bring the problem of overfitting (Figs 6 and 7). The effect of learning algorithm efficiency as well as proper choice of learning data set on the network performance is also demonstrated (Fig. 8). Apart from the general conclusions concerning neural approximation, the presented results show also the possibility of neural control of robotic manipulator trajectory

Transfer function models for a class of hyperbolic systems with boundary inputs

W artykule przedstawiono ogólną postać transmitancji operatorowych pewnej klasy układów o parametrach rozłożonych, opisanych dwoma równaniami różniczkowymi cząstkowymi typu hiperbolicznego. Zakładając istnienie w układzie dwóch wejść o charakterze wymuszeń brzegowych typu Dirichleta oraz dwóch wyjść rozłożonych wzdłuż osi zmiennej przestrzennej, przedstawiono wyrażenia opisujące transmitancje operatorowe układu dla dwóch różnych konfiguracji sygnałów wejściowych. Rozważania zilustrowano praktycznym przykładem wymiennika ciepła pracującego w układach: współ- oraz przeciwprądowym.Transfer function models for a class of distributed parameter systems described by the two hyperbolic partial differential equations defined on a one-dimensional finite spatial domain are considered. Assuming two boundary inputs of Dirichlet type, the closed-form expressions for the individual elements of the 22 transfer function matrix are proposed based on the decoupled canonical representation of the system. The influence of the location of the boundary inputs on the transfer function representation is demonstrated for two different input configurations. The first one is the so-called congruent arrangement, for which both inputs act on the system at the same spatial position, l=0 (Fig. 1). The second one is the incongruent arrangement, where both inputs act on the system at its opposite ends, l=0 and l=L, respectively (Fig. 2). The considerations are illustrated with a practical example of a shell and tube heat exchanger operating in parallel- and countercurrent-flow modes (Fig. 3), which correspond to the two abovementioned boundary input configurations. Based on the transfer function model, both frequency and time responses of the system can be determined, which can be useful e.g. in the case of the model-based fault detection scheme

Steady-state analysis for a class of hyperbolic systems with boundary inputs

Results of a steady-state analysis performed for a class of distributed parameter systems described by hyperbolic partial differential equations defined on a one-dimensional spatial domain are presented. For the case of the system with two state variables and two boundary inputs, the analytical expressions for the steady-state distribution of the state variables are derived, both in the exponential and in the hyperbolic form. The influence of the location of the boundary inputs on the steady-state response is demonstrated. The considerations are illustrated with a practical example of a shell and tube heat exchanger operating in parallel- and countercurrent-flow modes

Parallel implementation of artificial neural network with use of MPI protocol

W artykule wskazano na pewne aspekty związane z implementacją jednokierunkowej sieci neuronowej w architekturze równoległej z wykorzystaniem standardu przesyłania komunikatów MPI. Zaprezentowany przykład zastosowania sieci dotyczy klasycznego problemu aproksymacji funkcji. Zbadano wpływ liczby uruchamianych procesów na efektywność procedury uczenia i działania sieci oraz zademonstrowano negatywny wpływ opóźnień powstałych przy przesyłaniu danych za pomocą sieci LAN.In the paper some characteristic features concerning feed-forward neural network implementation in parallel computer architecture using MPI communication protocol are investigated. Two fundamental methods of neural network parallelization are described: neural (Fig. 1) as well as synaptic parallelization (Fig. 2). Based on the presented methods, an original application implementing feed-forward multilayer neural network was built. The application includes: a Java runtime interface (Fig. 3) and a computational module based on the MPI communication protocol. The simulation tests consisted in neural network application to classical problem of nonlinear function approximation. Effect of the number of processes on the network learning efficiency was examined (Fig. 4, Tab. 1). The negative effect of transmission time delays in the LAN is also demonstrated in the paper. The authors conclude that computational advantages of neural networks parallelization on a heterogeneous cluster consisting of several personal computers will become apparent only in the case of very complex neural networks, composed of many thousands of neurons

Methods of artificial intelligence in applications of automatic control

W artykule przedstawiono wybrane aspekty zastosowania metod sztucznej inteligencji w zagadnieniach automatyki. Zawarto ogólny przegląd metod identyfikacji i sterowania opartych na sztucznych sieciach neuronowych. Wskazano także na możliwość wykorzystania logiki rozmytej do optymalizacji pracy samochodowego układu napędowego.In this paper, some issues concerning application of neural networks and fuzzy logic in automatic control are presented. Neural identification and control methods are briefly reviewed. Fuzzy logic approach to powertrain control of a passenger car is also shown