Quasi-Equivalence of Width and Depth of Neural Networks

While classic studies proved that wide networks allow universal approximation, recent research and successes of deep learning demonstrate the power of the network depth. Based on a symmetric consideration, we investigate if the design of artificial neural networks should have a directional preference, and what the mechanism of interaction is between the width and depth of a network. We address this fundamental question by establishing a quasi-equivalence between the width and depth of ReLU networks. Specifically, we formulate a transformation from an arbitrary ReLU network to a wide network and a deep network for either regression or classification so that an essentially same capability of the original network can be implemented. That is, a deep regression/classification ReLU network has a wide equivalent, and vice versa, subject to an arbitrarily small error. Interestingly, the quasi-equivalence between wide and deep classification ReLU networks is a data-driven version of the De Morgan law

Airfoil analysis and design using surrogate models

A study was performed to compare two different methods for generating surrogate models for the analysis and design of airfoils. Initial research was performed to compare the accuracy of surrogate models for predicting the lift and drag of an airfoil with data collected from highidelity simulations using a modern CFD code along with lower-order models using a panel code. This was followed by an evaluation of the Class Shape Trans- formation (CST) method for parameterizing airfoil geometries as a prelude to the use of surrogate models for airfoil design optimization and the implementation of software to use CST to modify airfoil shapes as part of the airfoil design process. Optimization routines were coupled with surrogate modeling techniques to study the accuracy and efficiency of the surrogate models to produce optimal airfoil shapes. Finally, the results of the current research are summarized, and suggestions are made for future research

Constraining cosmological parameters from N-body simulations with Variational Bayesian Neural Networks

Methods based on Deep Learning have recently been applied on astrophysical parameter recovery thanks to their ability to capture information from complex data. One of these methods is the approximate Bayesian Neural Networks (BNNs) which have demonstrated to yield consistent posterior distribution into the parameter space, helpful for uncertainty quantification. However, as any modern neural networks, they tend to produce overly confident uncertainty estimates and can introduce bias when BNNs are applied to data. In this work, we implement multiplicative normalizing flows (MNFs), a family of approximate posteriors for the parameters of BNNs with the purpose of enhancing the flexibility of the variational posterior distribution, to extract $\Omega_m$, $h$, and $\sigma_8$ from the QUIJOTE simulations. We have compared this method with respect to the standard BNNs, and the flipout estimator. We found that MNFs combined with BNNs outperform the other models obtaining predictive performance with almost one order of magnitude larger that standard BNNs, $\sigma_8$ extracted with high accuracy ($r^2=0.99$), and precise uncertainty estimates. The latter implies that MNFs provide more realistic predictive distribution closer to the true posterior mitigating the bias introduced by the variational approximation and allowing to work with well-calibrated networks.Comment: 15 pages, 4 figures, 3 tables, submitted. Comments welcom

Automatic analysis of electronic drawings using neural network

Neural network technique has been found to be a powerful tool in pattern recognition. It captures associations or discovers regularities with a set of patterns, where the types, number of variables or diversity of the data are very great, the relationships between variables are vaguely understood, or the relationships are difficult to describe adequately with conventional approaches. In this dissertation, which is related to the research and the system design aiming at recognizing the digital gate symbols and characters in electronic drawings, we have proposed: (1) A modified Kohonen neural network with a shift-invariant capability in pattern recognition; (2) An effective approach to optimization of the structure of the back-propagation neural network; (3) Candidate searching and pre-processing techniques to facilitate the automatic analysis of the electronic drawings. An analysis and the system performance reveal that when the shift of an image pattern is not large, and the rotation is only by nx90°, (n = 1, 2, and 3), the modified Kohonen neural network is superior to the conventional Kohonen neural network in terms of shift-invariant and limited rotation-invariant capabilities. As a result, the dimensionality of the Kohonen layer can be reduced significantly compared with the conventional ones for the same performance. Moreover, the size of the subsequent neural network, say, back-propagation feed-forward neural network, can be decreased dramatically. There are no known rules for specifying the number of nodes in the hidden layers of a feed-forward neural network. Increasing the size of the hidden layer usually improves the recognition accuracy, while decreasing the size generally improves generalization capability. We determine the optimal size by simulation to attain a balance between the accuracy and generalization. This optimized back-propagation neural network outperforms the conventional ones designed by experience in general. In order to further reduce the computation complexity and save the calculation time spent in neural networks, pre-processing techniques have been developed to remove long circuit lines in the electronic drawings. This made the candidate searching more effective

Bio-Inspired Mechanism for Aircraft Assessment Under Upset Conditions

Based on the artificial immune systems paradigm and a hierarchical multi-self strategy, a set of algorithms for aircraft sub-systems failure detection, identification, evaluation and flight envelope estimation has been developed and implemented. Data from a six degrees-of-freedom flight simulator were used to define a large set of 2-dimensional self/non-self projections as well as for the generation of antibodies and identifiers designated for health assessment of an aircraft under upset conditions. The methodology presented in this paper classifies and quantifies the type and severity of a broad number of aircraft actuators, sensors, engine and structural component failures. In addition, the impact of these upset conditions on the flight envelope is estimated using nominal test data. Based on immune negative and positive selection mechanisms, a heuristic selection of sub-selves and the formulation of a mapping- based algorithm capable of selectively capturing the dynamic fingerprint of upset conditions is implemented. The performance of the approach is assessed in terms of detection and identification rates, false alarms, and correct prediction of flight envelope reduction with respect to specific states. Furthermore, this methodology is implemented in flight test by using an unmanned aerial vehicle subjected to nominal and four different abnormal flight conditions instrumented with a low cost microcontroller

Dynamic fuzzy rule interpolation and its application to intrusion detection

Fuzzy rule interpolation (FRI) offers an effective approach for making inference possible in sparse rule-based systems (and also for reducing the complexity of fuzzy models). However, requirements of fuzzy systems may change over time and hence, the use of a static rule base may affect the accuracy of FRI applications. Fortunately, an FRI system in action will produce interpolated rules in abundance during the interpolative reasoning process. While such interpolated results are discarded in existing FRI systems, they can be utilized to facilitate the development of a dynamic rule base in supporting subsequent inference. This is because the otherwise relinquished interpolated rules may contain possibly valuable information, covering regions that were uncovered by the original sparse rule base. This paper presents a dynamic fuzzy rule interpolation (D-FRI) approach by exploiting such interpolated rules in order to improve the overall system's coverage and efficacy. The resulting D-FRI system is able to select, combine, and generalize informative, frequently used interpolated rules for merging with the existing rule base while performing interpolative reasoning. Systematic experimental investigations demonstrate that D-FRI outperforms conventional FRI techniques, with increased accuracy and robustness. Furthermore, D-FRI is herein applied for network security analysis, in devising a dynamic intrusion detection system (IDS) through integration with the Snort software, one of the most popular open source IDSs. This integration, denoted as D-FRI-Snort hereafter, delivers an extra amount of intelligence to predict the level of potential threats. Experimental results show that with the inclusion of a dynamic rule base, by generalising newly interpolated rules based on the current network traffic conditions, D-FRI-Snort helps reduce both false positives and false negatives in intrusion detection

Literature Review For Networking And Communication Technology

Report documents the results of a literature search performed in the area of networking and communication technology