Deep Complex Networks

At present, the vast majority of building blocks, techniques, and architectures for deep learning are based on real-valued operations and representations. However, recent work on recurrent neural networks and older fundamental theoretical analysis suggests that complex numbers could have a richer representational capacity and could also facilitate noise-robust memory retrieval mechanisms. Despite their attractive properties and potential for opening up entirely new neural architectures, complex-valued deep neural networks have been marginalized due to the absence of the building blocks required to design such models. In this work, we provide the key atomic components for complex-valued deep neural networks and apply them to convolutional feed-forward networks and convolutional LSTMs. More precisely, we rely on complex convolutions and present algorithms for complex batch-normalization, complex weight initialization strategies for complex-valued neural nets and we use them in experiments with end-to-end training schemes. We demonstrate that such complex-valued models are competitive with their real-valued counterparts. We test deep complex models on several computer vision tasks, on music transcription using the MusicNet dataset and on Speech Spectrum Prediction using the TIMIT dataset. We achieve state-of-the-art performance on these audio-related tasks

Real-Time Machine Learning for Quickest Detection

Safety-critical Cyber-Physical Systems (CPS) require real-time machine learning for control and decision making. One promising solution is to use deep learning to discover useful patterns for event detection from heterogeneous data. However, deep learning algorithms encounter challenges in CPS with assurability requirements: 1) Decision explainability, 2) Real-time and quickest event detection, and 3) Time-eficient incremental learning. To address these obstacles, I developed a real-time Machine Learning Framework for Quickest Detection (MLQD). To be specific, I first propose the zero-bias neural network, which removes decision bias and preferabilities from regular neural networks and provides an interpretable decision process. Second, I discover the latent space characteristic of the zero-bias neural network and the method to mathematically convert a Deep Neural Network (DNN) classifier into a performance-assured binary abnormality detector. In this way, I can seamlessly integrate the deep neural networks\u27 data processing capability with Quickest Detection (QD) and provide real-time sequential event detection paradigm. Thirdly, after discovering that a critical factor that impedes the incremental learning of neural networks is the concept interference (confusion) in latent space, and I prove that to minimize interference, the concept representation vectors (class fingerprints) within the latent space need to be organized orthogonally and I invent a new incremental learning strategy using the findings, I facilitate deep neural networks in the CPS to evolve eficiently without retraining. All my algorithms are evaluated on real-world applications, ADS-B (Automatic Dependent Surveillance Broadcasting) signal identification, and spoofing detection in the aviation communication system. Finally, I discuss the current trends in MLQD and conclude this dissertation by presenting the future research directions and applications. As a summary, the innovations of this dissertation are as follows: i) I propose the zerobias neural network, which provides transparent latent space characteristics, I apply it to solve the wireless device identification problem. ii) I discover and prove the orthogonal memory organization mechanism in artificial neural networks and apply this mechanism in time-efficient incremental learning. iii) I discover and mathematically prove the converging point theorem, with which we can predict the latent space topological characteristics and estimate the topological maturity of neural networks. iv) I bridge the gap between machine learning and quickest detection with assurable performance

Dimension Reduction by Mutual Information Discriminant Analysis

In the past few decades, researchers have proposed many discriminant analysis (DA) algorithms for the study of high-dimensional data in a variety of problems. Most DA algorithms for feature extraction are based on transformations that simultaneously maximize the between-class scatter and minimize the withinclass scatter matrices. This paper presents a novel DA algorithm for feature extraction using mutual information (MI). However, it is not always easy to obtain an accurate estimation for high-dimensional MI. In this paper, we propose an efficient method for feature extraction that is based on one-dimensional MI estimations. We will refer to this algorithm as mutual information discriminant analysis (MIDA). The performance of this proposed method was evaluated using UCI databases. The results indicate that MIDA provides robust performance over different data sets with different characteristics and that MIDA always performs better than, or at least comparable to, the best performing algorithms.Comment: 13pages, 3 tables, International Journal of Artificial Intelligence & Application

4. generációs mobil rendszerek kutatása = Research on 4-th Generation Mobile Systems

A 3G mobil rendszerek szabványosítása a végéhez közeledik, legalábbis a meghatározó képességek tekintetében. Ezért létfontosságú azon technikák, eljárások vizsgálata, melyek a következő, 4G rendszerekben meghatározó szerepet töltenek majd be. Több ilyen kutatási irányvonal is létezik, ezek közül projektünkben a fontosabbakra koncentráltunk. A következőben felsoroljuk a kutatott területeket, és röviden összegezzük az elért eredményeket. Szórt spektrumú rendszerek Kifejlesztettünk egy új, rádiós interfészen alkalmazható hívásengedélyezési eljárást. Szimulációs vizsgálatokkal támasztottuk alá a megoldás hatékonyságát. A projektben kutatóként résztvevő Jeney Gábor sikeresen megvédte Ph.D. disszertációját neurális hálózatokra épülő többfelhasználós detekciós technikák témában. Az elért eredmények Imre Sándor MTA doktori disszertációjába is beépültek. IP alkalmazása mobil rendszerekben Továbbfejlesztettük, teszteltük és általánosítottuk a projekt keretében megalkotott új, gyűrű alapú topológiára épülő, a jelenleginél nagyobb megbízhatóságú IP alapú hozzáférési koncepciót. A témakörben Szalay Máté Ph.D. disszertációja már a nyilvános védésig jutott. Kvantum-informatikai módszerek alkalmazása 3G/4G detekcióra Új, kvantum-informatikai elvekre épülő többfelhasználós detekciós eljárást dolgoztunk ki. Ehhez új kvantum alapú algoritmusokat is kifejlesztettünk. Az eredményeket nemzetközi folyóiratok mellett egy saját könyvben is publikáltuk. | The project consists of three main research directions. Spread spectrum systems: we developed a new call admission control method for 3G air interfaces. Project member Gabor Jeney obtained the Ph.D. degree and project leader Sandor Imre submitted his DSc theses from this area. Application of IP in mobile systems: A ring-based reliable IP mobility mobile access concept and corresponding protocols have been developed. Project member Máté Szalay submitted his Ph.D. theses from this field. Quantum computing based solutions in 3G/4G detection: Quantum computing based multiuser detection algorithm was developed. Based on the results on this field a book was published at Wiley entitled: 'Quantum Computing and Communications - an engineering approach'

Coherent Optical OFDM Modem Employing Artificial Neural Networks for Dispersion and Nonlinearity Compensation in a Long-Haul Transmission System

In order to satisfy the ever increasing demand for the bandwidth requirement in broadband services the optical orthogonal frequency division multiplexing (OOFDM) scheme is being considered as a promising technique for future high-capacity optical networks. The aim of this thesis is to investigate, theoretically, the feasibility of implementing the coherent optical OFDM (CO-OOFDM) technique in long haul transmission networks. For CO-OOFDM and Fast-OFDM systems a set of modulation formats dependent analogue to digital converter (ADC) clipping ratio and the quantization bit have been identified, moreover, CO-OOFDM is more resilient to the chromatic dispersion (CD) when compared to the bandwidth efficient Fast-OFDM scheme. For CO-OOFDM systems numerical simulations are undertaken to investigate the effect of the number of sub-carriers, the cyclic prefix (CP), and ADC associated parameters such as the sampling speed, the clipping ratio, and the quantisation bit on the system performance over single mode fibre (SMF) links for data rates up to 80 Gb/s. The use of a large number of sub-carriers is more effective in combating the fibre CD compared to employing a long CP. Moreover, in the presence of fibre non-linearities identifying the optimum number of sub-carriers is a crucial factor in determining the modem performance. For a range of signal data rates up to 40 Gb/s, a set of data rate and transmission distance-dependent optimum ADC parameters are identified in this work. These parameters give rise to a negligible clipping and quantisation noise, moreover, ADC sampling speed can increase the dispersion tolerance while transmitting over SMF links. In addition, simulation results show that the use of adaptive modulation schemes improves the spectrum usage efficiency, thus resulting in higher tolerance to the CD when compared to the case where identical modulation formats are adopted across all sub-carriers. For a given transmission distance utilizing an artificial neural networks (ANN) equalizer improves the system bit error rate (BER) performance by a factor of 50% and 70%, respectively when considering SMF firstly CD and secondly nonlinear effects with CD. Moreover, for a fixed BER of 10-3 utilizing ANN increases the transmission distance by 1.87 times and 2 times, respectively while considering SMF CD and nonlinear effects. The proposed ANN equalizer performs more efficiently in combating SMF non-linearities than the previously published Kerr nonlinearity electrical compensation technique by a factor of 7