Exploring the Encoding Layer and Loss Function in End-to-End Speaker and Language Recognition System

In this paper, we explore the encoding/pooling layer and loss function in the end-to-end speaker and language recognition system. First, a unified and interpretable end-to-end system for both speaker and language recognition is developed. It accepts variable-length input and produces an utterance level result. In the end-to-end system, the encoding layer plays a role in aggregating the variable-length input sequence into an utterance level representation. Besides the basic temporal average pooling, we introduce a self-attentive pooling layer and a learnable dictionary encoding layer to get the utterance level representation. In terms of loss function for open-set speaker verification, to get more discriminative speaker embedding, center loss and angular softmax loss is introduced in the end-to-end system. Experimental results on Voxceleb and NIST LRE 07 datasets show that the performance of end-to-end learning system could be significantly improved by the proposed encoding layer and loss function.Comment: Accepted for Speaker Odyssey 201

Proceedings of the Second International Mobile Satellite Conference (IMSC 1990)

Presented here are the proceedings of the Second International Mobile Satellite Conference (IMSC), held June 17-20, 1990 in Ottawa, Canada. Topics covered include future mobile satellite communications concepts, aeronautical applications, modulation and coding, propagation and experimental systems, mobile terminal equipment, network architecture and control, regulatory and policy considerations, vehicle antennas, and speech compression

Energy Efficiency Optimization in Green Wireless Communications

The rising energy concern and the ubiquity of energy-consuming wireless applications have sparked a keen interest in the development and deployment of energy-efficient and eco-friendly wireless communication technology. Green Wireless Communications aims to find innovative solutions to improve energy efficiency, and to relieve/reduce the carbon footprint of wireless industry, while maintaining/improving performance metrics. Looking back at the wireless communications of the past decades, the air-interface design and network deployment had mainly focused on the spectral efficiency, instead of energy efficiency. From the cellular network to the personal area network, no matter what size the wireless network is, the milestones along the evolutions of wireless networks had always been higher-and-higher data rates throughout these years. Most of these throughput-oriented optimizations lead to a full-power operation to support a higher throughput or spectral efficiency, which is typically not energy-efficient. To qualify as green wireless communications, we believe that a candidate technology needs to be of high energy efficiency, reduced electromagnetic pollution, and low-complexity. In this dissertation research, towards the evolution of the green wireless communications, we have extended our efforts in two important aspects of the wireless communications system: air-interface and networking. In the first aspect of this work, we study a promising green communications technology, the time reversal system, as a novel air-interface of the future green wireless communications. We propose a concept of time reversal division multiple access (TRDMA) as a novel wireless media access scheme for wireless broadband networks, and investigate its fundamental theoretical limits. Motivated by the great energy-harvesting potential of the TRDMA, we develop an asymmetric architecture for the TRDMA based multiuser networks. The unique asymmetric architecture shifts the most complexity to the BS in both downlink and uplink schemes, facilitating very low-cost terminal users in the networks. To further enhance the system performance, a 2D parallel interference cancellation scheme is presented to explore the inherent structure of the interference signals, and therefore efficiently improve the resulting SINR and system performance. In the second aspect of this work, we explore the energy-saving potential of the cooperative networking for cellular systems. We propose a dynamic base-station switching strategy and incorporate the cooperative base-station operation to improve the energy-efficiency of the cellular networks without sacrificing the quality of service of the users. It is shown that significant energy saving potential can be achieved by the proposed scheme

Ultra Wideband Communications: from Analog to Digital

Ultrabreitband-Signale (Ultra Wideband [UWB]) können einen signifikanten Nutzen im Bereich drahtloser Kommunikationssysteme haben. Es sind jedoch noch einige Probleme offen, die durch Systemdesigner und Wissenschaftler gelöst werden müssen. Ein Funknetzsystem mit einer derart großen Bandbreite ist normalerweise auch durch eine große Anzahl an Mehrwegekomponenten mit jeweils verschiedenen Pfadamplituden gekennzeichnet. Daher ist es schwierig, die zeitlich verteilte Energie effektiv zu erfassen. Außerdem ist in vielen Fällen der naheliegende Ansatz, ein kohärenter Empfänger im Sinne eines signalangepassten Filters oder eines Korrelators, nicht unbedingt die beste Wahl. In der vorliegenden Arbeit wird dabei auf die bestehende Problematik und weitere Lösungsmöglichkeiten eingegangen. Im ersten Abschnitt geht es um „Impulse Radio UWB”-Systeme mit niedriger Datenrate. Bei diesen Systemen kommt ein inkohärenter Empfänger zum Einsatz. Inkohärente Signaldetektion stellt insofern einen vielversprechenden Ansatz dar, als das damit aufwandsgünstige und robuste Implementierungen möglich sind. Dies trifft vor allem in Anwendungsfällen wie den von drahtlosen Sensornetzen zu, wo preiswerte Geräte mit langer Batterielaufzeit nötigsind. Dies verringert den für die Kanalschätzung und die Synchronisation nötigen Aufwand, was jedoch auf Kosten der Leistungseffizienz geht und eine erhöhte Störempfindlichkeit gegenüber Interferenz (z.B. Interferenz durch mehrere Nutzer oder schmalbandige Interferenz) zur Folge hat. Um die Bitfehlerrate der oben genannten Verfahren zu bestimmen, wurde zunächst ein inkohärenter Combining-Verlust spezifiziert, welcher auftritt im Gegensatz zu kohärenter Detektion mit Maximum Ratio Multipath Combining. Dieser Verlust hängt von dem Produkt aus der Länge des Integrationsfensters und der Signalbandbreite ab. Um den Verlust durch inkohärentes Combining zu reduzieren und somit die Leistungseffizienz des Empfängers zu steigern, werden verbesserte Combining-Methoden für Mehrwegeempfang vorgeschlagen. Ein analoger Empfänger, bei dem der Hauptteil des Mehrwege-Combinings durch einen „Integrate and Dump”-Filter implementiert ist, wird für UWB-Systeme mit Zeit-Hopping gezeigt. Dabei wurde die Einsatzmöglichkeit von dünn besetzten Codes in solchen System diskutiert und bewertet. Des Weiteren wird eine Regel für die Code-Auswahl vorgestellt, welche die Stabilität des Systems gegen Mehrnutzer-Störungen sicherstellt und gleichzeitig den Verlust durch inkohärentes Combining verringert. Danach liegt der Fokus auf digitalen Lösungen bei inkohärenter Demodulation. Im Vergleich zum Analogempfänger besitzt ein Digitalempfänger einen Analog-Digital-Wandler im Zeitbereich gefolgt von einem digitalen Optimalfilter. Der digitale Optimalfilter dekodiert den Mehrfachzugriffscode kohärent und beschränkt das inkohärente Combining auf die empfangenen Mehrwegekomponenten im Digitalbereich. Es kommt ein schneller Analog-Digital-Wandler mit geringer Auflösung zum Einsatz, um einen vertretbaren Energieverbrauch zu gewährleisten. Diese Digitaltechnik macht den Einsatz langer Analogverzögerungen bei differentieller Demodulation unnötig und ermöglicht viele Arten der digitalen Signalverarbeitung. Im Vergleich zur Analogtechnik reduziert sie nicht nur den inkohärenten Combining-Verlust, sonder zeigt auch eine stärkere Resistenz gegenüber Störungen. Dabei werden die Auswirkungen der Auflösung und der Abtastrate der Analog-Digital-Umsetzung analysiert. Die Resultate zeigen, dass die verminderte Effizienz solcher Analog-Digital-Wandler gering ausfällt. Weiterhin zeigt sich, dass im Falle starker Mehrnutzerinterferenz sogar eine Verbesserung der Ergebnisse zu beobachten ist. Die vorgeschlagenen Design-Regeln spezifizieren die Anwendung der Analog-Digital-Wandler und die Auswahl der Systemparameter in Abhängigkeit der verwendeten Mehrfachzugriffscodes und der Modulationsart. Wir zeigen, wie unter Anwendung erweiterter Modulationsverfahren die Leistungseffizienz verbessert werden kann und schlagen ein Verfahren zur Unterdrückung schmalbandiger Störer vor, welches auf Soft Limiting aufbaut. Durch die Untersuchungen und Ergebnissen zeigt sich, dass inkohärente Empfänger in UWB-Kommunikationssystemen mit niedriger Datenrate ein großes Potential aufweisen. Außerdem wird die Auswahl der benutzbaren Bandbreite untersucht, um einen Kompromiss zwischen inkohärentem Combining-Verlust und Stabilität gegenüber langsamen Schwund zu erreichen. Dadurch wurde ein neues Konzept für UWB-Systeme erarbeitet: wahlweise kohärente oder inkohärente Empfänger, welche als UWB-Systeme Frequenz-Hopping nutzen. Der wesentliche Vorteil hiervon liegt darin, dass die Bandbreite im Basisband sich deutlich verringert. Mithin ermöglicht dies einfach zu realisierende digitale Signalverarbeitungstechnik mit kostengünstigen Analog-Digital-Wandlern. Dies stellt eine neue Epoche in der Forschung im Bereich drahtloser Sensorfunknetze dar. Der Schwerpunkt des zweiten Abschnitts stellt adaptiven Signalverarbeitung für hohe Datenraten mit „Direct Sequence”-UWB-Systemen in den Vordergrund. In solchen Systemen entstehen, wegen der großen Anzahl der empfangenen Mehrwegekomponenten, starke Inter- bzw. Intrasymbolinterferenzen. Außerdem kann die Funktionalität des Systems durch Mehrnutzerinterferenz und Schmalbandstörungen deutlich beeinflusst werden. Um sie zu eliminieren, wird die „Widely Linear”-Rangreduzierung benutzt. Dabei verbessert die Rangreduzierungsmethode das Konvergenzverhalten, besonders wenn der gegebene Vektor eine sehr große Anzahl an Abtastwerten beinhaltet (in Folge hoher einer Abtastrate). Zusätzlich kann das System durch die Anwendung der R-linearen Verarbeitung die Statistik zweiter Ordnung des nicht-zirkularen Signals vollständig ausnutzen, was sich in verbesserten Schätzergebnissen widerspiegelt. Allgemeine kann die Methode der „Widely Linear”-Rangreduzierung auch in andern Bereichen angewendet werden, z.B. in „Direct Sequence”-Codemultiplexverfahren (DS-CDMA), im MIMO-Bereich, im Global System for Mobile Communications (GSM) und beim Beamforming.The aim of this thesis is to investigate key issues encountered in the design of transmission schemes and receiving techniques for Ultra Wideband (UWB) communication systems. Based on different data rate applications, this work is divided into two parts, where energy efficient and robust physical layer solutions are proposed, respectively. Due to a huge bandwidth of UWB signals, a considerable amount of multipath arrivals with various path gains is resolvable at the receiver. For low data rate impulse radio UWB systems, suboptimal non-coherent detection is a simple way to effectively capture the multipath energy. Feasible techniques that increase the power efficiency and the interference robustness of non-coherent detection need to be investigated. For high data rate direct sequence UWB systems, a large number of multipath arrivals results in severe inter-/intra-symbol interference. Additionally, the system performance may also be deteriorated by multi-user interference and narrowband interference. It is necessary to develop advanced signal processing techniques at the receiver to suppress these interferences. Part I of this thesis deals with the co-design of signaling schemes and receiver architectures in low data rate impulse radio UWB systems based on non-coherent detection.● We analyze the bit error rate performance of non-coherent detection and characterize a non-coherent combining loss, i.e., a performance penalty with respect to coherent detection with maximum ratio multipath combining. The thorough analysis of this loss is very helpful for the design of transmission schemes and receive techniques innon-coherent UWB communication systems.● We propose to use optical orthogonal codes in a time hopping impulse radio UWB system based on an analog non-coherent receiver. The “analog” means that the major part of the multipath combining is implemented by an integrate and dump filter. The introduced semi-analytical method can help us to easily select the time hopping codes to ensure the robustness against the multi-user interference and meanwhile to alleviate the non-coherent combining loss.● The main contribution of Part I is the proposal of applying fully digital solutions in non-coherent detection. The proposed digital non-coherent receiver is based on a time domain analog-to-digital converter, which has a high speed but a very low resolution to maintain a reasonable power consumption. Compared to its analog counterpart, itnot only significantly reduces the non-coherent combining loss but also offers a higher interference robustness. In particular, the one-bit receiver can effectively suppress strong multi-user interference and is thus advantageous in separating simultaneously operating piconets.The fully digital solutions overcome the difficulty of implementing long analog delay lines and make differential UWB detection possible. They also facilitate the development of various digital signal processing techniques such as multi-user detection and non-coherent multipath combining methods as well as the use of advanced modulationschemes (e.g., M-ary Walsh modulation).● Furthermore, we present a novel impulse radio UWB system based on frequency hopping, where both coherent and non-coherent receivers can be adopted. The key advantage is that the baseband bandwidth can be considerably reduced (e.g., lower than 500 MHz), which enables low-complexity implementation of the fully digital solutions. It opens up various research activities in the application field of wireless sensor networks. Part II of this thesis proposes adaptive widely linear reduced-rank techniques to suppress interferences for high data rate direct sequence UWB systems, where second-order non-circular signals are used. The reduced-rank techniques are designed to improve the convergence performance and the interference robustness especially when the received vector contains a large number of samples (due to a high sampling rate in UWB systems). The widely linear processing takes full advantage of the second-order statistics of the non-circular signals and enhances the estimation performance. The generic widely linear reduced-rank concept also has a great potential in the applications of other systems such as Direct Sequence Code Division Multiple Access (DS-CDMA), Multiple Input Multiple Output (MIMO) system, and Global System for Mobile Communications (GSM), or in other areas such as beamforming

Adaptive antenna array beamforming using a concatenation of recursive least square and least mean square algorithms

In recent years, adaptive or smart antennas have become a key component for various wireless applications, such as radar, sonar and cellular mobile communications including worldwide interoperability for microwave access (WiMAX). They lead to an increase in the detection range of radar and sonar systems, and the capacity of mobile radio communication systems. These antennas are used as spatial filters for receiving the desired signals coming from a specific direction or directions, while minimizing the reception of unwanted signals emanating from other directions.Because of its simplicity and robustness, the LMS algorithm has become one of the most popular adaptive signal processing techniques adopted in many applications, including antenna array beamforming. Over the last three decades, several improvements have been proposed to speed up the convergence of the LMS algorithm. These include the normalized-LMS (NLMS), variable-length LMS algorithm, transform domain algorithms, and more recently the constrained-stability LMS (CSLMS) algorithm and modified robust variable step size LMS (MRVSS) algorithm. Yet another approach for attempting to speed up the convergence of the LMS algorithm without having to sacrifice too much of its error floor performance, is through the use of a variable step size LMS (VSSLMS) algorithm. All the published VSSLMS algorithms make use of an initial large adaptation step size to speed up the convergence. Upon approaching the steady state, smaller step sizes are then introduced to decrease the level of adjustment, hence maintaining a lower error floor. This convergence improvement of the LMS algorithm increases its complexity from 2N in the case of LMS algorithm to 9N in the case of the MRVSS algorithm, where N is the number of array elements.An alternative to the LMS algorithm is the RLS algorithm. Although higher complexity is required for the RLS algorithm compared to the LMS algorithm, it can achieve faster convergence, thus, better performance compared to the LMS algorithm. There are also improvements that have been made to the RLS algorithm families to enhance tracking ability as well as stability. Examples are, the adaptive forgetting factor RLS algorithm (AFF-RLS), variable forgetting factor RLS (VFFRLS) and the extended recursive least squares (EX-KRLS) algorithm. The multiplication complexity of VFFRLS, AFF-RLS and EX-KRLS algorithms are 2.5N2 + 3N + 20 , 9N2 + 7N , and 15N3 + 7N2 + 2N + 4 respectively, while the RLS algorithm requires 2.5N2 + 3N .All the above well known algorithms require an accurate reference signal for their proper operation. In some cases, several additional operating parameters should be specified. For example, MRVSS needs twelve predefined parameters. As a result, its performance highly depends on the input signal.In this study, two adaptive beamforming algorithms have been proposed. They are called recursive least square - least mean square (RLMS) algorithm, and least mean square - least mean square (LLMS) algorithm. These algorithms have been proposed for meeting future beamforming requirements, such as very high convergence rate, robust to noise and flexible modes of operation. The RLMS algorithm makes use of two individual algorithm stages, based on the RLS and LMS algorithms, connected in tandem via an array image vector. On the other hand, the LLMS algorithm is a simpler version of the RLMS algorithm. It makes use of two LMS algorithm stages instead of the RLS – LMS combination as used in the RLMS algorithm.Unlike other adaptive beamforming algorithms, for both of these algorithms, the error signal of the second algorithm stage is fed back and combined with the error signal of the first algorithm stage to form an overall error signal for use update the tap weights of the first algorithm stage.Upon convergence, usually after few iterations, the proposed algorithms can be switched to the self-referencing mode. In this mode, the entire algorithm outputs are swapped, replacing their reference signals. In moving target applications, the array image vector, F, should also be updated to the new position. This scenario is also studied for both proposed algorithms. A simple and effective method for calculate the required array image vector is also proposed. Moreover, since the RLMS and the LLMS algorithms employ the array image vector in their operation, they can be used to generate fixed beams by pre-setting the values of the array image vector to the specified direction.The convergence of RLMS and LLMS algorithms is analyzed for two different operation modes; namely with external reference or self-referencing. Array image vector calculations, ranges of step sizes values for stable operation, fixed beam generation, and fixed-point arithmetic have also been studied in this thesis. All of these analyses have been confirmed by computer simulations for different signal conditions. Computer simulation results show that both proposed algorithms are superior in convergence performances to the algorithms, such as the CSLMS, MRVSS, LMS, VFFRLS and RLS algorithms, and are quite insensitive to variations in input SNR and the actual step size values used. Furthermore, RLMS and LLMS algorithms remain stable even when their reference signals are corrupted by additive white Gaussian noise (AWGN). In addition, they are robust when operating in the presence of Rayleigh fading. Finally, the fidelity of the signal at the output of the proposed algorithms beamformers is demonstrated by means of the resultant values of error vector magnitude (EVM), and scatter plots. It is also shown that, the implementation of an eight element uniform linear array using the proposed algorithms with a wordlength of nine bits is sufficient to achieve performance close to that provided by full precision

An MDL framework for sparse coding and dictionary learning

The power of sparse signal modeling with learned over-complete dictionaries has been demonstrated in a variety of applications and fields, from signal processing to statistical inference and machine learning. However, the statistical properties of these models, such as under-fitting or over-fitting given sets of data, are still not well characterized in the literature. As a result, the success of sparse modeling depends on hand-tuning critical parameters for each data and application. This work aims at addressing this by providing a practical and objective characterization of sparse models by means of the Minimum Description Length (MDL) principle -- a well established information-theoretic approach to model selection in statistical inference. The resulting framework derives a family of efficient sparse coding and dictionary learning algorithms which, by virtue of the MDL principle, are completely parameter free. Furthermore, such framework allows to incorporate additional prior information to existing models, such as Markovian dependencies, or to define completely new problem formulations, including in the matrix analysis area, in a natural way. These virtues will be demonstrated with parameter-free algorithms for the classic image denoising and classification problems, and for low-rank matrix recovery in video applications

Simulations of Implementation of Advanced Communication Technologies

Wireless communication systems have seen significant advancements with the introduction of 3G, 4G, and 5G mobile standards. Since the simulation of entire systems is complex and may not allow evaluation of the impact of individual techniques, this thesis presents techniques and results for simulating the performance of advanced signaling techniques used in 3G, 4G, and 5G systems, including Code division multiple access (CDMA), Multiple Input Multiple Output (MIMO) systems, and Low-Density Parity Check (LDPC) codes. One implementation issue that is explored is the use of quantized Analog to Digital Converter (ADC) outputs and their impact on system performance. Code division multiple access (CDMA) is a popular wireless technique, but its effectiveness is limited by factors such as multiple access interference (MAI) and the near far effect (NFE). The joint effect of sampling and quantization on the analog-digital converter (ADC) at the receiver\u27s front end has also been evaluated for different quantization bits. It has been demonstrated that 4 bits is the minimum ADC resolution sensitivity required for a reliable connection for a quantized signal with 3- and 6-dB power levels in noisy and interference-prone environments. The demand for high data rate, reliable transmission, low bit error rate, and maximum transmission with low power has increased in wireless systems. Multiple Input Multiple Output (MIMO) systems with multiple antennas at both the transmitter and receiver side can meet these requirements by exploiting diversity and multipath propagation. The focus of MIMO systems is on improving reliability and maximizing throughput. Performance analysis of single input single output (SISO), single input multiple output (SIMO), multiple input single output (MISO), and MIMO systems is conducted using Alamouti space time block code (STBC) and Maximum Ratio Combining (MRC) technique used for transmit and receive diversity for Rayleigh fading channel under AWGN environment for BPSK and QPSK modulation schemes. Spatial Multiplexing (SM) is used to enhance spectral efficiency without additional bandwidth and power requirements. Minimum mean square error (MMSE) method is used for signal detection at the receiver end due to its low complexity and better performance. The performance of MIMO SM technique is compared for different antenna configurations and modulation schemes, and the MMSE detector is employed at the receiving end. Advanced error correction techniques for channel coding are necessary to meet the demand for Mobile Internet in 5G wireless communications, particularly for the Internet of Things. Low Density Parity Check (LDPC) codes are used for error correction in 5G, offering high coding gain, high throughput, low latency, low power dissipation, low complexity, and rate compatibility. LDPC codes use base matrices of 5G New Radio (NR) for LDPC encoding, and a soft decision decoding algorithm is used for efficient Frame Error Rate (FER) performance. The performance of LDPC codes is assessed using a soft decision decoding layered message passing algorithm, with BPSK modulation and AWGN channel. Furthermore, the effects of quantization on LDPC codes are analyzed for both small and large numbers of quantization bits