Using auxiliary sources of knowledge for automatic speech recognition

Standard hidden Markov model (HMM) based automatic speech recognition (ASR) systems usually use cepstral features as acoustic observation and phonemes as subword units. Speech signal exhibits wide range of variability such as, due to environmental variation, speaker variation. This leads to different kinds of mismatch, such as, mismatch between acoustic features and acoustic models or mismatch between acoustic features and pronunciation models (given the acoustic models). The main focus of this work is on integrating auxiliary knowledge sources into standard ASR systems so as to make the acoustic models more robust to the variabilities in the speech signal. We refer to the sources of knowledge that are able to provide additional information about the sources of variability as auxiliary sources of knowledge. The auxiliary knowledge sources that have been primarily investigated in the present work are auxiliary features and auxiliary subword units. Auxiliary features are secondary source of information that are outside of the standard cepstral features. They can be estimation from the speech signal (e.g., pitch frequency, short-term energy and rate-of-speech), or additional measurements (e.g., articulator positions or visual information). They are correlated to the standard acoustic features, and thus can aid in estimating better acoustic models, which would be more robust to variabilities present in the speech signal. The auxiliary features that have been investigated are pitch frequency, short-term energy and rate-of-speech. These features can be modelled in standard ASR either by concatenating them to the standard acoustic feature vectors or by using them to condition the emission distribution (as done in gender-based acoustic modelling). We have studied these two approaches within the framework of hybrid HMM/artificial neural networks based ASR, dynamic Bayesian network based ASR and TANDEM system on different ASR tasks. Our studies show that by modelling auxiliary features along with standard acoustic features the performance of the ASR system can be improved in both clean and noisy conditions. We have also proposed an approach to evaluate the adequacy of the baseform pronunciation model of words. This approach allows us to compare between different acoustic models as well as to extract pronunciation variants. Through the proposed approach to evaluate baseform pronunciation model, we show that the matching and discriminative properties of single baseform pronunciation can be improved by integrating auxiliary knowledge sources in standard ASR. Standard ASR systems use usually phonemes as the subword units in a Markov chain to model words. In the present thesis, we also study a system where word models are described by two parallel chains of subword units: one for phonemes and the other are for graphemes (phoneme-grapheme based ASR). Models for both types of subword units are jointly learned using maximum likelihood training. During recognition, decoding is performed using either or both of the subword unit chains. In doing so, we thus have used graphemes as auxiliary subword units. The main advantage of using graphemes is that the word models can be defined easily using the orthographic transcription, thus being relatively noise free as compared to word models based upon phoneme units. At the same time, there are drawbacks to using graphemes as subword units, since there is a weak correspondence between the grapheme and the phoneme in languages such as English. Experimental studies conducted for American English on different ASR tasks have shown that the proposed phoneme-grapheme based ASR system can perform better than the standard ASR system that uses only phonemes as its subword units. Furthermore, while modelling context-dependent graphemes (similar to context-dependent phonemes), we observed that context-dependent graphemes behave like phonemes. ASR studies conducted on different tasks showed that by modelling context-dependent graphemes only (without any phonetic information) performance competitive to the state-of-the-art context-dependent phoneme-based ASR system can be obtained

Use of Entropy for Feature Selection with Intrusion Detection System Parameters

The metric of entropy provides a measure about the randomness of data and a measure of information gained by comparing different attributes. Intrusion detection systems can collect very large amounts of data, which are not necessarily manageable by manual means. Collected intrusion detection data often contains redundant, duplicate, and irrelevant entries, which makes analysis computationally intensive likely leading to unreliable results. Reducing the data to what is relevant and pertinent to the analysis requires the use of data mining techniques and statistics. Identifying patterns in the data is part of analysis for intrusion detections in which the patterns are categorized as normal or anomalous. Anomalous data needs to be further characterized to determine if representative attacks to the network are in progress. Often time subtleties in the data may be too muted to identify certain types of attacks. Many statistics including entropy are used in a number of analysis techniques for identifying attacks, but these analyzes can be improved upon. This research expands the use of Approximate entropy and Sample entropy for feature selection and attack analysis to identify specific types of subtle attacks to network systems. Through enhanced analysis techniques using entropy, the granularity of feature selection and attack identification is improved

Design of static intercell interference coordination schemes for realistic lte-based cellular networks

Today, 3.5 and 4G systems including Long Term Evolution (LTE) and LTE-Advanced (LTE-A) support packet-based services and provide mobile broadband access for bandwidth-hungry applications. In this context of fast evolution, new and challenging technical issues must be e ectively addressed. The nal target is to achieve a signi cant step forward toward the improvement of the Quality of Experience (QoE). To that end, interference management has been recognized by the industry as a key enabler for cellular technologies based on OFDMA. Indeed, with a low frequency reuse factor, intercell interference (ICI) becomes a major concern since the Quality of Service (QoS) is not uniformly delivered across the network, it remarkably depends on user position. Hence, cell edge performance is an important issue in LTE and LTE-A. Intercell Interference Coordination (ICIC) encompasses strategies whose goal is to keep ICI at cell edges as low as possible. This alleviates the aforementioned situation. For this reason, the novelties presented in this Ph.D. thesis include not only developments of static ICIC mechanisms for data and control channels, but also e orts towards further improvements of the energy e ciency perspective. Based on a comprehensive review of the state of the art, a set of research opportunities were identi ed. To be precise, the need for exible performance evaluation methods and optimization frameworks for static ICIC strategies. These mechanisms are grouped in two families: the schemes that de ne constraints on the frequency domain and the ones that propose adjustments on the power levels. Thus, Soft- and Fractional Frequency Reuse (SFR and FFR, respectively) are identi ed as the base of the vast majority of static ICIC proposals. Consequently, during the rst part of this Ph.D. thesis, interesting insights into the operation of SFR and FFR were identi ed beyond well-known facts. These studies allow for the development of a novel statistical framework to evaluate the performance of these schemes in realistic deployments. As a result of the analysis, the poor performance of classic con gurations of SFR and FFR in real-world contexts is shown, and hence, the need for optimization is established. In addition, the importance of the interworking between static ICIC schemes and other network functionalities such as CSI feedback has also been identi ed. Therefore, novel CSI feedback schemes, suitable to operate in conjunction with SFR and FFR, have been developed. These mechanisms exploit the resource allocation pattern of these static ICIC techniques in order to improve the accuracy of the CSI feedback process. The second part is focused on the optimization of SFR and FFR. The use of multiobjective techniques is investigated as a tool to achieve e ective network-speci c optimization. The approach o ers interesting advantages. On the one hand, it allows for simultaneous optimization of several con icting criteria. On the other hand, the multiobjective nature results in outputs composed of several high quality (Pareto e cient) network con gurations, all of them featuring a near-optimal tradeo between the performance criteria. Multiobjective evolutionary algorithms allow employing complex mathematical structures without the need for relaxation, thus capturing accurately the system behavior in terms of ICI. The multiobjective optimization formulation of the problem aims at achieving e ective adjustment of the operational parameters of SFR and FFR both at cell level and network-wide. Moreover, the research was successfully extended to the control channels, both the PDCCH and ePDCCH. Finally, in an e ort to further improve the network energy e ciency (an aspect always considered throughout the thesis), the framework of Cell Switch O (CSO), having close connections with ICIC, is also introduced. By means of the proposed method, signi cant improvements with respect to traditional approaches, baseline con gurations, and previous proposals can be achieved. The gains are obtained in terms of energy consumption, network capacity, and cell edge performance.Actualmente los sistemas 3.5 y 4G tales como Long Term Evolution (LTE) y LTE-Advanced (LTE-A) soportan servicios basados en paquetes y proporcionan acceso de banda ancha m ovil para aplicaciones que requieren elevadas tasas de transmisi on. En este contexto de r apida evoluci on, aparecen nuevos retos t ecnicos que deben ser resueltos e cientemente. El objetivo ultimo es conseguir un salto cualitativo importante en la experiencia de usuario (QoE). Con tal n, un factor clave que ha sido reconocido en las redes celulares basadas en Orthogonal Frequency- Division Multiple Access (OFDMA) es la gesti on de interferencias. De hecho, la utilizaci on de un factor de reuso bajo permite una elevada e ciencia espectral pero a costa de una distribuci on de la calidad de servicio (QoS) que no es uniforme en la red, depende de la posici on del usuario. Por lo tanto, el rendimiento en los l mites de la celda se ve muy penalizado y es un problema importante a resolver en LTE y LTE-A. La coordinaci on de interferencias entre celdas (ICIC, del ingl es Intercell Interfe- rence Coordination) engloba las estrategias cuyo objetivo es mantener la interferencia intercelular (ICI) lo m as baja posible en los bordes de celda. Esto permite aliviar la situaci on antes mencionada. La contribuci on presentada en esta tesis doctoral incluye el dise~no de nuevos mecanismos de ICIC est atica para los canales de datos y control, as como tambi en mejoras desde el punto de vista de e ciencia energ etica. A partir de una revisi on completa del estado del arte, se identi caron una serie de retos abiertos que requer an esfuerzos de investigaci on. En concreto, la necesidad de m etodos de evaluaci on exibles y marcos de optimizaci on de las estrategias de ICIC est aticas. Estos mecanismos se agrupan en dos familias: los esquemas que de nen restricciones sobre el dominio de la frecuencia y los que proponen ajustes en los niveles de potencia. Es decir, la base de la gran mayor a de propuestas ICIC est aticas son la reutilizaci on de frecuencias de tipo soft y fraccional (SFR y FFR, respectivamente). De este modo, durante la primera parte de esta tesis doctoral, se han estudiado los aspectos m as importantes del funcionamiento de SFR y FFR, haciendo especial enfasis en las conclusiones que van m as all a de las bien conocidas. Ello ha permitido introducir un nuevo marco estad stico para evaluar el funcionamiento de estos sistemas en condiciones de despliegue reales. Como resultado de estos an alisis, se muestra el pobre desempe~no de SFR y FFR en despliegues reales cuando funcionan con sus con guraciones cl asicas y se establece la necesidad de optimizaci on. Tambi en se pone de mani esto la importancia del funcionamiento conjunto entre esquemas ICIC est aticos y otras funcionalidades de la red radio, tales como la informaci on que env an los usuarios sobre el estado de su canal downlink (feedback del CSI, del ingl es Channel State Information). De este modo, se han propuesto diferentes esquemas de feedback apropiados para trabajar conjuntamente con SFR y FFR. Estos mecanismos explotan el patr on de asignaci on de recursos que se utiliza en ICIC est atico para mejorar la precisi on del proceso. La segunda parte se centra en la optimizaci on de SFR y FFR. Se ha investigado el uso de t ecnicas multiobjetivo como herramienta para lograr una optimizaci on e caz, que es espec ca para cada red. El enfoque ofrece ventajas interesantes, por un lado, se permite la optimizaci on simult anea de varios criterios contradictorios. Por otro lado, la naturaleza multiobjetivo implica obtener como resultado con guraciones de red de elevada calidad (Pareto e cientes), todas ellas con un equilibrio casi- optimo entre las diferentes m etricas de rendimiento. Los algoritmos evolucionarios multiobjetivo permiten la utilizaci on de estructuras matem aticas complejas sin necesidad de relajar el problema, de este modo capturan adecuadamente su comportamiento en t erminos de ICI. La formulaci on multiobjetivo consigue un ajuste efectivo de los par ametros operacionales de SFR y FFR, tanto a nivel de celda como a nivel de red. Adem as, la investigaci on se extiende con resultados satisfactorios a los canales de control, PDCCH y ePDCCH. Finalmente, en un esfuerzo por mejorar la e ciencia energ etica de la red (un aspecto siempre considerado a lo largo de la tesis), se introduce en el an alisis global el apagado inteligente de celdas, estrategia con estrechos v nculos con ICIC. A trav es del m etodo propuesto, se obtienen mejoras signi cativas con respecto a los enfoques tradicionales y propuestas previas. Las ganancias se obtienen en t erminos de consumo energ etico, capacidad de la red, y rendimiento en el l mite de las celdas.Actualment els sistemes 3.5 i 4G tals com Long Term Evolution (LTE) i LTE- Advanced (LTE-A) suporten serveis basats en paquets i proporcionen acc es de banda ampla m obil per a aplicacions que requereixen elevades taxes de transmissi o. En aquest context de r apida evoluci o, apareixen nous reptes t ecnics que han de ser resolts e cientment. L'objectiu ultim es aconseguir un salt qualitatiu important en l'experi encia d'usuari (QoE). Amb tal , un factor clau que ha estat reconegut a les xarxes cel lulars basades en Orthogonal Frequency-Division Multiple Access (OFDMA) es la gesti o d'interfer encies. De fet, la utilizaci o d'un factor de re us baix permet una elevada e ci encia espectral per o a costa d'una distribuci o de la qualitat de servei (QoS) que no es uniforme a la xarxa, dep en de la posici o de l'usuari. Per tant, el rendiment en els l mits de la cel la es veu molt penalitzat i es un problema important a resoldre en LTE i LTE-A. La coordinaci o d'interfer encies entre cel les (ICIC, de l'angl es Intercell Interfe- rence Coordination) engloba les estrat egies que tenen com a objectiu mantenir la interfer encia intercel lular (ICI) el m es baixa possible en les vores de la cel la. Aix o permet alleujar la situaci o abans esmentada. La contribuci o presentada en aquesta tesi doctoral inclou el disseny de nous mecanismes de ICIC est atica per als canals de dades i control, aix com tamb e millores des del punt de vista d'e ci encia energ etica. A partir d'una revisi o completa de l'estat de l'art, es van identi car una s erie de reptes oberts que requerien esfor cos de recerca. En concret, la necessitat de m etodes d'avaluaci o exibles i marcs d'optimitzaci o de les estrat egies de ICIC est atiques. Aquests mecanismes s'agrupen en dues fam lies: els esquemes que de neixen restriccions sobre el domini de la freq u encia i els que proposen ajustos en els nivells de pot encia. Es a dir, la base de la gran majoria de propostes ICIC est atiques s on la reutilitzaci o de freq u encies de tipus soft i fraccional (SFR i FFR, respectivament). D'aquesta manera, durant la primera part d'aquesta tesi doctoral, s'han estudiat els aspectes m es importants del funcionament de SFR i FFR, fent especial emfasi en les conclusions que van m es enll a de les ben conegudes. Aix o ha perm es introduir un nou marc estad stic per avaluar el funcionament d'aquests sistemes en condicions de desplegament reals. Com a resultat d'aquestes an alisis, es mostra el pobre acompliment de SFR i FFR en desplegaments reals quan funcionen amb les seves con guracions cl assiques i s'estableix la necessitat d'optimitzaci o. Tamb e es posa de manifest la import ancia del funcionament conjunt entre esquemes ICIC est atics i altres funcionalitats de la xarxa radio, tals com la informaci o que envien els usuaris sobre l'estat del seu canal downlink (feedback del CSI, de l'angl es Channel State Information). D'aquesta manera, s'han proposat diferents esquemes de feedback apropiats per treballar conjuntament amb SFR i FFR. Aquests mecanismes exploten el patr o d'assignaci o de recursos que s'utilitza en ICIC est atic per millorar la precisi o del proc es. La segona part se centra en l'optimitzaci o de SFR i FFR. S'ha investigat l' us de t ecniques multiobjectiu com a eina per aconseguir una optimitzaci o e ca c, que es espec ca per a cada xarxa. L'enfocament ofereix avantatges interessants, d'una banda, es permet l'optimitzaci o simult ania de diversos criteris contradictoris. D'altra banda, la naturalesa multiobjectiu implica obtenir com resultat con guracions de xarxa d'elevada qualitat (Pareto e cients), totes elles amb un equilibri gaireb e optim entre les diferents m etriques de rendiment. Els algorismes evolucionaris multiobjectiu permeten la utilitzaci o d'estructures matem atiques complexes sense necessitat de relaxar el problema, d'aquesta manera capturen adequadament el seu comportament en termes de ICI. La formulaci o multiobjectiu aconsegueix un ajust efectiu dels par ametres operacionals de SFR i FFR, tant a nivell de cel la com a nivell de xarxa. A m es, la recerca s'est en amb resultats satisfactoris als canals de control, PDCCH i ePDCCH. Finalment, en un esfor c per millorar l'e ci encia energ etica de la xarxa (un aspecte sempre considerat al llarg de la tesi), s'introdueix en l'an alisi global l'apagat intel ligent de cel les, estrat egia amb estrets vincles amb ICIC. Mitjan cant el m etode proposat, s'obtenen millores signi catives pel que fa als enfocaments tradicionals i propostes pr evies. Els guanys s'obtenen en termes de consum energ etic, capacitat de la xarxa, i rendiment en el l mit de les cel les