Structured parameter estimation for LFG-DOP using Backoff

Despite its state-of-the-art performance, the Data Oriented Parsing (DOP) model has been shown to suffer from biased parameter estimation, and the good performance seems more the result of ad hoc adjustments than correct probabilistic generalization over the data. In recent work, we developed a new estimation procedure, called Backoff Estimation, for DOP models that are based on Phrase-Structure annotations (so called Tree-DOP models). Backoff Estimation deviates from earlier methods in that it treats the model parameters as a highly structured space of correlated events (backoffs), rather than a set of disjoint events. In this paper we show that the problem of biased estimates also holds for DOP models that are based on Lexical-Functional Grammar annotations (i.e. LFG-DOP), and that the LFG-DOP parameters also constitute a hierarchically structured space. Subsequently, we adapt the Backoff Estimation algorithm from Tree-DOP to LFG-DOP models. Backoff Estimation turns out to be a natural solution to some of the specific problems of robust parsing under LFGDOP

Linguistic Constraints in LFG-DOP

LFG-DOP (Bod and Kaplan, 1998, 2003) provides an appealing answer to the question of how probabilistic methods can be incorporated into linguistic theory. However, despite its attractions, the standard model of LFG-DOP suffers from serious problems of overgeneration, because (a) it is unable to define fragments of the right level of generality, and (b) it has no way of capturing the effect of anything except simple positive constraints. We show how the model can be extended to overcome these problems. The question of how probabilistic methods should be incorporated into linguistic theory is important from both a practical, grammar engineering, perspective, and from the perspective of ‘pure ’ linguistic theory. From a practical point of view such techniques are essential if a system is to achieve a useful breadth of coverag

Parsing Inside-Out

The inside-outside probabilities are typically used for reestimating Probabilistic Context Free Grammars (PCFGs), just as the forward-backward probabilities are typically used for reestimating HMMs. I show several novel uses, including improving parser accuracy by matching parsing algorithms to evaluation criteria; speeding up DOP parsing by 500 times; and 30 times faster PCFG thresholding at a given accuracy level. I also give an elegant, state-of-the-art grammar formalism, which can be used to compute inside-outside probabilities; and a parser description formalism, which makes it easy to derive inside-outside formulas and many others.Comment: Ph.D. Thesis, 257 pages, 40 postscript figure

Inducing Tree-Substitution Grammars

Inducing a grammar from text has proven to be a notoriously challenging learning task despite decades of research. The primary reason for its difficulty is that in order to induce plausible grammars, the underlying model must be capable of representing the intricacies of language while also ensuring that it can be readily learned from data. The majority of existing work on grammar induction has favoured model simplicity (and thus learnability) over representational capacity by using context free grammars and first order dependency grammars, which are not sufficiently expressive to model many common linguistic constructions. We propose a novel compromise by inferring a probabilistic tree substitution grammar, a formalism which allows for arbitrarily large tree fragments and thereby better represent complex linguistic structures. To limit the model's complexity we employ a Bayesian non-parametric prior which biases the model towards a sparse grammar with shallow productions. We demonstrate the model's efficacy on supervised phrase-structure parsing, where we induce a latent segmentation of the training treebank, and on unsupervised dependency grammar induction. In both cases the model uncovers interesting latent linguistic structures while producing competitive results. © 2010 Evangelos Theodorou, Jonas Buchli and Stefan Schaal

Mac-Phy Cross-Layer analysis and design of Mimo-Ofdm Wlans based on fast link adaptation

The latestWLAN standard, known as IEEE 802.11n, has notably increased the network capacity with respect to its predecessors thanks to the incorporation of the multipleinput multiple-output (MIMO) technology. Nonetheless, the new amendment, as its previous ones, does not specify how crucial configuration mechanisms, most notably the adaptive modulation and coding (AMC) algorithm should be implemented. The AMC process has proved essential to fully exploit the system resources in light of varying channel conditions. In this dissertation, a closed-loop AMC technique, referred to as fast link adaption (FLA) algorithm, that effectively selects themodulation and coding scheme (MCS) for multicarriermultiantennaWLAN networks is proposed. The FLA algorithm determines the MCS that maximizes the throughput while satisfying a quality of service (QoS) constraint, usually defined in the form of an objective packet error rate (PER). To this end, FLA uses a packet/bit error rate prediction methodology based on the exponential effective SNRmetric (EESM). The FLA algorithm performance has been evaluated under IEEE 802.11n systems that thanks to the incorporation of a feedbackmechanismare able to implement closed- loop AMC mechanisms. Initially, this AMC technique relies only on physical layer information but it is subsequently extended to also take into account themediumaccess control (MAC) sublayer performance. At the physical layer, the FLA algorithm has demonstrated its effectivity by performing very close to optimality in terms of throughput, while satisfying a prescribed PER constraint. The FLA algorithm has also been evaluated using imperfect channel information. It has been observed that the proposed FLA technique is rather robust against imperfect channel information, and only in highly-frequency selective channels, imperfect channel knowledge causes a noticeable degradation in throughput. At the MAC sublayer, the FLA algorithm has been complemented with a timeout strategy that weighs down the influence of the available channel information as this becomes outdated. This channel information outdate is caused by the MAC sublayer whose user multiplexing policy potentially results in large delays between acquiring the instant in which the channel state information is acquired and that in which the channel is accessed. Results demonstrate the superiority of FLA when compared to open-loop algorithms under saturated and non-saturated conditions and irrespective of the packet length, number of users, protocol (CSMA/CA or CDMA/E2CA) and access scheme (Basic Access or RTS/CTS). Additionally, several analytical models have been developed to estimate the system performance at the MAC sublayer. These models account for all operational details of the IEEE 802.11n MAC sublayer, such as finite number of retries, anomalous slot or channel errors. In particular, a semi-analytical model that assesses the MAC layer throughput under saturated conditions, considering the AMC performance is first introduced. Then, an analytical model that allows the evaluation of the QoS performance under non-saturated conditions is presented. This model focuses on single MCS and it is able to accurately predict very important system performance metrics such as blocking probability, delay, probability of discard or goodput thanks to the consideration of the finite queues on each station. Finally, the previous non-saturated analytical approach is used to define a semi-analytical model in order to estimate the system performance when considering AMC algorithms (i.e. whenmultiple MCSs are available)La darrera versió de l’estàndard deWLAN, anomenada IEEE 802.11n, ha augmentat la seva capacitat notablement en relació als sistemes anteriors gràcies a la incorporació de la tecnologia de múltiples antenes en transmissió i recepció (MIMO). No obstant això, la nova proposta, al igual que les anteriors, segueix sense especificar com s’han d’implementar elsmecanismes de configuraciómés crucials, un dels quals és l’algoritme de codificació imodulació adaptativa (AMC). Aquests algoritmes ja han demostrat la seva importància a l’hora demaximitzar el rendiment del sistema tenint en compte les condicions canviants del canal. En aquesta tesis s’ha proposat un algoritme AMC de llaç tancat, anomenat adaptació ràpida de l’enllaç (FLA), que selecciona eficientment l’esquema demodulació i codificació adaptativa per xarxes WLAN basades en arquitectures multiportadora multiantena. L’algoritme FLA determina el mode de transmissió capaç de maximitzar el throughput per les condicions de canal actuals, mentre satisfà un requisit de qualitat de servei en forma de taxa d’error per paquet (PER). FLA utilitza una metodologia de predicció de PER basada en l’estimació de la relació senyal renou (SNR) efectiva exponencial (EESM). El rendiment de l’algoritme FLA ha estat avaluat en sistemes IEEE 802.11n, ja que aquests, gràcies a la incorporació d’unmecanisme de realimentació demodes de transmissió, poden adoptar solucions AMC de llaç tancat. En una primera part, l’estudi s’ha centrat a la capa física i després s’ha estès a la subcapa MAC. A la capa física s’ha demostrat l’efectivitat de l’algoritme FLA aconseguint un rendiment molt proper al que ens proporcionaria un esquema AMC òptim en termes de throughput, alhora que es satisfan els requisits de PER objectiu. L’algoritme FLA també ha estat avaluat utilitzant informació imperfecte del canal. S’ha vist que l’algoritme FLA proposat és robust en front dels efectes d’estimació imperfecte del canal, i només en canals altament selectius en freqüència, la informació imperfecte del canal provoca una davallada en el rendiment en termes de throughput. A la subcapa MAC, l’algoritme FLA ha estat complementat amb una estratègia de temps d’espera que disminueix la dependència amb la informació de canal disponible a mesura que aquesta va quedant desfassada respecte de l’estat actual. Aquesta informació de canal desfassada és conseqüència de la subcapa MAC que degut a la multiplexació d’usuaris introdueix grans retards entre que es determina el mode de transmissió més adequat i la seva utilització per a l’accés al canal. Els resultats obtinguts han demostrat la superioritat de FLA respecte d’altres algoritmes de llaç obert en condicions de saturació i de no saturació, i independentment de la longitud de paquet, nombre d’usuaris, protocol (CSMA/CA i CSMA/E2CA) i esquema d’accés (Basic Access i RTS/CTS). Amés, s’han desenvolupat diversosmodels analítics per tal d’estimar el rendiment del sistema a la subcapa MAC. Aquests models consideren tots els detalls de funcionament de la subcapaMAC del 802.11n, comper exemple un nombre finit de retransmissions de cada paquet, l’slot anòmal o els errors introduïts pel canal. Inicialment s’ha proposat unmodel semi-analític que determina el throughtput en condicions de saturació, considerant el rendiment dels algoritmes AMC. Després s’ha presentat un model analític que estima el rendiment del sistema per condicions de no saturació, mitjançat elmodelat de cues finites a cada estació. Aquestmodel consideramodes de transmissió fixes i és capaç de determinar de manera molt precisa mètriques de rendimentmolt importants comsón la probabilitat de bloqueig de cada estació, el retard mitjà del paquets, la probabilitat de descart o la mesura del goodput. Finalment, el model analític de no saturació s’ha utilitzat per definir un model semi-analític per tal d’estimar el rendiment del sistema quan es considera l’ús d’algoritmes AMC

ERASE: Energy Efficient Task Mapping and Resource Management for Work Stealing Runtimes

Parallel applications often rely on work stealing schedulers in combination with fine-grained tasking to achieve high performance and scalability. However, reducing the total energy consumption in the context of work stealing runtimes is still challenging, particularly when using asymmetric architectures with different types of CPU cores. A common approach for energy savings involves dynamic voltage and frequency scaling (DVFS) wherein throttling is carried out based on factors like task parallelism, stealing relations, and task criticality. This article makes the following observations: (i) leveraging DVFS on a per-task basis is impractical when using fine-grained tasking and in environments with cluster/chip-level DVFS; (ii) task moldability, wherein a single task can execute on multiple threads/cores via work-sharing, can help to reduce energy consumption; and (iii) mismatch between tasks and assigned resources (i.e., core type and number of cores) can detrimentally impact energy consumption. In this article, we propose EneRgy Aware SchedulEr (ERASE), an intra-application task scheduler on top of work stealing runtimes that aims to reduce the total energy consumption of parallel applications. It achieves energy savings by guiding scheduling decisions based on per-task energy consumption predictions of different resource configurations. In addition, ERASE is capable of adapting to both given static frequency settings and externally controlled DVFS. Overall, ERASE achieves up to 31% energy savings and improves performance by 44% on average, compared to the state-of-the-art DVFS-based schedulers

Unsupervised Induction of Frame-Based Linguistic Forms

This thesis studies the use of bulk, structured, linguistic annotations in order to perform unsupervised induction of meaning for three kinds of linguistic forms: words, sentences, and documents. The primary linguistic annotation I consider throughout this thesis are frames, which encode core linguistic, background or societal knowledge necessary to understand abstract concepts and real-world situations. I begin with an overview of linguistically-based structured meaning representation; I then analyze available large-scale natural language processing (NLP) and linguistic resources and corpora for their abilities to accommodate bulk, automatically-obtained frame annotations. I then proceed to induce meanings of the different forms, progressing from the word level, to the sentence level, and finally to the document level. I first show how to use these bulk annotations in order to better encode linguistic- and cognitive science backed semantic expectations within word forms. I then demonstrate a straightforward approach for learning large lexicalized and refined syntactic fragments, which encode and memoize commonly used phrases and linguistic constructions. Next, I consider two unsupervised models for document and discourse understanding; one is a purely generative approach that naturally accommodates layer annotations and is the first to capture and unify a complete frame hierarchy. The other conditions on limited amounts of external annotations, imputing missing values when necessary, and can more readily scale to large corpora. These discourse models help improve document understanding and type-level understanding

Performance modelling and enhancement of wireless communication protocols

In recent years, Wireless Local Area Networks(WLANs) play a key role in the data communications and networking areas, having witnessed significant research and development. WLANs are extremely popular being almost everywhere including business,office and home deployments.In order to deal with the modem Wireless connectivity needs,the Institute of Electrical and Electronics Engineers(IEEE) has developed the 802.11 standard family utilizing mainly radio transmission techniques, whereas the Infrared Data Association (IrDA) addressed the requirement for multipoint connectivity with the development of the Advanced Infrared(Alr) protocol stack. This work studies the collision avoidance procedures of the IEEE 802.11 Distributed Coordination Function (DCF) protocol and suggests certain protocol enhancements aiming at maximising performance. A new, elegant and accurate analysis based on Markov chain modelling is developed for the idealistic assumption of unlimited packet retransmissions as well as for the case of finite packet retry limits. Simple equations are derived for the through put efficiency, the average packet delay, the probability of a packet being discarded when it reaches the maximum retransmission limit, the average time to drop such a packet and the packet inter-arrival time for both basic access and RTS/CTS medium access schemes.The accuracy of the mathematical model is validated by comparing analytical with OPNET simulation results. An extensive and detailed study is carried out on the influence of performance of physical layer, data rate, packet payload size and several backoff parameters for both medium access mechanisms. The previous mathematical model is extended to take into account transmission errors that can occur either independently with fixed Bit Error Rate(BER) or in bursts. The dependency of the protocol performance on BER and other factors related to independent and burst transmission errors is explored. Furthermore, a simple-implement appropriate tuning of the back off algorithm for maximizing IEEE 802-11 protocol performance is proposed depending on the specific communication requirements. The effectiveness of the RTS/CTS scheme in reducing collision duration at high data rates is studied and an all-purpose expression for the optimal use of the RTS/CTS reservation scheme is derived. Moreover, an easy-to-implement backoff algorithm that significantly enhances performance is introduced and an alternative derivation is developed based on elementary conditional probability arguments rather than bi-dimensional Markov chains. Finally, an additional performance improvement scheme is proposed by employing packet bursting in order to reduce overhead costs such as contention time and RTS/CTSex changes. Fairness is explored in short-time and long-time scales for both the legacy DCF and packet bursting cases. AIr protocol employs the RTS/CTS medium reservation scheme to cope with hidden stations and CSMA/CA techniques with linear contention window (CW) adjustment for medium access. A 1-dimensional Markov chain model is constructed instead of the bi-dimensional model in order to obtain simple mathematical equations of the average packet delay.This new approach greatly simplifies previous analyses and can be applied to any CSMA/CA protocol.The derived mathematical model is validated by comparing analytical with simulation results and an extensive Alr packet delay evaluation is carried out by taking into account all the factors and parameters that affect protocol performance. Finally, suitable values for both backoff and protocol parameters are proposed that reduce average packet delay and, thus, maximize performance