The Level-agnostic Modeling Language: Language Specification and Tool Implementation

Since the release of the Entity-Relationship modelling language in 1976 and the UML in the early 1990's no fundamental developments in the concrete syntax of general purpose modelling languages have been made. With today's trends in model-driven technologies and the rising need for domain specific languages the weaknesses of the traditional languages become more and more obvious. Among these weaknesses are missing support for modelling multiple ontological levels or the lack of built-in Domain Specific Language development capabilities. The Level-agnostic Modeling Language (LML) was developed to address these two needs. During its development care was taken to retain the strengths of traditional languages. This thesis is based on a collection of papers about multilevel modelling. The collection starts with a paper that identifies the need for multilevel modelling through a practical example of a language used to describe computer hardware product hierarchies. A later paper examines the problems of current technologies from a more theoretical point of view and suggestions to solve the identified issues are made. The latest work in this collection defines the LML based on previously made observations. The work on the LML has now reached a maturity level which makes it worthwhile to write an LML specification 1.0 and implement a tool to give other researchers the opportunity to use this new technology. The thesis provides the specification 1.0 of the LML. Additionally, a graphical editor based on one of today's leading model driven development platforms, Eclipse, is developed

A BPM Lifecycle Plug-in for Modeling Methods Agility

Business Process Management literature has proposed several BPM lifecycles on a level of abstraction that is modeling method -agnostic, i.e. they consider the modeling language and tool support an underlying invariant or technological concern. While remaining on the same abstraction layer, we highlight a method agility requirement observed in commercial BPM consulting projects - concretely, it manifests as change requests for the modeling language or tool, from one lifecycle iteration to the next, leading to situations of model value co-creation as customer demands are assimilated in the modeling method. Based on a conceptualization of such situations, a lifecycle plug-in is proposed in the form of a methodology and associated tool support, allowing for responsive evolution of the adopted modeling method with impact on several lifecycle phases. Historical examples from the evolution of a BPM product are provided to illustrate and classify the demands that motivate the existence of this lifecycle plug-in

Exploring Different Dimensions of Attention for Uncertainty Detection

Neural networks with attention have proven effective for many natural language processing tasks. In this paper, we develop attention mechanisms for uncertainty detection. In particular, we generalize standardly used attention mechanisms by introducing external attention and sequence-preserving attention. These novel architectures differ from standard approaches in that they use external resources to compute attention weights and preserve sequence information. We compare them to other configurations along different dimensions of attention. Our novel architectures set the new state of the art on a Wikipedia benchmark dataset and perform similar to the state-of-the-art model on a biomedical benchmark which uses a large set of linguistic features.Comment: accepted at EACL 201

Clue: Cross-modal Coherence Modeling for Caption Generation

We use coherence relations inspired by computational models of discourse to study the information needs and goals of image captioning. Using an annotation protocol specifically devised for capturing image--caption coherence relations, we annotate 10,000 instances from publicly-available image--caption pairs. We introduce a new task for learning inferences in imagery and text, coherence relation prediction, and show that these coherence annotations can be exploited to learn relation classifiers as an intermediary step, and also train coherence-aware, controllable image captioning models. The results show a dramatic improvement in the consistency and quality of the generated captions with respect to information needs specified via coherence relations.Comment: Accepted as a long paper to ACL 202

Word-Level Representation From Bytes For Language Modeling

Modern language models mostly take sub-words as input, a design that balances the trade-off between vocabulary size, number of parameters, and performance. However, sub-word tokenization still has disadvantages like not being robust to noise and difficult to generalize to new languages. Also, the current trend of scaling up models reveals that larger models require larger embeddings but that makes parallelization hard. Previous work on image classification proves splitting raw input into a sequence of chucks is a strong, model-agnostic inductive bias. Based on this observation, we rethink the existing character-aware method that takes character-level inputs but makes word-level sequence modeling and prediction. We overhaul this method by introducing a cross-attention network that builds word-level representation directly from bytes, and a sub-word level prediction based on word-level hidden states to avoid the time and space requirement of word-level prediction. With these two improvements combined, we have a token free model with slim input embeddings for downstream tasks. We name our method Byte2Word and perform evaluations on language modeling and text classification. Experiments show that Byte2Word is on par with the strong sub-word baseline BERT but only takes up 10\% of embedding size. We further test our method on synthetic noise and cross-lingual transfer and find it competitive to baseline methods on both settings.Comment: preprin

On the relation between linguistic typology and (limitations of) multilingual language modeling

A key challenge in cross-lingual NLP is developing general language-independent architectures that are equally applicable to any language. However, this ambition is largely hampered by the variation in structural and semantic properties, i.e. the typological profiles of the world's languages. In this work, we analyse the implications of this variation on the language modeling (LM) task. We present a large-scale study of state-of-the art n-gram based and neural language models on 50 typologically diverse languages covering a wide variety of morphological systems. Operating in the full vocabulary LM setup focused on word-level prediction, we demonstrate that a coarse typology of morphological systems is predictive of absolute LM performance. Moreover, fine-grained typological features such as exponence, flexivity, fusion, and inflectional synthesis are borne out to be responsible for the proliferation of low-frequency phenomena which are organically difficult to model by statistical architectures, or for the meaning ambiguity of character n-grams. Our study strongly suggests that these features have to be taken into consideration during the construction of next-level language-agnostic LM architectures, capable of handling morphologically complex languages such as Tamil or Korean.ERC grant Lexica