Continuous Decomposition of Granularity for Neural Paraphrase Generation

While Transformers have had significant success in paragraph generation, they treat sentences as linear sequences of tokens and often neglect their hierarchical information. Prior work has shown that decomposing the levels of granularity~(e.g., word, phrase, or sentence) for input tokens has produced substantial improvements, suggesting the possibility of enhancing Transformers via more fine-grained modeling of granularity. In this work, we propose a continuous decomposition of granularity for neural paraphrase generation (C-DNPG). In order to efficiently incorporate granularity into sentence encoding, C-DNPG introduces a granularity-aware attention (GA-Attention) mechanism which extends the multi-head self-attention with: 1) a granularity head that automatically infers the hierarchical structure of a sentence by neurally estimating the granularity level of each input token; and 2) two novel attention masks, namely, granularity resonance and granularity scope, to efficiently encode granularity into attention. Experiments on two benchmarks, including Quora question pairs and Twitter URLs have shown that C-DNPG outperforms baseline models by a remarkable margin and achieves state-of-the-art results in terms of many metrics. Qualitative analysis reveals that C-DNPG indeed captures fine-grained levels of granularity with effectiveness.Comment: Accepted to be published in COLING 202

Model granularity and related concepts

Models are integral to engineering design and basis for many decisions. Therefore, it is necessary to comprehend how a model’s properties might influence its behaviour. Model granularity is an important property but has so far only received limited attention. The terminology used to describe granularity and related phenomena varies and pertinent concepts are distributed across communities. This article positions granularity in the theoretical background of models, collects formal definitions for relevant terms from a range of communities and discusses the implications for engineering design

Temporal Data Modeling and Reasoning for Information Systems

Temporal knowledge representation and reasoning is a major research field in Artificial Intelligence, in Database Systems, and in Web and Semantic Web research. The ability to model and process time and calendar data is essential for many applications like appointment scheduling, planning, Web services, temporal and active database systems, adaptive Web applications, and mobile computing applications. This article aims at three complementary goals. First, to provide with a general background in temporal data modeling and reasoning approaches. Second, to serve as an orientation guide for further specific reading. Third, to point to new application fields and research perspectives on temporal knowledge representation and reasoning in the Web and Semantic Web

How can macroscopic models reveal self-organization in traffic flow?

In this paper we propose a new modeling technique for vehicular traffic flow, designed for capturing at a macroscopic level some effects, due to the microscopic granularity of the flow of cars, which would be lost with a purely continuous approach. The starting point is a multiscale method for pedestrian modeling, recently introduced in Cristiani et al., Multiscale Model. Simul., 2011, in which measure-theoretic tools are used to manage the microscopic and the macroscopic scales under a unique framework. In the resulting coupled model the two scales coexist and share information, in the sense that the same system is simultaneously described from both a discrete (microscopic) and a continuous (macroscopic) perspective. This way it is possible to perform numerical simulations in which the single trajectories and the average density of the moving agents affect each other. Such a method is here revisited in order to deal with multi-population traffic flow on networks. For illustrative purposes, we focus on the simple case of the intersection of two roads. By exploiting one of the main features of the multiscale method, namely its dimension-independence, we treat one-dimensional roads and two-dimensional junctions in a natural way, without referring to classical network theory. Furthermore, thanks to the coupling between the microscopic and the macroscopic scales, we model the continuous flow of cars without losing the right amount of granularity, which characterizes the real physical system and triggers self-organization effects, such as, for example, the oscillatory patterns visible at jammed uncontrolled crossroads.Comment: 7 pages, 7 figure

A Model-Driven Architecture Approach to the Efficient Identification of Services on Service-oriented Enterprise Architecture

Service-Oriented Enterprise Architecture requires the efficient development of loosely-coupled and interoperable sets of services. Existing design approaches do not always take full advantage of the value and importance of the engineering invested in existing legacy systems. This paper proposes an approach to define the key services from such legacy systems effectively. The approach focuses on identifying these services based on a Model-Driven Architecture approach supported by guidelines over a wide range of possible service types