Empirical Methodology for Crowdsourcing Ground Truth

The process of gathering ground truth data through human annotation is a major bottleneck in the use of information extraction methods for populating the Semantic Web. Crowdsourcing-based approaches are gaining popularity in the attempt to solve the issues related to volume of data and lack of annotators. Typically these practices use inter-annotator agreement as a measure of quality. However, in many domains, such as event detection, there is ambiguity in the data, as well as a multitude of perspectives of the information examples. We present an empirically derived methodology for efficiently gathering of ground truth data in a diverse set of use cases covering a variety of domains and annotation tasks. Central to our approach is the use of CrowdTruth metrics that capture inter-annotator disagreement. We show that measuring disagreement is essential for acquiring a high quality ground truth. We achieve this by comparing the quality of the data aggregated with CrowdTruth metrics with majority vote, over a set of diverse crowdsourcing tasks: Medical Relation Extraction, Twitter Event Identification, News Event Extraction and Sound Interpretation. We also show that an increased number of crowd workers leads to growth and stabilization in the quality of annotations, going against the usual practice of employing a small number of annotators.Comment: in publication at the Semantic Web Journa

Absorptive capacity and relationship learning mechanisms as complementary drivers of green innovation performance

This paper aims to explore in depth how internal and external knowledge-based drivers actually affect the firms\u2019 green innovation performance. Subsequently, this study analyzes the relationships between absorptive capacity (internal knowledge-based driver), relationship learning (external knowledge-based driver) and green innovation performance. This study relies on a sample of 112 firms belonging to the Spanish automotive components manufacturing sector (ACMS) and uses partial least squares path modeling to test the hypotheses proposed. The empirical results show that both absorptive capacity and relationship learning exert a significant positive effect on the dependent variable and that relationship learning moderates the link between absorptive capacity and green innovation performance. This paper presents some limitations with respect to the particular sector (i.e. the ACMS) and geographical context (Spain). For this reason, researchers must be thoughtful while generalizing these results to distinct scenarios. Managers should devote more time and resources to reinforce their absorptive capacity as an important strategic tool to generate new knowledge and hence foster green innovation performance in manufacturing industries. The paper shows the importance of encouraging decision-makers to cultivate and rely on relationship learning mechanisms with their main stakeholders and to acquire the necessary information and knowledge that might be valuable in the maturity of green innovations. This study proposes that relationship learning plays a moderating role in the relationship between absorptive capacity and green innovation performance

Fit for planning? An evaluation of the application of development viability appraisal models in the UK planning system

The aim of this paper is to critically examine the application of development appraisal to viability assessment in the planning system. This evaluation is of development appraisal models in general and also their use in particular applications associated with estimating planning obligation capacity. The paper is organised into four themes: · The context and conceptual basis for development viability appraisal · A review of development viability appraisal methods · A discussion of selected key inputs into a development viability appraisal · A discussion of the applications of development viability appraisals in the planning system It is assumed that readers are familiar with the basic models and information needs of development viability appraisal rather than at the cutting edge of practice and/or academ

Disentangling urban sustainability: the Flemish City Monitor acknowledges complexity

Nowadays, cities have to deal with complexity. In this article we argue that the City Monitor for Sustainable Urban Development in the Flanders (Belgium) acknowledges complexity. This set of almost 200 SDIs (Sustainable Development Indicators) contains actor-exceeding and policy-exogenous information. On that account this learning instrument is relevant for all actors involved in the urban (sustainable) development of their city and is able to enhance and to sharpen the quality of strategic urban debates and complex decision-making processes. Our intensive co-design approach of the City Monitor also succeeds to deal adequate with the tensions of complex catch-all terms such as (urban) sustainability

Openness and Growth: What's the Empirical Relationship?

There is still disagreement among economists concerning how a country's international economic policies and its rate of economic growth interact, despite a number of multi-country case studies utilizing comparable analytical frameworks, numerous econometric studies using large cross-country data sets, and important theoretical advances in growth theory. This paper briefly surveys this literature and points out the main reasons for the disagreements. Particular attention is given to an important study by Francisco Rodriguez and Dani Rodrik (2001) criticizing the conclusion of a number of recent multi-country statistical studies that openness is associated with higher growth rates. Rodriguez and Rodrik show that openness simply in the sense of liberal trade policies seems to be no guarantee of faster growth. However, the conclusion of most researchers involved in either country studies or multi-country statistical tests that lower trade barriers in combination with a stable and non-discriminatory exchange-rate system, prudent monetary and fiscal policies and corruption-free administration of economic policies promote economic growth still seems to remain valid.