Quantifying consensus of rankings based on q-support patterns

Rankings, representing preferences over a set of candidates, are widely used in many information systems, e.g., group decision making and information retrieval. It is of great importance to evaluate the consensus of the obtained rankings from multiple agents. An overall measure of the consensus degree provides an insight into the ranking data. Moreover, it could provide a quantitative indicator for consensus comparison between groups and further improvement of a ranking system. Existing studies are insufficient in assessing the overall consensus of a ranking set. They did not provide an evaluation of the consensus degree of preference patterns in most rankings. In this paper, a novel consensus quantifying approach, without the need for any correlation or distance functions as in existing studies of consensus, is proposed based on a concept of q-support patterns of rankings. The q-support patterns represent the commonality embedded in a set of rankings. A method for detecting outliers in a set of rankings is naturally derived from the proposed consensus quantifying approach. Experimental studies are conducted to demonstrate the effectiveness of the proposed approach

Semantic Stability in Social Tagging Streams

One potential disadvantage of social tagging systems is that due to the lack of a centralized vocabulary, a crowd of users may never manage to reach a consensus on the description of resources (e.g., books, users or songs) on the Web. Yet, previous research has provided interesting evidence that the tag distributions of resources may become semantically stable over time as more and more users tag them. At the same time, previous work has raised an array of new questions such as: (i) How can we assess the semantic stability of social tagging systems in a robust and methodical way? (ii) Does semantic stabilization of tags vary across different social tagging systems and ultimately, (iii) what are the factors that can explain semantic stabilization in such systems? In this work we tackle these questions by (i) presenting a novel and robust method which overcomes a number of limitations in existing methods, (ii) empirically investigating semantic stabilization processes in a wide range of social tagging systems with distinct domains and properties and (iii) detecting potential causes for semantic stabilization, specifically imitation behavior, shared background knowledge and intrinsic properties of natural language. Our results show that tagging streams which are generated by a combination of imitation dynamics and shared background knowledge exhibit faster and higher semantic stability than tagging streams which are generated via imitation dynamics or natural language streams alone

Quantifying the Contours of Power: Chief Justice Roberts & Justice Kennedy in Criminal Justice Cases

This Article seeks to contribute to the debate with an empirical analysis of voting behavior in criminal justice cases decided during the first ten Terms of the Roberts Court era. The following section presents the study’s case selection and introduces the types of measures used to illuminate influence on the High Court (Part II). Court- and individual-level tendencies (Part III) identify potential spheres of influence occupied by Chief Justice Roberts and Justice Kennedy. These bases of judicial power are examined separately in Part IV (Chief Justice Roberts) and Part V (Justice Kennedy). Some possible implications of Justice Scalia’s death on these power bases are addressed in Part VI

How Should Research Performance Be Measured? A Study of Swedish Economists

Billions of euros are allocated every year to university research. Increased specialisation and international integration of research and researchers has sharply raised the need for comparisons of performance across fields, institutions and individual researchers. However, there is still no consensus regarding how such rankings should be conducted and what output measures are appropriate to use. We rank all full professors in a particular discipline, economics, in one European nation using seven established, and some of them commonly used, measures of research performance. Our examination shows both that the rank order can vary greatly across measures, and that depending on the measure used the distribution of total research out-put is valued very differently. The renowned KMS measure in economics stands out among the measures analysed here. It exhibits the weakest correlation with the others used in our study. We conclude by giving advice to funding councils and others assessing research quality on how to think about the use of both quantitative and qualitative measures of performance.Impact of research; Ranking; Research productivity; Bibliometrics; Impact Factor

How Should Research Performance be Measured? A Study of Swedish Economists

Billions of dollars are allocated every year to university research. Increased specialisation and international integration of research and researchers has sharply raised the need for comparisons of performance across fields, institutions and individual researchers. However, there is still no consensus regarding how such rankings should be conducted and what output measures are appropriate to use. We rank all full professors in a particular discipline, economics, in one country using seven established, and some of them commonly used, meas-ures of research performance. Our examination shows both that the rank order can vary greatly across measures, and that depending on the measure used the distribution of total research output is valued very differently.Impact of Research; Ranking; Research Output; Research Productivity; Bibliometrics; Google Scholar; h-index; Impact Factor; SSCI

Grading of parameters for urban tree inventories by city officials, arborists and academics using the Delphi method

Tree inventories are expensive to conduct and update, so every inventory carried out must be maximized. However, increasing the number of constituent parameters increases the cost of performing and updating the inventory, illustrating the need for careful parameter selection. This paper reports the results of a systematic expert rating of tree inventories aiming to quantify the relative importance of each parameter. Using the Delphi method, panels comprising city officials, arborists and academics rated a total of 148 parameters. In order of total mean score, the top ranking parameters, which can serve as a guide for decision-making at practical level and for standardization of tree inventories, were: Scientific name of the tree species and genera, Vitality, Coordinates, Hazard class and Identification number. The study also examined whether the different responsibilities and usage of urban tree databases among organizations and people engaged in urban tree inventories affected their prioritization. The results revealed noticeable dissimilarities in the ranking of parameters between the panels, underlining the need for collaboration between the research community and those commissioning, administrating and conducting inventories. Only by applying such a transdisciplinary approach to parameter selection can urban tree inventories be strengthened and made more relevant

Benchmarking Sustainable Development: A Synthetic Meta-index Approach

The need for monitoring countries’ overall performance in Sustainable Development (SD) is widely recognized, but the methods for aggregating vast amounts of empirical data remain rather crude. This paper examines the so-called ‘benefit-of-the-doubt’ weighting method as a tool for identifying benchmarks without imposing strong normative judgement about SD priorities. The weighting method involves linear optimization techniques, and allows countries to emphasize and prioritize those SD aspects for which they perform relatively well. Using this method, we construct a meta-index of SD (MISD), which combines 14 existing aggregate SD indices (developed by well-established organizations and/or expert teams) into a single synthesizing overall SD index. Within a sample of 154 countries, our index identifies 6 benchmark countries (3 high-income countries and 3 upper-middle-income countries), but also a number of seriously under-performing countries. We view this approach as a first step towards more systematic international comparisons, aimed at facilitating diffusion of the best practices and policies from the benchmark countries to the less developed world.Sustainable Development, Integrated Assessment, Benchmarking, benefit of the doubt weighting, Data Envelopment Analysis