26,545 research outputs found
Too much noise in the Times Higher Education rankings
Several individual indicators from the Times Higher Education Survey (THES) data baseâthe overall score, the reported staff-to-student ratio, and the peer ratingsâdemonstrate unacceptably high fluctuation from year to year. The inappropriateness of the summary tabulations for assessing the majority of the âtop 200â universities would be apparent purely for reason of this obvious statistical instability regardless of other grounds of criticism. There are far too many anomalies in the change scores of the various indices for them to be of use in the course of university management
The Leiden Ranking 2011/2012: Data collection, indicators, and interpretation
The Leiden Ranking 2011/2012 is a ranking of universities based on
bibliometric indicators of publication output, citation impact, and scientific
collaboration. The ranking includes 500 major universities from 41 different
countries. This paper provides an extensive discussion of the Leiden Ranking
2011/2012. The ranking is compared with other global university rankings, in
particular the Academic Ranking of World Universities (commonly known as the
Shanghai Ranking) and the Times Higher Education World University Rankings.
Also, a detailed description is offered of the data collection methodology of
the Leiden Ranking 2011/2012 and of the indicators used in the ranking. Various
innovations in the Leiden Ranking 2011/2012 are presented. These innovations
include (1) an indicator based on counting a university's highly cited
publications, (2) indicators based on fractional rather than full counting of
collaborative publications, (3) the possibility of excluding non-English
language publications, and (4) the use of stability intervals. Finally, some
comments are made on the interpretation of the ranking, and a number of
limitations of the ranking are pointed out
Academic rankings: an approach to a Portuguese ranking
The academic rankings are a controversial subject in higher education. However, despite all the criticism, academic rankings are here to stay and more and more different stakeholders use rankings to obtain information about the institutionsâ performance. The two most well-known rankings, The Times and the Shanghai Jiao Tong University rankings have different methodologies. The Times ranking is based on peer review, whereas the Shanghai ranking has only quantitative indicators and is mainly based on research outputs. In Germany, the CHE ranking uses a different methodology from the traditional rankings, allowing the users to choose criteria and weights. The Portuguese higher education institutions are performing below their European peers, and the Government believes that an academic ranking could improve both performance and competitiveness between institutions. The purpose of this paper is to analyse the advantages and problems of academic rankings and provide guidance to a new Portuguese ranking.Academic rankings; CHE; higher education; performance evaluation; Portugal; Shanghai; THES
Utility Cost of Formal Privacy for Releasing National Employer-Employee Statistics
National statistical agencies around the world publish tabular summaries based on combined employer-employee (ER-EE) data. The privacy of both individuals and business establishments that feature in these data are protected by law in most countries. These data are currently released using a variety of statistical disclosure limitation (SDL) techniques that do not reveal the exact characteristics of particular employers and employees, but lack provable privacy guarantees limiting inferential disclosures. In this work, we present novel algorithms for releasing tabular summaries of linked ER-EE data with formal, provable guarantees of privacy. We show that state-of-the-art differentially private algorithms add too much noise for the output to be useful. Instead, we identify the privacy requirements mandated by current interpretations of the relevant laws, and formalize them using the Pufferfish framework. We then develop new privacy definitions that are customized to ER-EE data and satisfy the statutory privacy requirements. We implement the experiments in this paper on production data gathered by the U.S. Census Bureau. An empirical evaluation of utility for these data shows that for reasonable values of the privacy-loss parameter Ï”â„1, the additive error introduced by our provably private algorithms is comparable, and in some cases better, than the error introduced by existing SDL techniques that have no provable privacy guarantees. For some complex queries currently published, however, our algorithms do not have utility comparable to the existing traditiona
Why Are they Doing so Well while We Are Doing so Badly? A Comparison between the Canadian and Italian University Systems
The Italian university system is in a profound and dangerous crisis. The below par performance of Italian universities is compared with the increasingly successful accomplishments of Canadian Universities. The paper identifies the major source of this performance differential in the hiring and promotion procedures. Funding methods also facilitate the success of Canadian Universities. The paper recommends a radical reform of the Italian system and a move towards a more decentralized, independent, flexible and transparent system like that of Canada.
How should peer-review panels behave?
Many governments wish to assess the quality of their universities. A prominent example is the UKâs new Research Excellence Framework (REF) 2014. In the REF, peer-review panels will be provided with information on publications and citations. This paper suggests a way
in which panels could choose the weights to attach to these two indicators. The analysis draws in an intuitive way on the concept of Bayesian updating (where citations gradually reveal information about the initially imperfectly-observed importance of the research). Our study should not be interpreted as the argument that only mechanistic measures ought to be used in a REF
Balancing Speed and Quality in Online Learning to Rank for Information Retrieval
In Online Learning to Rank (OLTR) the aim is to find an optimal ranking model
by interacting with users. When learning from user behavior, systems must
interact with users while simultaneously learning from those interactions.
Unlike other Learning to Rank (LTR) settings, existing research in this field
has been limited to linear models. This is due to the speed-quality tradeoff
that arises when selecting models: complex models are more expressive and can
find the best rankings but need more user interactions to do so, a requirement
that risks frustrating users during training. Conversely, simpler models can be
optimized on fewer interactions and thus provide a better user experience, but
they will converge towards suboptimal rankings. This tradeoff creates a
deadlock, since novel models will not be able to improve either the user
experience or the final convergence point, without sacrificing the other. Our
contribution is twofold. First, we introduce a fast OLTR model called Sim-MGD
that addresses the speed aspect of the speed-quality tradeoff. Sim-MGD ranks
documents based on similarities with reference documents. It converges rapidly
and, hence, gives a better user experience but it does not converge towards the
optimal rankings. Second, we contribute Cascading Multileave Gradient Descent
(C-MGD) for OLTR that directly addresses the speed-quality tradeoff by using a
cascade that enables combinations of the best of two worlds: fast learning and
high quality final convergence. C-MGD can provide the better user experience of
Sim-MGD while maintaining the same convergence as the state-of-the-art MGD
model. This opens the door for future work to design new models for OLTR
without having to deal with the speed-quality tradeoff.Comment: CIKM 2017, Proceedings of the 2017 ACM on Conference on Information
and Knowledge Managemen
Capturing lexical variation in MT evaluation using automatically built sense-cluster inventories
The strict character of most of the existing Machine Translation (MT) evaluation metrics does not permit them to capture lexical variation in translation. However, a central
issue in MT evaluation is the high correlation that the metrics should have with human judgments of translation quality. In order to achieve a higher correlation, the identification of sense correspondences between the compared translations becomes really important. Given
that most metrics are looking for exact correspondences, the evaluation results are often misleading concerning translation quality. Apart from that, existing metrics do not permit one to make a conclusive estimation of the impact of Word Sense Disambiguation techniques into
MT systems. In this paper, we show how information acquired by an unsupervised semantic analysis method can be used to render MT evaluation more sensitive to lexical semantics. The sense inventories built by this data-driven method are incorporated into METEOR: they replace WordNet for evaluation in English and render METEORâs synonymy module operable in French. The evaluation results demonstrate that the use of these inventories gives rise to an increase in the number of matches and the correlation with human judgments of translation quality, compared to precision-based metrics
Employment Polarization and Job Quality in the Crisis
[Excerpt] European labour markets added nearly 30 million new jobs in a golden age of employment creation prior to the onset of the Great Recession in 2008. The markets have subsequently shed five million jobs and unemployment â rising rapidly once again â is at its highest since the late 1990s. This second annual European Jobs Monitor report looks in detail at recent shifts in employment at Member State and European level. The analysis covers three distinct periods: the pre-recession employment expansion (1995â2007); the Great Recession (2008â2010); the stalled recovery (2011â2012).
A âjobs-basedâ approach is applied to describe employment shifts quantitatively (how many jobs were created or destroyed) and qualitatively (what kinds of jobs)
- âŠ