Shapley Values for Assessing Research Production and Impact of Schools and Scholars. ESRI WP370, January 2011

Performance measures of individual scholars tend to ignore the context. I introduce contextualised metrics: cardinal and ordinal pseudo-Shapley values that measure a scholar’s contribution to (perhaps power over) her own school and her market value to other schools should she change job. I illustrate the proposed measures with business scholars and business schools in Ireland. Although conceptually superior, the power indicators imply a ranking of scholars within a school that is identical to the corresponding conventional performance measures. The market value indicators imply an identical ranking within schools and a very similar ranking between schools. The ordinal indices further contextualise performance measures and thus deviate further from the corresponding conventional indicators. As the ordinal measures are discontinuous by construction, a natural classification of scholars emerges. Averaged over schools, the market values offer little extra information over the corresponding production and impact measures. The ordinal power measure indicates the robustness or fragility of an institution’s place in the rank order. It is only weakly correlated with the concentration of publications and citations

Planning for self-employment at the beginning of a market economy: evidence from individual data of East German workers

We investigate the plans of individual workers concerning future self-employment in the former German Democratic Republic (East Germany) shortly before the economic, monetary and social union in June/July 1990. Our data base is the Socio-Economic Panel (SOEP) East. We find that the desire to become an entrepreneur is basically determined by individual and household characteristics, including income and asset indicators, and not as much by the current job situation of the individual. Furthermore, we find evidence of barriers to entry which may come from capital market constraints and institutional restrictions. Due to the ordinal nature of the answers, we used the ordinal logit model for estimation. The corresponding stochastic assumptions are tested extensively using pseudo-Lagrange multiplier tests against omitted variables, non-linearity, asymmetry of distribution, and heteroscedasticity. --

An Empirical Bayes Approach to Estimating Ordinal Treatment Effects

Ordinal variables — categorical variables with a defined order to the categories, but without equal spacing between them — are frequently used in social science applications. Although a good deal of research exists on the proper modeling of ordinal response variables, there is not a clear directive as to how to model ordinal treatment variables. The usual approaches found in the literature for using ordinal treatment variables are either to use fully unconstrained, though additive, ordinal group indicators or to use a numeric predictor constrained to be continuous. Generalized additive models are a useful exception to these assumptions (Beck and Jackman 1998). In contrast to the generalized additive modeling approach, we propose the use of a Bayesian shrinkage estimator to model ordinal treatment variables. The estimator we discuss in this paper allows the model to contain both individual group level indicators and a continuous predictor. In contrast to traditionally used shrinkage models that pull the data toward a common mean, we use a linear model as the basis. Thus, each individual effect can be arbitrary, but the model “shrinks” the estimates toward a linear ordinal framework according to the data. We demonstrate the estimator on two political science examples: the impact of voter identification requirements on turnout (Alvarez, Bailey, and Katz 2007), and the impact of the frequency of religious service attendance on the liberality of abortion attitudes (e.g., Singh and Leahy 1978, Tedrow and Mahoney 1979, Combs and Welch 1982)

Ordinal Welfare Comparisons with Multiple Discrete Indicators: A First Order Dominance Approach and Application to Child Poverty

We develop an ordinal method for making welfare comparisons between populations with multidimensional discrete well-being indicators observed at the micro level. The approach assumes that, for each well-being indicator, the levels can be ranked from worse to better; however, no assumptions are made about relative importance of any dimension nor about complementarity/substitutability relationships between dimensions. The method is based on the concept of multidimensional first order dominance. We introduce a rapid and reliable algorithm for empirically determining whether one population dominates another on the basis of available binary indicators by drawing upon linear programming theory. These approaches are applied to household survey data from Vietnam and Mozambique with a focus on child poverty comparisons over time and between regions.

A critical cluster analysis of 44 indicators of author-level performance

This paper explores the relationship between author-level bibliometric indicators and the researchers the "measure", exemplified across five academic seniorities and four disciplines. Using cluster methodology, the disciplinary and seniority appropriateness of author-level indicators is examined. Publication and citation data for 741 researchers across Astronomy, Environmental Science, Philosophy and Public Health was collected in Web of Science (WoS). Forty-four indicators of individual performance were computed using the data. A two-step cluster analysis using IBM SPSS version 22 was performed, followed by a risk analysis and ordinal logistic regression to explore cluster membership. Indicator scores were contextualized using the individual researcher's curriculum vitae. Four different clusters based on indicator scores ranked researchers as low, middle, high and extremely high performers. The results show that different indicators were appropriate in demarcating ranked performance in different disciplines. In Astronomy the h2 indicator, sum pp top prop in Environmental Science, Q2 in Philosophy and e-index in Public Health. The regression and odds analysis showed individual level indicator scores were primarily dependent on the number of years since the researcher's first publication registered in WoS, number of publications and number of citations. Seniority classification was secondary therefore no seniority appropriate indicators were confidently identified. Cluster methodology proved useful in identifying disciplinary appropriate indicators providing the preliminary data preparation was thorough but needed to be supplemented by other analyses to validate the results. A general disconnection between the performance of the researcher on their curriculum vitae and the performance of the researcher based on bibliometric indicators was observed.Comment: 28 pages, 7 tables, 2 figures, 2 appendice

Towards Validating Risk Indicators Based on Measurement Theory (Extended version)

Due to the lack of quantitative information and for cost-efficiency, most risk assessment methods use partially ordered values (e.g. high, medium, low) as risk indicators. In practice it is common to validate risk indicators by asking stakeholders whether they make sense. This way of validation is subjective, thus error prone. If the metrics are wrong (not meaningful), then they may lead system owners to distribute security investments inefficiently. For instance, in an extended enterprise this may mean over investing in service level agreements or obtaining a contract that provides a lower security level than the system requires. Therefore, when validating risk assessment methods it is important to validate the meaningfulness of the risk indicators that they use. In this paper we investigate how to validate the meaningfulness of risk indicators based on measurement theory. Furthermore, to analyze the applicability of the measurement theory to risk indicators, we analyze the indicators used by a risk assessment method specially developed for assessing confidentiality risks in networks of organizations

Analysis of Visual Communication of Telkom University Endowment and Alumni Directorate Based on User Satisfaction Level using The User Satisfaction Index Method

Most of the universities in Indonesia, especially private universities rely on tuition fees as the main income for their operational cost., one of the solutions is the Endowment Fund, Telkom university has it, namely Endowment and Alumni Directorate (EAD) of Telkom University. Telkom University has managed the Endowment Fund program, but until now it has not been very successful, because the achievement of the target funds collected is far under target. The results of the initial identification show that one of the main media for promotion is the website of the Telkom University Endowment and Alumni Directorate. In this study, prospective donors were assessed on the appearance, content, and message delivered on the website of Telkom University Endowment and Alumni Directorate using the user satisfaction index method. The result of the USI index is 43.64% of overall user assessment of the website, which indicates that the website is still low. Keywords- Endowment Fund, User Satisfaction Index, WebQual 4.

A geoadditive Bayesian latent variable model for Poisson indicators

We introduce a new latent variable model with count variable indicators, where usual linear parametric effects of covariates, nonparametric effects of continuous covariates and spatial effects on the continuous latent variables are modelled through a geoadditive predictor. Bayesian modelling of nonparametric functions and spatial effects is based on penalized spline and Markov random field priors. Full Bayesian inference is performed via an auxiliary variable Gibbs sampling technique, using a recent suggestion of Frühwirth-Schnatter and Wagner (2006). As an advantage, our Poisson indicator latent variable model can be combined with semiparametric latent variable models for mixed binary, ordinal and continuous indicator variables within an unified and coherent framework for modelling and inference. A simulation study investigates performance, and an application to post war human security in Cambodia illustrates the approach