Subjective and objective performance evaluation

We study executive compensation in an environment in which firms compete offering contingent contracts to managers with private information about their ability. We ask whether equilibrium executive compensation depends on subjective evaluations, i.e., on assessments made by the firm which are based on noncontractible information. We also allow for objective (i.e., contractible) performance measures and we depart from the rest of the literature on the topic by assuming that subjective evaluations are made before the uncertainty on the objective performance measures is resolved. We find that even in this case, equilibrium contracts ignore subjective evaluations regardless of their informativeness

Subjective Evaluation versus Public Information

This paper studies a principal-agent relation in which the principal's private information about the agent's effort choice is more accurate than a noisy public performance measure. For some contingencies the optimal contract has to specify ex post inefficiencies in the form of inefficient termination (firing the agent) or third-party payments (money burning). We show that money burning is the less efficient incentive device: it is used at most in addition to firing and only if the loss from termination is small. Under an optimal contract the agent's wage may depend only on the principal's report and not on the public signal. Nonetheless, public information is valuable as it facilitates truthful subjective evaluation by the principal

Subjective and objective quality evaluation of compressed medical video sequences

Existing objective video quality metrics such as VQM from NTIA [1] and MOVIE [2] are known to perform well for assessing compression degradation in natural scene and broadcast television sequences but their suitability for the quality evaluation of compressed medical video has not been studied extensively. In this work we assess the quality of compressed medical video sequences using objective metrics and a subjective evaluation study conducted with non-expert subjects. Test sequences consist of High Definition medical video of laparascopic surgery. Four compression types (Motion JPG and three variants of H.264) at four bit-rates (5, 12, 20, and 45 Mbps) are studied and compared to original uncompressed sequences. One reduced reference metric (VQM) and one full-reference metric (MOVIE) are studied. Subjective video evaluation consists of overall quality scores as well as difference scores between compressed and uncompressed sequences for similarity and five types of artifacts or attributes: blurring, blocking, noise, color fidelity, and motion artifacts. The results of the subjective and objective evaluations exhibit similar trends across the compression types and bit-rates, and may indicate that these objective quality metrics may be valid reflections of subjective quality judgments made by non-expert observers on compressed medical video sequences. In future work we will expand the subjective quality evaluation to include expert laparoscopic surgeons as subjects

Subjective Performance Evaluation and Inequality Aversion

Many firms use subjective performance appraisal systems due to lack of objective performance measures. In these cases, supervisors usually have to rate the performance of their subordinates. Using such systems, it is a well established fact that many supervisors tend to assess the employees too good (leniency bias) and that the appraisals hardly vary across employees of a certain supervisor (centrality bias). We explain these two biases in a model with a supervisor, who has preferences for the utility of her inequality averse subordinates, and discuss determinants of the size of the biases. Extensions of the basic model include the role of supervisor’s favoritism of one particular agent and the endogenous effort choice of agents. Whether inequality averse agents exert higher efforts then purely self-oriented ones, depends on the size of effort costs and inequality aversion.appraisals, inequality aversion, performance evaluation, centrality bias, leniency bias

Towards a tool for the subjective assessment of speech system interfaces (SASSI)

Applications of speech recognition are now widespread, but user-centred evaluation methods are necessary to ensure their success. Objective evaluation techniques are fairly well established, but previous subjective techniques have been unstructured and unproven. This paper reports on the first stage of the development of a questionnaire measure for the Subjective Assessment of Speech System Interfaces (SASSI). The aim of the research programme is to produce a valid, reliable and sensitive measure of users' subjective experiences with speech recognition systems. Such a technique could make an important contribution to theory and practice in the design and evaluation of speech recognition systems according to best human factors practice. A prototype questionnaire was designed, based on established measures for evaluating the usability of other kinds of user interface, and on a review of the research literature into speech system design. This consisted of 50 statements with which respondents rated their level of agreement. The questionnaire was given to users of four different speech applications, and Exploratory Factor Analysis of 214 completed questionnaires was conducted. This suggested the presence of six main factors in users' perceptions of speech systems: System Response Accuracy, Likeability, Cognitive Demand, Annoyance, Habitability and Speed. The six factors have face validity, and a reasonable level of statistical reliability. The findings form a userful theoretical and practical basis for the subjective evaluation of any speech recognition interface. However, further work is recommended, to establish the validity and sensitivity of the approach, before a final tool can be produced which warrants general use

Do different subjective evaluation criteria reflect distinct constructs?

This is not the published version. Published version available from: http://journals.lww.com/jonmd/pages/default.asp

Subjective Measures of Economic Well-Being and the Influence of Income Uncertainty

This paper provides evidence that subjective measures of individual well being can be used to study the impact of income uncertainty from an ex ante point of view. Two different measures of subjective well being are under study: Satisfaction with household income and the income evaluation question as developed by Van Praag. It can be shown that satisfaction with income is more affected by ex ante than by ex post volatility of income. The ordinal version of the Van Praag approach might be biased if income uncertainty is essential. The paper was written in 1994.income uncertainty, subjective well-being, satisfaction, income evaluation

Fuzzy subjective evaluation of Asia Pacific airport services

This paper presents a fuzzy decision-making model to determine the ranking of fourteen Asia Pacific airports based on the services provided to passengers. Airport services were represented by six attributes namely comfort, processing time, convenience, courtesy of staff, information visibility and security. Data for the attributes given by travel experts are in the triangular fuzzy number form. Based on fuzzy set and approximate reasoning, the model allows decision makers to make the best choice in accordance with human thinking and reasoning processes.The use of fuzzy rules which are extracted directly from the input data in making evaluation, contributes to a better decision and is less dependent on experts.Experimental results show that the proposed model is comparable to previous studies.The model is suitable for various fuzzy environments