Cross-evaluation has been touted as a powerful extension of Data Envelopment Analysis that provides, not only a unique ordering among the Decision Making Units (DMUs), but also eliminates unrealistic weighting schemes without requiring the elicitation of weight restrictions from application area experts. The goal of this paper is to prove, in the single-input, multiple-output case, cross-evaluation implicitly uses a single fixed set of weights. We demonstrate how this unseen fixed set of weights may still be unrealistic

Anderson, Timothy R.

Hollingsworth, Keith

Inman, Lane

PDXScholar (Portland State University)

Portland State UniversityPDXScholarEngineering and Technology Management FacultyPublications and Presentations Engineering and Technology Management4-1-2001The Fixed Weighting Nature of a Cross-Evaluation ModelTimothy R. AndersonPortland State University, tim.anderson@pdx.eduKeith HollingsworthMorehouse CollegeLane InmanPortland State UniversityLet us know how access to this document benefits you.Follow this and additional works at: http://pdxscholar.library.pdx.edu/etm_facPart of the Engineering CommonsThis Post-Print is brought to you for free and open access. It has been accepted for inclusion in Engineering and Technology Management FacultyPublications and Presentations by an authorized administrator of PDXScholar. For more information, please contact pdxscholar@pdx.edu.Citation DetailsAnderson, Timothy R.; Hollingsworth, Keith; and Inman, Lane, "The fixed weighting nature of a cross-evaluation model" (2001).Engineering and Technology Management Faculty Publications and Presentations. Paper 9. http://archives.pdx.edu/ds/psu/926003/21/13 JPA-Fixed Weighting April 18 2001-PostPrint.doc 1  The Fixed Weighting Nature of a Cross-Evaluation Model   Timothy R. Anderson Engineering Management Program Portland State University   Keith Hollingsworth (Contact) Department of Economics and Business Administration Morehouse College 830 Westview Drive, SW Atlanta, GA  30314  USA (404) 681-2800 ext. 2550 khollingsworth@morehouse.edu   Lane Inman Engineering Management Program Portland State University  03/21/13 JPA-Fixed Weighting April 18 2001-PostPrint.doc 2 Abstract Cross-evaluation has been touted as a powerful extension of Data Envelopment Analysis that provides, not only a unique ordering among the Decision Making Units (DMUs), but also eliminates unrealistic weighting schemes without requiring the elicitation of weight restrictions from application area experts.  The goal of this paper is to prove, in the single-input, multiple-output case, cross-evaluation implicitly uses a single fixed set of weights.  We demonstrate how this unseen fixed set of weights may still be unrealistic.  Keywords Data Envelopment Analysis, Cross-efficiency 03/21/13 JPA-Fixed Weighting April 18 2001-PostPrint.doc 3 1.  Introduction  Cross-efficiency, touted as a powerful extension of Data Envelopment Analysis (DEA), was first proposed by [10] in 1986.  Subsequent development appeared in [2-9, 11] and, as can be seen, its use has proliferated over the last few years.  In traditional DEA, each decision making unit (DMU) is evaluated against the performance of the remaining DMUs in the sample via a ratio of the sum of weighted outputs to the sum of weighted inputs.  Only two restrictions are applied.  The first restriction is that the weights must be non-negative.  The second restriction is that the weighting scheme used will be applied to all other DMUs in the sample and none of them may have a ratio greater than one.  Therefore, an inefficient DMU is one for which a weighting scheme cannot be found that evaluates it better than all other DMUs.  An attempt is made to find the weighting scheme for each DMU that casts it in the most favorable light possible and the resulting ratio is designated the DMU’s efficiency value. However, the values found by applying the chosen weighting scheme to the other DMUs are not retained or used later in traditional DEA.  Cross-evaluation finds a use for those values.  Under a cross evaluation, once the DMU has a chosen weighting scheme which has been applied to all DMUs, the efficiency value given to each DMU is set aside forming a cross-efficiency matrix.  Once the matrix is filled, each DMU has not only its own self-evaluation but also the peer evaluations it has received via the other DMUs in the sample.  The average across self and peer evaluations represents a DMU’s cross efficiency value.  A DMU which has a high cross efficiency value has, therefore, passed a more rigorous test since it can not only make itself look good but is considered efficient by the majority of its peers.  The reverse to this is that, while traditional DEA has placed its emphasis on the efficient DMUs which form the production frontier and represent the 03/21/13 JPA-Fixed Weighting April 18 2001-PostPrint.doc 4 best performing, cross evaluation gives inefficient DMUs a far greater voice in the identification of the best practices.  Cross efficiency proponents often state two main advantages to its use.  First, it usually creates a unique ordering among the DMUs [6].  Since, in traditional DEA, free reign is given when deciding for each DMU which outputs and inputs to emphasize, many different avenues are present by which a DMU can appear efficient.  Therefore, it is common to have many DMUs that are relatively efficient.  With cross-evaluation, on the other hand, since each DMU is rated not only by its own weighting scheme but the schemes of the others also, this amalgamation of weighting schemes makes it far more difficult to have ties and, in effect, creates a unique ordering in practice. Second, cross efficiency appears to eliminate unrealistic weighting schemes which might be used by the DMUs.  Again, since each DMU has its own set of weights, all of its weight might be put on a single output and input.  While this is permissible, it may not be realistic.  An example of this is shown later in this paper in Table I where it is shown that with DMU B, output 2 is disregarded to emphasize output 1 and, with DMU C, output 1 is disregarded to emphasize output 2.  In traditional DEA, one solution to this is to add weight restrictions, which prevent DMUs from having unrealistic weights.  However, the weights must be added externally to the problem relying on the expert knowledge of the modeler to create these restrictions.  Since DEA weights do not have normal straightforward financial interpretations, this elicitation can be challenging.  In cross-evaluation, since the cross-efficiency value is a function of all of the weighting schemes, it has been proposed that the unrealistic ones may{are,} in effect, cancel out.  In addition, rather than have an 03/21/13 JPA-Fixed Weighting April 18 2001-PostPrint.doc 5 external weight restriction applied via an expert, the dataset serves as the arbiter of good judgment by, in essence, creating its own weight restrictions. In fact, this paper demonstrates that cross evaluation, using the single input/multiple output model, creates an implicit fixed weighting scheme which is applied to all DMUs.  This implicit scheme is a weighted average of the weights used by each of the DMUs in the sample.  In [13], Talluri and Sarkis state cross evaluation “maintains the weighting flexibility in DEA”.  This is both true and false.  It is true in that the DMUs in the sample, acting in concert, have the flexibility to create their own weighting scheme without the interference of outside experts.  It is false in that each DMU is forced to use the implicit weighting scheme created by the sample as a whole as opposed to having the flexibility to choose its own as in traditional DEA in the single-input, multiple-output case.  A major problem occurs if this implicit fixed weighting scheme has its own unrealistic values that may not be checked by the modeler if they are unaware the weights are being applied. 2.  Performing a Cross Evaluation Cross-efficiency is often calculated as a two-phase process.  The first phase derives individual DMU weighting schemes through traditional DEA efficiency score calculations. However, these solutions are often highly degenerate, particularly for efficient DMUs, resulting in multiple possible weighting schemes.  The second phase attempts to mitigate this problem of multiple solutions and is a process by which for each DMU, given its initial efficiency score, one of the available weighting schemes is selected for application to itself and others.  Although not necessarily unique, the results significantly reduce the potential difficulty of multiple optima. 03/21/13 JPA-Fixed Weighting April 18 2001-PostPrint.doc 6 The first phase is calculated using the standard Charnes, Cooper, and Rhodes formulation [1978].  Given the results of the first phase, we could use the weights used by the DMU for itself to calculate the peer-rated efficiency for each of the other DMUs.  The cross-efficiency score, p,j, is the efficiency score for DMU j using the weights selected by DMU p.    mijipisrjrprjpxvy1,,1,,,  (1)  As described earlier, the solutions are often highly degenerate and result in non-unique optima in terms of weights.  Therefore, the scores p,j for jp would be arbitrary making it difficult to reproduce cross-evaluation results. To mitigate this problem, the second phase holds the self-rated efficiency score, p,p, fixed and uses a secondary objective function to select a particular set of weights to be used in (2).  The particular cross-evaluation technique used here was developed in [10] and further investigated in [6] where it corresponds to Doyle and Green's aggressive formulation option (II). This technique is referred to as an aggressive formulation as it seeks to minimize the efficiency of the population of the other DMUs while maintaining the efficiency of the DMU under consideration fixed.  In the paper, Doyle and Green showed that different aggressive formulations appear to provide relatively consistent results.  The following formulation, (2), is used as the second phase for the cross-evaluation of DMU p.  03/21/13 JPA-Fixed Weighting April 18 2001-PostPrint.doc 7    misrvxvypjxvyxvtsyprpimipipippsrprprmijipisrjrprmi pjjipisr pjjrprv,,1,,,10,001..min,,1,,,1,,1,,1,,1,,1,,,        (2)  The two-phase process is repeated for each DMU, p, where p(1,…, n).  The resulting weights, vi,p, and r,p, are then used to calculate p,j, for each DMU j using (2).  Once all of the peer evaluation scores, p,j, are calculated for each DMU, j, from the perspective of each DMU p, the cross-efficiency scores is then simply the mean of the peer and self evaluations from a table of Cross-Efficiency Scores as described in (4).  nCEnjkjk 1,  (3)  A benevolent formulation also exists in which a DMU, holding its own efficiency value constant, would seek to maximize rather than minimize the efficiency of the rest of the overall population.  This is accomplished by simply changing the objective function.  Similarly, other formulations have been proposed for the second phase to select a unique weighting scheme for each DMU.  However, the following derivation will hold true for any variation of (3) currently proposed since it depends only upon the values of the weights obtained and the equation for calculating cross-efficiency (2). 03/21/13 JPA-Fixed Weighting April 18 2001-PostPrint.doc 8 3.  Derivation of the Implicit Fixed Weighting Scheme We will examine the case of cross-evaluation with one input and multiple outputs.  The cross-efficiency score (CEk) is calculated as shown below:      njmikijisrkrjrxvyn 11,,1,,1 (4) In the single input case, this becomes,   nj kjsrkrjrxvyn 1 ,1,11,,1 (5)   njsr jkrjrk vynx 1 1 ,1,,,11  (6)  Rearranging terms through simple algebraic manipulation results in the following:     srnj jjrkrk vynx 1 1 ,1,,,11  (7) To clarify the derivation and interpretation of the fixed weights, we rewrite this as,  ksrnj jjrkrxvny,11 1 ,1,,1    (8)  03/21/13 JPA-Fixed Weighting April 18 2001-PostPrint.doc 9 Notice that the multiplier,    nj jjr vn 1 ,1,1  , for each output, yr,k, is independent of the DMU, k, being examined.  In other words, this multiplier is essentially a fixed weight that can be used in the calculation of each DMU’s cross-evaluation scores.  The n multipliers for the n outputs form a fixed, common set of weights for calculating cross-evaluation scores.  The multiplier for the input is unity.  Given this is a standard applied to all of the DMUs, it is incumbent upon the analyst choosing to use cross-evaluation to explicitly examine and justify these fixed weights as we can see in the following section. 4.  An Example of Fixed Weighting in Cross-Evaluation To confirm this derivation, we compared the cross-efficiency results obtained using the above equation with those obtained using the traditional cross-efficiency approach.  We will begin by re-examining the hypothetical example that Doyle and Green [6] used to demonstrate cross-evaluation.  The data, the weights from the second phase of the cross-evaluation, self-rated efficiency scores (regular DEA efficiency scores), the matrix of peer-related efficiency scores, and the cross-efficiency scores are provided in Table I. Table I:  Cross-Evaluation Results for Example 1     Weights Self- Peer Evaluations  x y1 y2 v1 u1 u2 Eval A B C X Y Z A 1 10.7 12.0 0.2 0.0015 0.0153 1.0000 1.0000 0.2809 1.0000 0.9679 0.9801 0.9579 B 1 11.6   2.5 0.2 0.0172 0.0000 1.0000 0.9224 1.0000 0.2414 0.9052 0.8707 0.8793 C 1   2.8 12.8 0.2 0.0000 0.0156 1.0000 0.9375 0.1953 1.0000 0.9063 0.9219 0.8984 X 1 10.5 11.6 0.2 0.0169 0.0016 0.9799 1.0000 1.0000 0.3390 0.9799 0.9477 0.9538 Y 1 10.1 11.8 0.2 0.0015 0.0153 0.9801 1.0000 0.2809 1.0000 0.9679 0.9801 0.9579 Z 1 10.2 11.5 0.2 0.0015 0.0153 0.9579 1.0000 0.2809 1.0000 0.9679 0.9801 0.9579        Avg. CE 0.9767 0.5063 0.7634 0.9492 0.9467 0.9342  These results are consistent with those of [6].  From these weights, we can determine the implicit fixed weights and calculate each DMU’s cross-efficiency score.  03/21/13 JPA-Fixed Weighting April 18 2001-PostPrint.doc 10 The fixed weight for the input is 6 and, for outputs 1 and 2, the weights were 0.03232 and 0.05257 respectively. Results using these fixed weights match those obtained using the standard column average method to four decimal places of accuracy.  It is interesting to note that the fixed weight for the second output is more than 50% greater than the weight of the first output.  Is this reasonable?  It depends on the judgement of the modeler.  However, what we do see here, is that cross-evaluation is not a replacement for careful thought on the part of the modeler. 5.  Conclusion In this paper, we have derived and demonstrated that cross-evaluation, in a single input situation, in effect applies an implicit fixed weighting scheme to each and every DMU which is a weighted average of the weights used by all of the DMUs in the sample.  This has the effect, contrary to popular claims concerning cross-evaluation, of reducing the flexibility inherent in a DEA evaluation since each DMU no longer has the ability to create its own weighting scheme. While the common set of weights is potentially interesting, it should be noted the applicability of this is rather limited since it is based on a single-input, multiple-output constant returns to scale model with input-orientation.  The same process can be used to prove that a common set of weights exist in the multiple-input, single-output constant returns to scale model with output orientation.  The multiple-input, multiple-output models do not exhibit this fixed weighting phenomena because of the inability to normalize the weights.  Similarly, scale considerations in models other than the constant returns to scale model prevent separating out a simple fixed multiplier independent of k, 03/21/13 JPA-Fixed Weighting April 18 2001-PostPrint.doc 11 and therefore, there is no set of fixed weights in these models either. 03/21/13 JPA-Fixed Weighting April 18 2001-PostPrint.doc 12 Appendix I Data and Variable Definitions n is the number of DMUs to be analyzed. s is the number of outputs to be analyzed. yrk is the value of output r for DMU k. xik is the value of input i for DMU k. rk is the weight for output r as determined by the evaluation of DMU k. vik is the weight for input i as determined by the evaluation of DMU k. pj is the efficiency of DMU j with the weights determined by DMU p. CEj is the cross-efficiency of DMU j.  03/21/13 JPA-Fixed Weighting April 18 2001-PostPrint.doc 13 Bibliography [1] T. R. Anderson, A. Uslu, and K. B. Hollingsworth, "Revisiting extensions in efficiency measurement of alternate machine component grouping solutions via data envelopment analysis," Working paper 1998. [2] R. C. Baker and S. Talluri, "A closer look at the use of data envelopment analysis for technology selection," Computers & Industrial Engineering, vol. 32, pp. 101-108, 1997. [3] W. D. Cook, J. Doyle, R. Green, and M. Kress, "Ranking players in multiple tournaments," Computers & Operations Research, vol. 23, pp. 869-880, 1996. [4] J. Doyle and R. Green, "Efficiency and cross-efficiency in DEA: derivations, meanings and uses," Journal of the Operational Research Society, vol. 45, pp. 567-578, 1994. [5] J. R. Doyle, "Multiattribute choice for the lazy decision maker: Let the alternatives decide!," Organizational Behavior & Human Decision Processes, vol. 62, pp. 87-100, 1995. [6] J. R. Doyle and R. H. Green, "Cross-evaluation in DEA: Improving discrimination among DMUs," INFOR, vol. 33, pp. 205-222, 1995. [7] R. Green and J. Doyle, "On maximizing discrimination in multiple criteria decision making," Journal of the Operational Research Society, vol. 46, pp. 192-204, 1995. [8] R. H. Green, J. R. Doyle, and W. D. Cook, "Preference voting and project ranking using DEA and cross-evaluation," European Journal of Operational Research, vol. 90, pp. 461-472, 1996. [9] M. Oral, O. Kettani, and P. Lang, "A methodology for collective evaluation and selection of industrial R&D projects," Management Science, vol. 37, pp. 871-885, 1991. [10] T. R. Sexton, R. H. Silkman, and A. Hogan, "Data envelopment analysis:  critique and extensions," in Measuring Efficiency:  An Assessment of Data Envelopment Analysis, vol. 32, New Directions for Program Evaluation, R. H. Silkman, Ed. San Francisco: American Evaluation Association, Jossey Bass, Inc., 1986. [11] S. Talluri and J. Sarkis, "Extensions in efficiency measurement of alternate machine component grouping solutions via data envelopment analysis," IEEE Transactions on Engineering Management, vol. 44, pp. 299-304, 1997. 

The Fixed Weighting Nature of a Cross-Evaluation Model

http://pdxscholar.library.pdx.edu/cgi/viewcontent.cgi?article=1008&context=etm_fac

The Fixed Weighting Nature of a Cross-Evaluation Model

Abstract

Similar works

Full text

Available Versions

PDXScholar (Portland State University)