Several publications have appeared in the field of\ud
Operations Management which rank Operations\ud
Management related journals. Several ranking systems\ud
exist for journals based on , for example, perceived\ud
relevance and quality, citation, and author affiliation.\ud
Many academics also publish at conferences but we have\ud
not come across publications that rank conferences.\ud
Conference rankings are generally more complicated than\ud
journal rankings. Journal rankings are primarily for\ud
publishing purposes. Conferences on the other hand are\ud
attended by people for different reasons. In this paper the\ud
first attempt is made in developing an operations\ud
management conference ranking based upon author\ud
affiliation. Ranking based on an analysis of author\ud
affiliation assumes that one important motive for\ud
participants is to attend a high quality research\ud
conference. With that assumption it is reasonable to use\ud
the author affiliation approach. Based upon an existing\ud
ranking of institutes that offer operations management\ud
programs a ranking list of affiliations is developed.\ud
Subsequently, we compare several operations\ud
management related conferences such as POMS,\ud
EurOMA, OSCM and the Operations Management\ud
Division of the Academy of Management based on that\ud
ranked list of institutes. The results provide information\ud
for authors that help in deciding which operations\ud
management oriented conferences to attend

Bruijn, Erik Joost de

Gupta, Sushil

Steenhuis, Harm-Jan

English

Several publications have appeared in the field of Operations Management which rank Operations Management related journals. Several ranking systems exist for journals based on , for example, perceived relevance and quality, citation, and author affiliation. Many academics also publish at conferences but we have not come across publications that rank conferences. Conference rankings are generally more complicated than journal rankings. Journal rankings are primarily for publishing purposes. Conferences on the other hand are attended by people for different reasons. In this paper the first attempt is made in developing an operations management conference ranking based upon author affiliation. Ranking based on an analysis of author affiliation assumes that one important motive for participants is to attend a high quality research conference. With that assumption it is reasonable to use the author affiliation approach. Based upon an existing ranking of institutes that offer operations management programs a ranking list of affiliations is developed. Subsequently, we compare several operations management related conferences such as POMS, EurOMA, OSCM and the Operations Management Division of the Academy of Management based on that ranked list of institutes. The results provide information for authors that help in deciding which operations management oriented conferences to attend

Steenhuis, H.J.

de Bruijn, E.J.

University of Twente Research Information

RANKING OPERATIONS MANAGEMENT CONFERENCES   Harm-Jan Steenhuis1, Erik J. de Bruijn2, and Sushil Gupta3 1Eastern Washington University 668 N. Riverpoint Blvd., Suite A, Spokane WA 99202, USA Email: hsteenhuis@mail.ewu.edu  2University of Twente PO Box 217, 7500 AE  Enschede The Netherlands Email: e.j.debruijn@bbt.utwente.nl  3Florida International University RB 255, 11200 S.W. 8th Street Miami FL 33199, USA Email: guptask@fiu.edu    ABSTRACT Several publications have appeared in the field of Operations Management which rank Operations Management related journals. Several ranking systems exist for journals based on , for example, perceived relevance and quality, citation, and author affiliation. Many academics also publish at conferences but we have not come across publications that rank conferences. Conference rankings are generally more complicated than journal rankings. Journal rankings are primarily for publishing purposes. Conferences on the other hand are attended by people for different reasons. In this paper the first attempt is made in developing an operations management conference ranking based upon author affiliation. Ranking based on an analysis of author affiliation assumes that one important motive for participants is to attend a high quality research conference. With that assumption it is reasonable to use the author affiliation approach. Based upon an existing ranking of institutes that offer operations management programs a ranking list of affiliations is developed. Subsequently, we compare several operations management related conferences such as POMS, EurOMA, OSCM and the Operations Management Division of the Academy of Management based on that ranked list of institutes. The results provide information for authors that help in deciding which operations management oriented conferences to attend.  KEY WORDS Conference Rankings, Rankings, Operations and Management   1.  Introduction  Most faculty members are required to publish research as part of their yearly workload. For example, faculty at AACSB accredited institutions are required to maintain intellectual qualifications through a.o. journal publications. Colleges can determine their own definitions of what is acceptable and what is not. At many institutions this has developed into categories of journals, such as an A-list, B-list, etc.. These lists are often based upon published ranking lists and are extremely important not only in determining faculty qualifications but also for retention and promotion decisions. Although maybe less regarded, conference publications are another method of intellectual contributions that can be evaluated for performance. To the best of our knowledge, no rankings have been published on operations management conferences. The intent of this paper is to contribute in this area. The paper is developed by first discussing different journal ranking systems. The type of ranking system provides insight into how conferences can be ranked. After this, the literature on conference rankings is discussed. This includes the selection of a method and what the result, i.e. the rankings, mean. This is followed by a discussion of university rankings which serve as a base for the conference rankings and subsequently the comparative findings for several operations management oriented conferences. Finally, some conclusions are drawn.   2.  Ranking Journals  Before discussing conference rankings, this section will first look at journal rankings since these methods can provide insight into how conferences can be ranked. There are three different types of methods for determining journal rankings. Each of these ranking systems has a different orientation which can lead to different ranking results. The three ranking systems are: based upon perceived relevance and quality, based upon citation, and based upon author affiliation.  2.1 Perceived Relevance and Quality  One system for ranking journals is using perceived relevance and quality. With this method, a group of people is identified as well as a set of journals. People are then asked through surveys how these journals score on relevance and how they score on quality. Examples of these type of rankings are provided in [1-3]. One of the subjective elements in this type of ranking is the initial selection of the respondents. For example, if POMS members are surveyed about operations management oriented journals then this ranking might differ from Operations Management Division members of the Academy of Management. This explains how journals can be ranked differently even though the same survey instrument is used for ranking purposes.  2.2 Citation Based  Another system for ranking journals is by looking at citations. This method is based upon determining which journals have been the most influential in a discipline. With this method a set of journals (set A) is identified for ranking purposes and additionally a set of journals (set B, which can overlap set A) is identified for citations analysis purposes. The articles in journal set B are analyzed to determine which journals from set A are referenced most often. This determines the ranking of the journals in set A. Examples of this type of ranking are provided in [4-5]. One of the subjective elements in this type of ranking is the initial selection of the journals used for citation analysis (set B). For example, authors that use mathematical modeling approaches are likely to cite articles that are also mathematical modeling oriented. Therefore, if more mathematical modeling oriented journals are in set B, then it is expected that mathematical oriented journals in set A will be referenced more often than non-mathematical oriented journals in set A and therefore will be ranked higher.  2.3 Author Affiliation  A third type of journal ranking is based upon author affiliation. This method is based upon ranking journals by examining who publishes in these journals and the affiliation of these authors. An example of this type of ranking, for operations management, is provided in [6].   One of the subjective elements in this type of ranking is the initial selection of author affiliations. In this system a ranking of affiliations is assumed and the journal ranking is based upon the relative number of authors from the ranked institutions.  2.4 Conclusion  Journal rankings have been determined to aid, among other things, tenure and promotion decisions by providing a sense of ‘quality’ of publications. There are at least three different methods for ranking journals. Each of these methods is based upon certain assumptions which influence the ranking outcome.   3.  Ranking Conferences  Although publications about journal rankings are relatively common, this is quite a different matter for conferences. Part of the problem with conferences is that conference attendance and conference publications serve more roles than just an intellectual contribution. For example, one motive to participate in an operations management conference can be to meet colleagues in the area of operations management. Another motive can be to learn about new developments in an area of interest. Another motive can be to satisfy publishing requirements. Another motive can be that the specific conference theme is of specific interest. Getting feedback and/or validation for on-going research can be another motivator. Lastly, in some instances, the specific location of the conference can be an incentive to participate. Overall, it is therefore much harder to develop a conference ranking than developing a journal ranking due to the number of different factors involved for participation.  In this paper a ranking of conferences is developed based upon the assumption that it relates to scientific ‘quality’. In other words, this assumption is similar to the system for ranking journals. Obviously, if these methods are used for conference ranking then the concerns as expressed in section two for journal rankings apply here as well. To the best of our knowledge, a ranking for operations management conferences has not yet been published. Nevertheless, some conference rankings in other disciplines have been published, in particular in the area of computer science. An example is provided by Citeseer (http://citeseer.ist.psu.edu/impact.html.) This ranking is based upon the citation analysis method. Another ranking is provided by Computer Science Conference Ranking (http://www.cs-conference-ranking.org/). They use an estimated impact of conference that uses the following weights: 40% for citation of papers, 20% for quality of referees’ reports, 20% for availability of resources for students, 10% for indexing and 10% for the percentage of conference papers that are accepted or appeared in reputable journals. This type of ranking combines some of the different types of motives for attending a conference, for example financial support for students and quality of papers. Much of the weight is contributed to the citation of papers similar to the citation method for journal rankings. A study by Mathis and Zech [7] looked at regional economics conferences and ranked based upon two indicators. First, they looked at the proportion of papers that were subsequently published in journals and second, they looked at the number of economists from top-rated universities who participated. This latter aspect is similar to the author affiliation method. In this paper the first attempt to rank operations management conferences is made. For this ranking, the intent is to stay  focused rather than combine multiple aspects such as by Computer Science Conference Ranking. Combining different aspects introduces the aspect of weight. Determining weight is difficult because not all attendees attend a conference for the same reason and at this point not enough is known about the participants’ motives. The primary focus will be one of ‘quality’. This means a more limited scope. The advantage of this is that the ranking will be more straightforward. The disadvantage is that the ranking will only be relevant to those attendees who are interested in the quality aspect of a conference. In this paper a choice is made to use the author affiliation method. Each method has pros and cons for ranking as described in section 2. The advantage of the author affiliation method is that it gives an indication of who is attending these types of conferences. Author affiliations provide two indications for quality. First, with the underlying assumption that affiliation is related to quality research, the authors from these affiliations are expected to present high quality research. Second, it indicates that researchers from top-affiliates are interested in a particular conference which is, assuming that these researchers are mainly interested in high quality research from others as well, another indication that the conference is of high quality. Note that high quality is equated here with the quality of the papers.   4.  Ranking Affiliations  In order to develop a ranking list based upon author affiliation, a ranking list of affiliations for operations management is required. Some rankings exist. For example the Financial Times publishes regularly a ranking list of the top-100 full time global MBA programs. To determine a conference ranking for operations management, by utilizing the author affiliation index, a discipline specific ranking of affiliations is required. Business Week and U.S. News & World Report publish these types of rankings but their rankings only include U.S. institutions. The School of Management at the University of Texas at Dallas provides a method for ranking operations management affiliations (see: http://citm.utdallas.edu/utdrankings). Their system allows selecting a range of journals and then a ranking is determined by looking at who publishes in operations management journals for a specific time period. This ranking can be used as a base to calculate the author affiliation index for conferences. Seven journals were selected that were appropriate for operations management. These also appeared in operations management journal rankings such as by Barman et al. [2]. The seven journals were: Management Science, Operations Research, Journal of Operations Management, Manufacturing and Service Operations Management, Production and Operations Management, Academy of Management Journal and Academy of Management Review. The last ten years was selected as the time period for tracking publications. The ranking included 100 global schools. Table 1 provides some insight by providing the top-10 schools.  Table 1: Worldwide top-10 operations management affiliations based on University of Texas database.  Rank Affiliation 1 University of Pennsylvania (Wharton School) 2 Columbia University (Columbia Graduate School of Business) 3 Massachusetts Institute of Technology (Sloan School of Management) 4 Michigan State University (The Eli Broad College of Business) 5 Duke University (Fuqua School of Business) 6 Harvard University (Harvard Business School) 7 New York University (Leonard N. Stern School of Business) 8 University of Maryland at College Park (Robert H. Smith School of Business) 9 University of Minnesota at Twin Cities (Minneapolis) Carlson School of Management 10 Stanford University (Graduate School of Business)   5.  Findings  The author affiliation index for conference x is computed based upon Gorman and Kanet [6], as follows. Let n(i) is total number of authors for article I, A(i) is the number of authors for article I from the top university set, B(i) is the number of authors not from the top university set, M is the set of conference articles for conference x, then the author affiliation index for conference x is:  ∑∑∈∈+=MiMiiniBiAiniAxAAI)(/)]()([)(/)()( In other words, the author affiliation index is the ratio of authors from top-schools divided by the total number of authors that appear on papers. As indicated before, one element for conferences is also who attends the conferences. Earlier analysis [8-9] has shown that in some cases over ten authors appear on a paper. To control for this aspect only the first author will be used. The first author affiliation index (FAAI) is computed as follows:  MAxFAAI Mii∑∈=)(   Where; FAAI(x) is first author affiliation index for conference x. Ai is a first author from the top-100 affiliations and M is the total number of articles at conference i. For the analysis the year 2005 is used to illustrate the concept, this is the only previous year that the OSCM conference was held. For the analysis, the FAAI index is calculated for POMS, the US operations management society, EurOMA, the Eureopean operations management association, AOM, the operations management division of the US based Academy of Management and lastly OSCM. The results are provided in table 2.  Table 2: Conference Rankings Based upon FAAI.  Rank Conference Authors from ranked affiliations Total papers FAAI 1 POMS 2005 218 550 0.396 2 AOM 2005 13 48 0.271 3 EurOMA 2005 22 243 0.091 4 OSCM 2005 7 146 0.048  The results show that POMS scores the best followed by the Academy of Management, EurOMA and OSCM. The low scores for EurOMA and OSCM can be explained by the ranking list used for affiliations. This ranking was based upon US journals which favoured US-based authors. It is therefore not surprising that non-US-based conferences have a lower FAAI score.   6.  Conclusions  In this paper a first attempt has been made to develop a ranking list for operations management conferences. A review of the literature revealed that no such ranking list exist. One of the methods used to rank journals, which has also been used in other fields to aid the development of conference rankings, is the author affiliation index. A similar approach was used here but limited to the first author. An existing global top-100 ranking list was used for determining the ranked affiliations. The year 2005 was used as a test year and several operations management oriented conferences were compared. It was found that POMS had the highest ranking and US-based conferences scored higher than non-US based conferences. Lessons for the future are that the FAAI index is only as useful as the affiliation ranking that is used. The affiliation ranking that was used in this paper was very much oriented on US affiliations because it was based upon US journals. To improve the conference ranking we propose that first a better affiliation ranking is developed. This ranking can be developed based upon OM journals but should include non-US journals as well. We propose a set of 15-21 journals including 1/3 from the US, 1/3 from Europe and 1/3 from other regions. Furthermore, the journals should not be primarily operations research oriented but ideally production and operations management oriented although it might also include operations research, manufacturing management and technology management oriented journals. Once this list is developed an improved conference ranking can be developed.   References  [1] S. Barman, R.J. Tersine & M.R. Buckley, An empirical assessment of the perceived relevance an quality of POM-related journals by academicians, Journal of Operations Management, 10(2), 1991, 194-212. [2] S. Barman, M.D. Hanna & R. L. LaForge, Perceived relevance and quality of POM journals: a decade later, Journal of Operations Management, 19, 2001, 367-385. [3] A.C. Soteriou, G.C. Hadjinicola & K. Patsia, Assessing production and operations management related journals: the European perspective, Journal of Operations Mangement, 17, 1999, 225-238. [4] C.H. Goh, C.W. Holsapple, L.E. Johnson & J.R. Tanner, Evaluating and classifying POM journals, Journal of Operations Management, 15, 1997, 123-138. [5] R.J. Vokurka, The relative importance of journals used in operations management research, A citation analysis, Journal of Operations Management, 14, 1996, 345-355. [6] M.F. Gorman & J.J. Kanet, Evaluating operations management-related journals via the author affiliation index, Manufacturing & Service Operations Management, 7(1), 2005, 3-19. [7] E.J. Mathis & C.E. Zech, The quality of regional economics conferences, Eastern Economic Journal, 18(2), 1992, 221-228. [8] H.J. Steenhuis, E.J. de Bruijn & S. Gupta, Location analysis of POMS conference contributions, POM, 17th annual conference, Boston, MA, 2006, 3. [9] H.J. Steenhuis & E.J. de Bruijn, Location analysis of EurOMA conference contributions, EurOMA, Vol. ii, Glasgow, UK, 2006. 435-443.  

Ranking Operations Management conferences

An empirical assessment of the perceived relevance an quality of POM-related journals by academicians,

Assessing production and operations management related journals: the European perspective,

Evaluating and classifying POM journals,

Evaluating operations management-related journals via the author affiliation index,

Perceived relevance and quality of POM journals: a decade later,

The quality of regional economics conferences,

The relative importance of journals used in operations management research, A citation analysis,

http://purl.utwente.nl/publications/73381

Ranking Operations Management Conferences

Ranking Operations Management Conferences

Abstract

Similar works

Full text

Available Versions

University of Twente Research Information