AbstractQuality and effectiveness are vital factors in health, safety and environment management systems. In order to move towards the goal of increasing quality, to achieve best results, to reach the continuous improvement of system services and productions, and also to make the internal and external customers satisfied, it is necessary to consider the system performance measurement. This study was aimed to represent Health, Safety and Environment Excellence Instrument to measure the performance of a wide range of health, safety and environment management systems. The development of the proposed tool overall structure, its principles, and its test results in three different contractors of MAPNA Group are presented in this article. Test results in three organizations revealed that, on the whole, the instrument has the ability to measure the performance and efficiency of health, safety and environment management systems in a wide range of systems

Kianfar, A.

Mahmoudi, S.

Mohammadfam, I.

Elsevier - Publisher Connector 

 Procedia Engineering  45 ( 2012 )  194 – 198 
1877-7058 © 2012 Published by Elsevier Ltd.
doi: 10.1016/j.proeng.2012.08.142 
2012 International Symposium on Safety Science and Technology 
Development of the health, safety and environment excellence 
instrument: a HSE-MS performance measurement tool 
I. MOHAMMADFAMa, b*, S. MAHMOUDIb†, A. KIANFARa 
aDepartment of Occupational Health and Faculty of Health, Hamadan University Of Medical Science, Hamadan , Iran 
bDepartment of HSE, MAPNA Group, Tehran 1918953651, Iran 
Abstract 
Quality and effectiveness are vital factors in health, safety and environment management systems. In order to move towards the goal of 
increasing quality, to achieve best results, to reach the continuous improvement of system services and productions, and also to make the 
internal and external customers satisfied, it is necessary to consider the system performance measurement. This study was aimed to 
represent Health, Safety and Environment Excellence Instrument to measure the performance of a wide range of health, safety and 
environment management systems. The development of the proposed tool overall structure, its principles, and its test results in three 
different contractors of MAPNA Group are presented in this article. Test results in three organizations revealed that, on the whole, the 
instrument has the ability to measure the performance and efficiency of health, safety and environment management systems in a wide 
range of systems.  
 
© 2012 The Authors. Published by Elsevier Ltd. Selection and/or peer-review under responsibility of the Beijing Institute of 
Technology. 
 
Keywords: performance measurement; health; safety; environment; management system 
1.  Introduction  
In modern management, human resources are the main axis of sustainable development [1]. To achieve industrial world 
and sustainable development different approaches have been considered, but regardless of human resources, no process 
would move toward the desired results and even the best designed system will fall down [2]. Modern organizations are in a 
competitive world, so in addition to a desire to promote customer satisfaction, they should seriously consider safety, health 
and welfare of their personnel as well as protection of environment [3]. Obviously, according to the principles mentioned, 
these elements have been emphasized in national and international regulation and litigation (including Iran's national 
regulations) [4]. Furthermore, nowadays Safety, health, and environmental factors for customers, employees and other 
stakeholders are delicate subjects [5]. Thus, observing these fundamentals can help organizations on the way to continuous 
improvement. In order to meet these necessities, Management Systems such as ISO 9000, ISO 14000, and OHSAS 18000 
have developed. 
A major objective in implementation of such management systems is to ensure that safety, health and environmental 
issues are being addressed in organizations strategies [6]. In each of the systems mentioned only one aspect of the 
organization considered (e.g., quality in ISO 9000, protection of environment in ISO 14000 or safety and health in OHSAS 
18000) and developed policies and strategies are just for that aspect of organization. Although the role of such systems in 
 
 
* Corresponding author. Tel.: +98-811-8255963; fax: +98-811-8255301. 
E-mail address: mohammadfam@umsha.ac.ir 
* Corresponding author. Tel.: +98-021-23151780; fax: +98-021-23151789. 
E-mail address: mahmoudi@mapna.com 
Available online at www.sciencedirect.com
Open access under CC BY-NC-ND license.
Open access under CC BY-NC-ND license.
195 I. MOHAMMADFAM et al. /  Procedia Engineering  45 ( 2012 )  194 – 198 
administrating and directing activities toward organizations’ policies is undoubted, however, multiplicity of systems can 
cause complexity and confusion of organizations, resource waste, duplication, creating conflict between the policies and 
goals that are developed by each system, etc [7, 8]. Therefore integration of different systems in the organization is 
obligatory [9].  
Considering the trend of significant investments in establishing different types of integrated management systems like 
Health, Safety and Environment Management System (HSE-MS) in MAPNA Group, a key question is whether An efficient 
method based on the principles of modern management and Appropriate scientific methods and techniques in selecting the 
contractor is used? Another factor that makes these evaluations necessary is the limited financial resources of organizations. 
These will reduce the motivation of contractors to participate in tenders for construction projects although some of them are 
capable and competent. According to what was stated before identifying factors affecting contractor selection and use of 
scientific methods of decision making in choosing the best contractor, and ultimately preventing the waste of resources is 
particularly important. 
Considering the HSE perspective required to evaluate criteria such as number of accidents, severity of accidents, safety 
training, safety equipments, having the safety system, etc. considering these conditions, one of the most common methods 
of decision making is multi criteria decision making that can measure consistency in judgments of decision makers and 
because of presentation of the critical aspects of the problem in a hierarchical structure, makes the decision-making process 
easy for senior managers in organizations. 
Using a health, safety and environmental ranking tool in the selection of contractors will provide an acceptable level of 
safety for employers and increasing the capabilities of contractor safety management in projects will be achieved. 
Despite the attention given to HSE management systems, there has been relatively little attention given to the 
measurement of HSE-MS performance. Few previous studies have not specifically addressed the issue of contractors. Some 
of these studies are following: 
(1) Brooks & Ragan Over a 10-year period (1991-2001), through working in a chemical company with about 6,000 
employees and over 50 plants researched the use of employee surveys to measure safety and as a diagnostic tool for 
improvement efforts. The survey measures important components of the safety management system [10]. 
(2) In 1998, The Michigan Occupational Health and Safety Management System Assessment Instrument (MAI) renamed 
in 2002 to Universal Assessment Instrument (UAI) [11, 12], which measures the performance of OHS management 
systems and which was developed by researchers from the University of Michigan. The authors tell of numerous 
cases of large companies that have improved their OHS results through the use of this tool.  
(3) Cadieux et al., (2005) introduced an instrument designed to conduct an Occupational Health and Safety (OHS) self-
diagnosis. Their tool was a questionnaire using a 10 anchor-point. The study done in three companies [3]. 
This study presents a tool for performance measurement of health, safety and environmental management systems in 
contractors and it will identify and evaluate different models and HSE expert opinions. 
The present article describes an instrument designed to measure the HSE-MS performance OF contractors based on three 
input models. MCAT is developed in response to the need for applying scientific methods to evaluate the contractors' health, 
safety and environmental performance in MAPNA Group. In addition, MCAT have been developed to make the 
organizations aware from the position of their contractors in the field of HSE and also to create an opportunity for 
contractors learning from the best contractor in the area of health, safety and environment management. 
2.  Materials and methods 
2.1. Input model selection 
Nine performance measurement models like Universal Assessment Tool (UAI), Balanced Score Card (BSC), Data 
Envelopment Analysis (DEA), European Foundation for Quality Management (EFQM), International Project Management 
Award (IPMA), and The Safety Culture Improvement Matrix (SCIM) were reviewed in an effort to define the MAPNA 
Contractors Assessment Tool (MCAT).  
2.2. Input model partitioning 
Three of these models were selected as instrument input models. These models are 1) EFQM, 2) IPMA, and 3) SCIM 
because they are widely applicable, universally so, easy to administer and their outcome determinations is easy to interpret. 
The instrument was first included 11 parts.  
 
196   I. MOHAMMADFAM et al. /  Procedia Engineering  45 ( 2012 )  194 – 198 
2.3. Tool structure 
The initial model was based on expert panel consisting of nine main criteria. 9 criteria including leadership, policy and 
strategy, human resources, risk management, resources, planning and implementation, people results, customers’ and 
society results, contractors’ results and key performance results by 30 HSE experts and projects were discussed using Focus 
Grouping method. During Focus Grouping meetings the experts were asked to evaluate criteria and prioritize them. Also the 
experts were asked to score each criterion relative to others using the Analytic Hierarchy Process (AHP). By collecting the 
expert opinions and examined them the HSE management system performance evaluation model based on five criteria that 
includes prioritization was done by experts and with their scores were determined. 
Sub-parts and guidance points were determined by seek the expert’s opinions and using EFQM, UAI, and SCIM models.  
To determine how the instrument can be used to measure the implementation of the HSE-MS established in an 
organization, the opinions of the HSE experts were sought and the structures of three input models were reviewed.  
2.4. Finalize the tool 
The proposed tool was tested in three different contractors of MAPNA Group in Iran. The goals of the tests were 1) to 
evaluate the ability of the MCAT to measure HSE-MS performance at organizations with different types of business natures, 
2) to evaluate and further develop of the MCAT, and 3) to make modifications to the MCAT as necessary based on test 
findings.  
3. Results
The MCAT based on the principles of three input models (EFQM, UAI and SCIM) and expert opinions. The MCAT is 
founded on parts to evaluate the performance of contractors HSE management system. These criteria are the core of the tool 
and the basis for performance measurement of contractors. The MCAT is not just an audit tool but also be a managerial tool 
that includes a variety of techniques and improvement frameworks in proportion to the level of excellence in the contracting 
organization to identify system problems and help their management to provide corrective actions.  
 Main parts of the MCAT are shown in Figure 1. Each parts of the MCAT divided into a number of sub-parts and each 
sub-part divided into a number of guidance points. Addressing all guidance points is not required by MCAT and 
organizations should consider them to their specifications. 
Fig. 1. The MCAT scheme. 
Of course, for each sub-part in results some guidance points are defined. Table 1 list the number of sub-parts and 
guidance points is used in HSEEI.  
The HSE experts' opinions and the structures of three input models (EFQM, IPMA and SCIM) reviewed to determine 
how the MCAT measures the performance of the HSE management system established in a contracting organization. To 
each question based on the evaluator's judgment will be assigned points from 0 to 200. Judgment is based on the content 
criteria or sub-part has been evaluated or even evaluator can consider guidance points but it is not mandatory. In order to 
assess the performance of the HSE management system established in a contracting organization, the SCIM method is used. 
SCIM scoring methods based on the EFQM scoring method. But there are different points for each element, as well as its 
scoring formula has changed. Scoring method involves weighted questions that each element alone and the overall score 
(Max 1000) could be calculated. First a committee was formed to evaluate the performance of the HSE management system 
197 I. MOHAMMADFAM et al. /  Procedia Engineering  45 ( 2012 )  194 – 198 
of contractors; the experts who were appointed by the Committee had received the necessary training of the tool concepts. 
These experts evaluated the contractors using specific questionnaires. Table 2 outlines the results of the evaluation of three 
contracting organizations. 
                                          Table 1. Number of sub-parts and guidance points in MCAT 
part Number of sub-parts Number of guidance points 
Initiation (HSE Inputs) 5 46 
Formulation (HSE Process) 4 37 
Implementation/Operations (HSE Process) 7 120 
Evaluation (Feedback) 6 57 
Improvement/integration (System Elements) 3 42 
Total 25 302 
                                       Table 2. Results of the third party audit 
Maximum score in HSEEI Org. 3Org. 2Org. 1part 
200 
8.2 8.4 44 Initiation (HSE Inputs) 
10 12.7 50 Formulation (HSE Process) 
10.4 12.7 50.7 Implementation/Operations (HSE Process) 
10.3 14.8 46.3 Evaluation (Feedback) 
9.6 11 47.3 Improvement/integration (System Elements)
1000 48.5 59.6 238.3Total score 
4. Discussion 
Survey that has been done in three organizations with different types of business nature, revealed the most important 
strengths and weaknesses of the MCAT. Contrary to Brooks & Ragan (1991-2001) approach that can be used only in 
chemical industries [10]. The MCAT due to its structure that is independent of the business nature of organizations can 
measure the performance and efficiency of a wide range of HSE management systems. The MCAT is not just based on a 
questionnaire with fixed questions like Brooks & Ragan (2003) approach and Cadieux et al., (2005) study and it can be used 
to measure the performance and efficiency of health, safety and environment management systems not just safety like 
Brooks & Ragan (2003) approach or safety and health like UAI [11, 12], and Cadieux et al., (2005) study. As mentioned 
above, in order to create the MCAT to evaluate the HSE-MS performance in a contracting organization, concepts and 
structures of three input models were especially reviewed. Widespread application of these models, especially the EFQM 
that is also widely used as the national productivity award in Iran, can be called as strength in MCAT, because of extensive 
awareness that exists in organizations particularly about EFQM. This awareness results in shorter time lost by organizations 
to gain knowledge, skill, and understanding in the MCAT through training.  
The major advantages of MCAT for organizations are as following: 
(1) Possibility of quantitative analysis and determining the current level of organization 
(2) Organizations could identify their strengths and weaknesses and classify the improvement programs as well as 
administering and directing the trade and pace in the way to HSE excellence.  
(3) The MCAT helps organizations to compare their contractors in each MCAT parts and provide them an 
opportunity to specify their position. 
(4) The MCAT helps organizations to balance the expectations and requirements of stakeholders, i.e. customers, 
general public, employees, local authorities etc. 
(5) The MCAT can be implemented in a wide range of systems and organizations due to its process approach 
(6) In the long run, the MCAT provides a systematic viewpoint on specifications of an excellent contractiong 
organization in HSE management system. 
The following notes should be primarily considered when utilizing the MCAT to evaluate the performance of HSE 
management system in a contracting organization:  
198   I. MOHAMMADFAM et al. /  Procedia Engineering  45 ( 2012 )  194 – 198 
 Organization should provide effective orientation and training for all personnel involved in MCAT activities 
specially evaluators. 
 Organization should allocate appropriate funds and assign people, or other resources for utilizing the MCAT.  
 Lack of extensive practical experience on the application of the MCAT in organizations may cause it to proceed 
slowly.  
5. Conclusion 
Using the MCAT in three different organizational structures revealed that the method used was effective in collecting the 
necessary information to further understand the complex nature of management system measurement. The MCAT provides 
a valid framework for contracting organizations to follow in their development of a HSE-MS management system. 
Acknowledgements 
Authors are thankful to MAPNA Group authorities. 
References 
[1] Behm, M., (2005). Linking construction fatalities to the design for construction safety concept. Saf. Sci., 43: 589–611. 
[2] Fang, D.P., Xie, X.Y., Li, H., (2004). Factors analysis-based studies on construction workplace safety management in China. Int. J. Proj. Manag., 22: 
43–49. 
[3] Cadieux, J., Roy, M., Desmarais, L., (2006). A preliminary validation of a new measure of occupational health and safety. J. Saf. Res., 37: 413–419 
[4] Azadeh, M.A., (2000). Creating Highly Reliable Manufacturing Systems: An Integrated Approach: Reliability, Quality and Safety. Int. J. Reliab. Qual. 
Saf. Eng., 7: 205-222.  
[5] Davis, M. L. and Cornwell, D. A., (1998). Introduction to Environmental Engineering. 3rd Ed. McGraw- Hill Inc, 63-78. 
[6] Blair, E.H., (1996).  Achieving a total safety paradigm through authentic caring and quality. Prof. Saf., 41: 24– 27.  
[7] Jackson, S.L., (1997). The ISO 14001 Implementation Guide: Creating an Integrated Management System. John Wiley and Sons Inc, 121-125.  
[8] Pollock, R.A., (1995). Making safety matter. Occup. Hazards., 57: 193–198.  
[9] Roberge, C.L., (1999). It’s all about attitude. Ind. Distrib., 88: 122.  
[10] Carder, B., Ragan, P., (2003). A survey-based system for safety measurement and improvement. J. Saf. Res., 34: 157–165. 
[11] Redinger, C., Levine, S., (1998). Development and evaluation of the Michigan Occupational Health and Safety Management System Assessment 
Instrument: A Universal OHSMS Performance Measurement Tool. Am. Ind. Hyg. Assoc. J., 59: 572-581. 
[12] Redinger, C., Levine, S., (2002). Evaluation of an occupational health and safety management system performance Measurement Tool-II: Scoring 
Methods and Field Study Sites. A m. Ind. Hyg. Assoc. J., 63: 34-40. 


Development of the Health, Safety and Environment Excellence Instrument: a HSE-MS Performance Measurement Tool 

file:///data/remote/core/dit/data/elsevier/pdf/2bc/aHR0cDovL2FwaS5lbHNldmllci5jb20vY29udGVudC9hcnRpY2xlL3BpaS9zMTg3NzcwNTgxMjAzMTU0Mg==.pdf

Development of the Health, Safety and Environment Excellence Instrument: a HSE-MS Performance Measurement Tool

Abstract

Similar works

Full text

Available Versions

Elsevier - Publisher Connector

Elsevier - Publisher Connector