The OIRA draft Risk Assessment Bulletin has worthy intentions and has stimulated useful review and discussion.; Most previous major documents in the development of the risk assessment field have been cited and used appropriately. In general, the formulation is too broad. The Revision should clarify the place of risk assessment as distinguished from hazard identification and from risk management.The category of "influential risk assessment" is unnecessary and confusing, and should be deleted. A single set of six standards would suffice, without the additional nine special standards for "influential risk assessments". Greater transparency within the EOP is desirable to give this process credibility and meet one of the explicit aims of the Bulletin. Finally, several omissions should be addressed: proactive engagement of stakeholders, public health context, deceptive use of quantitation, exclusion for research agencies, interagency steering committee and symmetry of risk assessment guidance for manufacturers as well as regulatory agencies.

Omenn, Gilbert S.

English

Research Papers in Economics

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
The attached material is posted on regulation2point0.org with permission.    
 
JOINT CENTER
AEI-BROOKINGS JOINT CENTER FOR REGULATORY STUDIES 
 
 
Public Comment on OMB Draft Risk 
Assessment Bulletin 
 
 
Gilbert S. Omenn, M.D., Ph.D.
 *
 
Regulatory Analysis 06-04 
June 2006 
                                                 
* Dr. Omenn is a professor of internal medicine, human genetics, computational biology, and public 
health at the University of Michigan.  
     
J   O   I   N   T     C   E   N   T   E   R 
AEI-BROOKINGS JOINT CENTER FOR REGULATORY STUDIES 
 
 
In order to promote public understanding of the impact of regulations on 
consumers, business, and government, the American Enterprise Institute and the 
Brookings Institution established the AEI-Brookings Joint Center for Regulatory 
Studies.  The Joint Center’s primary purpose is to hold lawmakers and regulators 
more accountable by providing thoughtful, objective analysis of relevant laws and 
regulations. Over the past three decades, AEI and Brookings have generated an 
impressive body of research on regulation. The Joint Center builds on this solid 
foundation, evaluating the economic impact of laws and regulations and offering 
constructive suggestions for reforms to enhance productivity and welfare. The 
views expressed in Joint Center publications are those of the authors and do not 
necessarily reflect the views of the Joint Center. 
 
 
 ROBERT W. HAHN 
Executive Director 
  ROBERT E. LITAN 
Director 
 
 
COUNCIL OF ACADEMIC ADVISERS 
KENNETH J. ARROW 
Stanford University 
  MAUREEN L. CROPPER 
University of Maryland  
 JOHN  D.  GRAHAM 
Pardee RAND Graduate 
School 
        
PHILIP K. HOWARD 
Common Good 
 
  PAUL L. JOSKOW 
Massachusetts Institute 
of Technology  
 DONALD  KENNEDY 
Stanford University  
 
        
ROGER G. NOLL 
Stanford University  
 PETER  PASSELL 
Milken Institute 
 RICHARD  SCHMALENSEE 
Massachusetts Institute 
of Technology 
        
ROBERT N. STAVINS 
Harvard University 
 CASS  R.  SUNSTEIN 
University of Chicago 
  W. KIP VISCUSI 
Harvard University 
 
All AEI-Brookings Joint Center publications can be found at www.aei-brookings.org 
 
© 2006 by the authors.  All rights reserved. 
 
    
Executive Summary 
  
The OIRA draft Risk Assessment Bulletin has worthy intentions and has stimulated 
useful review and discussion.  Most previous major documents in the development of the risk 
assessment field have been cited and used appropriately.  In general, the formulation is too 
broad.  The Revision should clarify the place of risk assessment as distinguished from hazard 
identification and from risk management.  The category of “influential risk assessment” is 
unnecessary and confusing, and should be deleted.  A single set of six standards would suffice, 
without the additional nine special standards for “influential risk assessments”.  Greater 
transparency within the EOP is desirable to give this process credibility and meet one of the 
explicit aims of the Bulletin. Finally, several omissions should be addressed: proactive 
engagement of stakeholders, public health context, deceptive use of quantitation, exclusion for 
research agencies, interagency steering committee, and symmetry of risk assessment guidance 
for manufacturers as well as regulatory agencies. 
 
 
 
       1
Public Comment on OMB Draft Risk Assessment Bulletin 
 
Gilbert S. Omenn, M.D., Ph.D. 
 
1. Introduction
 
This Bulletin has a worthy intent:  to enhance the named attributes of quality--objectivity, 
utility, and transparency--of risk assessments prepared by or for the federal agencies. 
There is a supportive objective of improving quantitative components of risk 
assessments, including methods of characterizing and estimating what the National Research 
Council report, “Science and Judgment in Risk Assessment” (1994), differentiated as variability 
and uncertainty.  Nevertheless, the Bulletin makes clear the primary importance of the qualitative 
or narrative characterization of risk, starting with the nature, significance, and reversibility of the 
potential adverse effects from exposure scenarios, as recommended by the OSTP (Calkins et al, 
1980) (Figure 1) and the National Research Council (NRC, 1983, “The Red Book”).   
The Bulletin recognizes and emphasizes the fact that many agencies have responsibilities 
for a variety of particular risk scenarios, clearly justifying the interest of OIRA/OMB and of 
OSTP in providing guidance and periodic oversight, though perhaps not in case-by-case 
management.  
Examples are described.  However, consistency in what is presented and meaningful 
evaluation of lessons learned from these case studies and value-added from the risk assessments 
would enhance this important element of the Bulletin.  Moreover, major examples—like the air-
only analysis of the addition of MTBE to gasoline and the protracted Corps of Engineers 
analyses of risks and risk management for levees—would be informative. 
The Bulletin appropriately cites many major reports from the development of risk 
assessment concepts and methods, especially a series of National Research Council reports, most 
from the Board on Environmental Studies and Toxicology (including years 1988-1991 when I 
was chair).  The Bulletin also draws extensively upon the reports of the 
Presidential/Congressional Commission on Risk Assessment and Risk Management (1994-1997, 
see www.riskworld.com/reports/1997; hereinafter, “The Risk Commission”).  The Bulletin 
should recognize early work from EPA and FDA  (see Alberts, 1994; Anderson, 1983).  
      2
Finally, OIRA has shown respect for the Public Comment process and enhanced its depth 
through cooperation with the National Academy of Sciences, among others, to hold well-planned 
workshops, as in this case and in the preceding case of the Peer Review Guidelines. 
 
2. Key Concepts from the Risk Commission
 
  The Framework for Risk Management presented by the Risk Commission and utilized by 
multiple agencies in the United States and around the world is shown in Figure 2. 
 
The special features of this framework, as shown in the hexagon of six steps, are two: 
 
a)  The requirement to put a “new problem” into public health (or ecological) context before 
launching into technical aspects of risk assessment. 
   “Context” at the more technical level means moving beyond analysis and regulation of 
one chemical, one environmental medium (air, water, soil, food), one health (cancer, birth 
defects, …) at a time to a comprehensive analysis. For each agent (a chemical, for example), it is 
necessary to describe and analyze the sources and pathways of exposure, the often multiple 
health effects, and the variation in susceptibility within exposed populations; then it is necessary 
to describe and analyze other causes of the same adverse effects; and finally to address aspects of 
environmental justice, different perceptions of risks, cultural practices influencing exposures, 
and statutory requirements. 
b) The central feature of proactively and broadly engaging stakeholders, so that their questions, 
their perceptions, their sometimes substantial information, and their often practical risk 
management/risk reduction suggestions can inform the whole risk assessment/risk 
management/risk communication process. 
The Commission addressed many cross-agency issues, as directed in its mandate in the 
1990 Clean Air Act Amendments, and then addressed various agency-specific and program-
specific issues.  The cross-agency topics were:  risk assessment science and models; risk-
management framework; communicating uncertainty; peer review; inter-and intra-agency 
consistency; “bright lines”; sensitive subpopulations; ecological risk assessment; comparative 
risk assessment; economic analysis; and judicial review.  The work was all presented openly, 
      3
with bimonthly public hearings during the findings phase and with a set of hearings after the 
main report was issued.  We prepared both a detailed scientific and policy report and a report for 
the general public, with important examples drawn from our public hearings [see 
www.riskworld.com/reports/1997]. 
 
3. Brief Comments on OMB/OIRA Criteria for Risk Assessments
  
Problem formulation/purpose of the risk assessment (RA): 
Yes, of course, the purpose should be stated clearly and should be put in public health, 
ecological, or other appropriate context, as outlined above (Risk Commission).  The RA is not 
just for the “agency decision-maker”, but must be useful for many other stakeholders. 
 
Completeness: 
This criterion is obscure in scope and inadequate in terms of resources required and 
opportunity cost. 
 
Effort and resources required:   
These points are too soft and likely to lead to misunderstandings and disputes. 
 
Peer review: 
OIRA admirably revised the draft Peer Review Guidelines after inputs from the National 
Academies Workshop and the public; it would be appropriate here to summarize and cite 
assessments of the experience to date. 
 
Missing criteria:  Earlier Inter-Agency Cooperation 
Improved risk communication with publics/stakeholders 
 
4. Major Issues to be Addressed in the Revision of this Draft Bulletin 
 
There is mission-creep in demanding, it seems, of hazard assessment/hazard 
identification (see Figure 1, above, from OSTP, 1980) the extensive analysis required for risk 
      4
assessment/risk characterization.   These two steps should not be merged.  The critical aspect of 
exposure assessment for risk assessment should be highlighted 
(Lioy et al, 2005; Weis et al 2005). 
The elaborate scheme of “risk assessments” and “influential risk assessments” is 
unhelpful, confusing, and likely to generate disputes about the classification in particular cases.  
In no way are these guidelines “minimal”! 
The Bulletin should hew to the Red Book and many agency positions that risk 
assessment, while informing risk management, should be kept separate from risk management. 
The Bulletin needs to reinforce the requirements to respect legal and budgetary context.  
A “value-of-information” concept would be helpful, as applied to carcinogenicity testing by 
Lave & Omenn (1986) and Lave et al (1988). 
The Bulletin should clarify what are “central estimates” or “expected risks”. 
The Bulletin should enhance transparency—one of the primary attributes of quality 
espoused by OIRA—of intra-governmental processes, starting from agency discretion right 
through the roles of OIRA and OSTP.   
 
5. Responses to Specific Questions: Recommendations 
 
Breadth 
  The formulation is much too broad.  On p.1, the Bulletin states that “Risk assessment is a 
useful tool for estimating the likelihood and severity…”  However, it then states that “Risk 
Assessment refers to a document that assembles and synthesizes scientific 
information….whether a potential hazard exists”.  “Potential hazard” is NOT “risk” (see Major 
Issues, above).  On p.8, there is mention of “an exposure or hazard assessment”; again, the term 
hazard is used loosely and with confusion.   
  Even if a more inclusive scheme is desired long-term, it would be wise to start with the 
most important and most relevant cases and gain experience.   
  We should remember and apply the wise counsel of U.S. First Circuit Chief Judge David 
Bazelon in his 1979 address at the University of Southern California entitled “The Perils of 
Wizardry”.  Starting from the premise that most regulatory issues tend to recur, Bazelon stated 
      5
that it is important to build a useful record.  Technical experts (including risk assessors), both 
inside and outside the agency, should: 
•  Stay away from the ultimate policy decision (risk management), which it not their charge 
or their special expertise 
•  Delineate the specific elements of the risk characterization 
•  Focus on those specific elements—including those known, those feasible to know with 
present methods, and those beyond our present methods—to build the record; and  
•  Expect to be asked again!   
And, he urged that policy makers respect the different roles of risk assessors and risk managers.   
 
Different standards for different classes of risk assessment 
  I would prefer a single set of guidance, with narrower application and with the proposed 
flexibility.  The nine special standards only embellish the basic set of six standards. 
  The classification introduces a fuzzy line that could lead to lots of unproductive disputes; 
for example, p.9 includes “hazard determinations” as “influential”. 
  “Regulatory analysis” primarily introduces risk management, which should not be 
lumped together with risk assessment. 
 
Omissions 
There are several significant matters which should be added: 
a)  Proactive engagement of stakeholders:  the text is too much directed at serving “agency 
decision-makers”.  See Risk Commission, above. 
b)  Public health context:  “comparative risk” guidance is too vague, and generally 
boomerangs when interpreted to include meteorite crashes or even cigarette smoking 
c)  Foolish or deceptive use of quantitation—one of the most frequent and most egregious 
misuses of numbers is the presentation of excessive significant figures, for example, 
using measurements or, worse, estimates in integers and then generating calculated 
estimates of risk to 4 or 6 decimal places!  It is offensive to brush off this practice as just 
computers run amok; such precision is deceptive and distracting.  Journals are just as 
guilty as agencies.  Scientists who claim they are simply copying the original numbers 
      6
should be admonished to round off unjustifiably precise numbers.   The rule is that the 
final figure cannot be more precise than the least precise number used in the calculation. 
d)  “Risk-related research”:  The Bulletin should provide an unambiguous exclusion for 
research agencies performing or funding what regulatory agencies may later, or 
concurrently, consider to be risk-related research.  The present scheme is much too far-
reaching. 
e)  Inter-agency and EOP review:  The Bulletin should identify or propose an inter-agency 
mechanism, like a National Science & Technology Council-appointed steering 
committee, to work with OIRA and OSTP and the agencies.  In the late 1970s there was a 
Regulatory Analysis Review Group, chaired by a member of the Council of Economic 
Advisers, which played a useful role.  The EOP role, especially in a time of expanded 
Executive Office prerogatives, should not be a “black hole”, especially in a Bulletin 
espousing transparency. 
f)  Symmetry:  The Bulletin places considerable burdens on the agencies to justify their 
conclusions, while a symmetrical set of requirements of transparency, objectivity, and 
utility is explicitly excluded for manufacturers seeking licenses, approvals, and 
registration of agents subject to regulation.  That is unwise and should be modified. 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
      7
References 
 
Albert, R.E. (1994) Carcinogen risk assessment in the U.S. Environmental Protection Agency.  
Critical Rev Toxicol 24:75-85. 
 
Anderson, E.L. (1983) Quantitative approaches in use to assess cancer risk.  Risk Analysis 
3:277-295. 
 
Calkins, D.R., Dixon, R.L., Gerber, C.R., Zarin, D. and Omenn, G.S. (1980). Identification, 
characterization, and control of potential human carcinogens:  a framework for federal decision-
making. J Nat Cancer Institute 61: 169-175. 
 
Charnley, G. and Omenn, G.S. (1997). A summary of the findings and recommendations of the 
Commission of Risk Assessment and Risk Management (and accompanying papers prepared for 
the Commission). Hum Ecol Risk Assess 3: 701-711. 
 
Faustman, E.M., Omenn, G.S. Risk assessment (2001).  In: Klaassen CD (ed.), Casarett    
and Doull’s Toxicology:  The Basic Science of Poisons, 6th ed.  New York: McGraw-Hill, 
pp.83-111. 
 
Lave, L.B., and Omenn, G.S. (1986).  Cost-effectiveness of short-term tests for carcinogenicity.  
Nature 324:29-34. 
 
Lave, L.B., Ennever, F., Rosenkranz, H.S., Omenn, G.S. (1988). Information value of the rodent 
bioassay.  Nature 336:631-633. 
 
Lioy P, Leaderer B, Graham J, Lebret E, Sheldon L, Needham L, Pellizzari E, Lebowitz M.  
(2005). International Society of Exposure Analysis 2004: The application of exposure 
assessment to environmental health science and public policy--what has been accomplished and 
what needs to happen before our 25th anniversary in 2014.  J Expo Anal Environ Epidemiol  
15(2):121-122. 
 
NRC (1983). Risk assessment in the Federal Government:  Managing the process. Washington 
DC, National Academy Press. 
 
NRC (1987). Committee of Biological Markers. Washington DC, National Academy Press. 
 
NRC (1994). Science and judgment in risk assessment. Washington, DC, National Academy 
Press.  
 
Omenn, G.S. (2003).  On the significance of “The Red Book” in the evolution of risk  
assessment and risk management. Human and Ecological Risk Assessment  
9:1155-67. 
 
      8
Omenn, G.S. and Faustman, E.M. (2002). Risk assessment and risk management. Oxford 
Textbook of Public Health, 4th ed.  R. Detels, J. McEwen, R. Beaglehole, and H. Tanaka (eds).  
New York, Oxford University Press: 1083-1103. 
 
Risk Commission, Presidential/Congressional Commission on Risk Assessment and Risk 
Management.  A Framework for Environmental Health Risk Management, vol. 1 (1997); Risk 
Assessment and Risk Management in Regulatory Decision-Making, vol. 2 (1997).  U.S. 
Government Printing Office, Washington, DC; http://www.riskworld.com/reports/1997. 
 
Weis, B.K., Balshaw, D., Barr, J., Brown, D., Ellisman, M., Lioy, P., Omenn, G.S., Potter, J., 
Smith, M., Sohn, L., Suk, W.A., Sumner, S., Swenberg, J., Walt, D., Watkins, S., Thompson, C. 
and Wilson, S. (2005).  Personalized exposure assessment: Promising approaches for human 
environmental health research. Environ Health Perspect. 113: 1-10.   
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
      9
Figure 1 
 
Framework for Regulatory 
Decision-Making (OSTP, 1980)
Epidemiology
Hazard Identification Lifetime rodent bioassays
Short-term, in vitro/in vivo tests
Structure / activity
Potency (dose/response)
Risk Characterization Exposure analysis
Variation in susceptibility
Information
Risk Reduction Substitution
Regulation / Prohibition
 
 
 
 
Figure 2 
 
      10
Figure 3 
 
MAJOR ISSUES
• Risk Assessment (RA) vs Hazard Assessment
• RA vs “Influential Risk Assessment”
• RA vs Risk Management (regulatory analysis)
• Need to respect legal and budgetary contexts
• Need to clarify what are “central estimates” or 
“expected risks”?  
• The guidance is not “minimal”!
• Need transparency to intra-governmental 
process, given agency flexibility/discretion
 
   

A summary of the findings and recommendations of the Commission of Risk Assessment and Risk Management (and accompanying papers prepared for the Commission). Hum Ecol Risk Assess

Carcinogen risk assessment in the U.S. Environmental Protection Agency. Critical Rev Toxicol

Cost-effectiveness of short-term tests for carcinogenicity.

Identification, characterization, and control of potential human carcinogens: a framework for federal decisionmaking.

Information value of the rodent bioassay.

International Society of Exposure Analysis 2004: The application of exposure assessment to environmental health science and public policy--what has been accomplished and what needs to happen before our 25th anniversary in

On the significance of “The Red Book” in the evolution of risk assessment and risk management. Human and Ecological Risk Assessment 9:1155-67.

Personalized exposure assessment: Promising approaches for human environmental health research. Environ Health Perspect.

Presidential/Congressional Commission on Risk Assessment and Risk Management. A Framework for Environmental Health Risk Management,

Quantitative approaches in use to assess cancer risk. Risk Analysis 3:277-295.

Risk assessment and risk management.

Risk assessment in the Federal Government: Managing the process. Washington DC,

Science and judgment in risk assessment.

Public Comment on OMB Draft Risk Assessment Bulletin

http://regulation2point0.org/wp-content/plugins/download-monitor/download.php?id=76

Public Comment on OMB Draft Risk Assessment Bulletin

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics