HRA is a method used to describe, qualitatively and quantitatively, the occurrence of human failures in the operation of complex systems that affect availability and reliability. Modeling human actions with their corresponding failure in a PRA (Probabilistic Risk Assessment) provides a more complete picture of the risk and risk contributions. A high quality HRA can provide valuable information on potential areas for improvement, including training, procedural, equipment design and need for automation

Hamlin, Teri L.

NASA Technical Reports Server

SPACE SHUTTLE PROGRAM
Space Shuttle Safety and Mission Assurance Office
NASA Johnson Space Center, Houston, Texas
THE IMPORTANCE OF HUMAN RELIABILITY 
ANALYSIS IN HUMAN SPACE FLIGHT: 
UNDERSTANDING THE RISKS
Teri L. Hamlin
NASA Johnson Space Center
Safety & Mission Assurance 
Space Shuttle Probabilistic Risk Assessment Lead
Presented at 
Department of Defense Human Factors Engineering Technical Advisory Group 
October 26 - 28, 2010
San Jose, CA
1
https://ntrs.nasa.gov/search.jsp?R=20100036673 2019-08-30T12:38:31+00:00Z
SPACE SHUTTLE PROGRAM
Space Shuttle Safety and Mission Assurance Office
NASA Johnson Space Center, Houston, Texas
HUMAN RELIABILITY ANALYSIS (HRA) OVERVIEW
• HRA is a method used to describe, qualitatively and quantitatively, the 
occurrence of human failures in the operation of complex systems that 
affect availability and reliability. 
• Modeling human actions with their corresponding failure in a PRA 
(Probabilistic Risk Assessment) provides a more complete picture of the 
risk and risk contributions.  
• A high quality HRA can provide valuable information on potential areas for 
improvement, including training, procedural, equipment design and need 
for automation.
– For Shuttle, the HRA was useful to show the importance of maintaining crew training at 
it’s current level in a time when budget reductions were threatening training levels.
– For Shuttle, the HRA showed areas where automation would be beneficial but 
considering the Shuttle program retirement, it was not worth pursuing  (e.g. H20 loop 
freeze protection given overcool event).
2
SPACE SHUTTLE PROGRAM
Space Shuttle Safety and Mission Assurance Office
NASA Johnson Space Center, Houston, Texas
• Modeling human error has always been a challenge
– Performance data is not always readily available
– When data is available , it is sensitive and must be handled with care
• For spaceflight, the challenge is amplified
– small number of participants
– limited amount of performance data available
– lack of definition of the unique factors influencing human performance in 
space
• Performance Shaping Factors (PSF), in HRA terminology, are used in HRA techniques 
to modify basic human error probabilities in order to capture the context of an 
analyzed task.  
• Many of the human error modeling techniques were developed within the context 
of nuclear power plants and therefore the methodologies do not address 
spaceflight factors, such as the effects of microgravity and long duration missions.
3
HUMAN RELIABILITY ANALYSIS (HRA) OVERVIEW (2)
SPACE SHUTTLE PROGRAM
Space Shuttle Safety and Mission Assurance Office
NASA Johnson Space Center, Houston, Texas
PHILOSOPHY FOR MODELING HUMAN ACTIONS
• Model human actions that are required for normal operation of a system
• In general, recoveries and work-a-rounds are only modeled if the scenario 
becomes dominate in the risk profile
• Use screening Human Error Probabilities (HEPs) for recoveries and work-a-
rounds in order to reduce the number events required for detailed 
analysis down to only those that are significant
- Screening analysis should be quick and inherently conservative so that lack of 
detailed modeling does not lead to underestimation of the risk
• There are many methodologies for performing HRA, NASA JSC has chosen 
to use an internally developed screening method and the Cognitive 
Reliability Error Analysis Model (CREAM)
- Key Factors for Selecting an HRA Approach
• Ability to model errors of commission and continuous feedback events as well as 
errors of omission
• Reproducibility of results
• Reasonable results
• Ability to perform the analysis in-house in a reasonable timeframe
4
SPACE SHUTTLE PROGRAM
Space Shuttle Safety and Mission Assurance Office
NASA Johnson Space Center, Houston, Texas
• Screening analysis should take < 1 hour per event
– Can be performed by either HRA lead or individual analyst.  If 
performed by analyst, it should be reviewed by HRA lead to ensure 
consistency
• Detailed analysis ~3 to 5 days per event
– Performed by a single HRA lead
– Requires research
– Interviews with Astronauts, Mission Operations, Trainers, interaction 
with the PRA system’s analyst
5
EXPECTED EFFORT FOR PERFORMANCE OF HRA
For Shuttle PRA, ~70% of HRA events are modeled with Screening Analysis
SPACE SHUTTLE PROGRAM
Space Shuttle Safety and Mission Assurance Office
NASA Johnson Space Center, Houston, Texas
EXAMPLE HRA SCREENING TABLE
(Means and Lognormal Distributions)
10.48EF 5
0.16
EF 5
0.048
EF 5
0.016
EF 5
0.0048
EF 5T > 30 minutes
110.48EF 5
0.16
EF 5
0.048
EF 5
0.016
EF 5
T > 10 minutes
T < 30 minutes
1110.48EF 5
0.16
EF 5
0.048
EF 5
T > 1 minute
T < 10 minutes
11110.48EF 5
0.16
EF 5T < 1 minute
4   Adverse 
Conditions
3   Adverse 
Conditions
2   Adverse 
Conditions
1 Adverse 
Condition
Nominal HEPIdeal HEPAvailable Time
Adverse conditions are degraded or extreme environments, unfamiliar tasks, 
high stress scenarios, and complex tasks
Screening values are used to determine whether a detailed calculation is worth 
pursuing 6
SPACE SHUTTLE PROGRAM
Space Shuttle Safety and Mission Assurance Office
NASA Johnson Space Center, Houston, Texas
7
1.E-05 1.E-04 1.E-03 1.E-02 1.E-01
MMOD
Orbiter Hardware / 
Software
SSME
Human Error
Ascent Debris
SRB
RSRM
External Tank
1:300
1:350
1:610
1:770
1:930
1:1500
1:1500
1:5000
SHUTTLE PRA ITERATION 3.2 CONTRIBUTIONS BY 
ELEMENT OR MAJOR AREA*
* Some overlap in risk exists.  For example, a cut set containing both a mechanical failure and a human error that 
result in failure to lower the landing gear is counted in both the Orbiter hardware contributor and the human error 
contributor.
Scenarios 
involving 
human error 
provide a 
significant risk 
contribution
SPACE SHUTTLE PROGRAM
Space Shuttle Safety and Mission Assurance Office
NASA Johnson Space Center, Houston, Texas
SHUTTLE PRA TOP CONTRIBUTORS INVOLVING HUMAN ERROR
Rank %age of Total
Cumulativ
e Total % Probability Description
1 7.3 7.3 8.2E-04(1 in 1200) Crew error during entry
2 1.2 8.5 1.3E-04(1 in 7700)
Collision of the Orbiter with the International Space Station (ISS) during 
rendezvous and docking
3 0.9 9.4 1.0E-04(1 in 9500)
Orbiter inspections (Flight Day 2 and late) produce false positive indications of 
damage, and failure of crew rescue
4 0.7 10.1 7.5E-05(1 in 13,000)
MPS component failures cause a catastrophic overpressure condition in the aft 
compartment during entry
5 0.3 10.4 3.3E-05(1 in 30,000)
Fuel cell leak and a subsequent failure of the crew to respond appropriately 
causes a catastrophic failure
6 0.3 10.7 3.1E-05(1 in 32,000) Orbit inspections (Flight Day 2 and late) result in damage to the TPS
7 0.2 10.9 1.9E-05(1 in 51,000)
Cabin depressurization due to leaks beyond the make-up capability of the 
Pressure Control System (e.g., penetration leaks) or pressure control system fails
8 0.1 11.1 1.2E-05(1 in 81,000) APU heater fails on and human error failure results in catastrophic failure on orbit
9 0.1 11.2 1.2E-05(1 in 83,000) Failure of Deorbit burn due to improper targeting of OMS burn (human error)
10 0.1 11.3 9.9E-06(1 in 100,000)
Cabin Fan System failure combined with a human error during landing brought 
about by high heat or humidity
11 0.1 11.4 6.6E-06(1 in 150,000) Landing Deceleration System (LDS) tire ruptures
12 0.1 11.5 6.6E-06(1 in 150,000)
Flash Evaporator System freeze up and failure to recover leads to LOCV during 
entry
Scenarios with some applicability to commercial crew.  The particular scenario may not be 100% 
applicable but similar scenarios would exist (for example may not have an APU heater but may 
have a heater scenario that could lead to LOC if not mitigated by crew) 8
SPACE SHUTTLE PROGRAM
Space Shuttle Safety and Mission Assurance Office
NASA Johnson Space Center, Houston, Texas
INSIGHTS FROM SHUTTLE PRA APPLICABLE TO 
COMMERCIAL SPACE FLIGHT
• Helpful to have a single HRA lead which models Human error events to 
maintain consistency
– Less important on screening events but necessary for events modeled in detail
• Special attention should be paid to critical crew/human actions required for 
a nominal flight 
– Contributed to ~10% of the overall Shuttle risk with the other 2% human error 
contribution relating to responses to failures
• If it is unknown whether or not an action is to be automated, evaluate it as 
manual
– Difficult to identify later on, and will lead to underestimates of the risk if action 
is actually manual
• Use of CREAM methodology only slightly modified to capture dependencies 
and uncertainty may be reasonable for short duration missions such Shuttle 
mission of 11-16 days.  However, the space environment may be a more 
important factor for longer duration missions of 6 months or more
9
SPACE SHUTTLE PROGRAM
Space Shuttle Safety and Mission Assurance Office
NASA Johnson Space Center, Houston, Texas
PLANS FOR EXPANDING CURRENT HRA METHODOLOGY 
• First step is working on identifying and quantifying a new set of 
Performance Shaping Factors which are relevant to spaceflight
– Recommendation from NASA 2006 HRA TIM was to address Performance 
Shaping Factors (PSFs) specific to spaceflight
– Graduate Student from University of Colorado at Boulder primarily funded by 
NASA Graduate Student Researchers Program (GSRP) is work ing with JSC
• Selected PSFs quantified by the end of 2011, with expected follow on work to be 
completed by NASA
• Second step is working on identifying and quantifying a new set of basic 
human error probabilities with a potential for a new cognitive model
– Work with University of Maryland to address limitations of current HRA model 
with respect to basic human error probabilities and cognitive model which are 
more focused on ground based activities
– Less effort has been spent on this activity to date
10
SPACE SHUTTLE PROGRAM
Space Shuttle Safety and Mission Assurance Office
NASA Johnson Space Center, Houston, Texas
Reference: Hollnagel, E. Cognitive Reliability and Error Analysis Method (CREAM).  Elsevier Science. 1998. 
EXAMPLE OF PSFS AND WEIGHT FACTORS
(COGNITIVE RELIABILITY AND ERROR ANALYSIS METHOD - CREAM)
11
First Step:  Expand and Modify These PSFs from CREAM
PSFs that  
may need to 
be modified 
for space 
applications
SPACE SHUTTLE PROGRAM
Space Shuttle Safety and Mission Assurance Office
NASA Johnson Space Center, Houston, Texas
CREAM QUANTITATIVE PERFORMANCE PREDICTION
• Is task mainly observation, interpretation, planning, or execution?
• Identify the likely cognitive function failure
12
Reference: Hollnagel, E. Cognitive Reliability and Error Analysis Method (CREAM).  Elsevier Science. 1998  
• Determine the 
failure probability by 
using nominal failure 
value given for each 
function failure.
• Adjust values based 
on performance 
shaping factors 
(PSFs)
Second Step:   Identify and Quantify New basic human error 
probabilities with the potential for adopting new 
cognitive model   
SPACE SHUTTLE PROGRAM
Space Shuttle Safety and Mission Assurance Office
NASA Johnson Space Center, Houston, Texas
BACKUP
13
SPACE SHUTTLE PROGRAM
Space Shuttle Safety and Mission Assurance Office
NASA Johnson Space Center, Houston, Texas
from: http://www.nsbri.org/HumanPhysSpace/
SPACE SHUTTLE PROGRAM
Space Shuttle Safety and Mission Assurance Office
NASA Johnson Space Center, Houston, Texas
COMPARISON OF CREAM TO SIMULATOR DATA
“Land Too Hard”
“Failure to Lower
Landing Gear”
“Brake at Wrong
Time”
For available data, CREAM compared well with Shuttle simulator 
data
– Added to credibility of the analysis
SPACE SHUTTLE PROGRAM
Space Shuttle Safety and Mission Assurance Office
NASA Johnson Space Center, Houston, Texas
CREAM QUANTITATIVE PERFORMANCE PREDICTION
• Is task mainly observation, interpretation, planning, or execution?
• Identify the likely cognitive function failure.
• Determine the failure probability by using 
nominal failure value
given for each 
function failure.
• Adjust values based on performance shaping factors (PSFs)
16Reference: Hollnagel, E. Cognitive Reliability and Error Analysis Method (CREAM).  Elsevier Science. 1998. 
SPACE SHUTTLE PROGRAM
Space Shuttle Safety and Mission Assurance Office
NASA Johnson Space Center, Houston, Texas
CREAM QUANTITATIVE EXAMPLE
• Basic Event: Collision of Orbiter with International Space Station (ISS) during 
rendezvous and docking
• Five cognitive activities were identified for this action  
– EXECUTE – The crew executes rendezvous and docking actions.
– OBSERVE – The crew observes an erroneous action has been taken.
– OBSERVE – The MCC observes an erroneous action has been taken.
– EXECUTE – The MCC fails to warn crew of the need to recover.
– EXECUTE – The crew recovers from erroneous action
• Each cognitive activity is matched to its dominant cognitive function and likely 
failure.
17Reference:  Hamlin, T.  Space Shuttle Program Human Reliability Analysis (HRA) Data Report.  Nov.  2008.   
Cognitive Activity Cognitive Function Predominant Failure Nominal Failure 
Probability (Median)
Execute Execution Action of a Wrong Type (Shuttle crew) 3.0E-03
Observe Observation Observation not made (Shuttle crew) 3.0E-03
Observe Observation Observation not made (MCC crew) 3.0E-03
Execute Execution Missed Action (MCC crew) 3.0E-02
Execute Execution Action at the Wrong Time (Shuttle crew) 3.0E-03
SPACE SHUTTLE PROGRAM
Space Shuttle Safety and Mission Assurance Office
NASA Johnson Space Center, Houston, Texas
• Once each predominant cognitive failure is identified, the Performance Shaping 
Factors (PSFs) are evaluated.  
• Evaluated for the Shuttle crew and MCC crew separately.
• The weighting factors corresponding to the PSF evaluations are found: 
18
PSF Name Level Exec 
Wrong 
Type
Obs Not 
Made
Obs Not 
Made 
(MCC)
Exec 
Missed  
(MCC)
Exec 
Wrong 
Time
A Adequacy of Organization Very Efficient 0.8 1 1 0.8 0.8
B Working Conditions Compatible/Advantageous 1 1 0.8 0.8 1
C Adequacy of MMI Supportive/Adequate 0.5 1 0.5 0.5 1
D Procedures/Plans Appropriate 0.8 0.8 0.8 0.8 0.8
E Number of Goals Fewer Than Capacity 1 1 1 1 1
F Available Time Adequate/
Continuously Inadequate
0.5 5 5 5 5
G Time of Day Daytime 1 1 1 1 1
H Training & Preparation Adequate, High Experience/Adequate, 
Low Experience
0.8 1 1 1 1
I Crew Collaboration Very Efficient 0.5 0.5 0.5 0.5 0.5
Total Influence of PSFs = A*B*C*D*E*F*G*H*I
(overall weighting factor) 0.064 2 0.8 0.64 1.6
Reference:  Hamlin, T.  Space Shuttle Program Human Reliability Analysis (HRA) Data Report.  Nov.  2008.   
CREAM QUANTITATIVE EXAMPLE (2)
SPACE SHUTTLE PROGRAM
Space Shuttle Safety and Mission Assurance Office
NASA Johnson Space Center, Houston, Texas
• The weighting factors are combined with the nominal failure probabilities to 
obtain adjusted values: 
• Dependencies between the Shuttle Crew and MCC crew are then taken into 
account in combining the activity probabilities
• Leads to overall probability for the event “Collision of Orbiter with 
International Space Station (ISS) during rendezvous and docking” 
– Max probability => 2.4E-04 * 0.55 = 1.3E-04 
19
Activity Predominant 
Failure
Nominal Failure 
Probability
Overall Weighting 
Factor
Adjusted Failure 
Probability
Adjusted for 
Dependency
Crew Error During 
Docking
Action of a Wrong 
Type (Shuttle crew)
3.0E-03 0.064 1.9E-04 Median
(2.4E-04 Mean)
N/A
Crew Fails to Observe 
need for recovery
Observation not 
made (Shuttle crew) 
3.0E-03 2 6.0E-03 Median 0.55 Mean
(High Depend)
MCC fails to Observe 
need for recovery
Observation not 
made (MCC crew)
3.0E-03 0.8 2.4E-03 Median 0.19 Mean
(Medium Depend)
MCC fails to Warn 
Crew
Missed Action (MCC 
crew)
3.0E-02 0.64 1.9E-02 Median 0.21 Mean
(Medium Depend)
Crew Fails to Execute 
Recovery
Action at the Wrong 
Time (Shuttle crew)
3.0E-03 1.6 4.8E-03 Median 0.55 Mean
(High Depend)
Reference:  Hamlin, T.  Space Shuttle Program Human Reliability Analysis (HRA) Data Report.  Nov.  2008.   
CREAM QUANTITATIVE EXAMPLE (3)


The Importance of Human Reliability Analysis in Human Space Flight: Understanding the Risks

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server