Levin, Dana

Mcguire, Kerry M.

English

NASA Technical Reports Server

National Aeronautics and Space Administration
Modelling Provider Knowledge Skills and Abilities 
(KSA)
Dana Levin MD MPH
Kerry McGuire, PhD
Human Research Program
Exploration Medical Capability Element
January 2020
“Expanding the Boundaries of Space Medicine and Technology”
https://ntrs.nasa.gov/search.jsp?R=20200001184 2020-03-28T18:59:18+00:00Z
Carayon et al., 2006
2
Designing for the Human
What are KSAs and Why Do We Care?
• This is a conceptual approach that is in 
progress
• Care systems depend on care providers
• Most sophisticated hospital staffed by 
coffee baristas 
• Best trained care team on a deserted 
island
• We need both
• Qualitative: 
– EMT, Paramedic, Nurse, Doctor, Specialist, Subspecialist
• Minimum training level based on standards
• Doctor           Emergency Physician          Psychiatrist
• Quantitative: ? 
How is KSA Measured?
• One way is to start with how attributes of each provider affect aspects of disease 
progression
• Diseases start with an initial insult or defect
• Progress to one of several end points 
Probability and progression
Disease Onset Stable
Worse
Better
50%
30%
Unable to work
Limited ability to work
Full Ability to Work
2
6
1-3 days
Infinite
7 10 Days
30-60 Minutes
20%
• Medical intervention alters;
– The probability of disease progression: Curative
– The time to progression: Stability
– Severity of Symptoms: Palliative
Probability and progression
Disease Onset Stable
Worse
Better
50%
30%
Unable to work
Limited ability to work
Full Ability to Work
2
6
1-3 days
Infinite
2-4 Hours
7 10 Days
30-60 Minutes
I
n
t
e
r
v
e
n
t
i
o
n
20%
• The probability of an intervention being successful depends on;
– Diagnosis
– Clinical data
• Lab data
• Imaging data
– Stabilization 
– Palliation 
– Repair/Healing/curative
• This may provide the basis for how KSAs can be integrated into a model system
Probability and Progression
Attribute Points
The minimum equipment needed to alter each intervention gate.
“Does the crew have the material and equipment they need to 
intervene”
Modifies the probabilities to each progression state at each gate 
“Did the crew get it right”
Modifies the incidence of the disease going in
“Can they stop it before it starts”
Assigns a modifier to each procedure (diagnostic and treatment) 
indicating average number of attempts needed for success. Also 
assigns a modifier to each disease state indicating chance of 
successful diagnosis without using consumable supplies 
“How much equipment did they use” 
Modifies diagnostic and treatment procedure probability of 
success. “How skilled is the CMO?”
Attributes 
Minimum Equipment 
Level
1-5
Stabilization Attribute 
Level
1-5
Diagnostic Attribute Level 1-5
Preventive Attribute Level 1-5
Equipment Stewardship 1-5
Specific Procedure 
Attribute level
1-5
Additional Attributes 1-5
Modifies the time to progression at each gate 
“how long do they have to get definitive care”
Added later as data fills in gaps
• Specific providers can be modelled with “Attribute Points”
• Each provider training level represents varied amounts of skill in specific areas
Probability and Progression
UTI Incidence: 3%/year
UTI Incidence: 1%/year
Preventive KSA Modifier
95% Resolved
<1% Pyelonephritis 
5% Still Infected 
10% 
Resolved
20% Pyelonephritis 
70% Still Infected 
Minimum Equipment
• 1st generation cephalosporin
• Med storage
• Procedure: med rec
• Procedure: PO medication 
INTERVENTION GATE 1
Diagnostic KSA Modifier
Stabilization KSA Modifier
Progression Time b Progression Time a  
Probability of Progression a Probability of Progression b
10% Resolved
70% 
Still Infected 
95% Resolved
5% Still Infected 
75% Resolved
5% Still Septic
20% Dead
5% Resolved
15% still Sepsic
80% Dead
10% Resolved
67% Still Infected 
95% Resolved
5% Still Infected 
INTERVENTION GATE 2a
INTERVENTION GATE 2b
Minimum Equipment
Diagnostic KSA Modifier
Stabilization KSA Modifier
Time bTime a
Probability a Probability
• Alternative PO antibiotic
• Med storage
• Procedure: med rec
• Procedure: PO medication 
Sepsis  Incidence
Minimum Equipment
Stabilization KSA Modifier
Time bTime a
• Alternative PO antibiotic
• Med storage
• Procedure: med rec
• Procedure: PO medication 
20% Pyelonephritis 20% Pyelonephritis 
20% Sepsis 20% Sepsis 
Minimum Equipment
Diagnostic KSA Modifier
Stabilization KSA Modifier
Time bTime a
Probability a Probability
<1% Sepsis 3% Sepsis 
The Challenge
Data
Do We Know Enough?
• If we have a list of diseases, we can cull the literature to see what data is out there, rate the 
quality of the data, and identify where there are gaps that we need to fill with new 
research/determine if this gap filling is worth the investment.
Do We Know Enough?: 
Example Data Search
Days
Incidence 1
Natural history flowchart 2
Natural history data collection 7
Diagnostics 2
Impact of practitioner on diagnostics 1
Treatments 1
Review, refinement 1
Total ~15
Time required
Do We Know Enough?: 
Example Data Search
Do We Know Enough?: 
Example Data Search
Incidence/yr n Date range Location Sex Age
Only 
First 
Stones Citation Comment
0.20% 468 1965-1974 Minnesota Males 35-49 Yes 5
Close to true incidence (retrospective, 
complete population data)
0.05% 204 1950-1974 Minnesota Females 35-49 Yes 5
Close to true incidence (retrospective, 
complete population data)
0.24% 157 1970-2000 Minnesota Males 40-49 Yes 8
Close to true incidence (retrospective, 
complete population data)
0.08% 91 1970-2000 Minnesota Females 40-49 Yes 8
Close to true incidence (retrospective, 
complete population data)
0.33% 121 1986-1990 US Males <45 Yes 6 True incidence (prospective survey)
0.38% 92 1986-1990 US Males 45-49 Yes 6 True incidence (prospective survey)
0.31% 77 1986-1990 US Males 50-54 Yes 6 True incidence (prospective survey)
1.90% 12 1998 Seoul, Korea Males 40-79 No 7 Point prevalence w/in 1 year, not incidence
0.30% 4 1998 Seoul, Korea Females 40-79 No 7 Point prevalence w/in 1 year, not incidence
0.25% 54 2006-2012 Sweden Men 20-39 Yes 9 From a control cohort w/o gout (sup. data)
0.36% 402 2006-2012 Sweden Men 40-59 Yes 9 From a control cohort w/o gout (sup. data)
0.28% 15 2006-2012 Sweden Women 20-39 Yes 9 From a control cohort w/o gout (sup. data)
0.17% 52 2006-2012 Sweden Women 40-59 Yes 9 From a control cohort w/o gout (sup. data)
0.15% * - US Navy - Pacific Both - No 30
Based on: US Navy After-Action Reports, 5-12 
cases per deploy of 5000-6000 crew
Mean ± SD (M)
0.60% ±
0.73%
Mean ± SD (F) 0.16% ± 0.12%
Mean ± SD (all) 0.38% ± 0.54% Note: example data, not a meta-analysis!
Do We Know Enough?: 
Example Data Search
6 > 8 In a smaller study, the rate of severe sepsis associated with stone-related pyelonephritis was 12.5% [citation 36].
In another, the rate of septic shock associated with stone-related pyelonephritis was 33%. [citation 33]
The rate of severe sepsis associated with all infected urolithiasis was 3.2% in a nationwide inpatient sample. [citation 32]
7 > 7b Unable to find data, however, this is equal to the complement of 7 > 8.
7 > 8 Unable to find data.
7 > 8b This statistic may be derived as the complement of the mortality, or 100-19% = 81% [citation 28].
6 > 8c
The mortality rate of obstructive pyelonephritis due to stones, in the subset whose initial drainage trial failed, was 5.6% (vs 2.5% where 
drainage succeeded). [citation 43]
The mortality rate was 1.1% in those who never were drained (these cases were in too dire condition for drainage) [citation 43].
However, as drainage was always available and used according to the patient's severity, these statistics do not reflect the natural history.
The more useful number may be the percentage who were drained, 67%. [43] This reflects the portion for whom drainage was indicated, 
and likely necessary.
In another study, those with obstructive pyelonephritis (with fever) due to calculi who did not undergo drainage had 100% survival. 
[citation 31]
In another, those with obstructive pyelonephritis due to calculi had 8% mortality [citation 37].
Again, these disparate numbers reflect a selection bias of patients deemed to require drainage.
The mortality rate from infected urolithiasis was 0.2% - 0.25% in a nationwide inpatient sample. [citation 32].
7 > 8c 19.2% of septic stone patients not undergoing surgical decompression died. [citation 28]
The mortality rate of obstructive pyelonephritis patients with qSOFA score 2 or greater was 18%. [citation 37]
8 > 8c The mortality rate from stone-related severe sepsis was 27% - 40% in a nationwide inpatient sample. [citation 32].
The mortality rates of those with obstructive pyelonephritis due to calculi with SOFA score 7 or greater was 28%. [citation 37]
Do We Know Enough?: 
Example Data Search
Procedure Notes / Data
Natural history 
nodes modified
Citatio
n
History and 
Physical
The most significant predictors of renal colic were hematuria, short duration of pain (<12h), loin or renal tenderness, 
normal appetite. 3>3b 45
Urinalysis
The presence of hematuria has 85% sensitivity for renal stone. 3>3a 11
The presence of pyuria gives a 36% chance of positive urine culture, versus 3% without. Large, medium and small/none 
leukocyte esterase correlated with chance of infection (78% vs 41% vs 2%). 2 > 2+ 21
CBC Elevated WBC count was not associated with UTI in renal colic. 2 > 2+ 29
BMP
The presence of acute kidney injury may be due to obstructive nephropathy, but this has a poor NPV, as the 
contralateral kidney compensates so that creatinine does not rise.
The prevalence of AKI in the setting of ureteral stones was 0.72%; of these 47% had a solitary kidney and 33% had 
bilateral stones. 39
Ultrasound
A totally negative sonogram predicts a very low need for urologic intervention. This can be seen as a surrogate for the 
presence of obstruction or severe medical complication. (0% - 3.5% (95% CI) risk for intervention if negative) 4>5 10
Pooled sensitivity and specificity of 45% and 94% for ureteral calculi.
New stone-specific ultrasound modes improve this performance.
46
46
Doppler 
ultrasound
The delta resistive index was 81% sensitive and 96% specific for obstruction. 4>5 47
Ureteral jet peak velocity above 15.25 cm/s had 85% sensitivity and 63% specificity for spontaneous passage within 4 
weeks. 4>4b 49
KUB X-ray
Sensitivity of 37% for stones <5mm, 88% for stones >5mm. Thus, useful in excluding stones that may not pass 
spontaneously. 4>4b 46
Effect of diagnostics on progression
Do We Know Enough?: 
Example Data Search
Treatment Notes
Natural 
history 
point(s) 
modified Citation
Tamsulosin
By meta-analysis, absolute risk reduction of 22% for non-passage of distal stones 5-10mm; however, 
many studies have null result. Given the heterogeneity of results, tamsulosin would not add much value 
to risk calculations; nonetheless, as it is widely used and favored, it may be prudent to consider it as a 
"bonus" medication.
4>5 12
Extracorporeal shockwave 
lithotripsy
Emergency ESWL has strong results in renal colic. However, technology and technique would differ 
significantly from terrestrial evidence. Analysis has shown ESWL failure predicted by stone burden, 
previous ipsilateral calculi.
51
Early ESWL for a mean stone size of 7mm produced 89% fragmentation, 86% 3 month stone free rate, 
and 10 day mean stone clearance. 4>5 53
Early ESWL for a mean stone size of 10mm produced 53% stone-free at 24 hours, 79% at 3 months. 4>5 52
Antibiotics Data not found on non-use of antibiotics.
Percutaneous nephrostomy
Surgical decompression in sepsis due to stone is associated with 8.9% mortality versus 19% mortality 
(p<.001). 7>8c 28
Effect of treatments on progression
Do We Know Enough?: 
Example Data Search
Comparing Provider Types
• Point-of-care Ultrasound with Radiology read Ultrasound[50]
– ER physicians (US trained): 85% sensitivity and 50% specificity
– Radiology ultrasound: 84% sensitivity and 53% specificity
– Not Significant
• Bedside ultrasound in the hands of novice residents vs. attendings [54]
– Attendings were 87% accurate, 
– Residents were 80% accurate
– Not Significant
• Number of exams until competency in detecting hydronephrosis [55]
– 11-20 exams: Negative Predictive Value of 78%
– 21-30 exams NPV of 93%
– 31-43: NPV of 97%
Summary
• A system needs to fit the needs of the people operating it, training will vary so how do we design to accommodate individual needs
• Care systems depend on the provider’s Knowledge Skill and Ability to operate them. 
• In the Medical world title (EMT, Nurse, Doctor, etc…) is a proxy for KSA and represents a providers minimum level of training
• KSA needs to fit into a model of disease progression
• Disease progression can be modeled as the probability of a specific outcome and time to reach that outcome.
• KSA would modify factors that affect the outcome probabilities and the time to reach them.
• One place to start is to define discrete categories of intervention 
– such as; diagnosis, clinical data interpretation, stabilization interventions, palliation interventions, and repair/curative interventions. 
• KSA’s could be modelled as provider attributes reflecting these categories with “skill points” representing proficiency in each
• The qualitative titles can be modelled with these attribute points to build provider type or system augmentation profiles
• Such profiles would enable tradespace and risk analyses including KSA
• All of this relies on data to make it evidence based and until we collect this data, this remains conceptual
• Collecting this data will take time, and there are significant holes in the existing literature 
• However, some work has already been done, and it is sufficient enough to show that this approach is workable 
• This is a complex, interdisciplinary effort that must coordinate and integrate across all existing ExMC project groups, but the end result 
will help create a more accurate picture of medical mission risk to enable tradespace, risk, and pre/intra mission planning. 
THANK YOU
Questions?


Modeling Provider Knowledge, Skills, and Abilities (KSA)

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server