Design of Experiments (DOE) is a systematic approach to investigation of a system or process. A series of structured tests are designed in which planned changes are made to the input variables of a process or system. The effects of these changes on a pre-defined output are then assessed. DOE is a formal method of maximizing information gained while minimizing resources required

Huddleston, Lisa L.

NASA Technical Reports Server

Resolution of an Orbital Issue:
A Designed Experiment
Lisa Hu«dleston
NASA KSC
o
https://ntrs.nasa.gov/search.jsp?R=20120000080 2019-08-30T18:38:09+00:00Z
Background
• During 5T5-114, on ,an issue came up that required the astronauts to
take a space walk visit to the underneath side of the orbiter
• This is story of the "Gap Filler" .....and how we used an advanced
statistical method to prevent it from happening again ....
• 50 let's see what happened.
!tl!P:/Ivideo.9.QQgle.com/videoplay?docid=1183929501511634226#
OR
!!!!p:/Iwww.youtube.com/watch?v=KmOp7Ab3Bds
k~A
, ".
I.
1
On-orbit issue
What was the issue?
Who cares if a hunk of ceramic sticks~ up?
There was extreme interest in not ccllowing this to happen again ...so
we used a designed experiment to determine the best way to address
it .
k~~, ,.
) .
2
Background For Design of Experiments (DOE)
• DOE is a systematic approach to inve~~tigation of a system or process. A
series of structured tests are designed ir which planned changes are made to
the input variables of a process or system. The effects of these changes on a
pre-defined output are then assessed.
• DOE is a formal method of maximizin~! information gained while minimizing
resources required.
k~~
" ", . 3
Background For Design of Experiments (DOE)
DOE can be used to find answers in situations such as:
• What is the main contributing factor to a problem?
• How well does the system/process perform in the presence of noise?
(defined as randomness in response due to un(ontrolled variation)
• What is the best configuration of factor values to minimize variation in a
response or outcome?
Information extracted from "Design of Experiments in brief', http://www.thequalityportal.com/qknow02.htm
4
Back round For Desi n of Ex eriments DOE
How to perform it
- Identify the input variables and the response (output) that is to be
measured.
- For each input variable, a number of levels are defined that represent
the range for which the effect of that variable is desired to be known.
- Produce an experimental plan wHich tells the experimenter where to set
each test parameter for each run of the test.
- Measure the response for each rlLn.
- Look for differences between response (output) readings for different
groups of the input changes.
- Attribute the differences to the in~ut variables acting alone (called a
single effect) or in combination with another input variable (called an
interaction).
k~!y.
"
) .
Information extracted from "Design of Experiments in brief', http://www.thegualityportal.com/gknow02.htm
5
Design of Experiments (DOE) Example
Gap Filler Pre Process Change Te~ting
(These samples were subjected to the "12 lb. pull load which was the
spec requirement prior to the process change. The current pull
load requirement is 5 Ibs.)
• Pre-Process Change Testing
- Bonded 20 1-ply per P-602 speo
- Bonded 20 2-ply per P-602 speo
- Bonded 20 4-ply per P-602 speo
• Results
- 1 ply failure rate: 12/20 failed =EO%
- 2 ply failure rate: 9/20 failed =45%
- 4 ply failure rate: 2/20 failed = 10%
• This showed us that the increased Ifumber of plies appeared to
increase bond reliability.
6
Design of Experiments (DOE) Example
Observations from Pre-test wl1ich also may have affected
the bondline integrity:
• lack of RTV on filler bar
- Armalon RTV squeeze out
• poor pressure application
• sidewall bonds
• gap filler protrusion after bond
• non-uniform load during pull test
• partial debonds
• multi-ply gap fillers may not deflect when debonds are present
7
Sandwich 0.05" Dip
Design of Experiments (DC(E) Example
• This is the kind of thing we all learne«( about in engineering schooL ...
• So ....the science is well understood
• The methods are well understood
• The selection techniques are well un<cerstood
• the primary difficulty in performing a set up and test like this is getting
management to understand what you're doing; why you're doing it that way;
and why it's important to follow the rules.
k~~, . 9
Design of Experiments ~albF Example
Qualification Test of Process Enhancements
- 144 gap fillers distributed as follows
Factors Levels
Technicians 6 levels - randomly selected from
population of technicians with gap filler
cert
Inspectors 2 levels- randomly selected from
population of TPS quality inspectors
Fill type 2 levels- conrplete or nominal
Lengths 4 levels - 1.!!", 3", 5", and 7"
* Plys/flat panel 3 levels - 1, 3, and 4 ply
* Plys/curved panel 3 levels - 2, 5, and 6 ply
10
Design of Experiments {DOE} Example
Qualification Test Results
• Ordinary Least Squares Regression was performed on the
142 data points to predict and hypothesize on the effects that
the gap filler process improvements may have brought to gap
filler bonding at KSC
11
Design of Experiments (DOE) Example
120
•
•
•••
•
•
• ••
• •
,J'--'.1 II:·
•••
•
•
o~"ff"fDJJJJJJJJJJJJJJJJJJJJJJJJJf
o 20 40 60 80 100
Pull To Failure Load (Ibs.) Predicted
P<.0001 RSq=0.H2 RMSE=10.895
:::J
a..
m N2M......JJJJJJJJJ~~J....I.
:::J
~
U
« 100
..-..
.
en
.c
'-' 80
"'0
m
.3 60
Q)
s....
:::J 4M~JJJJJ~ ~~~JJJJ____t
m
u..
~ 20
12
Design of Experiments (DOE) Example
Source Nparm DF Sum of Squares F Ratio Prob> F Rank
Technician 5 5 1152.939 1.9425 0.0917 3
QC 1 1 107.005 0.9014 0.3442 4
Length (in.) 1 1 54580.025 459.7903 ~.lllf \ 1
Ply 5 4 9198.389 19.3722 \ <.0001 )- ..
Panel 1 0 0.000 NA ~ ./ NA1'1 A-
(0 = flat,
1 = curved)
Complete or 1 1 1.983 0.0167 0.8974 5
Nominal?
(0 = nominal,
1 = complete)
The smaller
the number
the less likely
the observed
effect
occurred by
chance.
13
Desi n of Experiments DOE Exam Ie
What does this mean?
• Ply, and length have the greatest effect on the pull to failure load.
• Fill type (complete or nominal), tecHnician, or QC do not have as great an
effect.
• The effect of the panel could not be statistically determined since ply and
panel type were not independent. Itowever, examination of the following
figures indicates that IML curvature does have an effect on bond strength.
• The following is the overall summany of the 142 data points:
Average Pull Value (Ibs.): 41.09
Minimum value (Ibs.): 3
Maximum value (Ibs.): 118
0fc, Complete Fill: 35.420/0
% Nominal Fill: 64.580/0
% Curved: 43.750/0
%Flat: 56.250/0
0/0 Failure Rate: 0.700/0
14
Desi n of Ex eriments DOE Exam Ie
130
120
110
....-
~ 100
-- 90
'U
~ 80
...J
(l) 70
L..
~
:-= 60
ro
u. 50
o
..... 40
~ 30a..
20
10
o
•
•
8
o :
• •
III
0 06.
6. +x • i Y.06.'SWMS.~ +x 06.+ •
•
+
• th +: T0'06.' +x• x~8 if ~ 06. + YB 06.
*sgh, i x Y
o ,
+
~~~~
, " " " ,
1.5 3 5 7
Ply within Length (in.)
15
,------------------------------ --
Design of Experiments (DOE) Example
130
120
110
....-
~ 100
-- 90
"'0
~ 80
.....J
Q) 70
s....
.::J 60
ro
L.L. 50
o
..... 40
::J 30a..
20
10
o
+
+
*
*+ +
=1=
+ Ii ;~ $
+ +$ 3E I I + x
* I
x
~ + xi I I ~i
o 1 0 1 0 1 0 1
'---_---I' , , , ' .....' _---'
1.5 3 5 7
Panel (0 = flat, 1 = curved) within Length (in.
16
------------------------------
Design of Experiments (DOE) Example
Major Process Enhancements
- Gap Filler length limited to 1.5 to 7 inches
- Clarified the number of pull loops ra:quired for varying lengths
- Moved location of pull loop holes atove OML trim line and increase hole
diameter
- Added Sandwich method for gap filler installation based on OPF technician
recommendations and process testing
- Added description for bonding pressure using tape method on the high gap
filler
- Added requirement to sever any siccewall bonds using 0.006 inch thick metal
shim
- Increased the pull loads to 5.0 +1.01-0.0 pounds
- Trim gap filler flush to 0.12 inch reoessed after pull test
17
Design of Experiments (DOE) Example
Conclusion of gap filler experiment
• New process allows for an average gap filler pull to failure
value of 41 Ibs.
- Range of pull to failure values rcange from 3 Ibs. To 118 Ibs.
• Length and to a lesser degree, ply, have the greatest effect
on the pull to failure load
- Curved or flat panel effect cannot be determined statistically but from
observation of trend on previou~ charts, IML curvature does seem to
have an effect on bond strengtH.
18
Conclusion
• So.....what did we get out of all of flhis?
- Proven enhancements to the process
- Better understanding of the ins~tallationprocess .
- AND ---
- Not a single gap filler issue dUling remaining missions for Space
Shuttle
- AND
- Better management support for use of advanced statistical
methods in the operational environment.
lyf~A
" .
L..-- _
19
BACK UP
20
I Gap cill~ 'High" Iinch
'..:'..: RTV I Gap cill~ High"l inch ::.. ;... RTV
Annalon Annalon
c=J FillerBar c=J FillerBar
SIP SIP
Gap Filler Gap Filler
Sandwich Technique Wlh Stt11clure Sandwich Technique Wlh Smlcture
/ RTV from!'Jl mi. I.~ppin!" tttlmiq~ / RTV from !'Jl1iI. 'Dippinf tttfmiq~
Tile Densified Layer Tile Densified Layer
21
,------------------ -------------
Sandwich & G/F High
G/F Bonded High and Trimming
Armalon Placement & ~~emco
Creased Arrmlon
I I
: I
I ~
, I
Ih
Anmlon creased to ensure
reach down into pi'
within 0, 13 in· oflML
I
Filler Bar
I
SIP
k~~
" .') .
Tile


Resolution of an Orbital Issue: A Designed Experiment

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server