Managing software development in large organizations has become increasingly difficult due to increasing technical complexity, stricter government standards, a shortage of experienced software engineers, competitive pressure for improved productivity and quality, the need to co-develop hardware and software together, and the rapid changes in both hardware and software technology. The 'software factory' approach to software development minimizes risks while maximizing productivity and quality through standardization, automation, and training. However, in practice, this approach is relatively inflexible when adopting new software technologies. The methods that a large multi-project software engineering organization can use to increase the likelihood of successful software technology insertion (STI), especially in a standardized engineering environment, are described

Lydon, Tom

English

NASA Technical Reports Server

I 
- - . /  
N 9 2 - % ! & W  
S O R W A R E T E C H N O L O Q V ~ :  
A SrUDY OF SUCC€SS FACTORS J3- 
Submitted to: 
lha fWemh Annul Softwn Engimdng Workbhop 
November 2829,1990 
NASAIGoddard Space Rfght Center 
Greenbit MD 
- .,.' 
I - 
by 
, - - r  - 7 -  
Tew&sbury, MA 01 876 
hknaging software development in large organizations has become increasingly d i m  
due to m y  increasing technical complexity, stricter government standards. a 
shortage of experienced software engineers, ctmpetitiva pressure for improved 
productivity and quality, the need to codevelop hardware and software together. and he 
rapid changes in both hardware and software technology. 
The 'software factorym approach to software development minimizes risks Mile 
maximizing productivity and quality through standadiitation, automation, and trairing 
However, in practice, this approach is relatively inflexible when adopting new software 
technologies How can a large multi-project software engineering organization incr- 
the likelihood of successful aoftwam technology Lwertkn (Sn), espeddly in a 
standardized engineering environment? 
HISTOGRAM of SOFIWARE TECHNOCOGY INSERTION CASES 
8 I N-49 Rated Cases I 1 
Failure 4 b Success 
In an attempt to amelale various success factors with levels of success. 59 cases of hew 
software technology insertion' in thirteen m m t  projects at a hrge U.S. Defanse 
electronics corrtractor were identified and accordrig to several criteria The 
rebtive success or failure of 49 of these cases (see Rgra, 1) was determired by 
https://ntrs.nasa.gov/search.jsp?R=19920010184 2020-03-17T12:45:16+00:00Z
having key prom personnd (Lsad Engineer, Dept M m ,  and tooi sqqmdem) rate 6 
aqmcb (added together for total rating) of the software technology inserbIon resuits. 
Maximum succsss was scored as +12, and maximum failure as -12 on the rabing scale. 
The histogram in Figure 1 illustrates the distribution of scores from the 49 tated cases. 
There were 21 dfferent nw softwars technologlea studied, most of them new  mob^ 
or methods, induding (in ap9roimate lifecyde order): 
The use of 000-STD-2167 or 2167A 
Stnrchrred analysis CASE tools 
Raptd-Prototyping in requirements or design 
In-House requirements tracegbility tool 
In-ttouse program design language (POL) took 
Reusable Software in design or coding 
The use of A&@ as an implementation language 
The use of M68Cn0 assembly language 
Microprocessor 2avelopment Stations (MDS) for integration testing 
InHouse configuration management (CM), source code control tool 
Worlcstation-based engineering doarmentailon WS 
The use of -ens as primary development platforms 
Though meaningful statistical canelations were not possible due to the limited sample 
size, ratings were compiled and empirically compared with several technology mars 
measured for each STl case, induding: 
Technology Type (CompetencsEn hanang or -Destroying) 
Support Type (In-House or External) 
Mahrrity of the Technology (Young, Mature, and Old) 
Prryect Size (SLOC) 
Rior GgmtWons (for success or failure) 
Reasons (for using the new software technology) 
M o d s  (af inserting the new software technology) 
Perceived Time Savings 
Perceived Labor Savings 
Perceived Computer Cost Savings 
Perceived Quality Improvement 
Met ExpecWons? (for success or failure) 
A doser look at the Top Ekven' cases of successful STI (ratings 2 +7), and the 
'Bottom Sevwr' cases of unsuccessful Sll (ratings 5 -7) shows that 
1. Perceived Ttms Savings and perceived Lrrbor Savings are the 
. . . . most of successful or unsuccessful Sll. 
2 Though users often complain about incmsd computer aHs, 
s r v l n g m n p u m a m t i s n a t o f ~  . . becauseit 
is not usually a goal or a motivator for the use of new tecl'imlogy. 
3. P e f w i v e d W H y l m p r o v e n w n r t i s a ~ o f ~  
success, but a q b m  of Si'l failulB. 
4. Ewminsuccessfu lSTIcases ,users 'R lar~abaut  
what a mu technology cadcannot do are 
In addirtion to the success ratings owsit0 structursd lntOmWs were used 20 profile 
each new technology, and coils! other q u a l ' i  information that was wed to danfy 
and complete the data 
Tushman[l] desaibes new as: (1) c o m w l w r d r r g  - 
inmentally different. ku'ldhg on e A n g  know-how, and subsbitubing for older 
tednologies without rarrdering their skills obsolete, or (2) compasrrcadesboylng - 
fundmentally different. requiring new sldlls, abilities, and kndedge for use. The main 
types of -are: (1) IM-, where the supporters work in the same 
ofganization as the users, or (2) Outdde, Mere the sopporters work in a different 
organization than the users. 
A sample of the dstritndion of suecessfuf ST1 cases w6f these two combined fadors 
(technology type and strpport type) is show in FQam 2: 
of New 1- A c r w  TwgDimensians 
IN-HOUSE OUTSIDE 
Support Support 
Competence 
EN.IANc1w 
Competence 
DEsmoYlNG 
#Total= 16 
#Rated= 13  
Tot Rating- 47.0 
Median- +5.0 
Ave Rating- @ 
BEST 
#Total= 9 
#Rated= 8 
Tot Rating- 11.5 
~ - 
#Total= 11 
#Rated- 8 
Tot Rating= -2.0 
Poor 
#Total= 23 
#Rated= 20 
Tot Rating= 14.5 
, E?~izf@ 
Marginal 
RATING SCALE: +12 = Maximum Success, -12 = Maximum Failure 
The new softwan, tedmdogies that had ttre m a t  wcmmfd Sll experience (hugh 
across a very limited set of cases) are summarked below: 
&sasBsAve- 
1 +9.5 In-House Automated Build Tool 
2 +7.5 Mlaoprocessor Dwdopment Statlm for I-Um 
6 +4.8 In-House Software Problem Reporting Tool 
3 +3.3 In-House Configm&n Management (CM) Tod 
7 +21 In-House Rogram Oesign Language Tool 
The nw,n software t&nologies that had the lerrt ~ u c a d u l  Sfl experience are: 
S a S B s A v e -  
1 -11 0 In-House Automated Code Doarmentation Tod 
2 -8.8 Workstation-based Engineering Doarmentation Tod 
4 -4.8 Workstation-based CASE Tool for Req'ts and Design 
Among the overdl andusions from the study are: 
1. Saving schedule dme and hbor caMs are necessary and sufficient 
conditions for successful STI 
2 lmprovlng quallty is a necessary, but not *dent condition for 
successful Software Technobgy Insertion (STI) 
3. Succsss with new software technology insertion (STI) is much greater for 
c o ~ ~ ~ c i n g  than for competence-destroying Eechndogies 
4. Successwith Sll is somewhat greater for lrrhousa supported 
technologies than for outside supported technologies 
5. Success *with ST is greater for mature technologies than for either young 
or old technologies (mature is >1 year after release, <5 yea's after release) 
6. Success with Sll is greater when userst expedations about 'new 
technology' are controlled to avoid emcting too much - exceeding wrs'  
expectafions is not necessary for ruccessful Sll, but llqt meeting 
expections (i.e., disappointing them) is a sufficient condition for failure 
7. Success with Sfl can be inaeased when there is synergy between 
multiple new tectmlogies, such as Ada and workstations 
These and other results and condusions, dong with some recommmdations fa large 
software development organaations, will be covered at the workshog. 
13 Software Projects 
21 New Software Technologies 
(most of them new tools. methods, languages) 
A - The use of Ada@ as an implementation language 
6 - InHouse automated krild twl(s) 
C - ln-tiouse automated code documenCation tool 
D - In-House program design language (PDL) tools 
E - ln-tiouse metrics tools for automatic data collection 
G - IMouse standard test reporting tool based on RDBMS 
I - Worlc;tationbased engineering documentation tool 
J - The use of M68020 assembJy language 
K - Miioprocessor Development Stations (MDS) for integration testing 
L - ln-tiouse project scheduling and reporting tool 
M - ln-tiouse configuralion management (CM), source code control tool 
N - ln-tiouse VarNnix documentation package using troff 
P - ln-tiouse Software Roblem Reporting Tool based on RDBMS 
R - Rapid-Prototyping in requirements or design 
S - stmtwed analysis graphical CASE tool 
T - Structured analysis graphical CASE tool 
U - Reusable Software in design or coding 
W - The use of workstations as primary development platforms 
X - WorWon-based engirmring documentation tool 
Y - InSlouse requirements traceability tool 
Z - The use of DoD-STD-2167 or 21 67A 
Projectfrechnology Matrix 
New Technology ID 
A B C D E G I J K L M N P R S T U W X Y Z  
1 
2 
3 
4 
5 
!50SnCzts~x Rated 0 Not Rated 
Measuring Perceived STl Success 
For each STI Case, 6 Questions were asked of: 
(1) Lead Engineer (ProjecWatn'x) 
(2) Dept Manager (FunctionaVMatrix) 
Total Ratlng for each Sl Case is sum (example =+I) 
i.e., maximum = +12, minimum = -1 2 
For Each S77 Csss: Agm.,..Disagm 
+2 +I 0 -1 -2 
(Note: Questions not weighted) 
1. I would use the new methodltool again 
2 The new m&od/tool saved schedule tlme 
3. The new methodAwl saved labor cost 
4. The new m&od/twl saved computer cost 
5. The new methodltool improved quallty 
6. The new methodltool met my e ~ o n s  
R t y d m  
R a y c b m  
Page 6 dfl 
----- J 
J 
----- 4 
J 
----- J 
J 
[I] Tushman, M., and Anderson, P.. 'Technological Discontinuities and 
Organirational Envir.onments', Administrabive Science Quarterly, Sept 1986. 
[21 Scacchi, W., and Babcod<, J., 'Unders!anQng Software Technobgy Transfef. 
MCC Technical Report STP-30487, October 1987. 
AckrowledgermMs 
To the Raytheon Company, for helping to sponsor this mxk at M.I.T. during 19841 990. 
To Dr. Ralph Katz, for his technical (and nontechnical) advice on this thesis work 
To the M.I.T. Management of Technology Pmgram, in which the work was performed. 
VlEWGRAPH MATERIALS 
FOR THE 
R. LYDON PRESENTATION 
UNCLASSIFIED 
Mleslle Systems Dlvlslon Raytheon 
The flttwnth Annud Somare Englneerlng Workshop 
NASAlGoddard Space Flight Center, Greenbelt, MD 
November 28-29,1990 
Software Technology Insertion: 
A Study of Success Factors 
Tom Lydon 
Raytheon MSD, Mailstop T3ML19 
50 Apple Hlll Dfive, Tewksbury, MA 01876 
UNCLASSIFIED 
Missile Systems Division Software Technology Insertion: Success Factors Rayt heon 
Software Laboratow Nov 28,1990 
Software Technology Insertion (STI) 
Software Technology Insertion "New" Software Technology 
+ Opportunity to Insert 
"New" Software Technology Tool or Method that is unfamiliar 
to the majority of a Project Team, 
usually replacing a more familiar one 
Opportunity to Insert A software develo ment activity on 
a new (most likely P or ongoing (less 
likely) software project 
... -.--.. . . . .. ..... - . , , ... . . , . . .-- ,. ; -.. -..,, I.. ,;.: , ., I.-  
\ , .- . , , 1. ii . 
-,+..,. .-..- . - ...-.-.. 
, - . .- 8 ,  .q 
. . ....( . .--.  --. - . , - .... .- - - ,  - -. .L.. .---- -. . o. 9 *-, " ..- , .. - .  
, "" ! . ' . ' 
UNCLASSIFIED 
Mlsslle Systems Dlvlslon 
Software Technology Insertion: Success Factors Raytheon 
Software Laboratow Nov 28,1990 
Successful ST1 
Perceived ST1 Success User's sense of Labor Cost Savings + 
User's sense of Computer Cost Savings + 
User's sense of Elapsed Tlme Savings + 
User's sense of Quality Improvement 
Real ST1 Success Measured Labor Cost Savings + 
Measured Computer Cost Savings + 
Measured Elapsed Time Savings + 
Measured Quality Improvement 
UNCLASSIFIED 
Missile Systems Divlsion 
Software Technology Insertion: Success Factors Raytheon 
Software Laboratow Nov 28,1990 
ST1 "Cases" Overview 
ST1 Case A single incident of ST1 
on a single project, usually 
within a single development phase 
59 ST1 Cases Identified Across 13 different projects; 
from 1 to 7 ST1 Cases per project 
49 ST1 Cases Rated for Some of the 59 identified cases 
Perceived Success were not able to be rated 
13 Different SW Projects Some ongoing, some just completed; 
using Ada, C, Fortran, Assembl ; 
ranglng in size from 2900 to 49 8 00 SLOC 
21 Different SW Technologies Most new tools, methods, langua es (e.g., 
CASE, 21 67A. Ada, ~a~id-~roto,%euse, ...) 
UNCLASSIFIED 
Mlsslle Systems Dlvlslon Rayt heon Software Technology Insertion: Success Factors 
&#ware Laboratow Nov 2tf& .* :
13 Software Projects 
, ,  UNCLASSIFIED 
, 
Missile Systems Division 
I Software Technology Insertion: Success Factors Rayt heon 
, \ '  
.i. 
I Software Laboratory Nov 28,1990 
21 New Sofitware Technolo ies a (most of them new tools, met ods, languages) 
A - The use of Ada@ as an implementation language 
B - In-House automated bulld tool(s) 
C - In-House automated code documentation tool 
D - In-House program deslgn language (PDL) tools 
E - In-House metrics tools for automatic data collection 
Q - In-House standard test reporting tool based on RDBMS 
I - Workstation-based englneerfng documentation tool 
J - The use of M68020 assembly language 
K - Microprocessor Development Stations (MDS) for integration testing 
L - In-House project scheduling and reporting tool 
M - In-House conflguratlon management (CM), source code control tool 
N - In-House Vax/Unix documentation padcage using troff 
P - In-House Software Problem Reporting Tool based on RDBMS 
R - Rapid-Prototyping in requirements or design 
S - Structured analysis graphical CASE tool 
T - Structured analysis graphical CASE tool 
U - Reusable Software in design or coding 
W - The use of workstatlone as primary development platforms 
X - Woikstation-based engineering documentation tool 
V - In-House requirements traceability tool 
Z - The use of DoD-STD-2167 or 21 67A 
UNCLASSIFIED 
Missile Systems Division Raytheon 
Software Technology Insertion: Success Factors 
1 
I + 
I 1 1  Software Laboratorv Nov 28.1990 
, ' I  
ProjecVTechnology Matrix 
New Technology ID 
A B C D E G I J K L M N P R S T U W X Y Z  
SO sn caw: @ Rated 0 Not Rated 
UNCLASSIFIED 
M ~ S S ~ I ~  Systems Division Raytheon 
Software Technology Insertion: Success Factors 
Software Laboratow Nov 28,1990 
Other Measured Factors 
Technology Qpe (Competence-Enhandng or -Destroying) 
Support Qpe (In-House or External) 
Maturlty of the Technology (Young, Mature, and Old) 
Pro)ect Slze (SLOC) 
Prlor Expectatlohs (for success or failure) 
Reasons (for ST1 choice) 
Methods (of ST1 insertion) 
Perception of TIT Savings 
Perception of Lavr Savings 
Perception of Computer Cost Savings 
Perception of Quallty Improvement 
Result vs. Prlor Expectations (for success or failure) 
UNCLASSIFIEO 
Mlasile Systems Division 
Software Technology Insertion: Success Factors Raytheon 
Software Laboratory Nov 2 8 . 1 9 a  
Measui ing Perceived ST1 Success 
For each ST1 Case, 6 Questions were asked of: 
(1) Lead Engineer (ProjecVMatrix) 
(2) Dept Manager (FunctionaVMatrix) 
Total Rating for each ST1 Case is sum (example =+I)  
i.e., maximum = +12, minimum = -1 2 
b 
For Each ST/ Cam: 
Statemer\t (Agree or Disaareo?) 
1. I would use the new methodRool again 
2. The new methodnool saved schedule Urn8 
3. The new methodhool saved labor cost 
4. The new methodnool saved computer cost 
5. The new method4001 Improved quality 
6. The new methodnool met my expectailons 
(Note: Questions not weighted) 
Agree ..... Dlsagree 
+2 +1 0 -1 -2 
- A,--- 
2L--  
dm--- 4 
4 
,,
- 
A -  
UNCLASSIFIED 
Missile Systems Division 
Software Technology Insertion: Success Factors Raytheon 
Software Laboratow Nov 28,1990 
Histogram Of Software Technology lnsertlon Cases 
I N-49 Rated Cases I I 
Failure + t Success 
I UNCL.ASSIFIED 
Missile Systems Division 
Software Technology Insertion: Success Factors Raytheon 
-re Lsboratorv Nov 28,1900 
8 
8 
6 W n  mesons for " s m * ' :  
'Synergy' within a project (4) 
Critical need for a capability (3) 
4 'Synergym between two techr,alogles (2) Mature and powerful tool (1) 
2 May or may not "Save Computer Costsn (+0.2) 
"Met Ewpectetlonsn (+0.5) not as critical as: 
'Save Tlme" (+1.8) 
0 "Save Labor (+I .7) 
'Improve QualW (+1.61 
UNCLASSIFIED 
Missile Systems Division Rayt heon 
Software Technology Insertion: Success Factors 
Software Laboratory Nov 28,1990 
"Bottom Seven" i I 
4 t  I (Rating 5 -7) -I--- I I ,---. 
-1 2 -6 0 +6 41 2 
Failure -r------------- )L SUCCBSS 
Maln reasons for "tellure": 
Immature technology (3) 
Interface problems (3) 
Technology not "needed" by LE (2) 
Wrong technical solution (1) 
May or may not "Improve Quallty" (-0.4) 
"Save Computer Costs" (-1 -1) not as crttical as: 
'Save Time' (-1.8) 
"Savo Labof (-1.9) 
'Met Expectallons" (-1.9) 
UNCLASSIFIED 
Missile Systems Division 
Software Technology Insertion: Success Factors Raytheon 
Software Laboratow Nov 28,1990 
Competence-Enhancing vs Competence-Destroying 
Competence-Enhancing technology - major improvement in pricelperformance 
that builds on existing know-how; a substitute for older technology, but does 
not render old skills obsolete; increase efficiency. 
Competence-Destroying technology - new way of making a given product; 
requires new skills, abrlities, and knowledge for use; ma combine previously dY discrete steps into continuous flow, or be a completely ifferent process 
Maturity of a New Software Technology 
Young technology - Released < 1 year, or prior to 2nd major release (V l  .x) 
Mature technology - Released > 1 year, and after 2nd major release (V2.x+) 
8':: Old technology - Released > 5 years, or after end of formal support iii 
UNCLASSlFiED 
Missile Systems Division Raytheon 
Software Technology Insertion: Success Factors 
Software Laboratow Nov 28,1990 
Ratings of New Technology Types Across Two Factors 
IN-HOUSE Support OUTSIDE Support 
I BEST 1 OK I 
'OrnptenU 
('lncremental") 
Tot Rating= 47.0 
Median. +5.00 
Mean Rating. +3.6 
cO"PtenU 
Dyrz$!f" 
RATING SCALE: +1 2 Maximum Success, -12 - Maximum Failure 
Poor 
Tot Rating= 1 1.5 
#Total= 11 
#Rated= 8 
Tot Rating= -2.0 
Median- + I  .Sa 
Mean Rallng- -0.2 
Marginal 
Mean = +2.8 
Mean = +2.1 Mean = 4 . 9  
#Total= 23 
#Rated= 20 
Tot Rating= 14.5 
Mean = +0.4 
LJNCLASSIFIED 
Missile Systems Division 
Software Technology Insertion: Success Factors Raythetn 
Software Laboratow Nov 28,1990 
Summary of Results 
(Focus on success factors rather than successful technologies) 
(Focus on perceived rather than real ST1 success) 
Saving schedule time and labor costs drive successful ST1 (obvious?) 
Improving quality is necessary, but not sufficient for successful ST1 
Exceedin users' expectations not necessary for successful STI, but 
not meet 9 ng expectations is sufficient for failure (i.e., must control) 
Much greater success for com etence-enhancing C: vs competence-destroying tec nologies 
Greater success for mature vs young or old technologies 
up 
:3c Somewhat greater success for in-house vs outside supported 
4 "P s" 
UNCLASSIFIED 
Missile Systems Division 
Software Technology Insertion: Success Factors Rayt heon 
Software Laboratory Nov 28.1990 
. ... Next Step: 
Linking Perceived Success with Real Success 
via Software Metrics Collection 
Corporate-wide effort to implernet r i  automatic collection of software 
metrics as a by-product of develop!r;ent - MSD is Lead Division 
10 current software metrics defined (similar to Mitre Metrics) 
Based mainly on DoD-STD-2167A 
AutoCollection in development for both prolect-specific and 
process-level (across multiple projects) software metrics 
UNCLASSIFIED 
Missile Systams Division 
Software Technology Insertion: Success Factors Raytheon 
Software Laboratorv Nov 28.1990 
Overview of Raytheon MSD's Software Metrics Collection 
Project 0 


Software technology insertion: A study of success factors

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server