Naval Postgraduate School Acquisition Research Progra

Cole, Scott

Housel, Tom

Wolff, Russel

English

Calhoun, Institutional Archive of the Naval Postgraduate School

Calhoun: The NPS Institutional ArchiveDSpace RepositoryAcquisition Research Program Acquisition Research Symposium2012-05-01Historical Analysis of Costs, Risks, andUncertainties: Moving From a Proprietary toan Open Architected Systems, Open BusinessAcquisitions Management ApproachHousel, Tom; Cole, Scott; Wolff, RusselMonterey, California. Naval Postgraduate Schoolhttp://hdl.handle.net/10945/54395This publication is a work of the U.S. Government as defined in Title 17, UnitedStates Code, Section 101. Copyright protection is not available for this work in theUnited States.Downloaded from NPS Archive: CalhounHistorical Analysis of Costs, Risks, and Uncertainties: Moving From a Proprietary to an Open Architecture System, Open Business Acquisitions Management Approach Russ Wolff (USMC) and Scott Cole (Navy) May 2012 Research Objectives 5/18/2012 • examine relationships between OA, SOA, and Navy OA (NOA);  • establish cost-savings benchmarks based on industry performance for traditional proprietary architecture models and SOA;  • identify the risks to PMs in the Defense Acquisition Systems (DAS) ecosystem, including various organizations involved with acquisitions • evaluate if an OA strategy assists or hinders acquisition programs; • ascertain if an OA strategy exposes a program to unique risks and uncertainties; and   • establish if OA has delivered its promised benefits to the DAS.  Context: The “Open” Movement 5/18/2012 • Defense Acquisitions wants better products for less • Open Solutions deliver costs savings and capability improvements to the private sector • Open Business systems may deliver these same benefits • anticipated that OA principles would enable small, innovative businesses to enter the defense market.  • Service-Oriented Architecture   • (SOA): “approach for sharing functions and applications across an organization by designing them as discrete, reusable, business-oriented services.” (GAO, 2005) • Open solutions  • offer new possibilities for solving business problems, provide business interoperability by standardization and technology transparency, and decrease time to market for key products and services.  Comparison of OA and SOA 5/18/2012 Open Architecture Characteristics Service-Oriented Architecture Characteristics Modular design and design disclosure Services are modular Life-cycle affordability Reliability and modifiability attributes decrease cost over the lifetime of the system. Easily upgradable systems Adaptability, extensibility, and modifiability provide ease of upgrading a system. Core concepts of scalability and portability, and stated goal of interoperability Quality attributes of scalability and interoperability Goal to optimize system performance Quality attribute of performance Reusable application software Reusable services Interoperable joint warfighting applications and secure information exchange (common services and information assurance) Quality attributes of usability (common services) and security Potential Quantitative Benefits 5/18/2012 • Case Studies Analyzed the ROI of SOA implementation in 18 different companies- 336% ROI with 72% ROI from Cost Savings       Company Reported ROICalcuated ROI from Cost Savings / Cost AvoidanceAverage Annual Cost SavingsAverage Annual Cost AvoidanceAverage Annual Productivity ImprovementBenefit (discounted)Investment (discounted) NPV Discount %Discount Period (Years)Payback (months)Blue Cross Blue Shield 332% 330% $2,380,000 $0 $90,000 $14,330,000 $3,320,000 $11,010,000 12% 6 16.7Mobile Telecom 625% 136% $1,100,000 $0 $3,570,000 $10,120,000 $1,400,000 $8,720,000 12% 3 15.3Global Provider for 470% -18% $0 $387,853 $2,827,485 $8,080,525 $1,417,846 $6,662,679 12% 3 43.9Services and Fac Mgmt Co 360% -100% $0 $0 $1,140,000 $2,744,982 $596,674 $2,148,309 12% 3European based 212% -18% $478,463 $0 $1,801,860 $5,472,842 $1,753,242 $3,719,600 12% 3 44.0International Finance Firm 252% -31% $101,015 $329,054 $2,669,439 $6,627,447 $1,882,568 $4,744,879 12% 3 52.5Global Media Consulting 244% 107% $111,609 $198,140 $332,626 $1,541,718 $447,938 $1,093,780 12% 3 17.4International Insurance 256% 7% $143,839 $0 $427,328 $1,428,180 $401,607 $1,026,573 12% 3 33.5Healthcare Services 346% 146% $0 $2,870,000 $3,720,000 $15,800,000 $3,500,000 $12,300,000 12% 3 14.6Global Consumer 265% 165% $270,689 $0 $195,366 $1,118,547 $306,370 $812,176 12% 3 13.6Average 336% 72% 27.9Potential Qualitative Benefits as Seen in the Commercial World 5/18/2012   Benefit Categories Examples of Qualitative Measurements Relationship to the DoD Business Staff Efficiency Information delivered to managers more quickly and accurately improves decision-making. Timely and accurate delivery of information vital to military leaders. Business Credibility Equates to more business because other companies view their system as available and reliable. Productivity improvement through availability and reliability of systems used by the DoD.  Reduced Duplication of Effort Information entered once, available to all users.  Ensures accuracy and consistency of data. It also saves time inputting data or fixing mismatched data. Faster Time-to-Market Difference in the amount of time a product is available compared to the current time to market. Faster delivery of vital intelligence or logistics when and where required. Scalability The ability to increase size or volume without degradation.  Scaling of service  in accordance with changing mission. Flexibility Flexibility is achieved through increased agility and potential for reuse. Flexibility allows for quick adaptation to environmental changes.  Small Business Participation 5/18/2012 • DoD anticipated greater Small-Business participation through OA • Small Business Innovative Research (SBIR) Contracts • Results showed actual Small Business participation sub-optimal • Contract Funds Awarded from 1 February 2012- 31 March 2012       $9,133,000,234  Award Total $53,409,358  Average Contract Awarded  $537,112,301 Small Business Award Total minus contracts over $100 million $8,346,994  Average Small Business Contract Awarded minus               contracts over $100 million 5.88% Percentage of Awards given to Small Businesses Types of Risk 5/18/2012 • When interviewed, program managers identified the following types of risk they have to mitigate:  Risk Number of Times Mentioned Risk Number of Times Mentioned Risk Number of Times Mentioned TECHNICAL 10 PERFORMANCE 6 ENVIRONMENT SAFETY & OCCUPATIONAL HEALTH 5 COST 5 SCHEDULE 4 MANUFACTURING 3 OPERATIONAL 2 INTEGRATION 2 ENTERPRISE ARCHITECTURE 2 SYSTEM LEVEL 2 MATERIAL 1 PROGRAM 1 Risk and Uncertainty • Determined that Risk Suppression excluded Small Companies from participation in an Open Business Model • Heavy bureaucracy makes it        difficult for small businesses to compete. • Because of these risk suppression measures, entry costs prohibit entry of small businesses into the acquisitions environment.    5/18/2012 9 Top Internal Barriers to OA • Current flow of money - Budget allocations through organization • Lack of defined reference architecture • Lack of OA understanding • Lack of resources • Unwillingness to break existing contracts • Lack of OA education and training • Lack of formal requirements • Perceived up-front investment too expensive  • Lack of incentives • Lack of contractual requirements • T&E and Certification issues • Fear of Change – ranked #1 by respondents w/A-RCI experience • Lack of centralized configuration management control • Lack of leadership support  • Current Organizational Structure • Lack of accountability   A-RCI Success Story 5/18/2012 • Demonstrated practical application of Open Systems methodology by successfully implementing an improved sonar system while beating budget and time goals: • Life cycle costs were improved by a factor of close to 5 • a seven-fold increase in processing capability • mean operator success rate increased by a factor of four • mean number of false alarms reduced by 40% • detection and classification time improved by 27 minutes  A-RCI Development Model 5/18/2012  A-RCI Best Practices and Risks 5/18/2012 Best Practices and Successful Strategies Introduced Risks A-RCI   COTS Security Risk  Program Risk (bureaucratic friction) Incremental Strategy Operational Risk (initial deployed system does not meet user requirements) Cost Risk Schedule Risk MOSA Integration Risk Open Capabilities-Based Model Integration Risk Program Risk (bureaucratic friction) Conclusions 5/18/2012 • DoD must address risk management to implement OA/SOA • Regardless of risk suppression measures, current programs still miss time and budget requirements • DoD should examine the commercial world risk construct that includes the upside as well as downside of risk • Until the total risk constructs is addressed, the DoD will never achieve true portfolio management nor will it ever fully realize the potential of  OA. • DoD must weigh other factors in addition to cost-savings when considering Open Systems approaches • Productivity Improvements- measured by meaningful, valid ROI projections that need to be established • Flexibility, Scalability, and Reusability- measured by Real Options Analysis  Recommendations 5/18/2012 • Implement New Metrics • Traditional ROI insufficient for the DoD’s needs: Current ROI estimates rely on cost-savings or avoidance data that does not provide a unique numerator. • Portfolio optimization and real options for forecasting future  • Incremental Implementation Approach • Attack the low-hanging fruit by introducing SOA services that provide the most bang for the buck. • Provide Greater Initiative for PMs • A-RCI showed how taking initial performance risks, security risks using a COTS strategy, and potential cost and schedule risks using a spiral development strategy could lead to program success  

Historical Analysis of Costs, Risks, and Uncertainties: Moving From a Proprietary to an Open Architected Systems, Open Business Acquisitions Management Approach

https://calhoun.nps.edu/bitstream/handle/10945/54392/NPS-AM-12-C9P17R02-077.pdf?sequence=1

Historical Analysis of Costs, Risks, and Uncertainties: Moving From a Proprietary to an Open Architected Systems, Open Business Acquisitions Management Approach

Abstract

Similar works

Full text

Available Versions

Calhoun, Institutional Archive of the Naval Postgraduate School

Calhoun, Institutional Archive of the Naval Postgraduate School