12th ECALAS (ECATA Alumni Association) International Aerospace Seminar: Risk and Opportunity Management in Programs presentatio

Valerdi, Dr. Ricardo

DSpace@MIT

Using Cost Models to Capture Project Risk:      A Knowledge-Based Approach Dr. Ricardo ValerdiMassachusetts Institute of TechnologyOctober 22 2009 , [rvalerdi@mit.edu]We Share A Goal:We Share A Goal: Enterprise ExcellenceExcellenceEnterprisehttp://lean.mit.edu © 2009 Massachusetts Institute of Technology   Valerdi  2009 - 44Risk Assessment Lessons Learned in the U.S. Department of Defense     1. Systems engineering can be the blessing or the curse• Resource estimation methods are being developed2 T h l t it d i t t bilit. ec no ogy ma ur y an  requ remen s s a y are controllable risks• Cost models help understand this relationship     3. People risks are often underestimated• Experience and capability are not interchangeable     4. By the time the risk is identified, it’s too late!• Need leading indicators (not lagging indicators)http://lean.mit.edu © 2009 Massachusetts Institute of Technology   Valerdi  2009 - 5     Cost Commitment on Projects100% Commitment to Technology,Configuration, Performance, Cost, etc.75Cost Incurred50System-Specific Know ledge25Ease of ChangeDetail DesignandDevelopmentConceptual-PreliminaryDesignConstructionand/orProductionSystem Use, Phaseout,and DisposalNEEDhttp://lean.mit.edu © 2009 Massachusetts Institute of Technology   Valerdi  2009 - 66Blanchard, B., Fabrycky, W., Systems Engineering & Analysis, Prentice Hall, 1998.COSYSMO Operational ConceptConstructive Systems Engi eering Cost Model# RequirementsSizeDriversEffort# Interfaces# Scenarios# Algorithms+COSYSMOEffortMultipliers3 Adj. FactorsCalibration- Application factors-8 factors- Team factors6 f t-  ac orshttp://lean.mit.edu © 2009 Massachusetts Institute of Technology   Valerdi  2009 - 7Systems Engineering ProcessesEIA/ANSI 632, Processes for Engineering a System (1999).• Acquisition and Supply • Supply Process• Product Realization• Implementation Process• Acquisition Process• Technical Management• Planning Process• Assessment Process • Transition to Use Process• Technical Evaluation• Systems Analysis Process • Control Process• System Design• Requirements Definition Process• Requirements Validation Process• System Verification Process• End Products Validation Process• Solution Definition Processhttp://lean.mit.edu © 2009 Massachusetts Institute of Technology   Valerdi  2009 - 8COSYSMO Data SourcesBoeing Integrated Defense Systems (Seal Beach, CA)COSYSMO Data Sources     Raytheon Intelligence & Information Systems (Garland, TX)Northrop Grumman Mission Systems (Redondo Beach, CA)Lockheed Martin Transportation & Security Solutions (Rockville, MD)     Integrated Systems & Solutions (Valley Forge, PA)Systems Integration (Owego, NY)Aeronautics (Marietta, GA)Maritime Systems & Sensors (Manassas, VA;  Baltimore, MD; Syracuse, NY) General Dynamics Maritime Digital Systems/AIS (Pittsfield, MA)Surveillance & Reconnaissance Systems/AIS    (Bloomington, MN)BAE Systems National Security Solutions/ISS (San Diego, CA)Information & Electronic Warfare Systems (Nashua,      NH)SAIC Army Transformation (Orlando, FL)Integrated Data Solutions & Analysis (McLean VA)http://lean.mit.edu © 2009 Massachusetts Institute of Technology   Valerdi  2009 - 99     , L-3 Communications Greenville, TXPolicy & ContractsCommercial ImplementationsCOSYSMO Model10 Academic Theses  141,,,,,, )(jjEkkdkdknknkekeNS EMwwwAPMAcademic CurriculaProprietary Implementations• SEEMaP• COSYSMO-R• SECOSTSystems Enghttp://lean.mit.edu © 2009 Massachusetts Institute of Technology   Valerdi  2009 - 10•  . Cost ToolTraditional Cost and Schedule Risk Estimation Assess/ EstimatePossible Estimate Cost & Schedule RisksDetermineSources ofRiskRisk Impacts•Ranges of Values ofSelect Cost and •Ranges of Cost & • Risk Taxonomy    Size, Productivity, & Other ParametersScheduleSchedule Values & Probabilities• Organizational Lessons Management Learned Person Months Confidence (Cumulative Probability)0%5%10%15%20%25%30%35%40%45%50%55%60%65%70%75%80%85%90%95%100%0 25000 50000 75000 100000Person MonthsCumulative Probability of Person Monthshttp://lean.mit.edu © 2009 Massachusetts Institute of Technology   Valerdi  2009 - 11Risk Assessment DecisionExpert COSYSMO Operation SizeDriversCostEstimatingCost Estimate withUncertainty RangesIntegrated Estimation and Risk AnalysisUser InputCostRelationshipRule-BasedRisk HeuristicsRisk Assessment- Identification- AnalysisP i iti ti Drivers  - r or za onRisk Control- Planning- Monitoringhttp://lean.mit.edu © 2009 Massachusetts Institute of Technology   Valerdi  2009 - 1212Madachy, R. & Valerdi, R., Knowledge-Based Risk Assessment for Systems Engineering: Expert COSYSMO, working paper, 2009.Initial Risk Conditions SIZERQMTARCHLSVCMIGRTRSKDOCUINSTRECUTEAMPCAPPEXPPROCSITETOOLSIZE (REQ + INTF + ALG + OPSC) 21 21 9 12 5 4 7 10 8 9 11 7 6 7Requirements Understanding 17 9 7 8 3 5 9 5 10 8 5 4 1Architecture Understanding 9 10 12 3 7 11 6 11 11 5 6 4 Level of Service Requirements (the ilities) 5 7 4 5 3 6 4 4 2 3 2Migration Complexity (legacy system considerations) 8 1 10 1 4 7 7 3 5 4Technology Risk (maturity of technology) 2 8 6 4 9 5 3 3 5Documentation match to life cycle needs 2 3 4 4 2 6 2 3Number and Diversity of Installations or Platforms 4 3 5 6 4 8 5Number of Recursive Levels in the Design 4 8 7 7 2 5Stakeholder Team Cohesion 7 9 3 8 3Personnel/team capability 12 9 8 5Personnel Experience and Continuity 10 8 3Process Capability 5 8Multisite Coordination 8Tool Support  high risk small x = 0.5; big X = 1  medium risk n = 19l i khttp://lean.mit.edu © 2009 Massachusetts Institute of Technology   Valerdi  2009 - 13 ow r sValerdi, R. & Gaffney, J., Reducing Risk and Uncertainty in COSYSMO Size and Cost Drivers: Some Techniques for Enhancing Accuracy, 5th Conference on Systems Engineering Research, Hoboken, NJ, March 2007.Risk NetworkARCH RECURisk Categories         Risk Items               Mitigation Guidance Items        Product _ PrototypeREProduct =  REPRR =  REHireARCH_PCAPPeoplePRR =  REREPeople=  RERescopeARCH_MIGRRECU PCAPPlatformPRR =  RE_RE =  Risk Exposure  REPlatform =  REhttp://lean.mit.edu © 2009 Massachusetts Institute of Technology   Valerdi  2009 - 14PRR = Potential Risk ReductionMadachy, R. & Valerdi, R., Knowledge-Based Risk Assessment for Systems Engineering: Expert COSYSMO, working paper, 2009.Expert COSYSMO Inputshttp://lean.mit.edu © 2009 Massachusetts Institute of Technology   Valerdi  2009 - 1515http://csse.usc.edu/tools/ExpertCOSYSMO.phpExpert COSYSMO Outputshttp://lean.mit.edu © 2009 Massachusetts Institute of Technology   Valerdi  2009 - 1616Outputs - Risk Mitigation Advicehttp://lean.mit.edu © 2009 Massachusetts Institute of Technology   Valerdi  2009 - 1717Risk Exposure Trends as Leading Indicators• Risk burndown tracked as mitigation actions are executed and other changes occur   http://lean.mit.edu © 2009 Massachusetts Institute of Technology   Valerdi  2009 - 1818Publicly Available Resources• U.S. General Accountability Office (http://gao.gov/)• I ti ti f th U S Cnves ga ve arm o  e . . ongress• RAND Corporation (http://rand.org/)• Public think tank• “Managing Risk in USAF Force Planning”• Defense Acquisition University (https://acc.dau.mil)O f l U S Milit U i iti• ne o  severa  . . ary n vers es• “DoD Risk Management Guidebook “http://lean.mit.edu © 2009 Massachusetts Institute of Technology   Valerdi  2009 - 19

Using Cost Models to Capture Project Risk: A Knowledge-Based Approach

http://dspace.mit.edu/bitstream/1721.1/84068/1/cps_091022_valerdi_ecalas_madrid.pdf

Using Cost Models to Capture Project Risk: A Knowledge-Based Approach

Abstract

Similar works

Full text

Available Versions

DSpace@MIT