NASA CP ."" F NASA CP -003 003 C.1 SYSTEMS RELIABILITY ISSUES --- I- 4- = 4 , 951 g "-B -111 FOR FUTURE AIRCRAFT LOAN COPY: RETURN TO AFWL TECHNICAL. LIBRARY KIRTLAND AFB, N. M. " A workshop sponsored by Ames Research Center held at Massachusetts Institute of Technology Cambridge, 'Massachusetts August 18-20, 1975 https://ntrs.nasa.gov/search.jsp?R=19770015864 2020-03-22T09:13:55+00:00Z 1 NASA Conference Publications (CP Series) contain compilations of scientif ic and tech- ~ n ical papers or transcripts arising from conferences, workshops, symposia, seminars and other professional meetings that NASA elects to publish. The text of these proceedings was reproduced from author-supplied manuscripts. NASA reviewed each paper for adherence to format, use of the Internat ional System of Units, and basic gram mat ica I accuracy. NASA CP -003 SYSTEMS RELIABILITY ISSUES FOR FUTURE AIRCRAFT A workshop sponsored by Ames Research Center held at Massachusetts Institute of Technology Cambridge, Massachuseffs August 18-20,1975 0 For sale by the National Technical Information Service Springfield, Virginia 22161 Price: $7.75 FOREWORD The trend toward increased automatic control of aircraft through the use of sophisticated on-board digital computers is clearly evidenced by the nationwide efforts on active control technology and integration of flight con- trol and vehicle configuration. Future high performance aircraft, both civil- ian and military, will be characterized by an increase in the number of sensors, aerodynamic control surfaces, and direct thrust devices used. Improved performance achieved through a reduction in weight is being sought for large transport aircraft by having active attitude and mode controls allow reduced margins of open-loop rigid and flexible stability. Great maneuver- ability with adequate ride and handltng qualities is being sought for STOL and VTOL vehicles through various methods .of providing high lift, which usually increase the number of actuators to be controlled. The increase in functions being sought for aircraft increases the number of sensors and actuators on board. The multiplicity of hardware alone induces a need for automatic processing - of data or of control - to aid the pilot. The multiplicity of choice that naturally arises also leads to increased com- putation and automation. Fortunately, modern control systems theory appears to be well suited for the design of such complex systems. When the critical- ity of survival of some of these functions is considered, however, the picture changes: there is a great increase in hardware, some increase in choice, but not yet a corresponding growth in constructive and relevant theory. Neverthe- less, the new organizational and functional questions are control and systems theoretic in nature and should be addressed by the control system community. A workshop bringing about 30 government, university, and industry researchers to encourage the necessary dialog was organized by MIT's Elec- tronic Systems Laboratory, sponsored by Ames Research Center, NASA, and held at MIT August 18-20, 1975. It consisted of formal and informal presentations ofosurveys and theoretical and practical developments and of discussions. This report contains the formal proceedings of that workshop. The first group of papers relates o implementing advanced systems. After Doolin's introduction pointing out the opportunities for control-system theoretic contributions to future flight control, Taylor reviews NASA's active control program. The review points out the performance benefits of active control as well as the stringent reliability requirements. Next Meyer describes a series of experiments planned at Ames Research Center for V/STOL (vertical or short takeoff and landing) aircraft. These experiments are to determine the efficacy of a new versatile digital-computer-oriented flight- control system for the various active control modes associated with V/STOL flight. Another approach to solving the problem of a flexible digital flight- control system is given next by Stein (which was delivered by Yore). It, too, with"a structure that can allow extra provisions for reliable operation. In a similar vein, Astrom discusses some of the research carried out at the Lund Institute in Sweden on self-tuning regulators and their experimental use in ,..A, aims, .: .... at... satisfying ..the requirements of operating throughout a flight .envelope ., .. , . iii controlling sea ships. Montgomery's paper, the final one of this group, addresses the problem of failure detection and the computational experiments-. being conducted at Langley Research Center for. feasible implementation of . maximum likelihood methods. Two informal presentations are not included in these proceedings. Berger discussed (see ref. 1) the present approach to reliable operation..as employed by McDonnell-Douglas. Requirements are satisfied by quadruple redundancy of hardware. Particular. advantage is taken of built-in test equip- ment. In some cases, this "firmware" detects degradation in hardware,perfor- mance and alerts the flight crew or maintenance accordingly. In a second informal discussion, Clark (ref. 2) recounted experiences of using observers to detect incipient failures in inertial instruments. Saridis' paper on learning systems provides a transition between the practically oriented papers above and the theoretical discussions to come. This paper, together with that of Tse and an informal discussion by Ho of team decision theory, provided an excellent view of the status of theory and heuristics in combined estimation and control. Learning is used to develop an input-output map when the intervening system is assumed to be inexplicably complex. Dual control (discussed next) deserves ,consideration when the mis- sion is sensitive to precise knowledge of parameters and the parameters in turn can best be determined if the system is perturbed. Team problems model decentralized control and decentralized information where different control- lers share the same goal. The formidable difficulties in dual control and team problems are illustrated by the paucity of definite quantitative results to date. The extensions to gain margin theorems in Athans' paper derive from con- siderations of decentralized control and robustness research currently under investigation in the Electronic Systems Laboratory. Kleinman's paper reviews the successes and promise of modern control systems theory in modeling human behavior. The text for the presentation is not available, but the figures that are reproduced herein substantially convey the information. The final paper along control systems theory lines is Wonham's review of the results in linear multivariable theory obtained both by his geometric approach and by the transfer matrix approaches of Wolovich and Rosenbrock. These proceedings conclude with two excellent reviews of current researsh efforts in nonlinear filtering and failure detection by Rhodes and Willsky. The discussion periods emphasized how very complicated software becomes as soon as an operation is automated. Lacking the great tolerance to error that a human operator provides means that the conditions of operation must be specified in extreme detail. It was recognized that the required automation would be unthinkable except for digital computation. Although hardware people did not attend the workshop, their claim to better reliability than software can give was voiced. Clearly, software- generated functional reliability will not replace all hardware replication, yet software and computation reliability is being improved. The confidence in the reliability of hardware operation is largely due to the inc'leased use iv of relatively simple software or "firmware" as in builtin test equipment. Hence the argument of hardware vs. software is really one of-degree. It is an argument that cannot be resolved definitively as yet .because we do not now have appropriate principles by which to organize the logical structure of the software. As principles of organization for the synthesis of functional reliabil- ity are missing, so are criteria and methods for its analysis. Available failure modes assessment and event tree methods, of course, provide much information.. They do not give rise to a dynamics that would enable a con- venient process for optimization or tradeoff studies.. Berger recalled that Honeywell had once begun a study that could have 1ed.to a dynamic finite- state space method of analysis (ref. 3) Apparently the work was discon- t inued . Participants on the aeronautics side had the benefit of an excellent review of the status of most control system theory. Participants on the control systems side got an excellent perspective on expectations and open problems in aeronautics. People of all backgrounds represented joined in the lively discussions. If this dialog concerning principles and methods of designing for reliability persists, it will be to the great benefit both of control-system theory and of aeronautics, and the workshop will have served its purpose. This workshop was made possible through NASA grant NSG-2076. We are grateful to Mr. R. A. Osborne, Ms. B. Peacock, and Mr. W. H. Lee for their tireless efforts in organizing the workshop and for keeping everybody happy. Michael Athans Brian F. Doolin Director, MIT Electronic Systems Laboratory NASA Ames Research Center REFERENCES 1. Hooker, D. S. et al.: Survivable Flight Control System, Final Report. AFFDL-TR-73-105, Dec. 1973. 2. Clark, R. N.; Fosth, D. C.; Walton, J. M.: Detecting Instrument Malfunc- tion in Control Systems, IEEE Trans. Aerospace Electronic Sys., vol. AES-11, no. 4, July 1975. 3. Pukite, J.; Anderson, C. G.; and Jones, K. C.: Practfcal Applications of Electromechanical Redundancy for Flight. Control Systems, Rept. AFFDL-TR-66-31, Oct. 1966, pp. 13-26. V . ~ ~~ TABLE OF CONTENTS Page FOREWORD................................ iii Michael Athans and Brian F. Doolin Massachusetts Institute of Technology and Ames Research Center Brian F. Doolin, Ames Research Center, NASA Lawrence W. Taylor, Jr., Langley Research Center, NASA George Meyer and William R. Wehrend,-Ames Research Center, NASA G. Stein, G. L. Hartmann, and R. C. Hendrick Honeywell, Inc., Minneapolis, Minnesota K. J. &tr6m, Lund Institute of Technology, Sweden INTRODUCTION.............................. . 1 APPLICATION OF ACTIVE CONTROLS TO CIVIL TRANSPORT AIRCRAFT . . . . . . . 9 NASA AMES ACTIVE CONTROL AIRCRAFT FLIGHT EXPERIMENTS (ACA) PROGRAM . . . 21 ADAPTIVE CONTROL LAWS FOR F-8 FLIGHT TEST . . . . . . . . . . . . . . . 27 SELF-TUNING REGULATORS . . . . . . . . . . . . . . . . . . . . . . . . . 51 FAILURE DETECTION AND CONTROL-SYSTEM RECONFIGURATION: PAST, PRESENT, AND FUTURE. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 FACTS AND FICTION OF LEARNING SYSTEMS . . . . . . . . . . . . . . . . . 79 STATUS OF DUAL CONTROL THEORY . . . . . . . . . . . . . . . . . . . . . 105 Raymond C. Montgomery, Langley Research Center, NASA George N. Saridis, Purdue University, Ind. Edison Tse, Stanford University, Calif. Poh Kam Wong and Michael Athans Massachusetts Institute of Technology, Mass. CLOSED-LOOP STRUCTURAL STABILITY FOR LINEAR-QUADRATIC OPTIMAL SYSTEMS. . 123 ANALYSIS OF PILOT-AIRCRAFT PERFORMANCE AND RELIABILITY VIA THE APPLICATION OF MAN-MACHINE MODELS . . . . . . . . . . . . . . . . . . 141 David L. Kleinman, University of Connecticut, Con. W. M. Wonham, University of Toronto, Canada Ian B. Rhodes, Washington University, Mo. Alan S. Willsky, Massachusetts Institute of Technology, Mass. RECENT PROGRESS IN MULTIVARIABLE CONTROL . . . . . . . . . . . . . . . . 157 A SUMMARY OF NONLINEAR FILTERING . . . . . . . . . . . . . . . . . . . . 165 A SURVEY'OF DESIGN METHODS FOR FAILURE DETECTION IN DYNAMIC SYSTEMS . . I81 LIST OF PARTICIPANTS . . . . . . . . . . . . . . . . . . . . . . . . . . 209 vii 0 - " . . . . . . . . . . . .__ - INTRODUCTION Brian F. Doolin Ames Research Center, NASA My thanks to all of you for coming to participate in what promises to be a stimulating and productive workshop. My particular thanks to our chairman, Mike Athans, for his recommendations that spawned this workshop and for his labors that brought it about. My purpose this morning is simply introductory and motivational. The workshop's title, "Systems Reliability Issues for Future Aircraft," covers a call for help in the.task of using systems theory to improve the reliability of aircraft flight. What is sought is direction in developing a unified ana- lytic approach to exploit the couplings between the many sources of informa- tion and between the many sources of control in airplanes. Much of the attention in Control Systems Engineering has focused on developing what is by now an extensive and effective theory to analyze end synthesize systems that are relatively simple structurally. The success of that theory, together with the improvements evident in the technology of digital computation, emboldens us to consider directly the challenge thrust by aeronautics. The source of the challenge is the increase in the number of control functions being demanded of many types of aircraft. Control Systems Engineering must find a rational approach to structuring these control functions o as to secure both order in their increased com- plexity and flexibility in their use. The task is important in that its suc- cess will allow other aircraft technologies to achieve their goals of perfor- mance and economy by reducing the replication of components that reliability considerations will otherwise require. For motivation, I.will point out opportunities for control that some aircraft provide, and lean on a little history to show the interplay between human and automatic operation. Many of you are intimately familiar with these topics and will discuss some of them to much greater depth. I ask your forbearance. Figures 1 to 4 are of aircraft or aircraft models that have been flown or tested at Ames. They illustrate various opportunities for aerodynamic or propulsive control. Figures 1 and 2 show some aerodynamic controls. Figure 1 shows our C-141 aircraft with its flaps, at the trailing edge of the wings, par t ia l ly deployed - t h e f l a p is s p l i t i n t o two sec t ions . The a i l e r o n s t h a t make t h e aircraft r o l l are l o c a t e d f a r t h e r a l o n g t h e t r a i l i n g edge of the wing. Besides their pr imary job of rol l ing the aircraf t or i nc reas ing its l i f t a t the ' lower f l igh t speeds , these sur faces can be used t o s h a p e the a i r l o a d s on t h e wing. Leading-edge su r faces are also used. For var ious reasons , the vertical cont ro l sur face of the t a i l i s sometimes s p l i t . Some aircraft ' a l so have small su r faces - canard surf aces - on the fo r - ward pa r t . o f t he fuse l age . An example i s the wind-tunnel model i n f i g u r e 2. They are used t o h e l p . k o n t r o 1 body bending or even t o improve a i r c r a f t con- t r o l a t low speeds. Similar controls have been instal led on special a i rcraf t t o g e n e r a t e c o n t r o l l a b l e s i d e f o r c e s . Perhaps f igure 3 por t r ays the p ropu l s ion man's dream of what a i r c r a f t should look l ike: a lift-fan'V/STOL vehicle, models of which have undergone wind-tunnel tes t ing. Each of s i x j e t engines dr ives a tu rb ine tha t sucks in air . The two rear engines have shrouds that deflect the a i r downward during f l i g h t a t low speeds and hover. They are the only engines opera t ing in normal f l i g h t . Louvers below the f ans modu la t e the f low fo r a t t i t ude con t ro l a t speeds too low f o r normal aerodynamic control. You can w e l l imagine that p i l o t e d d i g i t a l s i m u l a t i o n models have been used t o i n v e s t i g a t e v a r i o u s eng ine f a i lu re s and t h e a b i l i t y o f p i l o t s t o h a n d l e t h e f a i l u r e s . Figure 4 shows t h e Augmentor Wing Research Aircraf t . These f laps are not ordinary aerodynamic surfaces, but ducts containing a flow of cool a i r from the engines . This f low a l so en t raps a i r flowing over the wing. Perhaps you can see t h i s n o z z l e on the engine. There are four of them i n . a l l , two on each .engine. They c a n b e r o t a t e d t o d e f l e c t t h e d i r e c t h o t a i r flow of t he eng ines f rom s t r a igh t back to s t r a igh t down and even somewhat forward." The combination of direct cold engine a i r , entrapped flow over the wing, and d i r ec t ed th rus t g ives enough l i f t f o r t h e a i r c r a f t t o d e s c e n d s t e e p l y b u t gent ly a t 80 o r 90 mph. These photographs are shown t o i l l u s t r a t e t h a t a n increas ing number of aerodynamic and propulsive controls are evident on p r e s e n t a n d f u t u r e a i r c r a f t . I f t h e p i l o t were respons ib le for us ing a l l t hese con t ro l s , he would b e faced with enormous tasks of act ions and decis ions. Such regards have been the bas i c cause fo r cons ide r ing au tomat ion on a i r c ra f t . F igu re 5 summarizes t h e b a s i s q u a l i t a t i v e l y : I f t h i n g s happen too f a s t fo r t he p i lo t t o hand le , o r i f t h e r e are j u s t t o o many t h i n g s t o keep track of and make judgments on, then i t is n e c e s s a r y t o g i v e him r e l i e f . On the o the r hand , i f he on ly has minor funct ions to f iddle with over an extended per iod of time, he gets bored and calls fo r he lp . Figure 6 i l l u s t r a t e s t he t ime-sca l e spec t rum of operat ions associated w i t h f l i g h t . The middle column r e f e r s t o t h e a i r c r a f t , and that on t h e r i g h t t o t h e p i l o t . The du ra t ions o f f l i gh t ope ra t ions are from the order of hours f o r t h e c r u i s e p o r t i o n o f f l i g h t t h r o u g h m i s s i o n p h a s e s l i k e t a k e o f f and approach and landing that take minutes . Durat ion of f l ight regimes or m i s - s ion phases over laps the dura t ion of f l igh t -pa th dynamics whose time con- stants are minutes to seconds. Then a t t i t u d e dynamics takes over with short 2 periods of 1 to 2 sec. Structural effects overlap here with limit cycle flutter periods reaching do& to fractions of a second. The spectrum gives an understandable ordering of operation: each shorter period motion (except for the structural noise) acts as the driving function for the next slower motion. The human response is :characterized in the next column, which shows that the pilot's performance deterlorates on both sides of periods of the order of minutes. What could not be shown on the figure is a plot of human activity vs. operational phase - how many things are to be done or decisions to be made. Those functions depend on failure conditions (whose response times fall in the lower part of the time scale) and on the design of the control and information systems, including help given the pilot by automatic systems. The pilot wants control where he is most effective so long as the load is reasonably organized and not excessive. Be not only wants control, but the total package is most effective with him operating in his best region,. as the little, history of the early days of flight (fig. 7) illustrates (ref. 1). The first time period in the figure corresponds to the earliest days of this century when the vehicle was being developed as a tool for transporta- tion. Flights were short and limited to the best weather. As flight times increased, navigation followed roads and available maps. Since you cannot go very far or often without running into weather giving poor visibility, the next pilot aids had to do with his receiving information via radio, first voice and then also beacon signals that provided the earliest skyways. With improved vehicles and trip times, the pilot no longer had to pay con- stant attention to routine and dull tasks since autopilots now could maintain a status quo. The corresponding history of vehicle stabilization and control is con- veyed in the second part of figure 7. A major contribution of the Wright brothers was to relax the inherent stability that earlier designers were building into their vehicles. When the pilot was allowed to assume some of the stabilization duties, path control became reasonable. Stabilization required too much agility and attention - it was exhausting. So Sperry relieved the pilot of some of this task with his gyrostabilizer in 1913. He improved stabilization while retaining the improved configuration technology. This mechanism was done away with once the existence of a tradeoff between stability and control was understood and airframe designers built compromises into the configuration. The compromises were haphazard until the mid-thirties when researchers conceived of handling qualities as a design guide and began to regularize design conditions. Figure 8 brings us up to date. Our present highly developed ailway system that enabled long-distance flight began with the introduction.of simple ground beacons. Improvements in onboard instrumentation and computa- tion have recently relaxed the need for strict following of the one-dimensional airway system. Adding one degree of freedom in flight - two-dimensional, or 3 area, navigation - is becoming commonplace. In fact, at Ames,' four-dimensional '' feasibility.has been demonstrated. , \ -4 IMX (ROlt XSMPG) \ c c s - d - rnox SMPG I O O ' I ' I I I I I I I I I I I ' I I I I ' 50 O0 I50 200 Total seat miles per gallon . . Figure 9.- Return on investment and total seat miles per gallon for various designs. 18 '0°[ Modc 9 ? 360 8 Mode6 rad/sec 0 2.8 2.4 2.0 1.6 1.2 .8 .4 0 -.4 -.E ' I I I I I I 0 Figure 10.- Dynamic pressure locus using state space aeroelastic model. . t - . Maneuver load control Feedforward, minimize primarily wing weight by reducing wing bending moment due to 2 . 5 g maneuver Gust load alleviation, Measure gust, feedforward, desensitize the ride control aircraft response through feedback designed I using reduced state, quadratic cost algorithl Reduced static stability Optimal reduced state feedback using qua- dratic cost function based on pilot ratings Flutter suppression Include aeroelastic modes unsteady structural mode damping aerodynamics in models used for control synthesis to assure stability Consider various levels of complexity and at several flight and loading conditions. Use linearized design criterion to assess candi- date system configurations . ~~ " _ ~ Figure 11.- Approach taken in synthesizing active control laws. 19 STATE VARIABLES FOR LONGITUDINAL, SYMMETRICAL MODES: No. of states Rigid body modes (a , q , 0 , V) 4 Aeroelastic modes (9 second order modes) 18 Unsteady flow (+2 each mode, control) 26 Actuator dynamics 8 Sensor dynamics 10 Filters, compensators - 8 TOTAL 74 If all modes are coupled because of engine momen- tum or malfunctioning actuator, the total states num- ber is about 150. Conclusion: Need reduced state, insensitive control, 'synthesis techniques, and algorithms. Figure 12.- The problem of too many state variables. noise, uncertainties Turbulence, Proposed cost function J = E{o/r(Yp-YR)T Q(Yp-YR)dt} K ~ . KR=ARG MIN{J} Pilot input, desired response Feedback Figure 1 3 . - Control law synthesis using reduced state feedback. Figure 14.- Comparison of pilot opin- ion boundaries. 20 NASA AMEs ACTIVE CONTROL AIRCRAFT FLIGHT EXPERIMJ3NTS (ACA) PROGRAM George Meyer and William R. Wehrend Ames Research Center, NASA The objectives of the Ames ACA program are to develop the active control technology (ACT) for short-haul aircraft, to evaluate existing methods, to develop new techniques, and.to demonstrate the readiness of the technology in operational environment. Two concepts are basic to ACT: integrated aircraft design and integrated flight-control-system design. These two concepts form the basis of the Ames program. Studies have shown that significant reductions in induced drag and 'weight, improvement in passenger comfort, and reduction of flight hazards can be achieved with ACT. These benefits are possible due to (a) a reduction in the size of stabilizing surfaces, with stability provided by dynamically control- ling movable surfaces (reduced static stability design) rather than statically with larger fixed surfaces as in conventional design; (b) reductions in struc- tural strength requirements by applying maneuver load and gust load allevia- tion and flutter mode control; ( c ) improvement of ride by a ride quality control system; and (d) reduction in the occurrence of inadvertent flight hazard through automatic limitation of flight condition. The benefits are maximized when ACT is factored in the design and integrated with aerodynamics, structures, and propulsion early in the design cycle of the aircraft. But, before such an ACT-configured aircraft can be designed with confidence, existing mathematical modeling techniques must be evaluated. In comparison with the conventional flight-control-system design, the design of such ACT systems as flutter mode control and reduced static stability requires signif- icantly more accurate mathematical models of the process to be controlled. Of particular concern are the models of unsteady aerodynamics of the basic aircraft, control surfaces, and sensors in the flow field; models of flexible structures of the airframe, control surfaces, effects of flexibility on sensors and actuator linkages, and aeroelastic coupling; and the models of propulsion systems, including engine power dynamics and control, and power-aerodynamics coupling, the latter being particularly significant in powered lift configura- tions. Therefore, a part of the Ames program is directed at the evaluation of the complete aircraft modeling methodology. The process resulting in an optimum ACT-configured design is expected to require many iterations on the aircraft geometry, including size and location of control surfaces. At each such iteration, the effects of the control system must be determined, thus requiring a mathematical model of the controlled process. Since-the availability of efficient algorithms for transforming aircraft configurations into mathematical models for control- system design purposes is implicit in the ACT design approach, particular attention is given in the Ames program to digital computation methods based on potential flow theory (ref. 1). Experiments are being planned to evaluate 21 such methods by comparison with flight-test data. The methodology and algo- rithms of system identification are expected to play a rominent role in extracting model parameters from flight-test data. The second.basic concept of Active Control Technology is that maximum benefit is obtained from automatics when all functions are blended into a single, fully integrated flight-control system. A good working hypothesis is . that the next generation of flight-control systems will provide four sets of functions: (1) ACT modes, including gust load alleviation, maneuver load control, flutter mode control, and ride quality control, all of which improve aircraft performance without adversely affecting the pilot handling qualities; (2) stability and. control augmentaion system (SCAS), a set that provides an interface between the pilot and the basic aircraft dynamics, such as the stabilization of an aircraft with reduced static stability; (3) full-flight envelope autopilot, a set whose elements provide the pilot with a variable degree of automation, from SCAS (requiring maximum pilot participation) to completely automatic four-dimensional trajectory tracking, automatic envelope limiting, and configuration management; and ( 4 ) automatic fault regulator, which provides automatic fault detection, fault identification, and system reconfiguration to bypass the fault. Before such an integrated flight-control system can be implemented with confidence, several system-theoretic problems of varying degrees of difficulty must be solved. Powerful concepts (ref. 2) and computerized algorithms exist for the design of linear systems and can often be applied to the design of ACT modes and SCAS, which are primarily perturbation regulators so that linearity and even time invariance are reasonably valid. However, as already noted, there may be significant uncertainties in the actual plant parameters and plant structure. Methods for the reduction of input-output sensitivity to perturbations in the plant are needed. Linear methods are also helpful for the design of autopilots, but to a-much lesser degree. Increased trajectory tracking accuracy requirements of projected STOL and VTOL operations, ,coupled with inherently nonlinear force and moment generation processes in powered- lift aircraft, make it very difficult to cover the operational envelope of such aircraft with a set of linear models and control laws linked by a gain schedule. Alternate methods for coping with the nonlinearities must be developed. The wide range of lift and drag coefficients required in short-haul operation is achieved by in-flight modification of the aircraft configuration (ref. 3 ) . Inside the flight envelope there are, normally, infinit.ely many combinations of controls which result in identical lift and drag coefficients - that is, given values of lift and drag define a surface rather than a single point in the control space. The remaining freedom can be used to optimize system performance and increase both safety margin and aircraft maneuverability. The methodology for automatic configuration optimization and envelope limiting and for the integration within the autopilot f ACT modes and SCAS must be developed. A part of the Ames program is directed at the development and evaluation of such a methodology. 22 Normal operation of all system components was tachly assumed in the discussion thus far. When the possibility of component failures is introduced into the synthesis, the available synthesis methodology loses most of its power. There is a pressing need for major advances in this area, particularly in the development of a theoretical foundation for the design of fault regula- tors, which detect and identify failures and reconfigure the control logic into the next safest mode of operation. The need for fault regulators is evident in the case of advanced flight-control systems with high authority, flight critical functions. However, it would seem that the majority of control systems in the future, regardless of the particular field of application, will be controlling with high, if not complete, authority complex, critical systems and therefore will have to regulate faults automatically. Thus, advances to be made in flight-control systems will very likely have wide-ranging applications. A part of the Ames program is directed at the development and evaluation of the fault regulation methodology; The scope is limited to (1) synthesis of the reconfiguration logic, given a complete description of every failure from a given set; (2) sensor subsystems that provide, through hardware redundancy and functional redundancy, state estimates and their accuracy including extreme failures; and ( 3 ) actuator subsystems that provide, through hardware redundancy, estimates of command execution accuracy. Other topics, such as hardware failures in the flight computer, are excluded from the program. All three parts of the Ames program, namely, modeling methodology, integrated flight-control-system design assuming no component failures, and fault regulation methodology will be developed and evaluated by generaliza- tion from particular applications to specific research aircraft. .In the first phase of the program, research will be focused on the DHC-6 (Twin Otter) aircraft - an example of a low wing loading configuration with conventional lift. The aircraft has been modified to include a set of spoilers for direct lift, drag, and roll control. In addition, the aircraft has been equipped with a STOLAND system, including a flight computer with access to a large set of sensors and actuators. In the second phase, the research will be focused on powered-lift configurations similarly equipped with functionally powerful hardware. Candidates for that phase are XV-15 (tilt-rotor) research aircraft and possibly either YC-14 or YC-15 (advanced medium STOL transport). The first major objective of the program is to formulate a complete overall logical structure for automatic flight-control systems that can (1) admit any combination of ACT modes; (2) provide accurate, automatic cou- pling to air traffic control, which may select any trajectory from a large set of complex trajectories; ( 3 ) provide a spectrum of modes of pilot- automatic interaction; (4) accept a variable sensor set of variable accuracy and a variable actuator set; (5) accept a highly nonlinear plant with function- ally redundant controls; and (6) provide a framework for the design of fault regulators. The logical structure being developed at Ames is described briefly here. (A more detailed description is given in ref. 4.) The structure of the control logic is hierarchical in the sense that there is a nesting of subsystems. At each level in the hierarchy, relatively 23 simple commands are accepted from above and transformed into more detailed commands downward. The command is based on a relatively simple input-output model of lower levels, and the levels have a degree of autonomy. Figure 1 shows the control logic as viewed from the level of the trajectory control system. At the top of the hierarchy is the air traffic control (suitably generalized to include the pilot) which requests a rajectory that may or may not be executable from the existing state of the aircraft. The trajectory command generator transforms this request into an executable open-loop acceleration command and a corresponding four-dimensional. trajectory that may, depending on the'original request, have ignorable parameters. The accelera- tion command is sent downward in the hierarchy to the trajectory acceleration controller, which is expected to execute that command in accordance with a relatively simple input-output relation and error bounds. Errors arising from wind turbulence, uncertainties in plant parameters, intentional oversimplifi- cation of the dynamics, etc., are controlled by means of the trajectory perturbation controller in which estimated aircraft position, velocity, and acceleration are compared with the command, and a corrective acceleration generated and added to the open-loop command, thereby closing the loop around the errors. Figure 2 shows the structure of the trajectory acceleration controller for the case in which lift is controlled by angle of attack and side force is controlled by the roll angle, so that aircraft attitude becomes an indirect control variable. The closed-loop acceleration command enters the force trimmap, an algebraic (static) representation of the total (aerodynamic and propulsive) force generation process. At this level, configuration management and envelope limiting take place. The output is the attitude and remaining controls which, in the absence of errors, would trim the aircraft to the required acceleration. The trim attitude, possibly time-varying, is sent to the attitude control syst'em, which is similar to the trajectory control system. The trim attitude is transformed in the rotation command generator into an executable open-loop angular acceleration command and a corresponding angular velocity and attitude time histories. The angular acceleration command is sent down the hierarchy to the angular acceleration controller, which is expected to execute the command with a relatively simple input-output relation and error bounds. Attitude errors are controlled by means of the attitude perturbation controller in which estimates of aircraft attitude angular veloc- ity and angular acceleration are compared with the command, and a corrective angular acceleration is added to the command, thereby closing the loop around the errors. In figure 2, the angular acceleration controller is represented by the moment trimmap. The control surface servos, at the bottom of the hierarchy, are not shown. The proposed logical structure has several advantages, among which are the following. It is applicable to a large class of aircraft. The particu- lars of the aircraft affect primarily the force and moment trimmaps. The design approach is nearly algorithmic. Performance, sensor accuracy, and actuator accuracy enter as independent variables that may be varied over a wide range. There is an effective tradeof f between performance and flight computer requirements. Because the control logic is hierarchical, problems 24 associated-with the design of fault regulators are considerably simplified. A flight-test evaluation of the control logic is expected to occur in the summer of 1976. REFERENCES 1. Anon. : A Method for Predicting the Stability and Control Characteristics of Control Configured Vehicles, AFFDL-TR-74-91, Nov. 1974. 2. Anon.: Special Issue on Linear-Quadratlc-Gaussian Estimation and Control Problem, IEEE Trans. Automatic Control, vol. AC-16, no. 6, Dec. 1971. 3. May, F. W.; and Bean G. E.: Aerodynamic Design of the Boeing YC-14 Advanced Medium STOL Transport (AMST), AIAA Paper 75-1017, Aug. 1975. 4. Meyer, G.; and Cicolani, L.: A Formal Structure for Advanced Flight- Control Systems, NASA TN.D-7940, 1975. 25 Estimated aircraft Sensors I 9 Aircraft I I Open-loop acceleration command Figure 1.- Trajectory control system. Closed-loop acceleration commandw attitude rotation controlter trimmop Force Trim 1*- Rotation Executable Rotation command * perturbation Open-loop angular acceleration command command Flap, throttle, DLC Figure 2 . - Trajectory acceleration controller. 26 ADAPTIVE CONTROL LAWS FOR F-8 FLIGHT TEST* G. Stein, G. L. Hartmann, and R. C. Hendrick Honeywell, Inc. Minneapolis, Minnesota 55413 SUMMARY This paper describes an adaptive flight-control-system design for NASA's F-8 digital fly-by-wire research aircraft. This design implements an explicit parallel maximum likelihood identification algorithm to estimate key aircraft parameters. The estimates are used to compute gains in simplified quadratic- optimal command augmentation control laws. Design details for the control laws and identifier are presented, and performance evaluation results from NASA Langley's F-8 Simulator are summarized. SECTION I - INTRODUCTION NASA is presently conducting research in digital fly-by-wire flight- control technology. In the first phase of this research, an F-8C test air- craft was modified to use an onboard Apollo guidance computer for flight- control calculations. The computer was connected directly to the aircraft's electrohydraulic actuation systems, with no mechanical linkages remaining between pilot stick and control surfaces. This configuration was flight tested to demonstrate the practicality of digital fly-by-wire control in an actual test vehicle (ref. 1). For the second phase of the research program, the test aircraft is being outfitted with triply redundant: AP-101 computers, which are again connected directly to (improved) electrohydraulic actuation systems. This new config- uration serves as a test bed for studies and demonstrations of redundancy con- cepts to enhance reliability and advanced control laws to improve the perfor- mance and/or overall effectiveness of the aircraft. For control law studies, in particular, the new computer's speed, memory size, and floating-point capability removes most of the complexity constraints imposed by more typical flight-control machines. To date, two sets of control laws have been scheduled for flight experi- mentation in the new F-8 configuration - a "CCV package'' and an "adaptive package." The first package is described in reference 2. It provides basic *This work was supported by NASA Langley Research Center under Contracts NAS1-12680 and NAS1-13383. 27 command augmentation functions, outer loops, and control modes for various control-configured-vehicle (CCV) concepts applicable to fighter aircraft. The second control law package is described here with further details in refer- ence 3. It consists of the same command augmentation functions found in the. CCV package, but adds explicit on-line maximum likelihood identification-capa- bility for adaptive control. In addition to these packages, research has also proceeded on several other candidate adaptive algorithms for the F-8. .. (refs. 4-6) and on sensor fault detection and isolation algorithms (refs. 7. and 8). Some of these will reach flight experimentation later in the program. The purpose of this paper is to describe the contents of the Adaptive Control Law package. We begin the description (section 2) with a discussion of general design objectives-and ground rules. We then introduce the overall adaptive system structure and discuss the design of its major components - the control law (section 3 . 2 ) and the explicit identifier (section 3 . 3 ) , followed by simulator performance evaluations in section 4 a d by a summary of remain- ing issues and plans in section 5. Of necessity, many of the modeling, design, and evaluation details cannot be covered here. For these details, the interested reader is referred to references 2 and 3. SECTION I1 - OBJECTIVES AND GROUND RULES NASA's primary reasons for studying adaptive control laws for the F-8 aircraft are to identify potential performance and system effectiveness bene- fits offered by modem adaptation/identification concepts and to demonstrate these in an actual digital flight-control environment. Hence, the aim of the design effort was not to develop new theoretical procedures or algorithms but to turn existing concepts into flightworthy control laws for the specific test aircraft. Of course, the concepts and design processes should be general enough to apply to other aircraft as well. This is important because the F-8C itself is not a state-of-the-art fighter, nor is it difficult to control with standard nonadaptive techniques. Other more specific groundrules imposed on the design include the following. Inputs The adaptive system must operate in the presence of normal pilot inputs and also when such inputs are absent. Any test signals required for the latter case must be small enough not to interfere with the aircraft's mission. This generally means that test input normal accelerations, as sensed at the pilot station, should be below 0.02-0.03 g rms, and lateral acceleration should be even lower in the range of 0.01 g. This ground rule establishes a crucial distinction between identifiers designed for operational adaptive con- trols and those designed for postflight-test data-processing applications (refs. 9 and 10). In the latter case, test inputs are deliberately large and often optimized for identification accuracy (refs. 11 and 12). For opera- tional adaptive controls, the emphasis must be to make these inputs small and hopefully unnoticeable. 28 Sensors The system was constrained to operate with the aircraft's existing sur- face .(servo) position transducers and inertial sensors (rate gyros and accel- erometers), but without attitude measurements and without air data measure- ments (dynamic pressure, velocity, angle of attack). These limitations are imposed by the sensor complement available on the test aircraft and, in the case of air data, by the philosophical bent of the authors. 'We believe that removal of flight-critical air data measurements is one of the most tangible benefits adaptive controls can offer for aircraft (like the F-8) whose basic performance requirements are readily satisfied with air-data-scheduled control laws. 1 Computer Capacity and Sample Rate Since control law calculations typically consume only a small fraction of the total computational load in a flight computer (the bulk is I/O, self test, mode and redundancy control), the adaptive law's goals are restricted to 25 percent of the available frame time of the AP-101 and to a ..!'reasonable" allocation of memory. The sample rate of the computer was prespecified to b 50 Hz. This gives 5 msec real time per sample in which to complete all con- trol and identification calculations. The sample rate is also high enough to produce no substantial differences between direct digital design (discrete- time control laws designed for discrete-time models) and continuous time design with after-the-fact discretization. Control Surfaces and Actuators The standard elevator, aileron, and rudder surfaces were available for adaptive control. Leading-edge and trailing-edge flaps were assumed to remain undeflected or to folPow their open-loop scheduled minimum drag positions. Actuation systems for these surfaces were represented by models incorporating projected characteristics of the improved systems being installed on the test aircraft . SECTION I11 - ADAPTIVE SYSTEM DESIGN The overall structure chosen for the F-8's adaptive control system is based on explicit separation of identification and control (fig. 1). A stan- dard gain-scheduled control law was designed for the aircraft's complete flight envelope and the scheduling functions (gain adjustment rules) are 'In flight-critical digital fly-by-wire control systems , reliability requirements call for triply redundant or even quadredundant air data sensors. Aside from the cost and complication of these multiple sensors, suitable probe locations impose difficult compromises between data quality, channel tracking, arid dispersion to avoid common hazards. 29 supplied from a separate aircraft identifier. While such a priori separation is known to be theoretically imperfect in the dual-control sense (ref. 13), it is ideally suited, both historically and in terms of p actical features of the solution, to the flight-control problem. This will become evident as we describe aircraft models and features of the separate controller and identi- fier designs. Aircraft Models Mathematical models for the F-8C were developed from general six-degree- of-freedom nonlinear equations of motion together with geometric data, mass properties, and aerodynamic data tables (ref. 14). These equations were linearized numerically at 25 flight conditions throughout the operating enve- lope of the aircraft. Each flight condition is defined by the nominal alti- tude, Mach number, weight, geometric configuration, and load factor which characterize the models obtained by the linearization, that is, Q where, x denotes 12 aircraft states, 6 denotes 3 control surface deflec- tions, w denotes wind gust disturbances, and y denotes assorted outputs we wish to measure and/or control. The matrices F, G1, G2, G3, H, and D can be treated as continuous functions of the slowly changing nominals c, with values known at discrete points. Equation (1) is not usually used directly for control design. It is first partitioned into longitudinal and lateral-directional degrees of free- dom, and each partition is separated into dynamic modes that dominate the fre- quency range of interest in the design task (ref. 15). The resulting reduced models are then augmented with actuator, sensor, gust, and flexure character- istics as necessary to make up the complete design model. For the F-8C adaptive system, the control objective was to provide basic stability and command augmentation functions. The reduced models that suffice for this objective consist of second-order short-period dynamics plus a first- order power actuator for the longitudinal axes. A fourth-order dutch roll, roll subsidence, and spiral mode model suffices for the lateral-directional axes. The longitudinal model is summarized in figure 2 and table I, and the lateral-directional model can be found in reference 2. Figure 2 and table I provide model equations and approximate functional descriptions for the equation coefficients, respectively. The latter func- tions were derived by plotting numerical values from the 25 flight conditions against two dominant nominal flight condition parameters, dynamic pressure (i), and Mach number. The trends in these plots were approximated by simple functions, and the scatter was represented by additional small-perturbation parameters with bounded uncertainties. Hence all coefficients have the form 30 p i = f (q, Mach, ci) o r t h e form Pi - g(M60, Mach, c ) - i where equation ( 3 ) is obtained by subst i tut ing the very good l i n e a r r e l a t i o n - sh ip between su r face e f f ec t iveness (M60) and 7 j i n to equa t ion (2) . The poin t of inc luding these model d a t a h e r e is t o show how strongly the parameter Mso (or equiva len t ly {) dominates the F-8C's dynamics. Consequently, t h i s same parameter a lso plays a dominant r o l e i n t h e c o n t r o l l a w and i d e n t i f i e r d e s i g n s to fo l low. . : Control Law Design Both the long i tud ina l and l a t e r a l - d i r e c t i o n a l c o n t r o l l a w s were designed . . with modern LQG techniques (ref . 16) appropriately tempered with establ ished classical design requirements . In each case, a quadratic optimization problem was formulated and solved a t selected f l ight condi t ions under assumptions of noiseless , ful l -s ta te , cont inuous- t ime feedback. The r e su l t i ng con t ro l ga ins and dynamics were then s impl i f ied and approximated as functions of measurable nominal-f l lght condi t ion parameters to provide a standard gain-scheduled con- t r o l l a w . We descr ibe t h i s des ign process in more d e t a i l below fo r t he long i - t u d i n a l c o n t r o l P a ~ 7 only. The l a t e r a l - d i r e c t i o n a l d e t a i l s are again provided in re ference 2. Longitudinal Optimization problem- To gain ready acceptance as f l i g h t - worthy, the F-8C's l ong i tud ina l con t ro l l a w s were d e s i g n e d t o s a t i s f y t h e following l i s t of classical performance specifications: 1. Command augmentation functions a. C* s tep response b. S t i c k g r a d i e n t t o m e e t MIL-F-8785 ( r e f . 17 ) c. N e u t r a l s p e e d s t a b i l i t y 2. Regulator function Reasonably damped gust responses (short-period damping r a t i o > 0.3) 3 . Tole rance fo r unce r t a in t i e s a. Classical loop gain and phase margins (GM > 6 db, 4 > 30') b. Loop a t t enua t ion a t high frequencies (< -20 db a t 8 Hz) These requirements are obv ious ly no t i n the quadra t i c cos t func t ion form required by LQG theory. Hence an impor t an t s t ep in the des ign p rocess w a s t o r e i n t e r p r e t them i n more s u i t a b l e terms. 31 Consider the command augmentation requirements first. Item (la) in this group is a dynamic response specification on C*, a linear combination of pitch rate and normal acceleration: C* = n, + Vcoq , Vco = 324 In response to step commands from the pilot's control stick, this variable must stay within a specified response envelope (ref. 18) whose center is closely approximated by the step response of a second-order linear model with w Y 7 rad/sec, 5 = 0.9. This requirement was incorporated in the LQG frame- work by the common technique of appending an explicit "command model" and penalizing errors between it and the aircraft (refs. 19 and 20). Specifica- tion item (lb) was then used to set the DC gain (acceleration per unit stick force) of the model, and item (IC) was satisfied by penalizing an appended integral of the model-following error as well as the error itself. The latter approach is motivated by the fact that "neutral speed stability" is actually imposed to provide command insensitivity to trim changes. A pilot should not be obligated to readjust trim or to provide compensating stick force inputs as the aircraft changes speed (and hence as it changes the trim disturbance terms Mo and g/V in fig. 2)i A bit of integral control action provides this desired insensitivity. In addition to the command model and integrator appended to satisfy items (la) to (IC) , it was also necessary to append a low-pass filter on the normal acceleration signal used to construct C*. This filter provides high- frequency attenuation, item (3b), which is required to assure adequate stabil- ity margins in the presence of uncertain servo characteristics and unmodelled flexure dynamics of the airframe. Loop transfer functions of trial designs without such filters tended to drop off too slowly to meet the specification. Slow roll-off is, of course, a general property of quadratic-optimal control laws, formally recognized but frequently forgotten (ref. 21) The complete optimization problem with these appended states is illus- trated in figure 3. A s shown, the criterion function was taken as a weighted sum of four quadratic terms - model-following error (C* - C*,), integrated model-following error j(C* - C*,)dt , elevator rate (d/dt) 6, , and elevator command 6c. Quadratic weights were then selected iteratively until the resulting control law satisfied all performance specifications. Representa- tive weights and gains are summarized in table 11. ControZ Z a w simplification- A s indicated earlier, the quadratic optimal control law was next simplified and approximated as a function of nominal flight condition parameters. While several numerical algorithms have been developed to aid in these reductions (refs. 20, 22, and 23), the F-8C's longi- tudinal control law was compact enough to be reduced by standard block diagram manipulations, cancellation of nearly equal poles and zeros, and removal of low-gain paths. The result is shown in figure 4 . It consists of a basic proport ional-plus- integral C* feedback loop, modified slightly by the 2Neutral speed stability is desirable during all flight phases except power approach. For the latter, the integral action must be switched out. 32 appended lag network on measured acce le ra t ion and by a lead-lag network on p i t c h rate. The p i l o t ' s s t i c k f o r c e commands th i s loop through a feedforward f i l t e r t h a t c o n s i s t s of t h e . o r i g i n a 1 C* model with al tered numerator dynam- ics. A l l pa rame te r s i n the ; con t ro l l a w are independent of f l ight condi t ion except the loop gain, G p , h ich varies inverse ly wi th sur face e f fec t iveness ( o r dynamic pressure) . It ecomes des i r ab le to a ,d jus t , ' o the r 5a ins on ly i f t he a i r c r a f t is f lown with del iberately re laxed stat$c s t a b i l i t y . Foro such con- d i t i o n s , t h e d o t t e d l i n e p a t h i n f i g u r e 4 must be added t o t h e c o n t r o l l a w , with gain GRSS ad jus ted as a func t ion of M60 and M, (or and Mach). Numerical values for the gain Gc* are shown i n f i g u r e 5, where they are a l s o compared wi th the c r i t i ca l ga in a t which the ( l inear ) loop reaches ins ta - b i l i t y . It is clear from these curves tha t the F-8C is a ve ry to l e ran t a i r c r a f t . Gc* e a s i l y achieves 6 db of m a r g i n o v e r t h e e n t i r e f l i g h t enve- lope, and even a f ixed ga in sys tem could f ly the a i rc raf t adequate ly a t a l l but the lowes t dynamic pressure condi t ions. This fact deserves emphasis because i t complicates evaluat ions of an adapt ive sys tem des igned for th i s a i rc raf t . Tota l c losed- loop per formance does not provide a sensi t ive measure of adap t ive capab i l i t y , and o t h e r less d i r e c t cr i ter ia are r e q u i r e d t o evaluate the system. Lateral-directional control laws- Although d e t a i l s are l e f t t o r e f e r - ences 2 and 3, t he l a t e ra l -d i r ec t iona l con t ro l des ign p rocess r e su l t ed in s t ruc tu res ve ry similar t o f i g u r e 4 . The r o l l c h a n n e l i n f i g u r e 6 is a simple rol l - ra te feedback loop commanded through a f i r s t -o rde r model, whi le the yaw channe l i n the f igu re combines compensated yaw rate, r o l l rate, la teral accel- e r a t i o n , and a i le ron pos i t ion feedbacks wi th d i rec t rudder peda l commands. Both loops have one gain that changes with f l ight condi t ion in inverse rela- t i o n t o dynamic pressure and hence in d i rec t p ropor t ion to ga in GC*. I d e n t i f i e r Design In nonadapt ive f l ight-control implementat ions, var iable gains l ike those i n f i g u r e s 5 and 6 are usually adjusted with external measurements obtained f r o m t h e a i r c r a f t ' s a i r data system. This approach w i l l be used i n t h e F-8C's f i r s t f l i g h t tests of t h e CCV Control Law Package planned f o r mid-1976. A s already noted, however, a i r data systems can be troublesome components i n t h e overa l l f l igh t -cont ro l mechaniza t ion , and i t would be ve ry des i r ab le no t t o need them. The i d e n t i f i e r d e s i g n is intended to accomplish this end. Its primary function i s to provide an adequate sur face e f fec t iveness estimate (56,) t o s u b s t i t u t e f o r measured q. It a l s o estimates several oth$r quanti- t ies - p i t ch ing moment due t o a n g l e o f a t t a c k (e,) , t rue a i r speed (V) , and t o t a l a n g l e o f attack (G). These are sometimes needed to schedu le more com- p l ex con t ro l l a w s t h a n t h o s e t h a t s u f f i c e f o r t h e F-8. - 3This is a p o t e n t i a l CCV feature achieved by l a r g e a f t d i s p l a c e m e n t s of t h e c.g. Its b e n e f i t s are reduced t r i m drag a t the expense of basic a i r f rame s t a b i l i t y . I n t h e model c o e f f i c i e n t s o f t a b l e I, i t means rep lac ing wi th the func t ions c2Mdo and (0.92 + c2)Mgo f o r Mach < 1 and Mach > 1, respec- t ively. There are n o p l a n s t o f l y t h e F-8C with these modif icat ions. 33 0 Several identification schemes were examined to perform these functions, including approximated (second-order) nonlinear filters, Lyapunov-stable model trackers, maximum likelihood estimation (MLE), and self-excited limit cycle systems. Trial designs were actually carried out for the latter three (ref. 3) . The MLE procedure was selected out of these primarily because it' offers the greatest capability in terms of the numbers of pa ameters identi-, fied and their accuracy. Details of the MLE design are discussed below. ' . . Identification accuracy- Before proceeding with the identifier descrip- tion, we present some evaluations of theoretical limits on the accuracy to which parameters of the F-8C can be identified under the ground rules imposed in section 11. These evaluations provide justification for most of the design decisions made for the MLE identifier. The accuracy evaluations were per- formed for the identification problem illustrated in figure 7. The aircraft (as modeled in fig. 1) with a simple pitch rate feedback control loop was assumed to be excited by small test signals, wind gusts, and sensor noise. Discrete pitch rate, normal acceleration, and surface position observations were assumed available to identify unknown dynamic coefficients (as parameter- ized in table I), gust level, trim disturbances., and initial conditions. It is well known that the achievable identification accuracy for this problem is limited by the following lower bound (ref. 2 4 ) : where vky k = 1,2,..., is the sequence of residuals generated by a Kalman filter designed for the true parameter values; Bk, k = J - , 2 , m . . , is the covariance matrix of that sequence; Po is the covariance matrix of initial parameter'uncertainties; and T is the data length. The situation in figure 7 was simulated, and sample averages of the matrix M were computed for various combinations of data length, te'st signal, gust, and sensor noise conditions. The principal result of these calculations is shown in table 111, which compared square roots of selected diagonal ele- ments of Po with corresponding elements of M(c,T). The comparisons4 are made for T = 5 and T = 10 sec and correspond to the following environmen- tal conditions: Flight condition: 20,000 ft, Mach = 0.67, 4 = 305 lb/ft2 Test signal: Square wave at short-period frequency, 0.015 g RMS acceleration Gust level: 1 ft/sec la Sensor noise: 0 = O.O1S0/sec, an, = 0.02 g g 'Because parameters can change significantly over 10 sec, it is not meaningful to look at longer data lengths. 34 Table I11 shows that we can expect only limited identification capability under these conditions. Our a pr ior i knowledge improves for only two dynamic coefficients, Ms0 and c2 (the small perturbation of Ma), and for (Tug, trim, and initial conditions (not shown). Accuracies are limited to approximately 10 percent for both M60 and M,. Fortunately, these are adequate in most , . gain-scheduling applications. Using similar analyses for other test condi- tions, the following additional conclusions were established: Accuracy limits: M60 and M, errors are proportional to true parameter values. Accuracies fall into the same 10-percent range for all flight , conditions. Reduced parameter identification: Identifiers desigped to find only a few parameters (as few as Mgo, trim, and ICs) do almost as well as identi- fiers designed to find all parameters, even in the presence of errors in the ? unrecognized parameters; However, trim disturbances and initial angle of attack cannot be deleted without inducing major errors in M60. . Test signals: Randomized test signals are superior to cyclic ones for reduced parameter identification. Lateral-directional identification: Identifiers for the lateral- directional axes reduce a pr ior i uncertainties for only two parameters, 'q and NB (yawing moment due to sideslip). Accuracies are comparable to longitudinal identification. MLE ident i f ier s tructure- In view of the above identifiability limita- tions for the F-8C aircraft, it was decided to utilize a longitudinal-axis identifier only with a reduced parameter set of three components - M60, c2, and c3 (small perturbation on velocity) .6 It was also decided to avoid the iterative nature of conventional MLE algorithms (refs. 9 and 10) by adopting a parallel approach to likelihood minimization. This approach is illustrated in figure 8. It consists of a bank of Kalman filters, each generating suffi- cient statistics for the conditional probability distribution of the measure- ments given a different value for the unknowns, that is, T/At 1 = 1,2, ..., N ) 51t is important to recognize that the limitations cited above are F-8C specific and are caused primarily by the low test signal ground rules imposed in section 11. 6Though only weakly identifiable, c3 was carried along for flight experiments with larger test signals. 7N[x,B] denotes the usual multivariate normal distribution with mean x and covariance matrix B. 35 where ?k(3) and Bk(9) are predicted measurements and residual covariances, respectively, from the ith Kalman filter designed for c = si. Likelihood functions (L = -Rn p) are then computed for each filter channel, the minimum likelihood channel is selected, and a single (approximate) Newton-Raphson step is taken from there to predict the parameter value, that is, a i = 1,2, ..., N (8) '" 0 where M(c,T) is the approximate second-partials matrix defined in equa- tion (5). This parallel minimization approach is a modified form of several procedures published in the literature (refs. 25-27). Its primary advantages for onboard use are recursiveness and fixed structure. The filters and sensi- tivity calculations implicit in equation (9) process measurement samples as they appear and use a fixed set of repetitive program instructions. No data storage and conditional iterative reprocessing are required. They also remain fixed in parameter space, requiring no onboard calculation of new plant models, Kalman filter gains, and their associated sensitivities. These are all desirable features for real-time onboard computer code. Design specif ics- A specific implementation of figure 8 involves the fol- lowing design choices and issues: * Identification models Channel selection Kalman filter design parameters Adjustments for practical situations not handled by the basic theoretical approach These items are discussed below for the F-8C's specific design. Ident i f icat ion models- Each Kalman filter channel in the F-8C design uses a discretized version of the basic aircraft model in figure 1 with the actu- ator state deleted (6e is assumed measurable) and with the two trim distur- bances appended as Brownian motion "trim states." These trim states eliminate the need (identified above) to treat the disturbances as explicit unknown parameters. The model's dynamic coefficients are parameterized as shown in table I, but cl, c4, and c5 are assumed to be zero. Only c2, c3, and Mgo are recognized as unknowns. 36 Chunnel selection- The three unknowns generate a parameter space illus- trated in figure 9,. Two axes are defined by Mgo and c2 (or equivalently Ma) and the.third axis (not shown) is defined by c3. The problem of channel selection is to choose both the number and the location 05 po nts in this space at which to operate filters. We obviously want as few channels as pos- sible, yet they must be close enough together to be able to interpolate with one Newton-Raphson step (eq. ( 9 ) ) . A bit of experimentation showed that five channels suffice for the F-8C (located as shown in the figure). Four cover the subsonic range and are logarithmically distributed along the MgO axis, with both c2 and c equal to zero. The logarithmic arrangemen't is motivated by our earlier findzng that expected identification errors are proportional to true parameter values. The fifth channel covers supersonic flight regimes. Each.channe1 interpolates over an approximate k50-percent range about its nominal M60 value. , KaZman f i l t e r design- Kalman filters and sensitivity filters for each channel were designed under steady-state assumptions, using the following fixed values for gust intensity, sensor noise, and trim disturbance growth rates : u = 6 ft/sec wg ag = 0. l!iO/sec anz = 0.02 g (5 g/v = 0.017 (g/Wi/Jsec = 0.001 (Mso)i/G i = 1,2, ..., 5 uMO In addition, the surface position measurement was assumed to be corrupted by discrete additive white noise with 0.04O rms intensity. These numbers can be considered roughly representative of the F-8C's environment. Adjustments- The basic MLE design required suprisingly few after-the-fact ''fixes" to make it work in situations not wholly consistent with the theoreti- cal maximum likelihood problem formulation. Two such adjustments are worth mentioning here. The first deals with the fact that the aircraft's parameters are not really constant, as assumed in the design so far. This means that we cannot accumulate likelihood functions and their sensitivities indefinitely, as implied by equations (5) to ( 9 ) . Rather we must "forget" past data by some expedient such as the high-pass operations used in figure 8. The high-pass approach was selected primarily because it preserves the overall recursive nature of the algorithm. Time constants (T) were set at 5 sec, an experimen- tal compromise between systematic tracking errors occurring for T too large and random errors or false minimums occurring for T too small. The second adjustment deals with time-varying noise statistics. These can cause substantial estimation errors in identifiers designed for fixed statistics. Consider, for example, a case where gust, sensor noise, and trim disturbances are actually very small compared to the nominal design values. 37 Then the likelihood function in equation (7) will be dominated by the Rn det B(3) terms. These depend on a priori channel data oniy ,not on incom- ing measurements. For the F-8CY they are minimized at channel l., M60 = -2.34., regardless of the true M60 value. Hence they can produce large estimation. errors.. This potential problem was alleviated by scaling the Rn det B. terms (and their gradients) with the constant T/At which can be formally derived by considering u to be an unknown common scale,,, factor on all noise statistics to be identified as part of the MLE procedure (ref. 3 ) . The F-8C identifier incorporates several other less important adjustments not covered here. These include interpolation of state estimates between channels, automatic initialization and data transfer when channel changes occur, significance tests on channel selections, and high-frequency roll-offs on sensitivity accumulation to prevent sharp estimation transients. These are covered in reference 3. Reference 3 also provides computer sizing estimates for the complete identifier. These call for approximately 2200 words of memory and 6 msec of execution time per data sample. SECTION IV - ADAPTIVE SYSTEM PERFORMANCE The control laws and identification algorithms described in the last sec- tion are scheduled for flight evaluation during 1976-77. In preparation for these tests, they have been mechanized and evaluated on the F-8 simulator at Langley Research Center and are also scheduled for evaluation on the simulator at Dryden Flight Research Center. Only Langley's evaluations are available to date. Key results are described below. Langley's F-8 simulator is an extensive six-degree-of-freedom nonlinear digital simulation program. It includes complete aircraft dynamics, actu- ators, servos, flexibility, sensor models, and real-time interfaces with an F-8 iron-bird cockpit and with standard simulation consoles (ref. 14) This facility was used to evaluate four "measures of goodness" of the adaptive system: Identification accuracy in steady flight Convergence characteristics Tracking characteristics for standard flight transition *As in equations (5) to (9) , the accumulation in equation (10) and the growing divisor (2T/At) are actually high-passed. 38 Responses to major maneuvers and configuration changes These criteria deliberately concentrate on identifier performance. Overall closed-loop performance, as mentioned earlier, provides a less sensitive mea- sure of adaptive capability. Accuracy at Fixed Flight Condition Identification accuracy (and thus control gain accuracy) was verified by subjecting the closed-loop adaptive system to a s andard sequence of test con- ditions while in trimmed flight at several fixed flight conditions. The test sequence consisted of 'a period of quiet, followed by a 20-ft/sec2 pilot doublet C* command followed by a period of atmospheric turbulence, and then again by a 20-ft/sec3 doublet. This entire sequence was repeated with and without sensor noise and always includes a mall random C* command as a test signal. Accuracy resuits for these tests are summarized in table IV and verify the theoretically predicted identifier performance. Typical transient responses are. given in figure 10. Convergence Characteristics To verify convergence properties, the algorithm's min-select was initial- ized at a preselected channel and required to converge to a flight condition covered by another channel. This was done for several combinations of chan- nels and input conditions under normal closed-loop operation. Convergence times (to within k20 percent of final M60 value) were less than 1 sec for all cases. Typical examples are shown in figure 11. Tracking The system's ability to track changing aircraft parameters was tested by flying through standard flight transitions. One suchltransition is shown in figure 12. It consists of maximum level accelerating flight starting at low dynamic pressure, h = 20,000 ft, Mach 0.40, ?j = 109 psf, and terminating just above Mach 1.0. To achieve repeatability, the transition was flown without pilot inputs. We simply applied 100-percent military power plus full after- burner and held pilot-commanded C* and lateral-directional commands at zero. The resulting transients in figure 12 verify smooth tracking behavior and properAchannel changes as dynamic pressure builds up. Note, in particular, that M, is discontinuous as the identifier decides that the aircraft has gone supersonic. This jump is consistent with the parameterization in table I. Figure 12 includes sensor noise and 6-%t/sec RMS vertical gusts. Ma j or Maneuvers The closed-loop adaptive system was also tested under large maneuver con- ditions and configuration changes. These included various rolling maneuvers, high-g turns, step angle-of-attack and sideslip d,isturbances, and speed 39 brake-, gear-, and wing-transition transients. These were largely flown under pilot control from the F-8 iron-bird cockpit. They were intended to explore qualitative properties of the concept under large signal condition and to assure that no drastic upsets occur during normal flight operations. Tran- sient traces are documented in reference.3. SECTION V - CONCLUSIONS AND REMAINING ISSUES The nonlinear simulation evaluations in section IV verify that the paral- lel channel maximuwlikelihood identifier works well throughout the flight envelope and provides adequate gain-scheduling signals for the control law. This is ample motivation to proceed to flight test. Like most system designs, however, the complete adaptive controller is not ideal and it is important to highlight remaining issues and limitations. These include tracking errors, compromises due to fixed filter design statistics, and test signal requirements. The tracking error limitation is evident in figure 12. Because of our 5-sec (effective) likelihood accumulation time, parameter estimates tend to lag in the presence of rapid parameter changes. These lags are responsible for the largest errors seen in the simulation trials. Errors as high as 30 percent are evident in the A; trace in figure 12, and 50 to 70 percent errors have been observed during configuration changes and more dramatic flight transitions (e.g., high-g pulldowns). Research is presently under way to provide some means of dealing with these time variations. The second limitation is evident in most of the accuracy runs. Parameter estimates shift somewhat between quiet or sensor-noise-only conditions and turbulence conditions. This appears to be caused by inconsistencies between true environmental statistics and the statistics used to design Kalman filter channels. Sqme on-line adjustment of these statistics may be desirable. Test signal requirements pose a potential limitation not quantifiable at this point. Required levels are determined primarily by small nonlinearities and actual noise characteristics in the flight-control hardware, while accept- able levels are determined by pilot considerations. In the simulations at Langley Research Center, a small random C* command signal (white noise passed through the second-order filter s/(s2 + 15s + 36)) was found adequate. Its IWS magnitude was adjusted to C* = 4 ft/sec2, which produced the follow- ing approximate accelerations at the pilot station: Flight condition Test signal acceleration Low (h = 20,000, M = 0.4) 0.01 g RMS Medium '4 (h = 20,000, M = 0.67) .025 g RMS High 5 (h = 3000, M = 0.8) .04 g RMS Supersonic (h = 40,000, M = 1.2) .02 g RMS These levels can be lowered somewhat with tolerable accuracy degradations. They cannot be used with much certainty, however, to predict the levels 40 ' , needed for the actual aircraft. For these requirements, we must wait until '! ' flight. REFERENCES 1. Deets, D. A.; and Szalai, K. J.: Design and Flight Experience With a Digital Fly-by-Wire Control System Using Apollo Guidance System Hard- ware on an F-8 Aircraft, AIM Paper 72-881, Aug. 1972. 2. Hartmann, Gary L.; Hauge, James A.; and Hendrick, Russell C.: F-8C Digi- ' . tal CCV Flight Control Laws, NASA CR-2629, 1976. 3. Hartmann, G. L., et al.: F-8C Adaptive Flight Control Laws, Honeywell Final Rept. , Contract NAS1-13383. 4. Athans, M., et al.: The Stochastic Control of the F-8C Aircraft Using the Multiple Model Adaptive Control (MMAC) Method, IEEE Conference on Decision and Control, Houston, Tex., Dec. 10-12, 1975, Proceedings, pp. 217-228. 5. Montgomery, R. C.; and Dunn, H. J.: Adaptive Control Using a Moving Win- dow Newton-Raphson Parameter Identification Scheme, IEEE Conference on Decision and Control, Houston, Tex., Dec. 10-12, 1975, Proceedings, p. 0229. 6. Kaufman, H.; and Alag, G.: An Implementable Digital Adaptive Flight Con- troller Designed Using Stabilized Single Stage Algorithms, IEEE Con- ference on Decision and Control, Houston, Tex., Dec. 10-12, 1975, Proceedings, pp. 231-236. 7. Deyst, J. J., Jr.; and Deckert, J. C.: Application of Likelihood Ratio Methods to Failure Detection and Identification in the NASA F-8 DFBW Aircraft, IEEE Conference on Decision and Control, Houston, Tex., Dec. 10-12, 1975, Proceedings, p. 237. 8. Montgomery, R. C.; and Price, D. B.: Management of Analytical Redundancy in Digital Flight Control Systems for Aircraft, AIAA Paper 74-887, Aug. 1974. 9. Mehra, R. K.; Stepner, D. E.; and Tyler, J. S., Jr.: A Generalized Method for Identification of Aircraft Stability and Control Derivative From Flight Test Data, 13th Joint Automatic Control Conference, Stanford, Calif., Aug. 16-18, 1972. 10. Iliff, Kenneth W.: Identification of Aircraft Stability and Control Derivatives in the Presence of Turbulence, NASA TN D-7647, 1974, pp. 77-113. 11. Mehra, R. K. : Optimal Inputs for Linear System Identification, 13th Joint Automatic Control Conference, Stanford, Calif., Aug. 16-18, 1972. 41 12. Chen, R. T. N.: Input Design for Parameter Identification. Part I: A. i' I' New Formulation and Practical Solution, 15th Joint Automatic Control ' / Conference, AustinT x., Jun 18-21, 1974. i 13. FeZ'dbaum, A. A. : Dual Control Theory. I. Automation and Remote Control,' vol. 21, no. 9, April 1960, pp. 877-880; Dual Control Theory. 11. Automation and Remote Control, vol. 21, no. 11,May 1960, pp. 1033- 1039; The Theory of Dual Control. 111. Automation and Remote Control; vol. 22, no. 1, Aug. 1961, pp. 1-12; The Theory of Dual Control. IV. Automation and Remote Control, vol. 22, no. 3, Sept. 1961, pp. 109-121. 14. Woolley, C. T.: A Real-Time Hybrid Simulation of a Digital Fly-By-Wire Aircraft, 1974 Summer Computer Simulation Conference, Houston, Tex., July 9-11, 1974. 15. Etkin, Bernard: Dynamics of Atmospheric Flight, John Wiley & Sons, Inc., 1972. 16. Athans, M.: The Role and Use of the Stochastic L near-Quadratic-Gaussian Problem in Control System Design, IEEE Trans. Automatic Control, VO~. AC-16, 1971, pp. 529-552. 17. Chalk, C. R., et al.: Background Information and Users Guide for MIL-F-8785B (ASG),.Military Specifications - Flying Qualities of Piloted Airplanes, Cornel1 Aeronautical Lab. , AFFDL-TR-69-72, Aug. 1969, p. 441. 18. Anon.: New Short Period Handling Quality for Fighter Aircraft, Boeing Document DG17841T/N, Oct. 1965. 19. Asseo, S. J.: Application of Optimal Control to Perfect Model Following, J. Aircraft, vol. 7, no. 4, July/Aug. 1970, pp. 308-313. 20. Stein, G.; and Henke, A. H.: Design Procedure and Handling-Quality Cri- teria for Lateral-Directional Flight Control Systems, Honeywell, Inc., AFFDL-TR-70-152, May 1971. 21. Kalman, R. E.: When is a Linear Control System Optimal? J. Basic Engineering, 1974. 22. -VanDierendonck, A. J.: Practical Optimal Flight Control for Aircraft With Large Flight Envelopes, AIAA Paper 73-159, Jan. 1973. 23. Levine, W. L.; Johnson, T. L.; and Athans, M.: Optimal Limited State Variable Feedback Controllers for Linear Systems, IEEE Trans. Auto- matic Control, vol. AC-16, no. 6, Dec. 1971, pp. 785-792. 24. Tse, Edison: Information Matrix and Local Identifiability of Parameters, 14th Joint Automatic Control Conference, Columbus, Ohio, June 20-22, 1973, pp. 611-619. 42 25. Lainiotis, D. C.: Optimal Adaptive Estimation: Structure and Parameter Adaptation, IEEE Trans. Automatic Control, vol. AC-16, no. 2, April 1969, pp. 160-170. 26. Bucy, R. S . ; and Senne, D. K.: Digital Synthesis of Non-linear Filters,, Automatica, vol. 7 , no. 3, May 1971, pp. 287-298. 27. Stein, G.; and Saridis, G. N.: A Parameter Adaptive Control Technique, Automatica, vol. 5, no. 6, Nov. 1969, pp. 731-739. TABLE I.- F-8 LONGITUDINAL MODEL PARAMETERIZATION Equation coefficient % = -0.23 + 0.028 M60 + ~1 Mach < 1 = -0.23 + 0.010 M60 + c1 Mach > 1 M, = (0.61 + c ~ ) M ~ ~ Mach .c 1 = (1.53 + c ~ ) M ~ ~ Mach > 1 V (200 + c3) - M60 Mach < 1 = (260 + ~ 3 ) - M60 Mach > 1 Z,V = (53 + c4)M60 Allach z M60 = c5 All Mach z6v = (7.7 + c6) All Mach """"""""""". - q = (2 + C+fgo All Mach Small perturbation parameter c2 c3 c4 c5 '6 """""""", c7 Perturbation parameter uncertainty 0.065 0.135 31.5 5.0 15.0 0.75 .""""""- 2.0 a This is "unflexed" surface effectiveness. Actual surface effectiveness taking quasi-static structural bending into account is given by Ms = (1 + 0.016Mgo + 0.0002M~o)M60. 4 3 TABLE 11.- LONGITUDINAL WEIGHTS AND GAINS Flight condition Altitude (h) ft Mach M Dynamic pressure ( 3 PSf Selected control gains Q to 6 , x3 to 6, 8, to 6, n, to 6 , 20,000 20,00 40,000 0.67 0.40 1.2 305 100 10 1 , 000 1 , 000 109 395 1 $000 1 $000 100 100 1,000 10 $000 1,000 1,000 0.30 .0086 .040 .23 0.96 0.19 .0011 .0017 .13 .040 .ll .13 10,000 0.8 652 100 ' 1 1 $000 1,000 0. 24 .013 .056 -. 000012 I TABLE 111.- IDENTIFICATION ACCURACY LIMITS FOR THE F-8C . . Parameter (from table I) c1 =2 c3 =4 M60 c5 """""""" . w 0 M, True value 0 0 0 0 -13.8 0 .""" 1.0 0 Initial uncertainty 0.065 .013 31.5 5.0 15 .O - 7% """"""" 1.5 .125 1-0 1-0 uncertainty uncertainty at T = 5 at T = 10 0.063 0.062 .065 .056 30.5 29.7 4.6 4.4 1.46 1.13 .75 .75 .0063 .0050 I I 4 4 - TABLE 1V.- F-8C IDENTIFIER ACCURACY AT FIXED FLIGHT CONDITIONS " __ Conditions h' = 20,000, M = 0.67 Quiet Gusts & sensor noise Pilot commands h .= 20,000, M = 0.4 Quiet Gusts & sensor noise Pilot commands h = 40,000, M = 1.2 Quiet Gusts & sensor noise Pilot commands h = 3,000, M = 0 . 8 Quiet Gusts & sensor noise Pilot commands It erro' G 4 3 3 6 8 55 8 2 4 2 40 23 29 2 10 2 2 10 4 16 30 15 2 15 4 A 'a aErrors were computed by comparing linearized coefficients with estimates plotted in real time on strip charts. Many errors fall below the expected resolution of this process. 45 I I > . Figure 1.- Adaptive control law structure. .. . State Dynamics: lkaaure.ents (discrete): pitch rate a total angle of attack d elevator position elevator c-nd normal acceleration unit intensity white noise unit variance white sequences ZaV. Z6V, V “08 “g* “nz nq. ma. n6’a. . , m,, 8/V actuator bandwidth (12 rlsec) accelerometer displacement from C.G. (15 ft) vertical gust level (ftlsec rms) sensor noise levels dynamic coefficients (functions of 5) constant trim disturbances (functions of 5) Figure 2.- F-8C longitudinal design model. Command generator C*Model Aircraft and actualor Figure 3. - -Longitudinal optimization problem. 46 Stick "2 0 . 7 2 ~ ~ + 9.8s + 40 I s2 +13.5s +55 I "+ - / I " I i Lataal slick Log A i k m , C W Figure 6.- Lateral-directional control Figure 4.- Simplified longitudinal con- , -law. trol law. Figure 5.- Longitudinal gain. ne -$ " Sensitivity * "sM(Lt ) calculation _p r = E Controller kt r+kz S Figure 7.- Identification accuracy analysis. q nz - % I Ka1m9n Accum. filter "+ 0.2 L l - a* "e €=€I L k p 1) - s+0.2 -- 3 man finer * h'S, update/ " 3 Accum. - filler 0.2 L5 sec - LIE5' t ) c=c5 I - ,s+0.2 SanitivitY VU &cum. Newton VL,M o.2 filter -e €=Gi . VL, M - - s+0.2 "+ 50 update/sec 8.33 updale /sec 3 R a m correction L Figure 8.- Parallel-channel maximum likelihood/identifier. 47 0 Stick cm (in) a rad Idea) I 0 Surface effectiveness. Mu Figure 9.- F-8C identifier channel loca- tions. - . L -501 I I a$ percent 50p-4 O r- -50 - I%7 I I u 5 s e c - d t- With gusts I With pilot commands Figure 11.- Typical convergence Figure 10.- Typical accuracy test traces; transients; h = 20,000, h = 20,000, M = 0.67. M = 0.4. 48 Figure 12.- Tracking traces. 49 h . . ... . SELF-TUNING REGULATORS K. J. htriim Lund Institute of Technology, Sweden 1. INTRODUCTION This paper gives a brief review of the results of a research project on self-tuning regulators which has been carried out at the Lund 1nstitute.of Technology. The project is part of a larger research program on adaptive control which has followed three main lines: A - stochastic control, B - self-tuning regulators, and C - analysis of adaptive regulators proposed in literature. The approach via stochastic control, which legds to dual-control strategies, has been very useful to provide understanding and insight. So far the results have, however, not been carried out to the stage of implemen- tation. Self-tuning' regulators are a particular version of adaptive regulators based on real-time identification. They are a special case of nondual stochastic control algorithms. The work on self-tuning regulators has progressed quite far in the sense that these regulators are reasonably well understood theoretically. They have also been tried extensively in several industrial applications. On the other hand, much work remains to be done in exploring other aspects of these regulators. Project C is needed to stay abreast of the development of other adaptive schemes. It has also resulted in a long list of problems relating to understanding the strange behavior of some algorithms in certain circumstances. The basic idea underlying the self-tuning regulators is the following. If a description of a system and its environment is known, there are many procedures available to design a control system subject to given specifica- tions. When trying to remove the assumption that the models for the system and its environment are known, we are immediately led to the problem of controlling a system with constant but unknown parameters. This problem can, in principle, be solved by using stochastic control theory at the price of exorbitant calculations. It is then meaningful to ask if there are simple control algorithms that do not require information about the model parameters, such that the controller will converge to the controllers that could be designed if the model parameters were known. It is an empirical fact that such controllers exist in several cases. The investigation of their proper- ties has also led to powerful tools that can be used to analyze many other cases. The generation of self-tuning algorithms is partly heuristic. It turns out that many algorithms can be obtained by combining a'real-time identifier with a control scheme. In our work we have so far mostly considered regula- tors for the LQG regulator problem. This has been motivated by the particular 51 app l i ca t ions w e have considered. Many of the concepts and ideas can, however, be extended to many other design methods. 2. AN EXAMPLE The main ideas are f i r s t demonstrated by a simple example. Consider the s imple d iscre te time system: y ( t + 1 ) + a y ( t ) = bu( t ) + e ( t + 1) + ce(t) (1) where u is the i npu t , y the ou tput , and {e( t ) ) a sequence of independent, equa l ly d i s t r ibu ted , random va r i ab le s . The number c is assumed to be less than 1. L e t the c r i te r ion be to minimize the var iance of t he ou tpu t , t ha t is, 1 t k=1 min V = min Ey2 = min E - y2(k) It is easy to show t h a t t h e c o n t r o l law u ( t ) = - Y(t) a - c b is a minimum var iance s t ra tegy , and tha t the output of system (1) wi th feedback (3) becomes (see, e.g. , Astrsm ( r e f . 1)). Note t h a t t h e c o n t r o l l a w (3 ) , which r ep resen t s a proport ional regulator , can be character ized by one parameter only. A se l f - tun ing regula tor for the sys tem (1) can be described as follows: ALGORITHM (Self-Tuning REgulator) Step 1 (Parameter Estimation) A t each time t , f i t t h e parameter a i n t h e model by least squa res , t ha t is, s u c h t h a t t h e c r i t e r i o n t 52 where is minimal. The estimate obtained is denoted at to indicate that it $8 a function of time. Step 2 (Control) At each time t, choose the control u(t) f a t Y W where a is the estimate obtained in step 1. t Motivation There are several ways to arrive at the control strategy given above. The algorithm STURE can, for example, b interpreted as the certainty equivaZence control for the corresponding stochastic control problem. Analysis The properties of a closed-loop system controlled by a self-tuning regulator are now discussed. Since the closed-loop system is nonlinear, time- varying, and stochastic, the analysis is not trivial. It is fairly obvious that the regulator will perform well if it is applied to a system (1) with b = 1 and c = 0, because in this case the least-squares estimate at will be an unbiased estimate of a. The regulator (8) will thus converge to a minimum variance regulator if the parameter estimate at converges. It is surprising, however, that the regulator will also converge to the minimum variance regulator if c f O (as demonstrated below). There may also be some difficulties because the control law is of the certainty equivalence type. Because of the special model structure ( 5 ) , the feedback gain will, however, be bounded if the estimate at is bounded. The least-squares estimate is given by the normal equation Assuming that the estimate at converges toward a value that gives a stable closed-loop system, than it is straightforward to show that 53 Thus the closed-loop system has' the property . . t 1 l i m - z y ( k + l ) y ( k ) = 0 ( 9 ) t-too k=1 Furthermore, assuming that the system to be control led is governed by equat ion (l), the output of the closed-loop system obtained in the limit is given by y ( t ) + [ a - ab]y ( t - 1) = e ( t ) + c e ( t - 1 ) (10) The covariance of { y ( t ) ) a t l a g 1 i s then given by Condition (9) g ives f ( a ) = 0 A second-order equation for a which has the so lu t ions : a = a l = - .a - c b The corresponding poles of the closed-loop system are A 1 = c and A, = l / c , r e s p e c t i v e l y . S i n c e c was assumed less than 1, only the va lue a1 corresponds to a stable closed-loop system. Note t h a t a1 cor responds to the gain of the minimum va r i ance r egu la to r (3 ) . Hence, i f t he pa rame te r estimate at converges t o a va lue tha t g ives a s table c losed-loop system, then the closed-loop system obtained must be such that equation (9) holds. This means that the algori thm can be thought of as a r e g u l a t o r t h a t a t t empt s t o b r i n g the covariance of the output a t l a g 1, t h a t is , r y ( l ) , t o z e r o i n t h e same way as an i n t e g r a t i n g r e g u l a t o r b r i n g s t h e i n t e g r a l o f t h e c o n t r o l e r r o r t o z e r o . I f the sys tem to be cont ro l led is actual ly governed by equation (l), then the se l f - tun ing r egu la to r w i l l converge t o a minimum v a r i a n c e r e g u l a t o r i f it converges a t a l l . 54 Figure 1 shows the results of a simulation of the algorithm. It is clear from this simulation that the algorithm converges in the particular ase. The least-squares estimate will be a biased estimate of the model parameter a = -0.5 because of the correlation between the model errors. As can be expected from the previous analysis, the bias is, however, such that the limit- ing regulator corresponds to the minimum variance regulator. The lower part of figure 1' shows the asymptotic value 'of the 'loss function obtained if the regulator gain is fixed to the current value. It is clear from this figure that the loss function is very close to the minimum loss for the case of known parameter after 50 steps. . . 3. GENERALIZATIONS A regulator tion is shown in of three parts: that generalizes the simple self-tuner of the previous sec- figure 2. The regulator can be thought of as being composed a parameter estimator (block 1). a controller (block 3 ) , and a third part (block-2) , which relates the controller parameters to the esti- mated parameters. The parameter estimator acts on the process inputs and outputs and produces estimates of certain process parameters. The controller is simply a linear filter characterized by the coefficients of its transfer function. These coefficients are generally a nonlinear function of the estimated parameters. This function is frequently not one to one. This way of describing the regulator is convenient from the point of view of explaining how it works. The subdivision is, however, largely arbitrary, and the regula- tor can equally well be regarded simply' as one nonlinear regulator. The functions of blocks 1, 2, and 3 are also simple, but the interconnection of these blocks represents a system with a rather complex input-output relation. The partitioning of the regulator (fig.' 2) is also convenient from the point of view of implementation because the parameter estimator and the controller parameter calculation are often conveniently time shared between several loops. There are many different ways to estimate the parameters 0 and to calculate the regulator parameters, 8 . Some possibilities are shown in figure 3. The complexity of the algebraic equation that relates the control- ler parameters to the estimated parameters can vary significantly, from a simple variable substitution for minimum variance regulators to solution of an algebraic Riccati equation for the general LQG case. Analysis A brief statement of some properties of the self-tuning regulators are now given. The results are fairly technical and only a few main points are given here. A review of available results are,-given in reference 2. The major results were proven in references 3 to 5. For the analysis, it is assumed that the process to be controlled-is governed by 55 where ' ,A(q-l) and B(4-l) are polynomials i n t h e backward s h i f t o p e r a t o r q-' and {v(t) 1 is a sequence of random va r i ab le s wi th bounded f o u r t h moment. The a n a l y s i s w i l l b a s i c a l l y c o v e r t h e case p ( t ) + 0 as t + -, which corre- sponds t o t h e case when the parameters are cons tan t . The following problems can be r e so lved pa r t i a l ly by ana lys i s : Overall s t a b i l i t y of the closed-loop system Convergence of the regulator P rope r t i e s o f t he poss ib l e l imi t ing r egu la to r s The a n a l y s i s is f a r from t r ivial because the closed-loop system is a nonlin- ear, t ime-variable s tochast ic system. Even i f t h e r e c u r s i v e i d e n t i f i c a t i o n schemes used are w e l l known, their convergence propert ies are l a r g e l y unknown except for the leas t - squares case. The input is a l so gene ra t ed by a time- varying feedback, which introduces addi t ional di f f icul t4es . I f the process n o i s e , { v ( t ) ) is co r re l a t ed , t he l ea s t - squa res estimates w i l l be biased and t h e b i a s will depend on the feedback used. A g l o b a l s t a b i l i t y r e s u l t was proven by Ljung and Wittenmark (refs. 5 and 6 ) ( see f ig . 4 ) . Th i s r e su l t app l i e s t o a r e g u l a t o r composed of a least- s q u a r e s i d e n t i f i e r and a minimum va r i ance con t ro l l e r . The r e s u l t r e q u i r e s that the system (12) is minimum phase and that the time delay k and t h e parameter Bo are known. A key r e s u l t i n t h e a n a l y s i s is the observa t ion made by Ljung ( r e f . 4) t ha t t he pa ths o f t he estimates are c l o s e l y r e l a t e d t o t h e t r a j e c t o r i e s o f t h e d i f f e r e n t i a l e q u a t i o n : where f (0) = E['? ( t , O ) ~ ( t , O ) l G ( 0 ) = E['? (t,O)Y(t,O).] T T In the pa r t i cu la r ca t e o f t he r egu la to r LS"V, the con t ro l l a w is chosen i n such a way tha t y( t ,O) = 0 and t h e s t a t i o n a r y p o i n t s are then given by 0 = f (0) = E[y( t + l ) + ( t ) ] = 0 56 The regulator L S M thus attempts to zero the autocovariance of the output and the crosscovariance of the input and the output for certain lags. This result, which generalizes the simple example discussed in section 2, was shown in reference 7. It was also shown here that has only one stationary solution for the regulator of L S M if the orders of the system and the model are compatible. The differential equations (13) and (14) can be used in several different ways. Ljung has exploited them to construct both convergence proofs and examples which show that the parameter estimates do not converge. The dif- ferential equations have also been very useful in simulations (see, e.g., refs. 8 and 9) . The.simulations shown in figures 5 , 6 , and 7 illustrate the behavior of different versions of the self-tuner. 4. SERVOPROBLEM So far, the self-tuning regulator has been discussed only in the frame- work of the regulator problem. It is straightforward to apply self-tuning , to the servoproblem, too. Clarke and Gawthrop (ref. 10) propose to do so by posing a linear quadratic problem for a servoproblem. Another approach is simply to introduce the reference values by the standard procedure using feedforward and an inverse model. For known param- eters, the problem is handled as follows. Assume that the process is described by equation (12) and introduce the reference values ur(t) and yr(t) which satisfy the same dynamics as the process A(q-l)yr(t) = B(q-l)ur(t - k) Hence A(q-’)[y(t) - yr(t)] = B(q-l)[u(t - k) - ur(t - k)] + v(t) A design procedure for the regulator then gives the feedback If the command signal yr (t) is specified, 57 I This system cannot be realized unless the change in reference value is known or can be predicted k steps ahead. If this is not the case, a time delay in the response of k units must be accepted. Observe that the control law (17) can be written: The servoproblem can be incorporated into the self-tuning regulator simply by changing the model in the parameter estimation step to M: $(t) = - d(q-l)y(t - 1) -t B(q-l)u(t - k) + C(q-l)tz(t - 1) and making the modification (18) in the control step. 5. APPLICATIONS The self-tuning regulators are conveniently implemented using a di ital computer. The simple regulator LS+MV requires no more than 30 lines of FORTRAN code, while the regulator RMLS.LQ requires an order-of-magnitude more code because of the necessity of solving the algebraic Riccati equation in each iteration. The regulators have been applied to a number of industrial processes. Among the applications currently known to me are paper machine (refs. 11 and 12) digester (ref. 13) ore crusher (ref. 14) enthalpy exchanger (ref. 15) supertanker (ref. 16) Several of these applications have been in operation for a long time. A self- tuning regulator has, for example, been running as an adaptive autopilot for a supertanker for more than year. Even if the regulators discussed automatically tune their parameters, it is necessary to determine some parameters in advance; for instance, Number of parameters in the prediction model (p, r, and s) Initial values of the parameter estimates Value of any fixed parameters in the model Rate of exponential forgetting of past data in the estimation algo- Sampling rate rithm Experience has shown that it is fairly easy to make the proper choice in practice. These parameters are also much easier to choose than to directly 58 determine the coefficients of a complex control law. It is our experience that system engineers without previous exposure to this type of algorithm have been able to learn how to use it after only a short training period. There have also been several misapplications. The most common mistake is to attempt a self-tuner for a control design that will not work even if the parameters are known. REFERENCES 1. btrijm, K. J.: Introduction to Stochastic Control Theory, Academic Press, New York, 1970. 2. btrijm, K. J.; Borisson, U.; Ljung, L.; and Wittenmark, B. : Theory and Applications of Adaptive Regulators Based on Recursive Parameter Estimation, Preprint IFAC 6th World Congress, Boston, 1975. 3. btrim, K. J. ; and Wittenmark, B. : On Self-Tuning Regulators, Automatica, vol. 9, no. 2, March 1973, pp. 185-199. 4. Ljung, L.: Stochastic Convergence of Algorithms for Identification and Adaptive Control, Thesis Rept. 7406, Dept. Automatic Control, Lund Institute of Technology, Lund, Sweden, March 1974. 5. Ljung, L.; and Wittenmark, B.: Asymptotic Properties of Self-Tuning Regulators, Rept. 7404, Dept. Automatic Control, Lund Institute of Technology, Feb. 1974. 6. Ljung, L.; and Wittenmark, B.: Analysis of a Class of Adaptive Regulators, Preprint IFAC Symposium on Stochastic Control, Budapest, 1974. 7. iistrom, K. J.; and Wittenmark, B. : On the Control of Constant But Unknown Systems, Proceedings of the Int. Fed. Autom. Control 5th World Congress, Paris, June 12-17, 1972. 8. Wittenmark, B.: A Self-Tuning Regulator, Rept. 7311, Dept. Automatic Control, Lund Institute of Technology, Lund, Sweden, April 1973. 9. Wittenmark, B.: Self-Tuning Regulators, Rept. 7312 (thesis), Dept. Automatic Control, Lund Institute of Technology, Lund, Sweden, April 1973. 10. Clarke, D. W.; and Gawthrop, P. J.: A Generalization of the Self-Tuning Regulator, Rept.,Dept. Eng. Sci., Oxford Univ., England, 1974. Electron Lett, vol. 11, no. 2, Jan. 23, 1975, pp. 40-41. 11. Cegrell, Torsten; and Hedqvist, Torbjom: Successful Adaptive Control of Paper Machines, Preprints 3rd IFAC Symposium on Identification and . System Parameter Estimation, Hague, 1973, vol. 11, no. 1, Jan. 1975, pp. 53-59. 59 12. Borisson, U.; and Wittenmark, B.: An-Industrial Application,of a'.Self-: Tuning Regulator, Preprints IFAC Symposium on Digital Computer Appli- cations to Process Control, Ziirich, 1974. , . . 13. Cegrell, T.; and Hedqvist, T.: A New Approach to Continuous.Digester Control, Preprints 4th IFAC/IFIP International Conference on Digital Computer Applications to Process Control, ZGrich, 1914. ' ' ' " ' . 14. Borisson, Ulf; and Syding, Rolf: Self-Tuning Control of an-Ore Crusher," Preprint IFAC Symposium on Stochastic Control, Budapest; 1974': Automatica, vol. 12, no. 1, Jan. 1976, pp. 1-7. 15. Jensen, L.; and Hksel, R. : Computer Control of an Enthalpy Exchanger; Rept. 7417, Dept. Automatic Control, Lund Institute of Technology, Lund, Sweden, Oct. 1974. 16. Gllstriim, C. : The Sea Scout Experiments, Rept. 7407(C), Dept. Auto- matic Control, Lund Institute of Technology, Lund, Sweden, April 1974. , . BIBLIOGRAPHY . . . . Alsholm, 0.; Borisson, U.; Stavnes, 0.; and Wittenmark, B: A Feasibility .'. Study of Self-Tuning Regulators, Rept . 7338, Dept . Automatic ' Control; Lund Institute of Technology, Lund, Sweden, Oct. 1973. . . . Andersson, L.; Borisson, U.; Braun, R.; and Syding, R.: A-DDCLoop Between .' Lund and Kiruna for Control f an Ore Crusher, Rept. .7318(C), Dept. Automatic Control, Lund Institute of Technology, Lund, Sweden, June 1973. Astrom, K. J. : A Self-Tuning Parameter Estimator. Publication 74/55. (Research Report), Dept. of Computing and Control, Imperial College.of Science and' Technology, London SW7 2BT. Also available as Rept. 7419(C), Dept. Automatic Control, Lund Institute of Technology, Lund, Sweden, Aug. 1974. htrijm, K. J.: A Self-Tuning Regulator for Non-Minimum Phase Systems, Rept. 7411(C), Dept. Automatic Control, Lund Institute of Technology, Lund, Sweden, June 1974. htriim, K. J.: Theory and Applications of Self-Tuning Regulators. Proc. m&rie du Controle M6thodes Numgriques et ModtSlisation des Systems Informatiques, Institut de Rechercher d'Informatique et d'Automatique, Rocquencourt, France, June 1974, Springer 1975. Also available as Rept. 7417(C), Dept. Automatic Control, Lund Institute of Technology, Lund, Sweden, Aug. 1974. ktrgm, K. J.; and Eykhoff, P. : System Identification - A Survey. 'Automatica, vol. 7, no. 2, March 1971, pp. 123-162. Originally given as an invited survey paper Int. Fed. Autom. Control Symposium on System Identification and Parameter Estimation, Prague, June 1970. 60 btr&, K. J. ; and Wittenmark, B. : Problems of Identification and Control. J", vol. 34, no. 1, April 1971, pp. 90-113. &&m, K. J.; and Wittenmark, B. : Analysis of a Self-Tuning Regulator for Non-Minimum Phase Systems. Preprints IFAC Symposium on Stochastic Control, Budapest, 1974. Also available as Rept. 7418(C), Dept. Automatic Control, Lund Institute of Technology, Lund, Sweden, Aug. 1974. Athans, M., ed.: Special Issue on Linear-Quadratic-Gaussian Problem. 'IEEE Trans. on Automatic Control, vol. AC-16, Dec. 1971. Bar-Shalom, Y. ; Tse, Edison; and Larson, R. E. : Some Recent Advances in the Development of Closed-Loop Stochastic Control and Resource Allocation , Algorithms. IFAC Symposium on Stochastic Control Theory, Budapest, 1974. Bohlin, T.: Information Pattern fo r Linear Discrete-Time Model With Stochas- tic-Coefficients. IEEE Trans. Automat. Control, vol. AC-15, no. 1, Feb. 1970, pp. 104-106. Borisson, U.; and Wittenmark, B.: Moisture Content Control on a Papermachine -An Application of a Self-Tuning Regulator. Rept. 7337, Dept. Automatic Control, Lund Institute of Technology, Lund, Sweden, Nov. 1973. Borisson, U.; and Syding, R. Control of an Ore Crusher - A Feasibility Stud3 of Self-Tuning Control and Remote Processing. Rept. 7318, Dept. Automatic Control, Lund Institute of Technology, Lund, Sweden, June 1973. Borisson, U.: Self-Tuning Regulators - Industrial Applications and Multi- variable Theory. Rept. 7513 (thesis), Dept. Automatic Control, Lund Institute of Technology, Lund, Sweden, Oct. 1975. Clarke, D. W.; and Gawthrop, B. A,: A Self-Tuning Controller. Technical 0 Report, Dept. Eng. Sci., Oxford Univ., England, 1975. (Submitted to IEE). Farison, J. B.: Parameter Identification for a Class of Linear Discrete Systems. IEEE Trans. on Automatic Control, vol. AC-12, no. 1, Feb. 1967, p. 109. Florentin, J. J.: Optimal Probing, Adaptive Control of a Simple Bayesian System. Int. J. Electronics and Control, vol. 13, no. 2, Aug. 1962, pp. 165-177. Gallier, P. W.; and Otto, R. E.: Self-Tuning Computer Adapts DDC Algorithms in Progress in Direct Digital Control. T. J. Williams and F. M. Ryan, eds., ISA. Ckwthrop, P. J. ; and Clarke, D. W. : Simulation of a Generalized Self-Tuning Regulator. Report, Dept. Eng. Sci., Oxford Univ., England, 1974. Electron Lett, vol. 11, no. 2, Jan 23, 1975, pp. 41-42. 61 Gustavsson, I.; Ljung, L.; and Saderstram, T.: Identification of Linear, . . Multivariable Process Dynamics Using Closed Loop Estimation. Rept.-7401, Dept. Automatic Control, Lund Institute of Technology, Lund, Sweden, Jan. 1974. . . HetthGssy, J. ; and Keviczky, L. : Some Innovations to the Minimum Variance Control. Preprints IFAC Symposium on Stochastic Control, Budapest, Sept. 1974. Holst, 'J. : - Anvindande av Sjiilvinstillande Prediktorer f& Prognos av Belastning i Kraftsystem (Using Self-Tuning Predictors for Load Prediction) (in Swedish). Rept. 7433(C), Dept. Automatic Control, Lund Institute of Technology, Lund, Sweden, Dec. 1974. Hughes, D. J.; and Jacobs, 0. L. R.: Turn-off, Escape and Probing in Non- Linear Stochastic Control. Preprints IFAC Symposium on Stochastic Control Theory, Budapest, Sept. 1974. Hughes, D. J.; and Jacobs, 0. L. R.: Simultaneous Identification and Control Using a Neutral Control Law. Report, Dept. Eng. Sci., Oxford Univ., England, 1974. Jacobs, 0. L. R.: Structure of Stochastic Control Laws. IMA Conference on Stochastic Programming, Oxford Univ., England, July 1974. Jacobs, 0. L. R.; and Langdon, S. M.: An Optimal Extrema1 Control System.' Automatica, vol. 6, 1970, pp. 297-301. Jacobs,: ,O. L. R.; and Hughes, D. J. : Simultaneous Identification and Control Using 'a Neutral Control Law. Preprints IFAC Congress, Boston, Aug. 1975. Kalman, R. E.: Design of a Self-optimizing Control System. Trans. of the ASME, vol. 80, no. 2, Feb. 1958, pp. 468-478. Ljung, L.: Convergence of Recursive Stochastic Algorithms. IFAC Symposium on Stochastic Control, Budapest, 1973. Also Rept. 7403, Dept. Automatic Control, Lund Institute of Technology, Lund, Sweden, Feb. 1974. Nahorski, Z.: Simultaneous Identification and Control of Linear Discrete- Time System Using Dual Control Concept. IFAC Symposium on Stochastic Control, Budapest, Sept. 1974. Nahorski, Zbigniew; and Vidal, Rene V. V.: Stabilization of Linear Discrete- Time Systems Using Simultaneous Identification and Control. Int. J. Contro1;vol. 19, no. 2, Feb. 1974, pp. 353-364. 62 Panuska, V.: A Stochastic Approximation Method for Identification of Linear Systems Using Adaptive Filtering, 9th Joint Automatic Control Conf., 1968, pp. 1014-1021. Peterka, V.: Adaptive Digital Regulation of a Noisy System. Paper no. 6.2, Preprints 2nd IFAC Symposium on Identification and Process Parameter Estimation, Prague, 1970. Peterka, V.; and AstrGm, K. J.: Control of Multivariable Systems with Unknown but Constant Parameters. Preprint 3rd IFAC Symposium on Identification and System Parameter Estimation, The Hague, June 1973. ' ' , ," . . . SGderstrgm, T. ; Ljung , L. ; and Gustavsson, I. : A Comparative Study of . . Recursive Identification Methods. Rept. 7427, Dept. Automatic Control, Lund Institute of Technology, Lund,, Sweden, Dec. 1974. Tse, E. ; and Bar-Shalom, Y. : An Actively Adaptive Control for Linear Systems With Random Parameters via the Dual Control Approach. IEEE Trans., vol. AC-18, April 1973, pp. 109-117. Tse, E.; Bar-Shalom, Y.; and Meier 111, L.: Wide-sense Adaptive Dual.Contro1 for Nonlinear Stochastic Systems. IEEE Trans., vol. AC-18, April 1973, pp. 98-108. Vanecek, A. : Coupled Air/Fuel Dynamic Optimization and Temperature 'Regula- tion: A Case Study in Stochastic Control. Preprints IFAC Symposium on Stochastic Control, Budapest, Sept. 1974. Wellstead, P. E.; and Edmunds, J. M.: Least Squares Identification of Closed Loop Systems. Report, University of Manchester, 1975. Int. J. Control, vol. 21, no. 4, April 1975, pp. 689-699. Wellstead, P. E.; and Edmunds, J. M.: On-Line Process Identification and Regulation, Report, University of Manchester, 1975. Wieslander, J.; and Wittenmark, B.: An Approach to Adaptive Control Using Real Time Identification. Automatica, vol. 7, March 1971, pp. 211-217. Wittenmark, B.: Adaptive Control of a Paper-machine. Preprint Computer, Electronics and Control Conference, Calgary, May 1974. Wittenmark, Bjgrn: A Self-Tuning Predictor. IEEE Trans. on Automatic Control, vol. AC-19, no. 6, Dec. 1974, pp. 848-851. Wouters, W. R. E.: Parameter Adaptive Regulatory Control for Stochastic SISO Systems. Theory and Applications. Preprints IFAC Symposium on Adaptive Control, Budapest, Sept. 1974. 63 I I I I I Figure 1.- Example,of self-tuning regulator. I I LR.* Pmarnater calculation Jgulalor """""""" I - I I I I "I Figure 2.- Block diagram of a general self-tuning regulator. 64 PARAMETER ESTIMATION Model: ?(t) =,-A,(q-l)y(t - 1) + B(q-’)u(t -’ 1) + C(q-l)E(t - 1) - qt)o E(t,O) = y(t) - qt)o O(t + 1) = @(t) + p(t)S(t + l)$T(t)E(t,Q), S”(t + 1) = S’l(t) + U(t + 1) [JIT(t + l)$(t + 1) - s-1 (t)] For: Least squares, C f 0 Extended least ,squares, $(t) = ‘$(t) Recursive maximum likelihood, -gradOe(t,O) CONTROL STRATEGIES Regulator parameters: 9 = col[qlqp ... q,, ; f,, f, ... f2] Criteria: Minimum variance Linear quadratic Figure 3.- Some approaches to parameter estimation and control. 65 Let the system be where the parameters are estimated by least squares and control gives a minimum variance response if the time delay, k, and the lead coefficient of -1 the polynomial B(q 1, B o , are known if the system order is not underestimated and if then and if the system be minimum phase then also lim SUP ECu2(t) 1 < - Figure 4.- An example global stability result. 66 I - t[ E 0 81 """ " "" u o - =I .- E d L 4 -I I 0 I I 3 1000 2000 3 0 Time e E Figure 5.- Example regulator design using least squares and minimum 5 variance estimation and control. 2 J .- c E l 0 -I 3 r 2 - In B L I I I I " 0 I O 0 0 2000 30 IO I I Time 1000 2000 Time 3000 Figure 6.- Example regulator design Figure 7.- Regulator design using using least squares and linear- extended least squares and linear- quadratic estimation and control. quadratic estimation and control. 67 FAILURE DETECTION AND CONTROL-SYSTEM RECONFIGURATION: PAST, PRESENT, AND FUTURE Raymond C. Montgomery Langley Research Center, NASA . SUMMARY The history of failure detection and redundancy management in aircraft applications is reviewed. To date, techniques related to that subject have been based mainly on hardware duplication of like components with failure monitoring and switchover or averaging for redundancy management. Specific examples of these techniques are discussed as they have been. applied to the NASA F8 Digital Fly-by-Wire (DFBW) aircraft and are to be applied to the space shuttle vehicle in the near future. Recently, interest has arisen in new mathematical developments that promise optimal failure detection and control law reconfiguration through the use of system dynamical equations and measurements from dissimilar components. Practical use of those developments depends on the availability of advanced logic processing such as is offered by modern high-speed computers. Recent results of simulations using those developments are presented. They indicate that, in the simulation environment, failure detection and control-system reconfiguration through the use of dissimilar components is possible and func- tions satisfactorily for sensors used for primary flight control. However, computational requirements for the system studied are large a.nd exceed the capability of the F8-DFBW aircraft as presently configured. Due to the parallel nature of the algorithm needed for the system, the use of micropro- cessors as dedicated computational units should alleviate this problem. For the time being, however, to develop advanced redundancy management concepts to a technology application level, NASA is proceeding with the development of advanced suboptimal systems. Plans for a flight test of the advanced redun- dancy management system are presented and discussed. PAST Since the advent of aviation there has been concern regarding safety of flight. That concern is most apparent in the aircraft flight-control system. The purpose of most flight-control systems is to translate motions from the pilot's controllers into appropriate motions of control surfaces located on the aircraft. In the early days of aviation, this was accomplished by an all-mechanical control system as indicated in figure l(a). That type of sys- tem is in use today in most light aircraft. One feature of the mechanical 69 system i s t h a t l o a d s r e q u i r e d t o move t h e c o n t r o l s u r f a c e s are t ransmi t ted through mechanical l inkages to the pi lot , thereby giving him a f e e l of the air- craf t . A s a i r c r a f t became l a rge r , t he fo rces t r ansmi t t ed a l so became l a r g e r . Tha t l ed to the r equ i r emen t fo r t he fu l ly powered hydraulic system shown i n . f i g u r e l ( b ) . The f o r c e s a p p l i e d t o t h e c o n t r o l s u r f a c e s are n o t f e l t by t h e p i l o t i n t h e f u l l y powered system s o t h a t a r t i f i c i a l f e e l and t r i m had t o b e used. Also, because of dependence on the ac tua t ion sys t em, inc reased r e l i ab i l - i t y w a s needed . This led to tandem ac tua tors , mul t ip le hydraul ic supply sys- tems, hydraul ic vo t ing mechanisms (usua l ly force sum devices) and many o the r ad hoc procedures to meet the vehic le des ign requi rements from t h e r e l i a b i l i t y s tandpoint . Desire t o h a v e a i r c r a f t f l y i n c o n d i t i o n s w h e r e i t s f l y i n g q u a l i t i e s are undes i r ab le l ed to the development of the s tab i l i ty augmenta t ion sys tem (SAS) shown i n f i g u r e l ( c ) . T h a t s y s t e m u s e s s e n s o r s t o d e t e c t a i r c r a f t m o t i o n s and e lectr ical s ignals determined from t h e s e n s o r s t o a c t i v a t e t h e h y d r a u l i c system. The s i g n a l i s a p p l i e d i n a d d i t i o n t o t h e c o n t r o l commands from t h e p i l o t ' s c o n t r o l s . The SAS system introduced new r e q u i r e m e n t s f o r r e l i a b i l i t y s ince fa i lure of the sensors could adverse ly a f fec t the vehic le ' s mot ions . Again, these were resolved largely through ad hoc procedures usually involving l imi t ing the au tho r i ty o f t he SAS inputs. This has been done by a v a r i e t y of methods usually employing ingenious mechanical devices (or nightmares, depend- ing on your point of view). One approach to e l iminate the complex, weighty, and maintainably undesirable mechanized nightmare i s being developed by government and indus t ry - the f ly-by-wire control system (sh6m in f ig . l (d)) . That system w i l l employ e lectr ical wires t o r e p l a c e t h e complex mechanical l inkages indicated above. It enables a f lex ib le des ign wi th advantages of weight saving and simplicity. The main reasons that such systems are not used i n o p e r a t i o n a l a i r c r a f t t o d a y is t h a t t h e r e i s a genera l l ack of confi- dence i n t h e b a s i c r e l i a b i l i t y o f t h e e lectr ical and e l e c t r o n i c components used i n a fly-by-wire design. An i n c r e a s e i n t h e b a s i c r e l i a b i l i t y o f t h e fly-by-wire system w i l l be necessary before i t can be used operat ional ly . NASA is invo lved in two programs to develop fly-by-wire technology. A t t h e Dryden Fl ight Center (DFRC), a d i g i t a l fly-by-wire system is being instal led onboard an F8 a i r c r a f t as p a r t o f t h e F8-DFBW pro jec t . Ir,deed they are involved in the second phase of the i r ac t iv i ty , having comple ted a phase where t h e APOLLO d i g i t a l computer w a s used as t h e c e n t r a l c o n t r o l u n i t i n a fly-by-wire system. The current phase of the project involves replacing the APOLLO computer with a set of IBM AP-101 computers. Concerning r e l i a b i l i t y requirements for the basic fly-by-wire mechanization, it w a s determined that f l y by wire required quadruply redundant secondary actuators. The ac tua to r concept used i n t h e f i r s t p h a s e o f t h e i r work is i l l u s t r a t e d i n f i g u r e 2 ( r e f . 1 ) . The a c t u a t o r is cons t ruc ted wi th three tandem p i s tons 'on a common s h a f t . It is powered by two independent hydraulic supplies. It has four independent se rvosys tems to cont ro l the pos i t ion of the ac tua tor . Normal operat ion uses servosystem 1, which cons i s t s o f an ac t ive and a monitor, .two-stage, flapper nozzle servo valve. The active va lve i s monitored for f a i l u r e by the monitor valve using a hydraulic comparator. I f t h e d i f f e r e n c e i n o u t p u t of the monitor and act ive uni t exceeds a threshold , the supply pres - sure engaging the act ive valve w i l l be dumped t o r e t u r n ( s e e s l i d e v a l v e a t 70 lower part of servosystem 1 block). This will occur with motion of the com- parator spool in either direction. Simultaneously, the hydraulic engage valve for servosystem 3 is hydraulically enabled. The primary channel fail switch causes all remaining (servosystems 2 and 3) engage solenoids to be energized. Thus the actuator is configured as a force summed triple-tandem actuator when all engage valves are engaged. Additional failures will result in some degradation of performance. The real question is: what is the cost of providing redundancy in the form of hardware duplication? Some measure of that is obtained from the actuator example. This resulted from the existence of ground simulators required for F8 project support. Because of the lack of a requirement for quadruplex actuators in the simulator environment, Langley Research Center procured simplex actuators. The cost of the quadruplex actuators is approxi- mately $20,000 per actuator, whereas simplex actuators are available at one- quarter the cost. In addition to increased cost, there also remains a ques- tion of performance degradation in redundant systems. When Langley acquired their actuators, it was determined that their bandpass had to be reduced from 50 to 10 Hz to simulate the quadruplex actuators onboard the aircraft. PRESENT The other NASA activity to develop fly-by-wire technology is the space shuttle project. Early in the project, it was determined that mission requirements for the shuttle dictated fly by wire. In addition to flight con- trol requirements, the space shuttle has navigational reliability requirements that dictate advanced failure detection and redundancy management techniques not previously used. To meet rel-iability requirements, the shuttle avionics system evolved to a hardware configuration that uses hardware duplication for the different systems. For example, in hardware, the rate gyro assembly is duplicated three times and the computers five times. Management of he hard- ware redundancy for the components, a major problem, will be done by the computers. The computers operational flight program is divided into modules with al1,hardware interface handled in a module referred to as subsystem operations. That module acts as a system executive and controls all input/ output interfaces. It also passes data from the 1/0 interface to the appli- cations programs (guidance, navigation, flight control, etc.). A redundancy management module determines which data are to be passed to the remaining applications programs and to hardware. The techniques for doing that depend on the particular type of data being processed. Techniques used for sensor data, controller data, switch data, and actuator data tracking tests, builtin stimulus tests (BIST), and comparison tests. One unique feature related to redundancy management, not used in past aircraft applications, is the handling of the navigational data (fig. 3 ) . Three inertial measurement units are used in parallel to generate position and velocity estimates. The position and velocity estimates are then adjusted for new navigation data input by the navigational sensors using a Kalman-Bucy-type filter. A data selection algorithm is then used to determine data to be passed on to the remaining applications programs, namely, the guidance and control modules. 71 Methods used by the shuttle system to select data depend,on the level of data redundancy available. Candidate three-level methods are being considered which include simple average, sample middle select, Kaufmads weighted aver- age, and Landley's method. Two-level methods being Considered include aver- aging and voting using an alternate data source (i.e., using, say, NAV module generated pitch rate to determine which of two pitch rate gyros is functioning properly). In the two-level case, there was possibility of including an advanced redundancy management algorithm based on early elements of decision theory. For the two-level'case, a filter using Wald's sequential probability ratio test is being considered. Although it offers the potential of improved redundancy management (especially for small bias errors on.IMU data), the method was computationally burdensome. It, also relied heavily on the prior statistics determined from sensor bench tests. In addition to the navigation module, there is, of course, a requirement for management of redundancy in the more classical fashion for the shuttle for the controls and sensors. One proposed method being evaluated for han- dling the sensor redundancy is: under no failures, middle value select will be used with tracking test to detect failures. Failure thresholds will be taken about the selected signal. A failure to track within the threshold is considered to be a failure of the device if it fails to track for N consecu- tive samples nominally spaced at 40-msec intervals. After the first failure, the failure is displayed to the crew and tracking tests are mechanized the same as in the no failure case, except that thresholds are taken about the average of the two good sensors. Again, failure to track assumes that a failure has occurred if it occurs for N consecutive samples. The action to be taken following a second failure is to notify the crew and safety select a single unit. BIST would then be initiated on the remaining unit. If it is determined to be a good unit, an interchange would then be made of'the selected unit. FUTURE A s is being applied in the shuttle project, one important potential for digital control systems is their ability to reorganize themselves following failures in sensors and actuators. For sensors, this has been accomplished using duplication of components and simple voting process to manage the redundancy. Redundancy provided in that way can be termed "hardware" redun- dancy since it owes its existence to hardware duplication. Redundancy also exists between dissimilar components. However, to use that type of redun- dancy, one must be aware of the dynamic behavior of the.system and the inter- actions between system components. Such redundancy is more properly termed "analytical" redundancy since it owes its existence to analytical knowledge of the systems behavior. Langley Research Center is developing this type of redundancy management for use in aircraft flight control. It is anticipated that a system will be developed and flight tested on the F8-DFBW aircraft at FRC. Simulation tests of an "analytical" redundancy management system for that aircraft have been made at Langley. The theoretical approach used and 72 portions of the results of those tests are described here. The approach is reported in detail in reference 2. To use analytical redundancy requires that the designer have an analyt- ical model that expresses the response of the system to control command inputs. It also requires consideration of the level of confidence that the designer has in the modeling process. Figure 4 shows the farm of the model used for the F8 work. Basically, the model is a complete nonlinear six-degree-of- freedom model of, the aircraft's motion which has a good kinematic representa- tion using quaternions but uses an aerodynamic force representation that employs linear aerodynamic coefficient modeling. That is, the force and moment coefficients are linear functions of angle of attack and control posi- tions. Also note that the F8-DFBW is a digitally controlled aircraft and hence requires a discrete form representation of the aircraft dynamics. That discrete form was obtained using Euler's method using a step size of 1/32 sec to be consistent with Langley's real-time computer system. The selection of the statistics was made consider'ing that kinetic relations are more likely to be accurate than aerodynamic force and moment relations. Figure 5 outlines the approach taken for failure detection. That approach is derived from Bayesian decision theory and involves selecting a group of hypotheses that are checked for validity. The different hypotheses represent different assumptions regarding the status of different subsystems. For example, hypothesis HO might represent the assumption that all systems are operating normally. Hypothesis H1 might represent the hypothesis that a specific sensor has failed. Hypothesis H2 might represent the hypothesis that a multiple,failure has occurred where, say, both the roll and yaw rate gyros have failed. The rationale for making a decision is to construct a function that assigns a cost to making an incorrect decision and then to mini- mize that function. The weights Ci- shown onmthe slide are the weights that the designer assigns to deciding on iypothesis as true. Of course, if i = j, the decision is correct so that normally t e terms of the form Cii are taken to be zero. Figures 6 and 7 show simulation results that indicate the importance of the selection of the weighting coefficients. The results are generated from a simulation of an entry vehicle. Figure 6 shows the response of the vehicle with no augmentation on the left; on the right is the input response with augmentation. It is that response that we will look for in figure 7, which shows one continuous run of the simulation where different hypotheses are assumed and where a controller designed for the assumed hypo- thesis is used. In that run, Ho is the unfailed hypothesis, HI is failure of the roll rate gyro, H2 is failure of the yaw rate gyro, and Hg is failure of an angle-of-sideslip indicator. During the entire run, the yaw rate gyro was failed by increasing the noise superimposed on its output. Note the character of the response when that hypothesis (i.e., Hz) is used.' HA The resolution to the Bayesian hypothesis testing problem involves two items (fig. 8). First one must construct hypothesis-conditioned estimates of the aircraft's state for each hypothesis selected. This implies having a bank of conditional mean estimators parallel in operation. The second step involves examination of the performance of each hypothesis-conditioned estimator considering the performance realizable if that hypothesis is u ed. 73 This is mechanized as a test of the residuals of each estimator with weights that depend on'the expected error covariance of the estimates. A block diagram of a failure detection system that might be implyented on an aircraft is indicated in figure 9. To implement the procedure Fequires computational capability in excess of that available on flight computers of today. Figure 10 lists some practical considerations used to studykhe con- cept. Simulation results to be presented were generated on the F8ykimulation facility at Langley which involves a complete piloted six-degree-of-freedom simulation. The aerodynamic representation used in the simulation accounts for effects of Mach number, altitude, and aeroelasticity. Concerning the failure detection system, it used a nonlinear prediction model obtained using linear aerodynamics (as mentioned earlier). Filter gains for the different hypotheses-conditioned estimates were obtained using steady-state Kalman filter gains obtained at level flight equilibrium at 20,000-ft altitude and a Mach number of 0.6. Figure 11 shows the performance of that system when the simulated F8 aircraft is subject to a rapid roll maneuver. In these runs, the hypothesis tested were Ho - no failures, HI - roll rate gyro failure, and H2 - yaw rate gyro failure. Traces of the actual and estimated failure state are shown on the slide. NASA currently has a contract with the Draper Laboratory to mechanize and study requirements for an analytical redundancy management system relative to an F8-DFBW mechanization. It is anticipated that the output of that con- tract will lead to a flight test. The concept to be mechanized will be con- strained by computer requirements and will hence probably not be a mechaniza- tion of the bank of filters approach being studied at Langley. Systems employing banks of filters are items for the future. They will probably most easily be mechanized using dedicated microprocessors. Langley is initializing work on optimal decentralization of flight-control processes that use microprocessors as dedicated control units and signal processors. One experiment being considered is the construction of a microprocessor con- trol unit for a digital actuator test facility at Langley. That control unit would provide for actuator stabilization, failure detection, and redundancy management. It would, in conjunction with other processors, provide an advanced redundancy management system using ideas presented here. REFERENCES 1. Wiggins, Don: Redundant Hydraulic Actuation Systems for Fly-By-Wire Controls. Hydraulics Research and Manufacturing Co. Rept. HR79900058, Valencia, Calif., Aug. 1975. 2. Montgomery, R. C.; and Price, D. P.: Management of Analytical Redundancy in Digital Flight Control Systems for Aircraft. AIAA Paper 74-887, Aug. 1974. 74 Pilot's stick C** (a) Mechanical system . . (b) Fully Powered hydraullc system dll -by-wire cont ol ystem Figure 1.- Evolution of f l ight-control systems. Figure 2 .- Phase 1 quadruplex redundant secondary actuator for the F,b DFBW aircraft . 75 / ' ~ State vector update - State vector integralion Nov filter Covariance P Kalman gain .I Figure 3 . - Redundancy management of navigation for Space Shuttle. I I I X = F(X, U) t W xktl = @(Xl Uk) t W k kT ( k + l h T ime FOR THE PURPOSE 0F.DESIGNING THE REDUNDANCY MANAGEMENT LOGIC Wk IS ASSUMED TO BE GAUSSIAN AND ITS ST.4TISTICS ARE SELECTED BY THE DESIGNER OF THE SYSTEM THE SELECTED STATISTICS ALLOWS THE DESIGNER TO MAKE A TRADE BETWEEN CONFIDENCE I N THE ANALYTICAL MODEL AND CONFIDENCE IN THE SENSORS Figure 4.- Continuous and discrete problem consideration for management of analytical redundancy. 76 HYPOTHESIS SELECTION Hi: ylk) = hilxfk)) +vilk). i = 0. 1, , , M Elvi) ASSUMED UNKNWN. cw [Vilk)vifjq = Riijkj BAYESIAN RISK FUNCTION C.. IS COST OF ACCEPTING Hi WHEN H. IS TRUE Zi IS THE DECISION REGION WHERE Hi I S ASSUMED TRUE pH, IS THE A PRIORI PROBABILITY OF H. 1 11 1 J Figure 5.- Analytical redundancy man- agement using multiple hypothesis testing. 1. REQUIRES A BANK OF M CONDITIONAL MEAN ESTIMATORS OPERATING I N PARALLEL 2. DECISION LOGIC IS BASED ON EXAMINATION OF RESIDUALS SELECTING THE HYPOTHESIS WHICH, ON-LINE, CORRESPONDS TO THE M I N I M U M Si WHERE K 1 5. = - In 14.1 +- r'.(j) Q. r . l l ) -1 . K I 2 I 2 j;ll I I ri - RESIDUAL OF THE ESTIMATOR FOR Hi Qi - MEASUREMENT ERROR COVARIANCE K - NUMBER OF SAMPLES I N DATA BASE Figure 7.- Effects of accepting Ho, HI, Hz, and H3 when H2 is true. 10r AILERON STEP 2,- BANK ANGLE COMMAND Figure 6.- Rol l responses for simulated I re-entry vehicle. 2 r .4 r Figure 8.- Minimum Bayesian risk solution for analytical redundancy management. - T - : -T ORPER Y+ AIRCRAFT SENSOR - CYNAMICS - DYNAMICS - I ESTIMATES 7 RES I DUALS BANK ff LINEAR FILTERS Figure 9.- Analytical redundancy management system. SENSOR FAILURE LARGE AND MOST OF THE COMPUTING IS INVOLVED I N UPDATING THE COVARIANCE MATRICES. MEASURED THE SYSTEM SELECTED FOR SIMULATION STUDY USES: YAW RATE, 0 A NONLINEAR PREDICTION MODEL OBTAINED FROM THE NONLINEAR S I X DEGREE-W-FREEDOM EQUATIONS OF MOTION OF THE AIRCRAFT USING F LINEAR AERODYNAMICS. CONSTANTFILTER GAINS AND ERROR COVARIANCES OBTAINED USING STEADY STATE KALMAN FILTER RESULTS. ~ 0 10 20 30 40 50 60 TIME, sec Figure 10.- Some practical considera- Figure 11.- Failure detection perfor- tions in implementation. mance during maneuvers using asix- degrees-of-freedom simulation of the F8 DFBW aircraft. 78 FACTS AND FICTION OF LEARNING SYSTEMS* George N. Saridis Purdue University; Indiana 1. INTRODUCTION The latest developments in computer technology and the ensuing successes of space exploration has generated a widespread belief that the aircraft and spacecraft of the future will be completely automated machines that may reliably accomplish preprogrammed treks and explorations in our galactic system. It has been also widely assumed that modem optimal control theories pro- vide the appropriate methodology to generate such automated controls. This may be so, but only in the far future when the computer technology would have progressed sufficiently more in sophistication and miniaturization. However, to meet the realistic demands of modern highly sophisticated high-speed air- craft and spacecraft requiring for their operation advanced system'reliabil- ity, a man-machine interactive control system represents the optimal design. Such a system would require high-precision electromechanical controls as well as higher level decision-making and task coordination and planning, accepting only qualitative commands and task override by the human operator. Reliabil- ity of this nonconventional control system requires almost anthropomorphic functions for the controller such as training capabilities for the hardware for performance improvement in the presence of uncertainties, software optimal decision-making, and interface with the human operator. The methodology that will provide the updated precision for the hardware control and the advanced decision-making and planning in the software control is called "learning systems and intelligent control." It has been developed, theoretically as an alternative for the nonsystematic heuristic approaches of artificial intelligence experiments and the inflexible formulation of m dem. optimal control methods. Its basic concepts are discussed in the sequel and some feasibility studies of some practical applications are presented. 2. LEARNING SYSTEMS, SELF-ORGANIZING, AND INTELLIGENT CONTROL . . The concept of "learning" has been used in- system design and control whenever the a pr ior i information of the underlying process is uqcertain or unknown (ref. 1). This concept may be applied with equal effectiveness to identify uncertain parameters or improve directly the performance of th *This research was supported by NSF Grant GK-36607. 7 9 I I I 111 11111.".1 I 1 1 1 I .111.1 I. .".I ,I . . - . . .. - I system by appropr i a t e h ighe r level decision-making which is sequen t i a l ly updated by previous experience. The fo l lowing def in i t ions regard ing learn ing sys tems and l ea rn ing con t ro l systems have been formulated by a subcommittee of Control Systems Society of IEEE and are presented here for completeness ( ref . 2 ) . Definition I. A system is c a l l e d Zeaming i f t he informa- t i o n p e r t a i n i n g t o t h e unknown fea tu res o f a p rocess o r i ts environment i s acquired by the system, and the obtained experience i s used fo r fu tu re e s t ima t ion r ecogn i t ion , classi- f ica t ion , dec is ion or cont ro l such tha t the per formance of the system w i l l be improved. Definition 2. A learning system is c a l l e d a Zeaming control system i f t h e a c q u i r e d i n f o r m a t i o n i s used t o c o n t r o l a pro- cess wi th unknown fea tu res . Note tha t l earn ing sys tems represent a wide class of processes , wider than control systems. A s such they may serve as high-level decision-makers, p rob lem so lve r s , and t a sk p l anne r s and o rgan ize r s i n a r t i f i c i a l ly in t e l l i gen t sys t ems l ike autonomous r o b o t s , i n t e r a c t i v e man-machine systems, bionic devices , and pat tern recognizers . Such learn ing sys tems tha t may perform on-line as w e l l as off-Zine are charac te r ized as inteZZigent systems or con- trol and are intended for highly sophis t icated anthropomorphic funct ions. When l ea rn ing i s appl ied to control hardware systems, one may devise a c o n t r o l l e r t h a t would on-line improve i t s 'performance regardless of uncertain- ties governing i t s mathematical modeling. Such control systems have been charac te r ized as self-organizing controZ systems and are d e f i n e d i n r e f e r - ence 2 as fol lows : Definition 3. A cont ro l p rocess is c a l l e d self-organizing i f r e d u c t i o n o f t h e a priori u n c e r t a i n t i e s p e r t a i n i n g t o t h e e f f ec t ive con t ro l o f t he p rocess i s accomplished through information accrued from subsequent, observations of the access ib l e inpu t s and outputs as the control process evolves . A c o n t r o l l e r s t r u c t u r e d t o p e r f o r m t h e above t a s k on-line i s c a l l e d a seZf-organizing controZZer. Definition 4 . A se l f -organiz ing cont ro l p rocess i s c a l l e d parameter-adaptive i f i t is poss ib l e to r educe the a p r i o r i unce r t a in t i e s o f a parameter vec tor charac te r iz ing com- pletely the process through subsequent observat ions of the access ib l e inpu t s and outputs of the system as the con t ro l process evolves. A con t ro l l e r s t ruc tu red to pe r fo rm the above t a sk on- Zine is c a l l e d a parameter-adaptive S. 0. controZler. Definition 5. A se l f -organiz ing cont ro l p rocess i s c a l l e d performance-adaptive i f i t i s p o s s i b l e t o r e d u c e d i r e c t l y t h e u n c e r t a i n t i e s p e r t a i n i n g t o t h e improvement of the perfoqnance 80 of the process through subsequent observations of the acces- sible inputs and outputs of the system as the control pro- cess. A controller structured to perform the above task on-line is called a performance-adaptive S.O. controller. It is obvious from the previous discussion that learning systems cover a wide area of decision-makers, ranging from highly sophisticated intelligent machines to advanced control systems. Generally, however, they deal with com- plex processes for which simple classical control techniques cannot be applied. Actually, this is one of the reasons why learning systems have not been popular with scientists and engineers until recently. The 'processes under their consideration have been too simple and too well defined to require any advanced decision-making. In the last few years, the study of large-scale systems in "soft sciences," energy environment, space technologies, etc., have found the exist- ing analytic tools inadequate. Therefore, scientists started looking for new methodologies, one of them being "learning systems," which provide a new dimension in the various control problems. In particular, in aircraft and space technology with the new high-speed sophisticated aircraft and space vehicles operating in unknown environment, two main reasons prompted the use of self-organizing control as a learning system for reliable operation: the uncertainties involved with the mathematical model and the need of lower-order models to represent high-dimensional systems. Several feasibility studies have proven the advantages of such control- lers and certain actual tests are already in progress by Honeywell and will be reported in these proceedings. A review of the most successful self-organizing control methods is pre- sented in the next section, while two feasibility studies performed by the author and his colleagues are discussed in the sequel. However, a word of caution is appropriate here; learning systems and self-organizing controls should not be used to drive systems for which a simple controller designed by classical methods is adequate. Sophisticated techniques are needed only when complex systems require them. 3. A REVIEW OF SOME SELF-ORGANIZING CONTROL METHODOLOGIES Numerous methods that qualify under self-organizing control definitions have appeared in the literature since 1950. Most of these methods have appeared in expository publications (refs: 3-17) under various names such as adaptive, optimalizing, self-optimizing, etc., systems. They may be sub- divided into two categories, the first one treating systems operating in a deterministic environment and the other for systems operating in a stochastic environment. The first category covers systems like Margolis and Leondes' parameter tracking servo, the polynomial expansion methods, the automatic spectrum analyzer approach, the functional analysis approach, the model refer- ence adaptive control methods, error correcting methods, and many others (ref. 12). 81 The second category, which is the most realistic of the two since it incorporates in the formulation the irreducible uncertainties introduced by the environment, covers a collection of algorithms with various degrees of uncertain,ties about. the.plant dynamics.. In the subcategory of parameter- adaptive self-organizing (SO) controls (ref. 12), one may have to select among the linearized stochastic optimal-control algorithms (ref. 18), the open-loop feedback.optima1- control.algorithms (refs. 19 and 20), the Bayesian learning algorithms (refs. 21-23); the parallel identification and control (refs. 24 and 25), and.the actively .adaptive.control algorithm (ref. 26). The. charac- teristic of. this subclass i .that a parameter identification is necessary to provide the information about.the uncertainties of the process. In the sub- class oE.perfo-rmance .adaptive.SO controls, the choice may be made among the crosscorrelations approach (refs. 27 and 28), sto.chastic automata methods, (refs. 29 and 30),.the stochastic approximation algorithm (refs. 31 and 32), the expanding subinterval algorithm (ref. 331, etc. The characteristic prop- erty of this subcategory is that the search for the reduction of uncertainties pertains to the evaluation of the performance criterion which indicates the improvement of the performance of the systems. Review monographs that give a detailed account of the formulation of these algorithms have recently appeared in the control literature ,(refs. 7 and'lz), and several applications in man- machine and robotic systeuis as well as,spacecraft control (refs. 19 and 34) have been discussed. S,ince the space is rather limited here, no detailed account is given and the interested reader is referred to the literature. Instead, two space vehicle studies, utilizing SO control techniques and being produced by the author and his colleagues as feasibility studies, are pre- sented in the sequel in terms of simulation results. 4 . ' PARAMETER-ADAPTIVE SO CONTROL FOR BOOSTER STAGE OF A SPACE VEHICLE Parallel identification estimation and control is used as a typical parameter-adaptive SO control Scheme,to demonstrate the feasibility and appli- cation of the method to a booster,stage of a Saturn space vehicle (ref. 2 4 ) . In this application, the goal is the simultaneous flight parameter identifica- tion and control of the vehicle during its mission, that is, from launch to stage separation while operating in a stochastic environment, generated by external wind gusts and control input disturbances. Considering a pitch plane motion only, the booster is shown diagrammati- cally in figure 1, were a is angle of attack (deg), 8 is attitude angle (deg), and 6 is gimbal angle of the rocket engine thrust chambers (deg). All. of the ab0v.e variables are perturbation angles from a reference trajec- tory. Thus the control problem will be one of maintaining small deviations about a nominal, flight path. For small-angle variations and other simplifying assumptions, the rigid-body equation of motion and the dynamics of the engine actuator are given, respectively, by 82 where pa and ~6 are unknown time-varying flight parameters and 6 c is the gimbal command (in deg) and l/kC is the engine thrust chamber gimbaling time constant. As it stands, the rigid-body equation of motion in equation (1) represents an.aerodynamically unstable vehicle. Therefore, a stabilization autopilot is initially synthesized using 'tworst-case't flight parameters pa and p6 to ensure stability of the uncontrolled system throughout i s flight profile. Generally, the values for the worst-case flight parameters are known from prior launch data. Figure 2 is a block diagram of the engine actuator- vehicle dynamics and autopilot gains. The constants kA, kR, and kc are chosen to ensure stability of the overall system for worst-case flight param- eters. The variable 8c represents the attitude conrmand input and is used to control the state of the system. Letting q = 6 , the system in figure 2 can be written in state variable form as 1 where Oc represents the feedback control input. The newly introduced terms y and w correspond to the perturbation noise input and system disturbance input, respectively. Using attitude and rate gyros, the output measurement equation for the system in equation (3) is given by The two output measurement signals are therefore contaminated by additive noise disturbances to account for external wind gusts. Using the numerical values in reference 24 and a sampling time of 0.005 sec, the system is converted to the discrete-time form: 1.0009 0.0049794 0.00221] [" (k)l 0.29102 0.98778 0.8546 q (k) 6 (k+ 1) -0.028015 0.8052 6 (k) 0.0003998 [ 0.48036 ] + 0.23582 [BC(k) + y(k) + w(k)] 83 where z(k) = [I :;! and v(k) = [ v1(k) e ] The random variables y(k), w(k) ,. and v(k) are assumed to be jointly v2 (k) independent and have Gaussian distributions with the statistical properties: k,j = 0, 1, 2, ... J For this problem, the performance criterion to be minimized s given by 3000 J(U) = E 1 Ilx(i + 1>112 + reg(i) 3000 i= 0 Q where Saturation bounds on eC were set at +5' to ensure that the system variables would not become unrealistically large during the period of identification, To set the matrices in a form suitable for identification, the following transformation is necessary for which the controllability and observability subindices are Using the selection procedure in equation ( 6 ) , define a matrix 84 Through the following change of coordinates the system (eqs. (3) and ( 4 ) ) is transformed into the canonical form: x(k+l) = Aox(k) +B0[OC(k) +y(k) +w(k)] = I 0.00039986 b + 0.23582 [eC(k) + y(k) + w(k)] ; m = 2 [ 0.643 ] z(k) = ]x(U + [:: = Hox(k) + v(k) 0 / (12) The parallel identification, estimation, and control scheme is presented in figure 3. Observing that system (10) is completely controllable and com- pletely observable and that if the random variables y(k), w(k), and v(k) have finite moments up to the fourth order, the identifier will produce esti- mates on the matrices A, and Bo of system (1) by use of a stochastic approx- imation algorithm such as where T ' T Bo, hy AoBo, . . . , hy AE-'B0 .; i = 1, 2, . . . , m 1 T T hy , AoDor . . , hi A, '-'Do] ; Do 5 Bo U( , ) (k+g- l ) = [ u ( k + g - l ) , u ( k + g - 2 ) , ..., u ( k ) ] T T T T W(k)(k+ g- 1 ) = [W ( k + g - l ) , w ( k + g - 2 ) , .. . , w (k) ] T T T T The term H, in equat ion (10) has been par t i t ioned as H: = [hy, hz, ..., hm] 0 and Si(O) i s chosen arbitrari ly. For convergence of equation (11) , The o r i g i n a l parameters are recovered by the following general transformations used here for m = 2: ... ... ... """- " " " _ e: . .. e; ... \ P1 p2 , Pm i 86 e?' .. . e? ... e:*?. . e:*2 .. . 1 pir+l el ... p r+2 1 ... ... A. = X e:+' . . . Pmrfl em ... Pmr+2 'm ... -1 qrr .. . el e: qrr . .. e, A quick glance shows that the above algorithms are dependent on knowledge of the statistics of the noises w(k), 'v(k), y(k), or of the matrix Do. The structure of the feedback controller can also attain an arbitrary form; in fact, it could be made dependent on the past of the identification estimates in equation (14). The asymptotically optimal controller designed in parallel of the identi- fier is given by the following. The resulting form of the controller consists of a cascaded discrete-time Kalman filter and deterministic control gain, both of these elements using estimates of the unknown system matrices from an iden- tifier. Thus the controller structure attains the following simple form: where K"(k) is the steady-state deterministic optimal gain at time k and is obtained by a backward iterative procedure on the discrete-time version of the matrix Riccati equation 87 Estimates of the unknown system matrices from the identifier $(k) , Bo(k) are used in the above equations in place of A,, Bo. At any time k, the station- ary valgle of Ki(k) that results by iterating backward with respect to i is designated K"(k). I A The state estimate G(k) in equation (15) is obtained using a discrete- time Kalman filter and again uses estimates of the unknown system matrices in place of A,, Bo. The equations for the filter are then given by G(k+ 1) = ko(k) G(k) + Bo(k) [u(k) + y(k)l + KF(k+ 1) z ( k + 1) ( - Ho{io(k) G(k) + io(k) [u(k) + ; G(0) = 0 (19) and KF(k+ 1) = P(k+ Ilk) HoTIHoP(k+ llk)Ho + U211-1 P(k+ Ilk) = Ao(k) P(kl k) A, (k) + q2DoDo T T P(k+lIk+l) = [l - KF(k+l)Ho] P(k+llk), P(o/o) specified where A saturation-type nonlinearity is inserted in the feedback loop to induce a stabilizing effect on the overall system and to more naturally depict the characteristics of all physical feedback controllers (i.e., they inherently saturate at a certain level). Simulation of the system in figure 3 using the above controller was per- formed on a CDC-6500 digital computer. The results of the identification are presented in figure 4 and the performance cost results in figure 5. In these figures, the average normalized error represents a 15-run ensemble average and hence is an approximation to the mean-square normalized error. It is apparent after 10,000 iterations that the identification is consistent. For a sampling interval of 0.005 sec, this corresponds to a real time of 50 sec which repre- sents about 1/3 of the total flight time. To shorten the identification i n t e r v a l , 0.001 s e c / i t e r a t i o n i s achievable with present-day general purpose, onboard d i g i t a l computers (for more d e t a i l s , see ref . 24) . 5 . PERFORMANCE ADAPTIVE SO CONTROL FOR A SPACE ATTITUDE CONTROL SYSTEM The f e a s i b i l i t y of application of the expanding subinterval performance adapt ive SO con t ro l method has been invest igated in conjunct ion with a s ingle- axis a t t i tude cont ro l problem of an o rb i t i ng satell i te wi th random switching de lay ( re f . 35). The method w a s selected because i t y i e l d s a g loba l minimum of accrued performance, it is easy to implement, and does not depend on t h e knowledge of t he sys t em o r the no i se dynamics, thus generating a r e l i a b l e algorithm for control of processes with extremely uncertain dynamics. In the p a r t i c u l a r case of a t t i t u d e c o n t r o l , t h e problem may b e i n t e r p r e t e d as a double-integral plant with equations of motion: The i n i t i a l c o n d i t i o n x ( t o ) w a s assumed to be zero , and the random v a r i a b l e s E(t) and n ( t ) were assumed to be Gauss i an wi th s t a t i s t i c s : E [ S ( t ) l = 0; cov [ S ( t ) ; 5 ( ~ ) 1 = s ( t - T ) E [ n ( t ) l = 0; cov [ n ( t ) ; n ( ~ > l = E :]a(f- 'f) The form of t h e SO c o n t r o l l e r i s a s impl i f ied vers ion of the dead-zone con t ro l l e r ob ta ined in r e fe rence 35 f o r a g e n e r a l s t o c h a s t i c f u e l r e g u l a t o r problem of which t h i s i s a s p e c i a l c a s e ( s e e f i g . 6 ) : u*(G,t) = -DEZ{i*[%(t),c,t]) (21) where w [ x ( t ) , c , t ] is the p red ic t ed va lue of the approximate switching func- t i o n w* [ i i ( t ) ,a, t 1 , included to minimize the inf luence of the random switching ** A 89 delay with unknown statistics caused by component imperfections. The two func t ions are def ined as . . The functions (9i[G(t) , t ] , i=l,. . , s , $ e (w*,t), j=l,. . . ,v are proper ly chosen t o s a t i s f y t h e c o n d i t i o n s f o r r e a l i z a b i l i t y o f t h e e x p a n d i n g s u b i n t e r v a l a lgori thm Consequently, for which the parameter space Qc was taken from real d a t a for o r b f t i n g satellites t o b e IC: 0.05 s a,, 5 0.5, Ia11/a121 1 2, "1 3 Ec: 0.05 ,< a l l 5 0.5, lall/u121 I 2 a13 - a12, B,, = 1, 0 5 B 1 2 i 0.5) (case 3) - - a12, B l l = 1, Sl2 = 0) (cases 1 and 2) I (27) - where cases 1 and 2 treat the problem without compensating the random switch- ing delay (e .g . , w* = i i n eqs. (20) and (21)) , while case 3 does. 90 The random search was implemented by where Ti is the expanding subinterval Ti = ti- ti- l , i=1,2, . . . p Ti-l I Ti wi and lim Ti = (29) i+- over which the following "subgoal" is 'defined: In equation (28),.pi is the ith sample value of a vector random variable defined at every subinterval Ti on S-2, = (C/U~(~)ES-~~~, with probability den- sity function p(P)-f 0 for all PEL',; p is an arbitrary positive number. It has been shotm in reference 33 that, if the performance cost accrued during the search is defined by then in the limit and under certain conditions of convergence (ref. 33), lim Prob [V(k) - Imin < 361 = 1 ; k-tco I(uz) = Min I(uc) = Irnin < 03 C An EAI-$80 analog computer was used in the simulation. As expected, the system is very sensitive to changes in the switching delay. In fact, for time scaling beyond 1000:1, the computer switching tolerance and dynamic response are such that analog simulation is not feasible. On the other hand, the sys- tem is quite.insensitive to the exact filter gain. To analyze the "transient" aspects, three cases of the proposed control were considered. For cases.1'and 2 , no attempt was made to compensate for the random switching delay. The difference in these two cases is in the form of the probability density p(p). In case 1, the expected value of the-"test-point" p ( k ) is the present value of the parameter vector a(k) , while in case 2 a bias is included to reinforce the motion in a successful direction. Case 3 I is similar to case 2, but it adds the "predictive" aspect to reduce the effect of the random switching delay. The experimental results are given in terms of an ensemble average in a normalized per iteration cost instead of V(k), as defined in equation (29), because it is more meaningful during the transient phase. The average per iteration cost and its standard deviations are plotted in figures 7 and 8. The statistics depicted in figures 1 and 2 are based on sample sets of 100 independent simulations. Each run was sufficiently long to establish the steady-state per iteration cost. In all runs the limiting accu-.. racy was governed by the accuracy of reading the analog records. It 'is ' ' observed that both the average and standard deviations of the per iteration cost have converged, in a practical sense, within the first 100 iterations. . Acceleration schemes, based on heuristic selection of the search variable pi in equation (28) have been used in the simulation (ref. 35). Figure 9 demon- strates the steady-state responses of the system before and after the tran- sient period. 6. RELIABILITY CONSIDERATIONS The reliability problem in the design of feedback controllers for air- craft and spacecraft systems may be considered from a different aspect, in view of the previous discussion. Modern high-speed vehicles with.uncertain- . ties in their dynamic models operate in a random environment. Stochastic solutions (refs. 36 and 37) are based only mostly on average a pr ior i informa- tion about the uncertainties and are usually poor approximations of.the desired nonlinear and optimal solution (ref. 38). Such solutions are highly.. ' unreliable for practical design purposes. Considerable improvement in relia- bility may be obtained if the process uncertainties may be reduced through a learning scheme in the controller, designed to acquire asymptotically its optimal value. SO controls provide such controllers that are also easy to implement and require considerably smaller computers than the general stochas- tic solutions. The previously presented feasibility studies should provide sufficient arguments to support this statement. From other comparative studies (ref. 12), it was found that the parallel identification estimation and control algorithm is 6 times faster, requires 18 times less core memory, has a 30-percent lower value of the performance cost over a linearized approx- imation of the stochastic optimal with unknown parameters, is 3 times slower, has the same core memory requirements, and gives 4 times higher performance cost than the stochastic optimal with known parameters. The expanding sub- interval algorithm designed for systems with almost unknown dynamics is rather slower than the parameter adaptive algorithm but compares favorably otherwise (ref. 12). Additional acceleration may further speed up the random search technique. The above results were obtained from simulations on a CDC-6500 digital computer. 92 7. DISCUSSION In conclusion, improvement in the reliability of the control algorithms may be obtained by using learning techniques to reduce the uncertainties of the systems and obtain an asymptotically optimal solution in a stochastic environment. These ideas have already been explored with various degrees of success by various areas of modem large-scale systems, as bioengineering, socio-economics, environmental, man-machine systems, etc. It is appropriate to mentiori that learning system approaches are superior to class'ical and other modern control methods only when the system involved is complex, when there are uncertainties in the system that can be reduced by learning instead of using their average values, and if an optimal solution could be obtained if the model of the system was completely known. Then, by use of l arnYng tech- niques, one obtains solutions that are more reliable asymptotically than otherwise. However, if the system is simple enough that a stochastic or dual optimal solution is available, such a method will produce an overdesigned con- troller with very poor engineering characteristics. For this reason, certain over-designed learning controllers have created some skepticism already about learning and SO control to engineering circles. This skepticism, although justifiable, should not be extended to the problems considered here. The problem of designing a SO control is by no means completed in the two feasibility studies presented. Stability analysis safety.specification, etc., are required for a reliable engineering design. Some of these problems are discussed in reference 12. Most of the problems encountered in the design of SO control systems are still under investigation. However, the purposes of this paper are to motivate interest and to inspire enough confidence in the researchers and development engineers in the area to use these ideas in modem aircraft and spacecraft technology. REFERENCES 1. 2. 3. 4. 5 . Bush, Robert R.; and Mosteller, Frederick: Stochastic Models for Learn- ing, John Wiley & Sons, 1955. Saridis, G. N.; Mendel, J. M.; and Nikolic, Z. J.: Report on Definition of Self-organizing Control Processes and Learning Systems, IEEE Control System SOC. Newsletter #48, April 1973. Fu, K. S.: Learning Control Systems. Advances in Information Systems Science, Julius T. Tou, ed., vol. 1, Plenum Press, 1969, pp. 251-292. Fu, K. S.: Learning Control Systems and Intelligent Control Systems: An . . Intersection of Artificial Intelligence and Automatic Control. IEEE Trans. Auto. Control, vol. AC-16, no. 1, Feb. 1971, pp. 70-72. Fu, K. S.: Learning Control Systems. Computer and Information Sciences, J. T. Tou and R. H. Wilcox, eds., Spartan Books, 1964. 93 6. Fu, King-Sun: Learning Control Systems - Review and Outlook, IEEE Trans. Auto Control, vol. AC-15, April 1970, pp. 210-221. 7. Fu, K. S., ed.: Learning Systems, a Symposium of the AACC Theory Com- mittee. ASME Publ., June 1973. 8 . Mendel, J. M.; and Fu, K. S.: Adaptive Learning and Pattern Recognition Systems: Theory and Applications, Academic Press, 1970. 9. Mishkin, Eli; and Braun, Ludwig, Jr., eds.: Adaptive Control Systems, McGraw-Hill Book Co., 1961. lo-., Pugachev, V. S.: Optimal Training Systems. Soviet-Physics-Doklady, vol. 12, no. 8, Feb. 1968, pp. 762-764; Translation of Akademii nauk SSSR Doklady, vol. 175, no. 5, Aug. 1967, pp. 1022-1025. 11. Saridis, G. N.: On-Line Learning Control Algorithms. Learning Systems, K. S. Fu, ed., ASME Publ., June 1973. 12. Saridis, G. N.: Self-organizing Control of Stochastic Systems, Marcel- Dekker, New York, 1976 (in press). 13. Schweppe, Fred C.: Uncertain Dynamic Systems, Prentice-Hall, 1973. 14. Sklansky, Jack: Learning Systems for Automatic Control, 1EEE.Trans. Auto. Control, vol. AC-11, no. 1, Jan. 1966, pp. 6-19. 15. Sworder, David D.: Optimal Adaptive Control Systems, Academic Press, 1966. 16. Tsypkin, Ya. Z.: Adaptation and Learning in Automatic Systems. Trans. by Z. J. Nikolic, Academic Press, 1971. 17. Tsypkin, Ya. Z.: Foundations of the Theory of Learning Systems. Nauka, Moscow, 1970. English translation, Academic Press, 1973. 18. Jenkins, K. W.; and Roy, R. I.: A Design Procedure for Discrete Adaptive Control Systems. Preprints of Conference Papers, 7th Joint Automatic Control Conference, Aug. 17-19, 1966, p. 624. 19. Mendel, J. M.: Applications of Artificial Intelligence Techniques to a Spacecraft Control Problem. NASA CR-755, 1967. 20. Tse, E.; and Athans, M.: Adaptive Stochastic Control for a Class of Linear Systems, IEEE Trans. Auto. Control, vol. AC-17, Feb. 1972, pp. 38-51. 21. Jones, Lloyd E. 111; and Fu, K. S.: On the Selection of Subgoals and the Use of a Priori Information in Learning Control Systems, Automatica, J., vol. 5, no. 6, Nov. 1969, pp. 705-720. 94 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. Saridis, G. N.; and Dao, T. K.: A Learning Approach to the Parameter Adaptive Self-organizing Control Problem. Automatica, vol. 9, no. 5, Sept. 1972, pp. 589-597. Stein, G. ; and 'Saridis, G. N. : A Parameter Adaptive Control Technique. Automatica, vol. 5, no. 6, Nov. 1969, pp. 731-739. Lobbia, R. N.; and Saridis, G. N. : Identification and Control of Multi- variable Stochastic Feedback Systems, J. Cybernetics, vol. 3, no. 1, Jan.-Mar. 1973, pp. 40-59. . . . Saridis, G. N. ; and Lobbia, R. N. : Comments on Parameter Identification and Control of Linear Discrete Time System, IEEE Trans. Auto. Control, vol. AC-20, no. 3, June 1975, pp. 442-443. Lee, Robert C. K.: Optimal Estimation Identification and Control, M.I.T. Press, 1964. Narendra, K. S . ; and McBride, L. E.: Multiparameter Self-optimizing Systems Using Correlation Techniques. IEEE Trans. Auto. Control, vol. AC-9, no. 1, Jan. 1964, pp. 31-38. Narendra, K. S . ; and Streeter, D. N.: An Adaptive Procedure for Con- trolling Undefined Linear Processes, IEEE Trans. Auto. Control, vol. AC-9, no. 4, Oct. 1964, pp. 545-548. McLaren, R. W.: A Stochastic Automation Model for the Synthesis of Learning Systems, IEEE Trans. System Sci. Cybernetics, vol. 2, no. 2, Dec. 1966, pp. 109-114. McLaren, R. W.: A Continuous-Valued Learning Controller for the Global Optimization of Stochastic Control Systems. Seminar on Learning Processes in Control Systems, Nagoya, Japan, Aug. 1970, pp. 77-81. Nikolic, Z. J.; and Fu, K. S . : An Algorithm for Learning Without External Supervision and Its Application t Learning Control Systems, IEEE Trans. Auto. Control, vol. AC-11, July 1966, pp. 414-422. Tsypkin, Ya. Z.: Adaptation, Learning and Self-Learning in Automatic Systems. Avtomatika i Telemekhanika, no. 1, 1966. Saridis, George N.: On a Class of Performance-Adaptive Self-organizing Control Systems. Pattern Recognition and Machine Learning, K. S . Fu, ed., Plenum Press, 1971, pp. 204-220. Mendel, Jerry M.: Survey of Learning Control Systems for Space Vehicle Applications, Preprints of Conference Papers, Joint Automatic Control Conference, Univ. of Washington, Seattle, Aug. 17-19, 1966, pp. 1-11. , 95 I I 35. S a r i d i s , George N.; and Gilber t , Harold D.: Self-organizing Approach t o the Stochast ic Fuel Regulator Problem, IEEE Trans. Systems Sci. Cybernet ics , vol . 6 , no. 3, July 1970, pp. 187-191. 36. Aoki, Masano: Optimizat ion of Stochast ic Systems; Topics in Discrete- Time Systems. Academic P res s , 1967. 37. Kushner, Harold J.: S t o c h a s t i c S t a b i l i t y and Control, Academic P res s , 1967. 38. Fel'dbaum, A. A.: Optimal Control Systems. Translated by A. Kraiman, Academic Press, 1966. BIBLIOGRAPHY istrijm, K. J.; and Wittenmark, B. : On Self Tuning Regulators, Automatica, vol. 9, no. 2, March 1973, pp. 185-199. Badavas, Paul C.;.and Sar.idis, George N. : A Performance Adaptive Self- Organizing Control for a Class of Distr ibuted Systems, IEEE Trans. SMC, vol. SMC-1, no. 2, April 1971, pp. 105-110. Bar-Shalom, Yaakov; and Sivan, Raphael: On the Optimal Control of Discrete- T ime Linear Systems with Random Parameters, IEEE Trans. Auto. Con- t r o l , v o l . AC-14, no. 1, Feb. 1969, pp. 3-8. Bellman, Richard: Adaptive Control Processes: A Guided Tour, Princeton Univers i ty Press , 1961. Cooper, G. R.; Gibson, J. E . , e t al.: A Survey of the Philosophy and State of t h e A r t of Adaptive Systems. Tech. Report, PRF 2358, Purdue Univ., J u l y 1960. J Cox, Henry: On the Est imat ion of State Variables and Parameters for Noisy Dynamic Systems. IEEE Trans. Auto. Control, vol. AC-9, no. 1, Jan. 1964, pp. 5-12. Curry, Renevick E.: A New Algori thm for Subopt imal Stochast ic Control . IEEE Trans. Auto. Control, vol. AC-14, no. 5, Oct. 1969, pp. 533-536. Deshpande, J. G.; Upadhyay, T. N . ; and L a i n i o t i s , D. G.: Adaptive Control of Linear Stochastic Systems. Automatica, vol. 9 , no. 1, Jan. 1972, pp. 107-115. Eveleigh, Virgil 'W.: Adaptive Control and Optimization Techniques. McGraw- H i l l Book Co., 1967. Far ison, James. B.; Graham, Richard E.; and Shelton, Roy C.: I d e n t i f i c a t i o n and Control of Linear Discrete-Time Systems. IEEE Trans. Auto. Control, vol . AC-12, no. 4, Aug. 1967, pp. 438-442. 96 Florentin, J. J.: Optimal Probing, Adaptive Control of a Simple Bayesian System. J. Electronics and Control, vol. 13, no. 2, Aug. 1962, pp. 165-177. Fu, K. S.: Learning Techniques in System Design - A Brief Review. Fifth World Congress of IFAC, Paris, June 12-16, 1972. Fu, K. S.: Learning System Theory, Chapter 11, System Theory, L. A. Zadeh and E. Polak, eds., McGraw-Hill Book Co., 1969. Fu, King-Sun, ed.: Pattern Recognition and Machine Learning. Plenum Press, 1971. Fu, King-Sun: Sequential Methods in Pattern Recognition and Machine Learning Academic Press, New York, 1968. Fu, K. S., ed.: Pattern Recognition and Intelligent Robots, Plenum Press, 1974. Fu, K. S.: A Class of Learning Control Systems, Using Statistical Decision Functions. Plenum Press, 1966, pp. 3-13. Fu, K. S.: Stochastic Automata as Models of Learning Systems, Computer and Information Sciences-11, Julius T. Tou, ed., Academic Press, 1967, pp. 177-191. Fu, K. S.; and Li, T. J.: Formulation of Learning Automata and Automata Games. Information Sciences, vol. 1, July 1969. Fu, K. S.; and McLaren, R. W.: An Application of Stochastic Automata to the Synthesis of Learning Systems. Tech. Rept. TR-EE 65-17, School of Electrical Engineering, Purdue University, Sept. 1965. Fu, K . S.; and McMurtry, G. J.: A Study of Stochastic Automata as Models of Adaptive and Learning Controllers. Rept. TR-EE 65-8, Purdue University, June 1965. 0 Fu, K. S.; and Nikolic, Z. J.: On Some Reinforcement Techniques and Their Relations with Stochastic Approximation. IEEE Trans. Auto. Con- trol, vol. AC-11, no. 4 , Oct. 1966, pp. 756-758. Fu, K:S.; Nikolic, Z. J.; et al.: On the Stochastic Approximation and Related Learning Techniques. TR-EE-66-6, School of Electrical Engineer- ing, Purdue University, April 1966. Gibson, J. E.; Fu, K. S.; Hill, J. D.; Luisa, J. A.; and Raible, R. H: Philosophy and State of the Art of Learning Control Systems, Tech. Rept. TR-EE 63-7, Purdue University, Nov. 1963. 97 Gorman, Daniel; Zaborsky, John: Stochastic Optimal Control of Continuous Time Systems with Unknown Gain. IEEE Trans. Auto. Control, vol. AC-13, no. 6, Dec. 1968, pp. 630-638. Hill, J. D.; and Fu, K. S.: A Learning Control System Using Stochastic Approximation for Hill-Climbing, Preprints of Conference Papers, 6th Joint' Automatic Control Conference, June 22-25 , 1965, pp. 334-340. Ho, Y. C.; and Lee, R. C. K.: A Bayesian Approach to Problems in Stochastic Estimation and Control. IEEE Trans. Auto. Control, vol. 9, no. 5, Oct. 1964, pp. 333-339. Ivakhnenko, A. G.; and Lapa, V. G.: Cybernetic Predicting Devices. Transla- tion TR-EE-66-4, School of Electrical Engineering, Purdue University, April 1966. Jones, L. E., 111: On the Choice of Subgoals for Learning Control Systems, IEEE Trans. Auto. Control, vol. AC-13, Dec. 1968. Kopp, Richard E.; and Orford, Richard T.: Linear Regression to System Identi- fication for Adaptive Systems. AIAA J., vol. 1, no. 10, Oct. 1963, pp. 2300-2306. Ku, R.; and Athans, M.: On the Adaptive Control of Linear Systems Using the Open-Loop-Feedback-Optimal Approach. Proceedings 11th Conference on Decision and Control and Symposium on Adaptive Processes, New Orleans, La., Dec. 13-15, 1972, Proceedings, IEEE, 1972, pp. 628-636. Lainiotis, D. G.; Deshpande, J. G.; and Upadhyay, T. N.; Optimal Adaptive Control: A Non-linear Separation Theorem, Inter. J. Control, vol. 15, 1972, pp. 877-888. Leondes, C. T.; and Mendel, J. M.: Artificial Intelligence Control. Survey of Cybernetics, R. Rose, ed., ILLIFE Press, London, 1969. Mendel, J. M.; and McLaren, R. W.: Reinforcement-Learning Control and Pattern Recognition Systems. Adaptive, Learning, and Pattern Recognition Sys- tems: Theory and Applications, J. M. Mendel and K. S. Fu, eds., Academic Press, 1970, pp. 287-318. Mendel, Jerry M.; and Zapalac, James J.: The Application of Techniques of Artificial Intelligence to Control System Design. Advances in Control Systems: Theory and Applications, C. T. Leondes, eds., Academic Press, 1968. Murphy, William J.: Optimal Stochastic Control of Discrete Linear Systems with Unknown Gain, IEEE Trans. Auto. Control, v l. AC-13, no. 4, Aug. 1968, pp. 338-344. McLaren, R. W.: A Stochastic Automaton Model for a Class of Learning Con- trollers. Procedures of 1967 Joint Automatic Control Conference, Univ. of Pennsylvania, 1967, pp. 267-273. 98 McLaren,. Robert W.: Application of a Continuous-Valued Control Algorithm to the On-line Global Optimization of Stochastic Control Systems. Proceed- ings of National Electronics Conference, Chicago, Ill., Dec. 8-10, 1969, VO~. 25, 1969, pp. 2-7. McMurtry, G. J.; and Fu, K. S.: A Variable Structure Automaton Used as a Multimodal Searchfng Technique, IEEE Trans. Auto. Control, vol. AC-11, no. 3, July 1966, pp. 379-387. Narendra, Kumpati S.; and Viswanathan, R.: A Two Level System of Stochastic Automata for Periodic Random Environments, IEEE Trans. Systems, Man and Cybernetics, vol. SMC-2, April, 1972, pp. 285-289. Narendra, Kumpati S.; and Viswanathan, R.: Learning Models Using Stochastic Automata, International Conference on Cybernetics and Society, Washing- ton, D.C., Oct. 9-12, 1972, Proceedings, pp. 378-385. Nikolic, 2. J.; and Fu, K. S.: A Mathematical Model of Learning in an Unknown Random Environment, National Electronics Conference, Chicago, Ill., Oct. 3-5, 1966, Proceedings, vol. 22, pp. 607-612. Patrick, Edward A,: On a Class of Unsupervised Estimation Problems. IEEE Trans. Inform. Theory, vol. IT-14, no. 3, May 1968, pp. 407-415. Pugachev, V. S.: A Bayes Approach to the Theory of Learning Systems. Third World Congress of IFAC, June 1966. Raible, R.; and Gibson, J.: A Computer Study on a Learning Control System. Preprints 3rd IFAC London 1966, paper 14E. Riordon, J. S . : An Adaptive Automaton Controller for Discrete-Time Markov Processes. Automatica J., vol. 5, no. 6, Nov. 1969, pp. 721-730. Saridis, George N.: Stochastic Approximation Methods for Identification and Control - A Survey. IEEE Trans. Auto. Control, vol. AC-19, no. 6, Dec. 1974, pp. 798-809. Saridis, George N.; and Lobbia, Robert N.: Parameter Identification and Control of Linear Discrete-Time Systems. IEEE Trans. Auto. Control, vol. AC-17, no. 1, Feb. 1972, pp. 52-60. Saridis, G. N.; and Kitahara, R. T.: Comparison of Per-interval and Overall Parameter-Adaptive SOC. Proceedings, 8th Symposium on Adaptive Pro- cesses, Penn. State, Dec. 1969. Saridis, G. N.; and Fensel, P. A.: Stability Considerations on the Expanding Subinterval Performance Adaptive SOC Algorithm. J. Cybernetics, vol. 3, no. 2 , April-June 1973, pp. 26-39. 99 Shapiro, I. J.; and Narendra, K. S.: Use of Stochastic Automata for Parameter . Self-Optimization with Multimodal Performance Criteria. IEEE Trans. Sys. Sci. Cybernetics, vol. SSC-5, Oct. 1969, pp. 352-360, and Witten, Ian H.: Comments .on Use of Stochastic Automata for Parameter Self-Optimization with Multimodal Performance Criteria. IEEE Trans. Systems, Man, and Cybernetics, vol. SMC-2, April 1972, pp. 289-290. Simon, H.: Dynamic,Programming under Uncertainty with a Quadratic Function. Econometrica, vol. 24, 1956, pp. 74-81. Spang, H. A.: Optimum Control of an Unknown Linear Plant Using Bayesian, Estimation of the Error. IEEE Trans. Auto. Control, vol. AC-10, no. 1, Jan. 1965, pp. 80-83. . .. Stein, G . ; and Saridis, G. N.: An Approach to the Parameter Adaptive Control Problem. Tech. Report TR-EE 68-35, School of Elect. Engr., Purdue Univ., Jan. 1969. Tou, Julius T.: Systems Optimization via Learning and Adaptation. Inter. J. Control, vol. 2, no. 1, 1965, pp. 21-31. Tou, J. T.; and Hill, J. D.: Steps Toward Learning Control. 7th Joint Auto- matic Control Conference, Aug. 17-19, 1966, Preprints of Conference Papers, pp. 12-26. Tse, Edison; Bar-Shalom, Yaakov; and Meier, Lewis, 111: Wide-Sense Adaptive Dual Control of Stochastic Nonlinear Systems. IEEE Trans. Auto. Control, vol. AC-18, no. 2, April 1973, pp. 98-108. Tse, Edison; and Bar-Shalom, Yaakov: An Actively Adaptive Control for Linear Systems with Random Parameters via the Dual Control Approach. IEEE Trans. Auto. Control, vol. AC-18, no. 2, April 1973, pp. 109-117. Tse, E.; and Bar-Shalom, Y.: Adaptive Dual Control for Stochastic Nonlinear Systems with Free End-Time. IEEE Trans. Auto. Control, vol. AC-20, no. 5, Oct. 1975, pp. 670-675. Tse, E. T.-S.: On the Optimal Control for Linear Systems with Incomplete Information. Ph.D. thesis, Report ESL-R-412, Electronic Systems Labora- tory at MIT, Cambridge, Mass., Jan. 1970. Tsypkin, Ya. Z.: All the Same, Does a Theory of Synthesis of Optimal Adaptive Systems Exist? Automation and Remote Control, no. 1, Jan. 1968, pp. 93-98. Tsypkin, Ya. Z.: Generalized Algorithms of Learning, Automation and Remote Control, no. 1, Jan. 1970, pp. 86-92. Tsypkin, Ya. Z.: Principles of Dynamic Adaptation in Automatic Systems. Fifth World Congress of IFAC, Paris, June 12-17, 1972. Tsypkin; Ya. Z.: Self-Learning - What is it? IEEE Trans. Auto. Control, vol. AC-13, no. 6, Dec. 1968, pp. 608-612. Tsypkin, Ya. Z.; and Kelmans, G. K.: Recursive Self-Training Algorithms. Engineering Cybernetics, no. 5, 1967, pp. 70-80. Tsypkin, Ya. Z.; Kelmans, G. K.; and Epstein, L. E.: Learning Control Sys- tems, Preprints, Fourth World Congress of IFAC, Warsaw, Poland. June 16-21, 1969. Ula, N.; and Kim, M.: An Empirical Bayes Approach to Adaptive Control. J. Franklin Institute, vol. 280, no. 3, Sept. 1965, pp. 189-204. Viswanathan, R.; and Narendra, Kumpati S.: Stochastic Automata Models with Applications to Learning Systems. IEEE Trans. Systems, Man and Cybernetics, vol. SMC-3, no. 1, Jan. 1973, pp. 107-111. Viswanathan, R.; and Narendra, Kumpati S.: Comparison of Expedient and Opti- mal Reinforcement Schemes for Learning Systems. J Cybernetics, vol. 2, no. 1, Jan./March 1972, pp. 21-37. Waltz, M. D.; and Fu, K. S . : A Computer-Simulated Learning Control System. IEEE International Convention Record, voi. 12, pt. 1, 1964, pp. 190-201. Waltz, M. D.; and Fu, K. S.: A Heuristic Approach to Reinforcement Learning Control Systems. IEEE Trans. Auto. Control, vol. AC-10, no. 4, Oct. 1965, pp. 390-398. 101 Engine octuotw d y m i c s Vehicle dvnamics thrust Fixed / / / Velocity vector / / Figure 1.- Booster vehicle. I I ' I Figure 3.- Schematic diagram of the parallel identifications, estima- tion, and control. I Figure 2.- Block diagram uncontrolled system. of Figure 4 . - Identification of the para- meters of a third-order booster stage problem. 102 Figure 5.- '4 t X a I - 8 - 2 - ------------ known system matrices Optimal solution with I 1 I I I I I I I 0 5 10 15 20 25 30 35 40 I6 - 14 - 12 - lo- Norm Performance results - third-order booster stage problem. t Figure 6.--.Performance adaptive self-organizing control for the stochastic fuel regulator problem. 103 0 cmel A - 2 v cme3 40 5 1 0 coscl A - 2 v coK3 ltemtians Figure 7.- Experimental results (average per iteration costy. Iterotiom Figure 8.- Experimental results (standard deviation of per iteration cost). I I I I I -2 - I 0 I 2 (a). Case I and 2 L I I -2 -I 0 I h X I (b) . Case 3 Figure 9.- Steady-state responses before and after self-organization. 104 STATUS OF DUAL CONTROL THEORY* Edison Tse Stanford Universi ty , Cal i f . I. INTRODUCTION I n many processes ar is ing in engineer ing, economic, and b io logica l sys- tems, t h e problem of decision-making (or control) under various sources of u n c e r t a i n t i e s is inherent . These uncertaint ies prevent exact determination o f t he e f f ec t o f a l l present and future act ions, and therefore determinis t ic cont ro l theory is n o t a p p l i c a b l e . I f t h e e f f e c t of t h e s e u n c e r t a i n t i e s is small, one can still use opt imal cont ro l theory to obta in a feedback control l a w based on de terminis t , i c cons idera t ions . The feedback nature of the control would tend t o r e d u c e t h e s e n s i t i v i t y t o u n c e r t a i n t i e s b u t would r e q u i r e t h e state of the system to be measured exactly. Again, this assumption is good only when t h e measurement e r r o r is small i n comparison with the signal being measured. I n many cases, t h e phenomena of uncertainty ( including measurement e r r o r ) can be appropriately modelled as s tochast ic processes , a l lowing them to be considered via s tochas t ic op t imal cont ro l theory . A very important concept i n s t o c h a s t i c c o n t r o l is t h e information pattern a v a i l a b l e t o a c o n t r o l l e r a t a s p e c i f i c t i m e , for the purpose of decision-making. A s the process unfolds , addi t ional information becomes a v a i l a b l e t o t h e c o n t r o l l e r . T h i s information may come about acc identa l ly through pas t cont ro l ac t ions or as a r e s u l t of active probing which i tself is a poss ib le cont ro l po l icy . Thus "learning" is present, whether it is "accidental" or "del iberate ." The infor - ma t ion pa t t e rn ava i l ab le to the con t ro l l e r i nd ica t e s no t on ly what type of l ea rn ing i s poss ib l e a t each instant of time, bu t , more importantly, whether fu tu re l ea rn ing can be an t i c ipa t ed and how i t could be influenced by present action (i .e. , whether probing would be he lp fu l i n fu tu re l ea rn ing ) . S ince more l ea rn ing may improve overal l control performance, the probing s ignal may i n d i r e c t l y h e l p i n c o n t r o l l i n g t h e s t o c h a s t i c s y s t e m . On the other hand, excessive probing should not be allowed even though it may promote l ea rn ing because it is "expensive" i n t h e s e n s e t h a t it w i l l gene ra l ly inc rease the expected cost performance of the system. This interplay between learning and c o n t r o l is t h e key i s sue o f s tochas t i c con t ro l t heo ry . This "dual" purpose of the control w a s po in ted out by Fel'dbaum ( r e f . 1 ) u s i n g t h e s t o c h a s t i c dynamic programming approach (ref. 2) . Because of t h i s d u a l e f f e c t , Fel'dbaum c a l l e d i t "dual control theory." The sub jec t was then s t u d i e d by Aoki ( r e f s . 3 and 4), h t r G m ( r e f . 5), F lo ren t in ( r e f . 6 ) , and *Research supported by Off ice of Naval Research under O M N00014-75-C-0738. 105 Meier (ref. 7), via some simple two-stage examples. These studies, however, did not display the full force of the dual effects since the control period is too short for any extra learning effort to be paid off in the future. To fully appreciate the interplay between 1earning.and control, a N-period con- trol problem (N >> 3) must be considered. Unfortunately, the optima1,dual control solution cannot be obtained numerically for N > 3. This motivates several recent developments of dual control methods by Tse e t aZ. (refs. 8-10), Chow (ref. 11), MacRae (ref. 12) Alster and Belanger (ref. 13), and'Alspach- (ref. 14). The method developed by Tse e t aZ. has been studied to a great extent and is applicable to a fairly large class of nonlinear stochastic systems, whereas the other methods are developed for specific cases of inter- ests. We shall discuss here the recent developments in dual control methods. The method proposed by Tse e t aZ. (ref. 8) is discussed in detail, whereas the other methods are only described briefly due to their specific nature. Simulation studies are summarized and the future of dual control is discussed. 11. DUAL CONTROL: AN ACTIVE CONTROLLER To begin our discussions, let us first introduce the concept of dual control. Consider a linear static system given by % = auk+ ck ; k = 1, 2 where xk is the observation, a is the unknown parameter which is Gaussian with mean a. and variance oo, uk is the control, and (ck) is a sequence of equidistributed and independent Gaussian variables with mean zero and variance q. The stochastic control problem is to find a control law J = E(xl + x2 + U: + - u2 1 1 1 2 2 (2) is minimized. First, note that since equation (1) is a static linear system, if a is known, the optimal closed-loop law would be u1 * = U* = -a 2 (3) For unknown a, the certainty equivalence (CE) controller is CE CE 1 0 ' u =-a - u2 = -a 1 where a is the updated mean of a after (X~,U;~) are being observed. 1 106 , L e t u1 be an a rb i t r a ry con t ro l va lue app l i ed t o equa t ion (1). Afte r the ob,servation of x , one can compute t h e a posteriori d i s t r i b u t i o n f o r a which I s Gaussian w i t k mean a q + u u x 0 0 1 1 a1 - u: Go + q and var iance From equation ( 6 ) , n o t e t h a t , i f l u l I is large, the updated variance f o r a w i l l be small; i n f a c t , i f l u l I + OJ , u1 + 0. This impl ies tha t a l a r g e con t ro l va lue a t k = 1 helps earn ing of the unknown parameter a. On t h e o ther hand , i f u1 = 0, a1 = a, and u = u t h a t is, no a d d i t i o n a l knowledge on a w i l l be provided i f a sma l loc in t ro l va lue is used a t k = 1. To see what should be the proper or optimum v a l u e f o r u , one has to calcu- la te the value, or the usefuZnes6, of additional informakon on a, i n t h e context of the given control problem, so t h a t one can t r ade o f f p re sen t con- t r o l ( s e t u1 = -a ) wi th add i t iona l l ea rn ing fo r fu tu re con t ro l ( s e t Iu l I l a r g e ) . 1 0 L e t u1 be the va lue o f the cont ro l a t k = 1 and x1 be t he r e su l t i ng (random) observation. In minimizing the remaining cost to go, + 2 1 2 u 1x1,ul} = E{au21x1,u1) + T 1 u; = + - 1 u2 a1u2 2 2 (7) where a1 is given by equat ion ( 5 ) , t h e optimum con t ro l l a w a t k = 2 i s a q + u u x u2 - -a = - - 0 0 1 1 1 u2. + q 1 0 Note t h a t u2 i s random since x1 is random. The minimum c o s t o go: is a random variable through x The cos t o f l earn ing i s negat ive and is represented by the expected minhkm cost to go: 107 . . This i nd ica t e s , t ha t l ea rn ing is bene f i c i a l (nega t ive cos t ) The ove ra l l cos t o f app ly ing u1 i s ; i \ J = E x1 + $ u:} + E{J5(xl ,ul)I { ' (10) t o f u t u r e c o n t r o l . The o p t i m k v a l u e f o r u1 t h a t w i l l balance between present control (minimizing aoul + (1/2)u:) and add i t iona l l ea rn ing fo r fu tu re con t ro l "dual cost" given by equation (11). The two components of t h e c o s t c1 (u) = a u + - u2 (d i rec t cont ro l cos t ) 1 0 2 (12) are p l o t t e d i n f i g u r e 1 (where a0 i s assumed p o s i t i v e ) . From the f i gu re , i t is clear t h a t (1) i f q is small, u1 = -ao, s i n c e , i n t h i s case, unplanned (or acc iden ta l ) l ea rn ing i s su f f i c i en t t o r educe the upda ted va r i ance ( eq . ( 6 ) ) . * (2) i f q is moderate, u1 is t o t h e l e f t o f -ao, i n d i c a t i n g t h a t * planned learning does pay off . (3) i f q is ve ry l a rge , UT -ao, s i n c e , i n t h e case of high noise i n t e n s i t y , a d d i t i o n a l l e a r n i n g does not pay o f f and thus the control ler might as well look a t it as a one-stage optimization problem. Similar conclusions can be drawn f o r d i f f e r e n t r a n g e of v a l u e s f o r uo. I n f a c t , t h e optimum u1 is determined by t h e r a t i o q/oo and u . 0 1.G E I n t h i s s t a t i c example, t he e f f ec t o f t he con t ro l va lue does no t p ropaga te in t h e through the system, and thus the effects of learning and cont ro l can be n ice ly d is t inguished . For a dynamic system, control appl ied a t a c e r t a i n t i m e w i l l i n f luence the fu tu re state t r a j e c t o r y , and thus the d u d effects of the cont ro l would b e d i f f i c u l t t o s e p a r a t e . This example serves to point out one very important concept of dual control, namely, that planned learning should be done only i f (1) acc identa l l earn ing is no t adequate or (2) planned learning does pay o f f . This is t h e c e n t r a l theme of dua l cont ro l theory . 111. PROBLEM FORMULATION AND OPTIMAL SOLUTION In th i s s ec t ion , t he fo rmula t ion and so lu t ion of the opt imal s tochas t ic con t ro l problem for d i scre te - t ime sys tems are discussed. Consider a d i sc re t e - time nonlinear system described by and nonlinear observation described by k = 0, 1, ..., N-1 k = 1, ..., N A where x is t h e i n i t i a l c o n d i t i o n , a random var iab le wi th mean x and covariance &lo , * {SI and {x) are the sequences of process and measurement no ses, respect ively, mutual ly independent , white and wi th known statistics. For s implici ty , they may be assumed t o have zero mean. Consider further the performance measure -0 -010 where the expec ta t ion E{*) is taken over a l l underlying random q u a n t i t i e s . Final ly , consider admissible control of the closed-loop type: N- 1 The goal is t o f i n d t h e o p t i m a l c o n t r o l l a w {$ik=o which has the form of equat ion (17) and minimizes the cost (eq. (16)) subject to the constraints (eqs. (14) and (15)). Suppose t h a t a c e r t a i n c o n t r o l l a w * ( * ) is chosen f o r k = 0, 1, ..., N-2, and the observat ion sequence YN-l is observed. The remaining problem is t o select so tha t the condi t iona l expec ted cos t 109 . . N- 2 is minimized. Since the summation term is independent of %-1, the optimum u+-l i s obtained by minimizing Note t h a t p ( ~ + - ~ ) i s known a p i o r i , and thus the only on-l ine quant i ty needed i n t h e c a l c u l a t i o n of equation (19) is e i t h e r ,u o r p(x+ - IYN-1,$-2); thus the opt imal cont ro l must be of t h e form (yN- 1 -N- 2 Subs t i tu te equat ion (20) i n t o (19) and def ine the opt imal cos t to go: IN-~(QN-~) = '('N-1 [%-I ($ N-1 + K{fN-l 9%-l(qN-l)1 + G-1 } l q N - l ) * * (21) where $N-l is t h e information state tha t can e i ther be represented by ( Y N - ~ , E ~ - ~ ) o r p ( 3 - 1 J Y N-1 ,U -N-2) . The cos t o f us ing the cont ro l l a w {U (* ) }k=o , wi th the knowledge tha t the remain ing cont ro l law is optimal, is N- 2 -k 1When a c o n t r o l l a w i s s p e c i f i e d , n o t a t i o n zi is used to deno te the - rea l iza t ion of the cont ro l va lue a t s t e p i; U +&, . . . , when Yk is r ea l i zed . -k& 110 To ca r ry t h i s p rocedure backward i n time, assume t h a t a(*) is chosen, k = 0, 1, . . . , N-3, and YNS2 is observed. I f a p a r t i c u l a r c o n t r o l v a l u e f lk2 informat ion s ta te is ava i l ab le : i s chosen, then, from equations (14) and (15), a dynamic descr ip t ion of where $N-2 is the in format ion state a t N-2 and YN - is t h e random’ ’ . observat ion to be observed a t time N-1. If, f o r example, ‘ t he cond i t iona l dens i ty fo r t he cu r ren t s ta te is used as an information s ta te , then . . - J The opt imal cos t to go is given by IN-2 (lcl N-2 = min E { LN-2(EN-2’%-2) + 1 N - 1 [ J I N - 1 ( $ N - 2 ’ ~ - 2 ’ Y N - 1 ) 1 1QN-z } *-2 (25) and the cos t o f us ing the cont ro l l a w {u+(-) 1 , with the knowledge t h a t t h e remaining control law is optimal , i s k= O N- 3 N- 3 \\ Induct ively, the condi t ional expected opt imal cost to go, I ($ ), s a t i s f i e s t he dynamic programming equat ion: k k with end condi t ion IN($N) = EEK(+) I$NI (28) The opt imal control law is obtained by solving equat ions (27) and (28). It is w e l l know tha t t he so lu t ion o f equa t ions (27) and (28) is f e a s i b l e on ly fo r a small class of problems (refs. 3 and 8 ) . I n most cases, one is 111 unable to solve equations (27) and ( 2 8 ) due to the curse of dimensionality (ref. 2), and thus suboptimal procedures must be developed for the problem. However, one can deduce the dual nature of th optimal.contro1 from the dynamic programming equation (27). Depending on the actual realization of $k, the overall cost of applying the control 's at time k consists of two components: direct cost, E{Lk(a,a) I $kl , and indirect cost, "1k+I k+l k [$ ($ ,U+J~+~)] I $kl. The direct cost represents the conditional expected cost that would'have been incurred if the problem is a on -shot static problem, whereas the indirect cost represents the conditional expected cost of the future information state, which depends OR % because of the dynamic description of the system equation (14) and the observation relation equation (15). The dual effects of are interrelated and embedded in the indirect cost component. IV. A DUAL CONTROL METHOD In this section, we shall describe a dual control method due to Tse et aZ. (refs. 8 and 9). Note that the main difficulty in solving equation ( 2 7 ) hinges on the calculation of the indirect cost E { I k + l [ 9 k + l ( $ k , ~ , . y k + l ) ~ 1$k}. If, however, this quantity can be obtained exactly or approximately, then at every time k, it looks like we have a two-stage stochastic control problem. With this motivation, Tse e t aZ. propose to approximate equation ( 2 7 ) as follows: (1) Approximate the information state $k so that the dimension of the information state space stays constant in time. It is suggested that the conditional mean and covariance be used as the appzoximated information state. Denote such an approximated information state by $J k = {&jk,&lk}'2 ( 2 ) Approximate the optimal cost-to-go function associated with an approximated information state by performing a second-order perturbation analysis around a nominal trajectory that is di f ferent for different predicted approx+nated infcymation s t a t e s . Denote the approximation for the optimal to go by Ik+l [$k+l (Jlk9u+9y++l) 1, which is given by (see refs. 8 and 15 for details) where is the deterministic part of the cost along the nominal 2We denote the conditional error covariance by either &] k or - C(k1k). 112 are the stochastic terms in the cost evaluated along the nominal. The matrices $( j ) ,AO,* ( j ) and the scalar y o ( j ) are given by - with 113 (3) At each time k, an on-line optimization problem . - is solved in real time. Denote the minimizing % by % ($k) % k k -t- 1. DC ,. "($ ), update jk+, when q+, is observed, and resolve,the problem for One 'distinguishing characteristic of this approach is that it explicitly takes into account that future measurements will be made, and these future 2 measurements will provide additional information about the uncertain param- eters that will enhance future control. Therefore, learning is planned in accordance with the control objective. This gives the control law a closed- loop characteristic (see refs. 15 and 16): To see this, first note that yo(k + i), E(k + l), and I&(k + 1) are not stochastic and depend only on the nominal control sequence being chosen. The function H o ( j ) defined by equation (36) is the Hamiltonian for the corresponding deterministic optimal control problem. Therefore, if the nominal control is a local minimum for the deterministic problem, then Ho,u(j) = 0 and thus yo(j) = 0; also %9uu (j) is positive definite for aT1 j L k + 1. The latter implies that theyeuturbation analysis is valid. In fact, a necessary and sufficient condition for the existence and uniqueness of the optimum perturbatson solu- t ion is that uu(j). must.be positive definite (regardless of whether the nominal control k a local minimum). If q,uu(j) is positive definite, then Ao,xx(j) (defined by eq. (40)) must brnonnegative definite, with rank equal to the dimension of the control vector. Now observe that the first stochastic term (eq. (31)) reflects the effect of the uncertainty at time k and subsequent process noises on the cost. These uncertainties cannot be affected by u(k) ,. but their weightings depend on it. The effect of these uncontrollable uncertainties on the cost should be minimized by the control; t h i s term indicates the need for the control t be cautious and thus is called the caukion tern. The second stochastic term, equation ( 3 2 ) , accounts for the effect of uncertainties when subsequent decisions will be made. As discussed above, if the perturbation problem has a solution, then the weighting of these future uncertainties is nonnegative is positive semidefinite). If the control. can reduce by probing (experimenEtion) the future updated covariances, it can thus reduce the cost. The weighting matrix AO,= yields approximately the value of future information for the problemmder consideration. Therefore, this is called the probing tern. Since the rank of A,,,x(j> is equal to the dimension of the control vector, this indicates that, alsz a certain nominal sequence, certain combinations of state uncertainties do not contribute (up to second order) to the total average cost. - . . The benefit df probing is weighted by its cost and a compromise is chosen to minimize the sum of the deterministic, caution, and probing terms. The minimization will also achieve a tradeoff between the present and future actions according to the information available when the corresponding deci- sions are made. 114 Figure 2 is a flow chart of the algorithm. V. OTHER DUAL CONTROL METHODS Several other dual control methods are suggested in the literature. But almost all are developed for limited classes of problems. Chow (ref. 11) and MacRae (ref. 12) developed two different dual control methods mainly for the control of linear econometric models with unknown parameters. Chow's method is based on an approximation of the optimal cost to go. .Inthis respect, his algorithm is similar to the one described in section ISJ. There are some basic differences between Chow's method and the one developed by Tse et aZ. One is that Chow employed a single nominal whereas Tse'et aZ. allowed to have multi- nominals; another difference is that Chow proposed a step-by-step numerical approximation of the optimal cost to go, whereas Tse et aZ. us d perturbation analysis to obtain the optimal cost to go. MacRae's approach is also based on dynamic programming, but whenever a future updated covariance appeared in the equation, she replaced it by a first-order approximation about a nominal trajectory. Alster and Belanger (ref. 13) developed an ad-hoc dual control method for the control of a class of linear discrete time systems with unknown constant parameters. They first rewrite the expected cost conditioned to the current time period, and suggested to minimize the open-loop expected cost . subject to an added constraint that the next period's updated covariance is smaller than some threshold. While this is a plausible approach to the prob- ~ lem, the problem of selecting the threshold is nontrivial and probably highly problem dependent. Kalaba and Detchmendy (ref. 17) suggested a similar approach where the original cost is modified to include penalty of having bad estimates. Alspach (ref. 14) used perturbation methods to develop a dual con- trol method for the control of linear systems with known dynamics but with multiplicative noise. All methods described in this section assumed a quadratic criterion. VI. SIMULATION STUDIES Up until now, no simulation study using the methods proposed by Chow, MacRae, and Kalaba and Detchmendy has been reported in the literature. Limited simulation studies were reported in reference 13 where their algorithm is compared to the certainty equivalence (CE) control algorithm for the control of single-input/single-output linear systems with unknown parameters under the minimum output variance criterion. Alspach (ref. 14) also compared his algorfthm with the CE algorithm for the control of single-input/single-output linear systems with multiplicative noise under the quadratic criterion. In both studies, the proposed dual control methods perform better than the CE method. 115 Tse e t aZ. applied their dual control methods to several representative cases in three different classes of problems. In each case, the dual control method is an order of magnitude better than the CE method. Moreover, the similation studies revealed several important characteristics of dual-control.' The first class .of.problems is the control of linear systems with unknown parameters. Tse e t aZ. (refs. 8 and 9) studied the control of single-input/' single-output linear systems with unknown parameters under the quadratic criterion. By varying the weighting matrices, the active learning character- istics of the dual control is vividly displayed. The second class of problems is that of.controlling nonlinear systems. For such a class of p'roblems, there is no unknown parameter to be learned. However, there is -another '- important characteristic of the dual control: the capability of bringing the system state to a certain region that will improve state estimation. This is clearly shown in the studies by Tse and Bar-Shalom (ref. lo), where they applied their dual control method to an interception problem. The third class is the control of dynamic systems with drifting parameters. Pekelman and Tse (ref. 18) used the dual control method in the problem of optimum decision on advertisement .expenditure. In this case, the system is nonlinear with drifting (unknown) parameters. This study also indicates that when the knowledge of the drifting parameters is crucial to the control performance, the dual control method will induce enough learning to track the variation of the parameters. ' I VII. CONCLUDING REMARKS AND THE FUTURE OF DUAL CONTROL THEORY Generally, it is found that dual control methods give a better perfor- mance, but they are also considerably more complicated than the CE control method. The question then is how much we want to trade off between perfor- mance improvement and complexity. In some cases, it can be shown that the CE method gives reasonably good results (ref. 19) or even the optimal solu- tion (refs. 16, 20, and 21), and thus the dual control is not needed. Clearly, one important question to be answered in any stochastic control problem is how much we can expect to gain by using dual control instead of CE control. Theoretical studies in this direction are not found in the literature. From personal experience, a problem with the following characteristics may need dual control approach : (1) The control period is relatively short, but not too short, so that the benefit of learning does not contribute substantially to the indirect cost . (2) State estimation performance is highly dependent on the state trajectory . (3 ) Control performance is highly dependent on the knowledge of the unknown parameters in the system. ( 4 ) The natural tendency of the system is incompatible with the system performance, for example, regulating the output of an unstable system close to zero or steering the output of a stable system to a specified value. 116 To understand the similarities and differences of all the proposed dual control methods, comparison studies of these methods should be carried out using the same data base. Bar-Shalom and Tse (ref. 22) initiated effort in this direction by comparing the methods of Tse et aZ. (ref. 81, Chow (ref. 11), and MacRae (ref. 12) via a simple three-stage linear system with unknown feed- forward gain parameter. Comparison studies .for more complicated problems will be reported in the future. Throughout the discussions, it was assumed that the structural form of-the system dynamic and the observation relation are known to the controller. While this assumption is valid in many engineering applications, it may not be valid in many processes arising in economic, chemical, and biological systems where a unique description of the system structure is impossible. A formula- tion of combined system identification and control for systems with ambiguity in structural description seems to be needed. One possible. formulation is to assume that the underlying system can have M different possible structural descriptions. These descriptions may be derived from several equally possible hypothetical assumptions. The problem then is that of minimizing a certain cost criterion, subject to the constraint that the t rue dynamic equation can be one of the M possible difference equations. This problem has been considered in references 23 through 26. These approaches are equivalent to separation of detection and control. While the methods seem to be useful in some applications (ref. 27), in some cases, the method faces the problem of identifiability where the true" system cannot be identified even at k -f 00. (For a discussion of identifiability problems, see refs. 28 to 3 2 . ) An active approach similar to the dual control (as described in section VI) would be useful. In this case, the control would have the following "dual" purposes. 11 (1) To allow resolvability of model structures and to help in estimating the unknown parameters of a given model structure. (2) To regulate or control the system to follow some desired path or performance measure. For M different possible model structure descriptions, the problem should be formulated as simultaneous hypothesis testing and control problem. Such an active approach has not yet been attempted. 117 REFERENCES 1. Fel'dbaum, A. A.: Dual Control Theory I-IV, Automation and Remote Control, VO~. 21, 1960, pp. 874-880,,pp. 1033-1039; VO~. 22, 1961, pp. 1-12,, pp. 109-121; also Optimal Control Systems, translated by A. Kraimen, Academic Press, 1965. 2. Bellman, Richard E.: Adaptive Control Processes: A Guided Tour, Princeton Univ. Press, 1961. , . 3. Aoki, Masanao: Optimization of Stochastic Systems; Topics in Discrete- Time Systems, Academic Press, 1967. 4. Aoki, Masanao: Optimal Control of Partially Observable Markovian Systems, J. Franklin Inst., vol. 280, no. 5, Nov. 1965, pp. 367-386. 5 . htrom, K. J.: Optimal Control of Markov Processes with Incomplete State Information, J. Math:Anal. Appl., vol. 10, no. 1, Feb. 1965. pp. 174-205. 6. Florentin, J. J.: Optimal, Probing, Adaptive Control of a Simple Bayesian System, J. Electr. Control, vol. 13, no. 2, Aug. 1962, pp. 165-177. 7. Meier, Lewis: Combined Optimal Control and Estimation, 3rd Allerton Conference on Circuit and System Theory, Oct. 20-22, 1965, Monticello, Ill., Proceedings, M. E. Van Valkenburg, ed., 1965, pp. 109-120. 8. Tse, Edison; Bar-Shalom, Yaakov; and Meier, Lewis, 111: Wide-Sense Adaptive Dual Control of Stochastic Nonlinear Systems, IEEE Trans. Automatic Control, vol. AC-18, no. 2, April 1973, pp. 98-108. 9. Tse,, Edison; and Bar-Shalom, Yaakov: An Actively Adaptive Control for Linear Systems with Random Parameters via the Dual Control Approach, IEEE Trans. Automatic Control, vol. AC-18, no. 2, April 1973, pp. 109-117. 10. Tse, Edison; and Bar-Shalom, Yaakov: Adaptive Dual Control for Stochastic Nonlinear Systems with Free End-Time, IEEE Trans. Automatic Control, vol. AC-20, no. 5, Oct. 1975, pp. 670-675. 11. Chow, Gregory C.: Analysis and Control of Dynamic Economic Systems, John Wiley and Sons, Inc., -1975. 12. MacRae, E. C.: Linear Decision with Experimentation, Ann. Economic Social Measurement, vol. 1, pp. 437-488. 13. Alster, Jacob; and Bilanger, Pierre R.: A Technique for Dual Adaptive Control, Automatica, vol. 10, no. 6, Dec. 1974, pp. 627-634. 118 14. Alspach, D:L.: Dual Control Based on Approximate A Posteriori Density Functions, IEEE Trans. Automatic Control, vol. AC-17, no. 5, Oct. 1972, pp. 689-693. 15. Bar-Shalom, Yaakov; and Tse, Edison: Concepts and Methods in Stochastic Control, in Control and Dynamic Systems: Advances in Theory and Applications, C. T. Leondes, ed., Academic Press, vol. 12, 1976, pp. 99-170. 16. Bar-Shalom, Yaakov; and Tse, Edison: Dual Effect, Certainty Equivalence and Separation in Stochastic Control, IEEE Trans. Automatic Control, vol. AC-19, no. 5, Oct. 1974, pp. 494-500. 17. Kalaba, R. E.; and Detchmendy, D. M.: Direct Conversion of Observational Histories into Control Signals, Information Sciences, vol. 1, 1968, pp. 1-5. 18. Pekelman, D.; and Tse, E.: Experimentation and Control in Advertisement: An Adaptive Control Approach, in preparation. 19. htrom, K. J.; and Wittenmark, B.: On Self-Tuning Regulators, Automatica, vol. 9, no. 2, March 1973, pp. 185-199. 20. Tse, Edison; and Bar-Shalom, Yaakov: Generalized Certainty Equivalence and Dual Effect in Stochastic Control, IEEE Trans. Automatic Control, vol. AC-20, no. 6, Dec. 1975, pp. 817-819. 21. Tse, E.: Sequential Decision and Stochastic Control, Symposium on Stochastic Systems, Univ. of Kentucky, June 1975. 22. Bar-Shalom, Y; and Tse, E.: Caution, Probing, and Value of Information in the Control of Uncertain Systems, 4th NBER Conference on Stochastic Control and Economics, Boston, May, 1975; to appear in Ann. Economic Social Measurement, 1975. 23. Saridis, G. N.; and Stein, G.: An Approach to the Parameter Adaptive Control Problem, TR-EE68-35, Purdue Univ., Dec. 1968. 24. Saridis, G. N.; and Dao, T. K.: A Learning Approach to the Parameter- Adaptive Self-organizing Control Problem, Automatica, vol. 8, no. 5, Sept. 1972, pp. 589-597. 25. Deshpande, J. G.; Upadhyay, T. N.; and Lainiotis, D. G.: Adaptive Control of Linear Stochastic Systems, Automatica, vol. 9, no. 1, Jan. 1972, pp. 107-115. 26. Athans, M.; and Willner, D.: A Practical Scheme for Adaptive Aircraft Flight Control Systems, Proc. Symposiirm on Parameter Estimation Techniques and Applications in Aircraft Flight Testing, NASA TN-D-7647 1974, pp. 315-336. . . . 119 27. Athans, M., et ai?.: The Control of the F-8 Aircraft Using the Multiple Model Adaptive Control Method, NASA Report. 28. Bellman, R. ; and ibtriim, K. J. : On Structural Identifiability, Math. Bioscience, vol. 7, 1970. 29. Tse, Edison; and Anton, John J.: On the Identifiability of Parameters, ' IEEE Trans. Automatic Control, vol. AC-17, no. 5,'Oct. 1972, pp., 637-646. 30. Tse, E.: Bounds on Identification Errors and A Quantitative Measure of Identifiability, 1974-IEEE Conference on Decision and Control, Phoenix, Arizona, Nov. 1974. 31. Koopmans, T. C.: Identification Problems in Economic Model Construction, in Ch. 11, Studies in Econometric Method, W. C. Hood and T. C. Koopmans, eds., John Wiley and Sons, Inc., 1953. 32. Fisher, Franklin M.: The Identification Problem in Econometrics, McGraw-Hill Book Co., 1966. 120 - 112 az 1 Figure 1.- Decomposition of dual cost. Compute the C.E. control for time its cwariance if u(k) is used Obtain the nominol trajectory and the cwariances along it rneasuremerd Evaluate the Dual cost I‘ Lr’ k = k + l Apply the control No Compute new value u(k) at which the dual cost is to be evaluated Figure 2.- Flow chart of dual control algorithm. 121 I 0 CL ,OSED-L NOOP STRUCTUR STABILITY FOR AL LINEAR-QUADRATIC OPTIMAL SYSTEMS* Poh Kam Wong and Michael Athans Massachusetts Institute of Technology SUMMARY This paper contains an explicit parametrization of a subclass of linear constant gain feedback maps that never destabilize an originally open-loop stable system. These results can then be used to obtain several new struc- tural stability results for mulPiinput linear-quadratic feedback optimal designs. . I. INTRODUCTION AND MOTIVATION This paper presents preliminary results which, in our opinion, represent a first necessary step in the systematic computer-aided design of reliable control systems for future aircraft. It is widely recognized that advances in active control aircraft and control configured vehicles will require the auto- matic control of several actuators so that future aircraft can be flown char- acterized by reduced stability margins and additional flexure modes. As a starting point for our motivation, we must postulate that the design of future stability augmentation systems must be a multivariable design prob- lem. AS such, traditional single-input/single-output system design tools based on classical control theory cannot be effectively used, especially in a computer-aided design context. Since modern control theory provides a concep- tual theoretical and algorithmic tool for design, especially in the linear- quadratic-Gaussian (LQG) context (see, e.g., ref. l), it deserves a special look as a starting point in the investigation. In spite of the tremendous explosion of reported results in LQG multi- variable design, the robustness properties have been neglected. Experience has shown that LQG designs "work" very well if the mathematical models upon which the design is based are somewhat'accurate. There are several sensitiv- ity studies involving "small-parameter perturbations" associated with the LQG problem. We submit, however, that the general problem of sensitivity and even stability of multivariable LQG designs under Zarge parametric and strmcturaz changes is an open research area. *This research was conducted in the M.I.T. Electronic Systems Laboratory with partial support extended by NASA Ames Research Center under grant NGL-22-009-124 and by the Air Force Office of Scientific Research under grant AF-AFOSR-72-2273. 123 It is useful to.reflect on the basic methodology in classical servo- - ' . mechanism theory which dealt with such large-parameter changes. The overall..,.,^ sensitivity and stability cons'iderations were .captured in. the definition of ., . , I . gain and phase margins. If a closed-loop system were characterized by reason- able gain and phase margins, then (a) reasonable changes in the parameters of the open-loop transfer functions and (b) changes in the loop gains due, for example, to saturation and other nonlinearities could be accommodated with guaranteed stability and at the price of somewhat degraded performance. . Although LQG designs are time-domain oriented, nonetheless their frequency-domain interpretations are important, although not universally- appreciated. For example, for the single-input/single-output linear-quadratic (LQ) optimal designs, Anderson and Moore (ref. 2) have shown that LQ-optimal designs are characterized by (a) an infinite gain margin property and (b) a -.: ... :..: phase margin of at least 60". Such. results are valuable because it can be readily appreciated that,-at least in the single-inputjsingle-output case, modern control theory designs tend to have a good degree of robustness, as measured by the classical criteria of gain and phase margin. Advances in the multiinput/multioutput case, however, have been scattered ' and certainly have not arrived at the cookbook design stage. Multivariable system design is extremely complex. To a certain 'extent, the numerical so&,+:;: .. t ion of LQ optimal is very easy. However, fundamental understanding of the- :, , ' structural interdependencies and their interactions with the weighting . I matrices is not a trivial matter. We believe that such fundamental under- standing is crucial for robust designs as well as for reliable desi,gns that involve a certain degree of redundancy in controls and sensors. . . .;,. . '. . I The recent S.M. thesis by Wong (ref. 3)'represents a preliminary yet : positive contribution in this area. In fact, the technical portion. of this . , . . I paper represents a slight modification of some of the results reported in reference 3. In particular, we focus our attention on the stability proper- ties of closed-loop systems designed on the basis of LQ-optimal techniques when the system matrices and loop gains undergo large variations. The main contributions reported here are the eventual results of gener- alizing the concepts of gain margin and of performing large-perturbation sen- sitivity analysis for multivariable linear systems designed via the LQ approach. We warn the reader that much additional theoretical and applied research is needed before the implications of these theoretical results can (a) be fully understood and (b) translated into systematic ''cookbook" pro- cedures that have the same value as the conventional results in classical servomechanism design. This paper is organized as follows: in section 2 we present an explicik parametrization of a subclass of linear constant feedback maps that never destabilize an originally open-loop stable system and establish some of its properties. In section 3, we apply this construct to obtain several new lEven the notion of what constitutes a "zero" of a multivariable transfer matrix was not fully appreciated until recently. 124 closeddoop structural stability characterizations of multiinput LQ-optimal feedback maps. We conclude in section 4 with a brief discussion of the rele- vance of the present results for computer-aided iterative feedback design. No tat ion . . (1) The linear time-invariant system where . , . - x(t) E Rn - x(*) = state vector - u(t) E Rm - u(=) = control vector - z(t) E Rr - z(*) = output vector . and A E Rnm, - B E Rnm, and ET E Rrm will be denoted by C (A,g,gT). Where HT is irrelevant to the discussion, we shorten the notation to C(A,g) and, where the choice A,B is clear from the context, we simply use E.' If the matrix A is stable (i.e., all eigenvalues of A have strictly negative real parts), we refer to C(A,B,HT) -" as a stable system. (2) R(&) is range space of X, N(K) is null space (kernel) of 5, and Rk(K) is rank of X. - - (3) Given system C(A,B,HT), "- R(A,B) = controllable subspace of the pair (A,B) A " 4 R(B) + + . . . + - A"-~R(B) - N(€JT,A) 4 unobservable subspace of the pair (HT,A) - n i= 1 4 n N(€J A ) T i-1 (4) If 9 E Rnm is positive semidefinite, we write Q 2 0. If Q is positive definite, we write Q > 0. - 125 11. PARAMETRIZATION OF NONDESTABILIZING. FEEDBACK MAPS . We begin our discussion as follows. . . Definition 1 Given the stable system C(A,B), let that is, S(C) is the set of all feedback maps tkat never destabilize an originally open-loop stable system, where g(t) = cT&(t). Ideally, one would like to be able to explicitly parametrize S(C), but 0 as this is a well-known intractable problem, our strategy here is to look for a simple parametrization of a (hopefully) sufficiently genera2 subset of S(C) We begin by first recalling some standard Lyapunov-type results: Lemma 1 (Wonham, ref. 4): (i) If A .is stable, then the Lyapunov equation - PA + " ATP + Q = 0 with Q 2 0 has a unique solution 2 0. If, in addition, ! (Q1l2,A) is observable, then P > 0. - - " (ii) If Then A is stable. . - (iii) If 4 L 2 and (%I2,A) , is P 2 0, R > 0 and for all g,F , the - - I observable Tdetectable). Proof: For (i), see reference 4, page 298. !.For.(ii), I . see reference 4, page 299. For (iii), see reference 4, page 82. 126 To proceed , the fo l lowing def in i t ion w i l l be useful,. , . , . . . Defin i t ion 2 . . . , . . , . ' . ' , For any s t a b l e A, l e t . LP@) i CK 2 " O ~ K A+ " A T K 5 0.3 LP+(A) CK > O ~ K A + ATK < 01 , ,-. . " " , . Remark: LP(A) . is genera l ly a proper subse t of ' the set of a l l pos i t ive- semidefint tematr ices of dimension n. . . . . . Example: . . . . . . I . . . . . ' > i, .. . , , I . ' , ' . . . .., . . Suppose t h a t ' Then . . = { [I, :J K1 O Y KlK, 1 [ (X, 4X1X, + X,) 3 K:,} . .' K2 I 0 Note t h a t r:, :J 1 0 i f f K1 I 0, K, 1 0, K1K2 L K:, and t h a t Lemma 2 : ( i ) LP(A) i s a convex cone; t h a t is, X,, X, E Lp (A) implies .' . . a& + ",IC2 E LP,(A) f o r a l l u1 2 0, a, 0 . Proof: Straightforward. We are now ready t o i n t r o d u c e o u r f i r s t c r u c i a l ' r e s u l t . . . I . . 127 . . Lemma 3: L e t A b e s t a b l e . . , Then [A + (E M)K] is s t a b l e f o r a l l K E LP (A) and f o r a l l 1 0, N = -NT and where RTE) C R ( g ) . - - - - I f - K E LP+(A), - then the condi t ion R(E) . C R(M) can be omitted. Proof: L e t 4 = - (H + ATg). Since K E LP(A), we have 9 1 0, and A s t a b l e A impl ies (g1I2, A) is always detectable. - but E + KNTK = 0 s i n c e - N = -NT. - so (m, [A + (E - M)K] is observable, which implies [A + (E - M)K] i s s t a b l e by Lemna 1 ( i i ) . Otherwise , assume R(E) C R(M), which impl ies tha t there exists 1 such tha t E = E o r t h a t " " By de f in ing i n Lemma 1 (iii), we therefore have [Is + 2 m , - A + (E - g)g] is de tec t ab le . By Lemma 1 (ii) , [A + (E - " M)K] s t a b l e . QED Remark : A s p e c i a l case of Lemma 3 w a s e'stablished by Barne t t and Storey ( re f . 5 ) . By s p e c i a l i z i n g Lemma 3 , w e immediately obtain an expl ic i t parametr izat ion of a subclass of s tab i l iz ing feedback . F i r s t we introduce the fol lowing. 128 Def in i t i on 3 Given the s t ab le sys t em C(A, E), l e t and e i t h e r X E LP+(A) o r else - K E LP@) wi th R(S) C R(L) - 1 We can now state o u r r e s u l t as Theorem 1 - Given the s t ab le sys t em C(A, E), then ( i ) ST E S-l ( X ) impl ies (A + ET) is s t a b l e . ( i i ) Jm gTt Q At d t L f e - - (A+BGT) Tt Q e - - (A+BGT) t dt 0 where Q 2 - 0 is such tha t a + AT& + Q = - 0 and - GT (5 - - L)BTK - - E sl(c). Proof: ( i ) L e t E = ETy = ET i n Lemma 3, and the r e su l t fo l lows d i r e c t l y . ( i i ) L e t Q 2 0 be such that Then we have - K = lm gTt Q eAt d t Next rewrite (*) as ” K(A + xT) + (A + STlTg + (2KBLBTK + 4) = 0 where ST (2 - &)ET& E S ( E ) which implies hence . . ' 1 . . . or gTt Q gt dt 2 fe-- (A+BGT)Tt Qe-- (A+BGT) t dt . . Remark: I t can be easily shown that all eigenvalues of the feedback .term . . - B(2 - &)ET& have nonpositive real parts (term -BLBTK has only real eigen- values, while BSBTK has only pure imaginary. (conjugate pairs) eigenvalues or zero eigenvalues). This observation and the content of theorem l(ii) makes it convenient to interpret S1(C) as a natural generalization of the concept of " " II negative" feedback to the multivariable and multiinput case. The next two corollaries are easy consequences of theorem l., Corollary 1.1 - Let C(A, B) be a system with a single input, that is, let 11 be a column (nx17 vector b. If x:, . . . , - &T E S1 (E@, b)), then j j C aigi E s ~ ( c ) for all. ai z 0 , i = 1, '..., j i= 1 Proof: Each is of the form ribTKi for some admissible ri, Ei, so j But, from Lentma 2(i), K+ E LP (A) implies airi$ E LP (A> for all j i= 1 i= 1 airi 1 0 ; hence ais E LP (A) for all ai 2 0 . . QED 130 Corollary 1.2 - Suppose there exists L 0 such that gT E LP (AT). Then [A - =T(K + E) I is stabl; for all E L 0 and = -$ such that R(K) 3 R@) - If gT E LP+(AT) actually, then the condition R(5) 3 R(N) can be omitted. Pro0 f : Immediate from "taking the transpose" in Lemma 3. QED i 1 Theorem 1 has illustrated the importance of LP(A). It is therefore use- ful to have an alternative characterization of LP(A)T Proposition 1 - , . . ' . s . L . : : . . . , . 1 . ' L i . . . .. LP (A), is AT-invariant, that is, for all 5 E LP (A) . I .. . I I . . . . - .. i _ ' I 2 , .. , , . r ( . j ' . . ' , . , , : \ , ,.,. . . Proof: for all t E R , which implies 2 E N(K) . Conversely, 5 E N(K) implies IH e- nI2 dt = 0 or e- 2 = 0 for all t E R, that is, c - -T At T At - x E N(gT, A). To complete the proof, note that Remark: The s igni f icance o f p ropos i t ion 1 is t h a t i t provides a systematic . . : means fo r gene ra t ing a l l members of LP(A). For example, i f A has p i s t inc t : real eigenvalues , then every 5 E LP(A) i s o f t he form - . . - K = PTMP - " where t h e rows of P are l e f t e i g e n v e c t o r s o f A, t h a t is, . . . - . . - PA = E, = diagonal ( A l , ..., An) and M = diagonal (m ..., %), 9 1 0, i = 1, ..., n. Thus, a l l members of LP(A)- can be t r iv ia l iy genera ted once - P is known. While membership i n S1(C) is suf f ic ien t to guarantee c losed- loop sta- b i l i t y , i t i s , of course , no t necessary , tha t i s , S1(C) is a s t r i c t l y p r o p e r subset of S(C). I n t u i t i v e l y , i f t h e open-loop system is s t a b l e enough t o begin with, i t can to l e ra t e a c e r t a i n amount of "posit ive" feedback without lead ing to c losed- loop ins tab i l i ty . In o ther words , the poles of the open- ' l oop sys t em can be sh i f t ed to the r igh t by feedback without destroying s tabi l - i t y o s o l o n g as none of them are s h i f t e d i n t o t h e c l o s e d r i g h t - h a l f p l a n e . By' al lowing such addi t ional nondestabi l iz ing feedback, therefore , w e o u g h t t o be able to "enlarge" S1 (C) . More p rec i se ly , we have the following. Def in i t i on 4 Given the s t ab le sys t em C(A -3 - B) and any - L 2 0, L ER-, let LP(C, L) = {& 1 OlKA + ATK + 2KBLB 5 01 A T "" Def in i t i on 5 Given the s t ab le sys t em C(A, B), l e t - L 1 0, L L c y and e i t h e r K E LP+(C , L) o r else - K E xP(y, &T with R(f, + 3) C R(L - E) 1 . Theorem 2 - Given the s t ab le sys t em C(A, E), then cT E S2(C) implies (A + ET) is s t a b l e . Proof: The proof follows by a s t ra ightforward extension of the proof of Lemma 3 and theorem 1 and hence i s omitted. QED 13 2 Remark.:. 1 t : c a n b e e a s i l y s e e n t h a t theorem 1 is j u s t a s p e c i a l theorem q2 (with L 3 ,O and 2 5 0, S2(C) w i l l be reduced to t h a t , i n !he geneyal case covered by theorem 2 , no def in i teness made of - L, and thus various "mixtures" of positive and negativ allowed. The next propo ' s i t ion provides fur ther c la r i f ica t ion on our t i o n scheme. F i r s t d e f i n e F1 (E) {ST E Rmxn(z = DB K, D E R- a r b i t r a r y , - K E Rnxn and 1. 0) T T " 1 . , Propos i t ion 2 : t h a t is, any feedback map ST E Rmxn is e i t h e r Fp (E) Proof: We need only to Necessity: Suppose - K E R n a , such show t h a t F1 (2) = €5 E R" I RK (ST.> - GT E F1 (B) , t h a t is , t h e r e t h a t - G r = " DBTK. Then I G ~ B D ~ = DB KBD 2 T T " " i n t h e set F1( 0 - Suff ic iency: Take D = GTB and observe tha t ST = GTBBTE has a s o l u t i RK(gTg) = G ( G q . - "- We now relate the content of proposi t ion 2 t o theorem 2. and hence our parametrization scheme f a i l s t c feedback map E F2(;). That S(C) n F2@) f 4 s t r a t e d by the fol lowing t r i v i a l example: . . . 133 Example : . . . . . . . . . . . . . I . . . : . . . ! .I. . . . . . . , . . . . . I : .. ? . , i . . . . . . . . . . . . .. , i. . . . . . . : , . . ' . I .'I : '. :. '., .~ T and (A + hT) = i s s t a b l e . , - . . . . . 1 ~. . - . , . . . . . L . . . . . . . . . . . .. , . ,,> 2 - . . . . . . . . . . , I. ' . . .'. .:. _. Note, however, t h a t . i . f B . is of f u l l r a n k , t h e n . t h e set F2 (B) . ,'is' not ...... :. g e n e r i c i n R-. The more i n t e r e s t i n g q u e s t i o n - Is S2(C) gener ic (i..e..;, . ' . I' dense) i n S ( C ) n F1(E)? - i s present ly unsolved. . . . . . Our r e s u l t s so. f a r have been on systems C (A, E) , which :.are 'open-loop -: s t ab le ; t he ques t ion nex t arises as t o what t h e s i t u a t i o n would be for ' sys tems. , not open-loop s t a b l e ( i . e . , A has unstable poles). For A uns.tab.ie, i t is , of course , no t poss ib le t o write Lyapunov-type equations. One is reminded, however, of the a lgebra ic Ricca t i equa t ions ; indeed , w e have the fol lowing interpretat ion of the t radi t ional LQ-opt imizat ion problem: , I . . . . Def in i t i on 6 Given (4, E), a s t a b i l i z a b l e p a i r , l e t LQ(& E) = {X 2 015 = &(A, E, R, 11 ) f o r some :'R > '0 and some A T 7 - HT so t h a t (g , A) is a d e t e c t a b l e p a i r ) T - where K(A, B, E, ) ' d eno te s t he un ique pos i t i ve s emide f in i t e so lu t ion t o the a lgeb ra i c R icca t i equa t ion : T KA + A ~ K - K B R - ~ ~ ~ K + HH = 0 T - -- - - For - R f ixed , w e denote the corresponding set as LQ(& E; E). Def in i t i on 7 Proposi t ion 3 - Given any s t ab i l i zab le sys t em C (A, B) G E S3(C) implies. (A +' BGT) T " - 9 - - . - is s t a b l e . Remark: The above proposition merely summarizes the well-known "standard" r e su l t s o f LQ-optimal feedback theory (refs. 1 and 4 ) . However, t he i n t e rp re - t a t ion he re o f t he LQ-optimal feedback class (S3(C.)) as a parametr izat ion of a subc lass of s tab i l iz ing feedback is i n t e r e s t i n g . . . . . ., .. 134. Remark: . It can be easily seen that theorgm 1 is just a special case of theorem ;2 (with & 0 and & 5 0, S2 (C) will be reduced to S1 ( E ) ) . Note that, in she general case covered by theorem 2, no definiteness assumption is made of &, and thus various "mixtures" of positive and negative feedbacks are allowed. The next propokition provides further clarification on our parametriza- tion scheme. First define that is, any feedback map ST E R- is either in the set F1(B) or else F, (E) . - Proof: We need only to show that Necessity: Suppose GT E F1 (B) , that is, there exists 2 E R- and - K E Rnm, K 2 0 such that ST= "_ DBTK. - Then Sufficiency: Take D = GTB and observe that GT = GTBBTK has a solution 5 2. 0 if RK(gTg) = E(Gq.- - "- - QED We now relate the content of proposition 2 to theorem 2. Observe first and hence our parametrization scheme fails to capture any feedback map E F2(g) . That S(C) r7 F2@) Z I$ is demon- strated by the following trivial example: 133 I I I I I 111 1111.11 111111111 -I ,I I 1",.,. ...... 1 1 1 ............ . . . . . . ..".... . . . ................ ~ Example : > . - "9, . . . . . : . . I . . . . <, . . . . I . , . I . . . . . . . . . . . , . :. : . . . . ,, ,l . . - . . . . . . . . . I 'I : ..... .. 5 and '(A + & ) = E is s t a b l e . . . . .: . . 4 . . . Note, however, t h a t . i f B is of f u l l r a n k , t h e n t h e set ' F2 (B) ' ,'is' no t . '.-: g e n e r i c i n R-. The more iE te re s t ing ques t ion - Is S2(C) gener ic (i,.e.-; ' I I ' dense) i n S(C) r7 F1(g)? - is presently unsolved. ' T . . . . . . . t . . . . . . . , . . , . . ,,> . . , :.t: . . . . . . . . . .... -. IT . . - . . . :. . . . , . . , Our r e s u l t s so. f a r have been on systems C(A B ) , which :are ' o~en- loop : s t ab le ; t he ques t ion nex t arises as t o what t h e s i t u a t i o n would be fo r ' sys t ems . . not open-loop s t a b l e ( i .e . , A has unstable poles) . For A unstab$e, i t is, of course, not possible t o write Lyapunov-type equations. One is reminded, however, of the a lgebra ic Ricca t i equa t ions ; indeed , w e have the fol lowing -3 - i n t e r p r e t a t i o n o f t h e Given (A, E) , a LQ(A, 2) = A where X@, E, E, ET> the a lgeb ra i c Riccati t r a d i t i o n a l LQ-optimization problem: .:. Def in i t i on 6 s t a b i l i z a b l e p a i r , l e t {X 2 = &(A, E, g, E ) f o r some :'R > '0 and some ' T T .7 - - HT so t h a t (E , A) is a d e t e c t a b l e p a i r ) - . deno tes the un ique pos i t i ve semide f in i t e so lu t ion to equat ion : KA + A ~ K - K B R - ~ B ~ K + HH = o T - " - " - - For - R f ixed , w e denote the corresponding set as LQ(A, E; E). Def in i t i on 7 Propos i t ion 3 - Given any s t a b i l i z a b l e s y s t e m C (A, E), s E S3(C) implies. (A + BGT) T i s s t a b l e . , - . - Remark: The above proposition merely summarizes the well-known "standard" r e s u l t s o f LQ-optimal feedback theory (refs. 1 and 4 ) . However, t h e i n t e r p r e - t a t ion he re o f t he LQ-optimal feedback class ( S 3 ( c > ) as a paramet r iza t ion of a subc lass of s t ab i l i z ing f eedback is i n t e r e s t i n g . - . . . . . - . , .: ., . 134. ,111. ' . STRUCTURAL STABILITV CHARACTERIZATION OF LINEAR QUADRATIC (LQ) OPTIMAL FEEDBACK MAPS In this section, we show how the parametrization scheme developed re- 0 tural Stability properties of systems under LQ-optimal feedback. More pre- viously can be applied to obtain a characterization of the closed-loop struc- cisely,, we:establish an explicit parametrization of a general class of structural'perturbations .in.the control feedback gains as well as in the con- trol actuation matrix (2) that leave the closed-loop system stabilized. These new results, we believe,.are the natural generalizations of some earlier results of Anderson and Moore (ref. 2). We begin by first recalling from Lenrma 2(iii) that, for A stable, 4 K E LP@) always implies K E LP (A - BLBTK) ; however, for A unstable and K 2 0 such th t (4 - BLBE) is stable, it need not be true that K E LP@ - BLB K). The following example underscores this unfortunate state of affairs: " L I Example: -1 3 Then (A - BLBTK) = [ -d is stable, but K(A - BLB K) + (A - BLB T "" " However, we have the following interesting observation. Lemma 4: Proof: Immediate from the Riccati equation. QED In other words, the above unfortunate state of affairs cannot occur if - K is an LQ solution. We are now ready to state our first main result of the section. 135 I - N = -E R(N) C R@) - 6 a r b i t r a r y Proof: We have K E LP(4 - g-'gTK), so by Lemma 3, [A - BRmlBTK + (V - W)K] is s t a b l e f o r all 1 0, 1 = -3 such that R(1) 7 R(W). - " - "- - Take = B (L - E-1 )gT + g ~ T and 1 = + HT and we are done. T - - - QED Remark: For 4 F 0, theorem 3 is a gene ra l i za t ion of t h e " i n f i n i t e g a i n margin" property of LQ-optimal feedback for s ing le- input sys tems f i r s t no ted b Anderson and Moore ( r e f . l), who showed tha t the feedback ga in vec tor J = - ( l / r )hTE can be mult ipl ied by any s c a l a r a 2 1 without destroying s tab i l i ty ; the proof they used involves c lass ica l Nyquis t t echniques . Theorem 3 no t on ly gene ra l i zes th i s p rope r ty to mul t i i npu t sys t ems , bu t a l lows more complicated alterations of the feedback gain vectors. Moreover, i t makes the proof of this property much more t ransparent . Remark: For # 0, theorem 3 a l lows for changes in the B m a t r i x i t s e l f w i thou t des t roy ing s t ab i l i t y . One u s e f u l i n t e r p r e t a t i o n is the following. Suppose that the opt imal feedback gain matr ix has been computed f o r a nominal &, b u t t h a t the ac tua l va lue of 2 during system operation i s changed t o B = & + gl. Then the feedback term becomes ( U - ~ ~ T E C g I g - l s l ~ ) . SO long as g1 = go(^ + M)R for some N = -NT, M 2 0, theorem 3 guarantees that the system w i l l remain s tab le (e .g . , B1 = a&, a > 0). More complicated cases are allowed. I Remark: A l t e rna t ive ly , t he ca se 1 f 0 can be in te rpre ted as a l lowing for . the poss ib i l i ty of adding ex t ra cont ro l le rs , and us ing these ex t ra feedbacks to "fine-tune" the closed-loop behavior of the or ig ina l sys tem. (A more sys temat ic explo i ta t ion of th i s idea w i l l b e ' d e a l t w i t h i n a future publ ica- t i o n ; see a l s o r e f . 3 ) . -- - - Theorem 3 has dea l t w i th the case when the "negative" feedback gains, negative" feedback gains are reduced i n magnitude (or when addi t ional "posi- etc., are allowed to increase i n magnitude; the converse s i tuat ion, when t h e tive" feedbacks are in j ec t ed ) i s examined i n the next proposi t ion. I 1 Theorem 4 (gain reduction and robustness property): L e t K 2 0 be the Ricca t i so lu t ion to the LQ problem (A, E, E, 9) where & > O - and (Q1I2, - A) detec tab le . Then ( i ) [A - B(M + N)BTK] is s t a b l e f o r a l l M > 0 such tha t - " "_ " M > 5 E-' N = -E 1 T - - 136 I ( i i ) I f (Q1l2, A) i g actual ly observable , then [A - g(g + z)gT& + + - N) 3 is s t a b l e where E, are as above, and 4 = aT is such tha t a < (1/2)Q, R[ (1/2)Q - a)] 3 R(Q) and 8 = -@T is such t h a t R[ (1/2)Q - $1 3 R@). Proof: - ( i ) L e t C = C [ (A - 13J-’ET&), 21 = X(&, E). Then w e have A - K E LP[Cc; (1/2);-l] from the Ricca t i equa t ion; so, ‘by theorem 2, [& + B ( i - rJ)g X] is s t a b l e f o r a l l fi < - R-l T 1 - 2 - - N = -ST o r [A - .g(g-l - + N)BTK] is s t a b l e . L e t = - > (l/Z)$-’, and t h e proof is complete. A - ”_ - ( i i ) L e t AC = [A - E(% + g)ET&]. From t h e Riccati equation, we have A . . Kii, + &Tg + @(2% - g-l)gT& + Q = 0 . Since (Q1i2, A)-observable implies & > 0, x-’ e x i s t s , so we have “4 K[k + - K-l(fi + - fi)] + [&+&-’@++)lT5 + E(2FJ-g-1)2Tg + (9- 24) = - 0 Hence, sub jec t t o the cond i t ion (1 /2 )Q 2 4, R[ (1/2)Q - $1 3 R(Q + &) it can be shown t h a t r 1 Remark: Theorem 4(i) is a genera l iza t ion of the known gain reduction toter- m c e property of LQ-optimal feedback. This interpretat ion is most t r anspa ren t i n t h e s p e c i a l case when = d i a g ( a l , . . . , a,,J and M = d iag (il, . . . , &), - N 0. Then the or ig ina l ind iv idua l feedback loops are of t h e form the system remains s table i f the feedback gains are reduced t o 9 - -SAT& so long as ii > (1 /2 )a i . More complicated cases are, of course, allowed. Remark: By i n t e r p r e t i n g t h e a d d i t i o n a l term + fi) as a model perturba- t i o n term SA of the open-loop matrix A, w e can use theorem 4(i i ) t o perform f i n i t e p e r t u r b a t i o n s e n s i t i v i t y a n a l y s i s . The following simple example i l l u s - trates the usefulness of this approach. Ei = -.AT&, i = 1, . . . , m. The theorem states t h a t , i n t h i s s p e c i a l c a s e , Example : I f we take Then we ob ta in the a lgeb ra i c Riccati so lu t ion as 0 and the optimal feedback gain &*T = -211 11. For any where we are assured, by theorem 4 ( i i ) , t h a t . is s t a b l e f o r a l l a > 1/2. Consider the following special cases: (a) y = B,,, B1 = 8, = 0. We have s t a b l e f o r a l l a > 1 / 2 and B,, so t h a t ( 1 - B,,), < 1.5 or 1 - 6 < B,, < 1.g (b) y = B,, = 0, w e have " 1 + a&* T -2 + B, 138 stable for ail a > 1/2 and B , , B, so that (i) B1 < 0.5,: B2 < 3 . . (ii) (0.5 - B 1 ) ( 3 . - 6,) > 1 7 . Thus if B , = 0, the perturbed system is stable for all B, < .,I. 'Other more-.,!. general cases are, of course, allowed. The above example thus shows that the combined effect of feedback gain reduction and perturbation or uncertainty of the open-loop system parameters (poles and coupling terms) can be tolerated by a linear quadratic design with- out leading to closed-loop instability. This' robustness prbperty ,of the " ' ' . ' LQ-feedback design deserves more atiention. , . . . I IV. CONCLUDING, REMARKS I Since fur"ther applications of the parametrization results established here to reliable stabilization synthesis and decentralized stabilization coordination will be made in a future publication, we reserve a fuller discus- sion of the implications of our approach until.then. However, we would like to point out an important implication for practicaZ design that is immediate: the ability to perform feedback "loop-shaping" analysis. . In any realistic synthesis problem (keeping a syst-em stabilized, local- izing particular disturbances, etc.), there is usually a large number of fea- sible solutions. While the use of cost-criterion optimization (e.g., LQ) in theory allows the designer to pick exactly one..such solution, in practice, the difficulties of judging or fully incorporating the relevant cost considera- tions and their tradeoffs as well as the often gross model uncertainties and physical variabilities of the system and the controllers necessitate further , sensitivity analysis or trial-and-error "hedging" about the nominal solution. It is therefore very important in the computer-aided design context that the feasible solution space" structure be known in some detail to facilitate and guide the.conduct of iterative search. In this regard, a major merit of a "classical" design technique like root locus is that it provides an explicit functional dependence of the closed-loop system structures (distribution of poles and zeros) on the control structure (feedback gain). However, such classical approaches become totally intractable when there is a multiple num- ber of controllers, while llmodern," "state-space"' linear feedback design tech- niques like "pole-placement" algorithms and "dyadic-feedback" design s,u,f,fer the serious drawback of providing little structural information about th'e solutions they generate. Moreover, such techniques are guided more by mathe- matical convenience' than'by physical interpretation. From this perspective, the parametrization results established earlier show promise in providing the basis for a new iterative design algorithm that will overcome the last- . , mentioned drawbacks. I . I 1 0 Several years ago, Rosenbrock,(ref. 6 ) suggested .a frequency-domain multiloop feedback design technique (''inverse Nyquist .array" method) which he ,139 . motivated also .as an attempt to overcome some of the abovementioned drawbacks. His approach is in contrast with ours, which is a "time-domain" approach. It will be interesting to investigate the connection, if any; betweesthe two approaches. REFERENCES 0 1. Athans, Michael: The Role and Use of the Stochastic Linear-Quadratic- Gaussian Problem in Control System Design, IEEE Trans. Automatic Con- trol, vol. AC-16, no. 6, Dec. 1971, pp. 529-552. 2. Anderson, B. D. 0.; and Moore, J. B.: Linear Optimal Control, Prentice- Hall, New York, 1971. 3. Wong, P. K.: On the Interaction Structure of Linear Multi-Inp,ut Feedback Control Systems, S. M. Thesis,' Electrical Engineering and Computer Science Dept., M. I T., 1975. 4. Wonham, W. M.: Linear Multivariable Control: A Geometric Approach, Springer-Verlag, Berlin, 1974. . , . .. . , ..I . . . . . 5 . Barnett, S.: Matrices in Control Theory, Van Nostrand, New York, 1973. 6. Rosenbrock, X. H.: Design of Multivariable Control Systems Using the Inverse Nyquist Array, Proc. IEEE, vol. 116, no. 11, Nov. 1969, pp. 1929-1936. 140 . ANALYSIS OF PILOT-AIRCRAFT PERFORMANCE AND RELIABILITY VIA THE APPLICATION OF MAN-MACHINE MODELS David L. Kleinman University of Connecticut SUMMARY Analytic modeling of human performance has progressed to- the, point where it can be used with some confidence in the design and performance analysis of man-machine systems. This presentation first reviews the present approach to t h i s analytic modeling, which utilizes human response theory together with mcdern control theory. Then an example of its use in helping design a flight director for the hover control of a VTOL vehicle is described. Finally, how the method could be used for analysis of human response to syst m failure is aket ched . 141 gb1eCtiVeS . . . 0 Develop and apply analysis procedures for manually controlled System 0 Understand human as an information processor, controller and decision . . . . I maker Approach 'Combine human response theory with modern control and estimation theory Assumption . I 0 Experienced humag behaves optimally subject to his inherent limitations Applications to Manned Vehicle Systems Basic manual control tasks VTOL precision hovering . . Failure detection and control Carrier and land-based approach Anti-aircraft gunnery STOL flare and landing Figure 1.- Modem control approach to human modeling. Input disturbances I Display t I Control input, u(t) """"_ Human I I I I I Motor noire Obxryation n o w I Figure 2.- Man as a generalized feedback control element. 142 X(t) = Desired state trajectory x(t) = Deviation of system response from nominal . ~. . . . 8 ' . . , . . , , . , ... . . . . . , Linearization about X(t) . . . %(t) = Ax(t) + Bu(t) + Ew(t) + Fz(t) distsbances I I - , , . , ' .. . . . . . . , , . w(t) = Gaussian, zero mean white noise , i ,!. I , I . ., ' . . . . , . . . . . , . .. I * . E{W(t)W(T) 1 W(t)6(t -: T) . . . . . I . . ' . , . , . > . ' . z(t) = Non-random, bias 'inputs ' ' . . . , . . . . . . h(t) = Ah h(t) -t Fh Zit) , . .. - , , . . . Y(t) = Available information concerning x(t) , , .:. r , , I . , ' , ' . .. . . . , . I .e contains position ' and velocity information ' Y(t) = Cx(t) + Du(t) . - . . , , I . . . , .: . Figure. 3 .- Vehicleldisplay .dynamics. Lump sensory, central processing, perceptual limitations into perceptual equivalents Lumped human time-delay, T Small signal thresholds, F(*) "Observation" and central processing noise, vy Yp(t) = "perceived" signal . .. ~~~~ ~ ~ . , . . . . Yp(t) F[Y(t " T)] i- vy(t - T) . . , . . . Figure 4. - Human perceptual limitations. 143 Most important element in human randomness model Related to subject training level For foveal viewing of a single display p y i = noiselsignal ratio Scanning and attention allocation w .e c. nk = fractional attention to display k NI I Pyi = P p n k I Human picks nk to optimize information "Neuro-motor" dynamics; h(u) - uc o(Z) + T N & + u = uc uc = .commanded control u = actual control Implicit modeling via limits on control rate "Motor" noise, vu White, Gaussian noise Second component of human randomness model e TN& + u = uc + vu Figure 6.- Human control limitations. Figure 5 . - Equivalent observation noise. Display T p i + u = -L;(tI t) + Vu(t) TN = "neuromotor" lag from 6 weighting r"f"""" I-1 I i I lnformtion Processor - l(1) L = optimal feedback gains Figure 7.- Optimal control model of human response. G(tlt) = best estimate of x(t) from {Yp(u) , u s t) Generation of $(t It) constitutes human's information processing behavior Kalman filter Generates best estimate of , &t - T I t) Compensates for. vy(t) . Predictor . , - . Generates f;(tl t) from i(t - 'I It) Compensates for delay Separable feedback control behavior Figure 8.- Feedback control solution. I . . . . . . . . . . \ . Time delay, T 0.15-0.25 sec Observat ion noise ra t ios , p y i 0.01 (-20 k 3 dB) 4: ' . 1 . . . . . . . . . . . I- . . . Thresholds, a; 0.05' visual a r c . . . . . (posi t ion) % . ) . . . . . . _. i. " . ,<- : . , .. O.OS'/sec ( r a t e . : Typical Y, = k, k/s , k / s2 . 6 Motor noise r a t i o s , p u i 0 T ( s ) = , no f se shap ing f i l t e r "Neuromuscular" lag , TN; 0.08-0.12 sec i,, = y + w ' y, =:x2 Assumpt.ion: Human parameters do &t depend on & 2 , = x , + u y2 = ,g2 = XI + u -. . . . . . . . . . . . . . . .. - . . . . ., - . . . . . . . . . sz 01"003 (-25 dB) , . i d . . . . : . . . . , . . . . . vehic le dynamics o r co t ro l cos t .... . . . . . . , . . . . * ' :-- Q\ func t iona l .I ...... Human's task: minimize mean squared error ! . . , ; A ; . . :.;.; { * . . ,_, ';-.';. ._ . . Human parameters do depend on ex terna l J (u) = E[y: + g c2] inf luences - l i g h t , h e a t , stress, vibra- t ion , d i sp lay reso lu t ion , e tc . Steady-state, stationary behavior Applications u ( s ) = h , (s)y, (s) + h 2 ( S ) y , ( S ) + noise . . . , . = [h l (s ) + sh2(8) ly , ( s ) + noise Figure 9.- Typical human response parameter values. " . ' v measurable . . . . . . . , . . _ , . . . . . . . , . I . < - Figure 10.- Simple SISO t racking tasks . I _ . . . . . . . . . . i . . . . . . . . G U . . Figure 11.- Measured and predicted performable measures k/s2 ' dynamics (Avg of 3 subjects). rn Refinement of existing model Analyze model-data mismatches Effect of a priori information Attentional allocation/scanning . . Human decision making - Decisions based on 2 Fault detection System monitoring . . H u m a n adaptivity and learning . . Explore relation to observation noise Training effects Environmental effects . - . - . , , - : . . . - Relate to changes. in human parameters Stress, age, alcohol, vibration, g-forces . . . Multi-human control tasks Human modeling in socio-economic systems . . . .a I Figure 12.- Further efforts in man-modeling. " "" . _" Objective: allocate tasks between pilot and automatic system Approach: 1) Select pilot/automatic configurations 2) Determine information requirements for control and monitoring 3) Perform comparative evaluatfon Figure 13.- VTOL controlldisplay design. Monitoring workload metric fM fMi = monitoring workload i fMi= fractional monitoring attention on ith display Figure 14.- Pilot monitoring with fixed total workload. 148 Equal attention fq = fH/ny Equivalent to residual monitoring I Peak excursion A Generalized quadratic index Instrument failure detection Relative estimation errors Performance assessment Miscellaneous Figure 15.- Monitoring model concepts. Control performance predictions .Control workload metric fc = fci = control workload i . fci = fractional control attention on ith display element Attention allocation min fci JC Figure 16.- Optimal control pilot model. 149 x = J, = J M = . . f ' I . , . . .I . . , . . . .. , . . , - . . . Ax + 'Bu; + Ew , . y = cx ;. uc =. -Lji . . . Jcc + tr[LeCLL] L, = diag'(qr1/2)LeAT ' , ? . . . . . ., 0 . . = CA' + AC + EWE' - C C ' P I C C Control: Vi - Ni ;, oi - g(fOl, fC2 . . .) Monitoring: V. = p . 0 2 / ( f c i , + f .'.)Ni ; ai, f, = constant 1 l i ml i I Figure 17.- Expressions for :cost measures J, JM. 7 Closed-form expressions for gradients . aJc 3% O D A - w - eA" L,!=L, e" da G] ' i aJM vi afm + f, " - e"' C i C e e'' do G i i i 1 G = CC'V-l = Kalman f i l t e r g a i n ; 2 = A - GC Nonl inear th reshold e f fec ts inc luded as RIDF Gradient project ions Cfi = cons tan t ; fk - f m f o r p o s i t i o n s / r a t e s Bas ic g rad ien t op t imiza t ion for f i 1 Figure 18.- Algorithm for optimizing f i . . . 150 . - I. Longitudinal axis, pitch hold, fc - 0.3 0 Predictions (A) without FD,. (B) with FD Instrument Altimeter Pitch IVSI mJK 9 6, 6, wfo FD 25.5 ft 4b7 ft 1. oo 114 fps "- "_ .13 in. .25 in. aY .# I FD 16.6 ft. 4.7 ft .7O 1.2 fps .13 in. .14 in. .10 in. .22 in. T fc, /fc I A wjo FD 0.35 .17 .03 .45 "_ "- Figure 19.- Helicopter hover task. FD 0.03 .1 .03 .13 .30 .40 0 Figure 20.- Predicted performance CH-46 hover flight director used. 153 . . . . . 4.0 3.5 3.0 - a Ei 8 2.5 2.0 1.5 1.0 o o 0 cx without F.D. 0 I(u) withcut F.D. I I I I Attention to longitudinal task .3 4 .5 .6 Figure 21.- Scores vs attention CH-46 hover longitudinal axis. - 3.0 - 2.5 - 2.0 - - - U 0 1.5 - B 1.0 - ~ .5 I- o Figure 22.- Scores vs attention CH-46 hover lateral axis. Performance measures and workload metrics defined for control and/or monitoring Procedures developed for attention allocation - avoids previously arbitrary choice of fi Continuing efforts Further validation Determine information requirements Tradeoff studies Figure 23.- Summary. Context Detection and identification of unexpected system failures Objective Model the human's decision process and adaptive control behavior Combine Human operator modeling 9 Decision and control theory Human response results Planned experimentation Figure 24.- Models for human decision-making and control. 153 . .. . H u m adaptive process . . r""""""""" ! I I I I 1 I '.. I 1 Conirol of pe-failure plant dynamics + I - , I Defection of a change o!fag- .I . L - - - - - - - - ________-I Identification or diagnosis of the fault 1 I I Corrective action modification of control strategy appropriate to post-failure dynamics Figure 25.- Human adaptive process. t < tf : G(t> = Aox(t) + Bou(t> + w(t) 1 S €{Si, i = O , 1 ..., N} k Humans task: Continuously determine true Sk Models : 1. Sequential failure detection and'identification 2. Simultaneous failure detection and identification Figure 26.- The adaptive control problem. 154 Internal dynamic model in Kalman filter c( t ) = Ap(t) + BG(t - T) + G[yp (t) - Cp(t) ] p(t) = i(t - T) State estimate G(t) G(t) = best internal estimate of vehicle state (x-i) (t) = error covariance Residual or innovations r(t) r(t) = yp(t) - cG(t - T) Nominally r(t) = zero mean, white noise Figure 27.- Features for adaptive modeling and decision making. If sk = SO then r (t I So) is zero-mean, white, with covariance vY Testing r(t) over t - T < T 2 t 1. Mean value 2. Whiteness 3. Covariance Determine model parameters via experiment Limited model Figure 28.- Failure detection by innovations testing. 155 Model structure for control and decision Experimental planning, model validation Implications for automation 1. Human monitoring 2. Automatic status information 3. Allocation of decision making Figure 29.- Conclusions. 156 RECENT PROGRESS I N MULTIVARIABLE CONTROL W. M; Wonham Universi ty of Toron.to, Canada SUMMARY The las t f ive years have wi tnessed a remarkable development i n linear mul t iva r i ab le con t ro l , f rom both the theoret ical and applied viewpoints. This paper summarizes the major new re su l t . s . i n s t ruc - tu ra l syn thes i s and des ign which are now avai lab le in tex tbook form, and suggests a number of promising d i r e c t i o n s f o r f u t u r e r e s e a r c h . 1. INTRODUCTION The las t . f ive years have witnessed a remarkable development i n l i n e a r mul t iva r i ab le con t ro l , f rom both the theoret ical and applied viewpoints. Defer r ing for a moment the app l i ed a spec t , we n o t e f i r s t t h a t s i g n i f i c a n t advances have taken place in our theoret ical understanding of system struc- tu re . Under th i s head ing w e group two major topic areas: r ea l i za t ion theo ry and system synthesis. Rea l iza t ion theory is concerned with the axiomatic def ini t ion of "system," and with relat ing " internal" system descr ipt ions ( i .e . , s ta te desc r ip t ions ) with "external" ( i .e . , input-output) descr ipt ions. For this , a un i f i ed mathe- matical framework w a s suppl ied by Kalman e t aZ. ( r e f . l ) , e x p l o i t i n g t h e con- nect ions (wel l known t o a l g e b r a i s t s ) between l inea r a lgeb ra and the theory of modules over a polynomial r ing. The general theme has since been pursued in the context of delay systems (ref . 2 ) , b i l inear sys tems ( re f . 3 ) , and what might be called "categorical" systems (ref. 4 ) . While rea l i za t ion theo ry supp l i e s t he in f r a s t ruc tu re , i t is system syn- t h e s i s and design that i s perhaps of most d i r ec t r e l evance to eng inee r s , and the remainder of this paper is devoted to ske tch ing the s i tua t ion here , t oge the r w i th the imp l i ca t ions fo r p rac t i ca l app l i ca t ions . Be fo re ge t t i ng down t o s p e c i f i c s , i t may be usefu l to d is t inguish be tween "synthes is" and "design," a t least in the contex t of cont ro l sys tems (c f . re f . 5, ch. 2 ) . I would l a b e l s y n t h e s i s as the process by which one e s t a b l i s h e s t h e q u a l i t a t i v e s t r u c t u r a l p o s s i b i l i t i e s : t h a t is, one determines whether and how one can achieve such desirab1e"system properties as nonin terac t ion (among func t iona l ly d i s t inc t subsys t ems and the i r a s soc ia t ed con t ro l l e r s ) , l oop s t a b i l i t y , r e g u l a t i o n o f s p e c i f i e d o u t p u t s w i t h r e s p e c t t o d i s t u r b a n c e s , t rack ing by s p e c i f i e d o u t p u t s t o r e f e r e n c e s i g n a l s i n some preassigned class, 157 and , finally, erties within structural stability (i.e., preservation of the foregoing prop,-:' a range of parameter variations). . . . . . On the other hand, design would.refer to the numerical massaging (ideally, optimization) of system parameters, within the structural frame- work established by synthesis, to meet quantitative design specifications, related (in this context) to transient response, stability margin, saturation levels, and so on. While the distinction is obviously not clear cut, we' have at least two working definitions. The point of making them is to stress that a control theory, qua theoret- ical construct, deals largely with synthesis, and thus may be expected to operate on a level of relative generality and abstraction. Its function is precisely to clarify the structural possibilities. Design may then bring in a host of special techniques, many of them heuristic, to ensure that a speci- ,: fic gadget works in practice. It is hoped that these simple-minded remarks - , : may forestall useless arguments about the ''gap. I 1 We shall now review some recent theories of linear multivariable system , ' ~ synthesis, turning finally to their potential impact on design. . . 2. APPROACHES TO SYNTHESIS A theory of synthesis will start from an assumed system model or class of models together with an assumed class of admissible controls, and supply , constructive answers to a series of precisely posed questions of the following kind: Does an admissible control exist such that the resulting (synthesized) system displays such-and-such desirable characteristics? To qualify as a theory, and not just a library of algorithms, our constructive procedures .. should somehow exhibit the answers in terms of a reasonably small number of , basic system concepts. While these must, of course, be made theoretically precise, they must also embody enough intuitive content o render the process of discovering new answers, by new procedures, easy and natural for the prac- titioner. Finally, the variety of questions amenable to attack must be wide enough to embrace a significant range of potential applications, and our con- structive procedures must translate readily into computational algorithms: these will provide the framework for design, as noted earlier. I think it is fair to claim that two complementary theories exist today which go some way to meeting these criteria. These are the geometric state- space approach (ref. 6 and references cited therein), and the polynomial matrix-frequency domain approach (ref. 5, 7 , 8, and cited references). I should emphasize that in neither of these theories is the notion of optimization at all central, and I omit discussion of the familiar "linear- quadratic" optimization problem, on which an extensive textbook literature already exists. It suffices to say that quadratic optimization is merely one method of achieving the rather limited goal of stabilization via state 158 feedback, and this theory, useful as it may be, contributes little to the primary objectives of structural synthesis. 2.1 Geometric State-Space Theory Here we take for granted a (discrete or continuous-time) state.mode1, of the familiar type % = A x + B u + E r + . . . ' y = ~ x + & + . . . . . z = D x + G u + . : . As usual, x and u denote the state and control vectors, r .= r(t) is a disturbance vector, y is the vector of measured' variables, and z the vector of variables to be regulated (e.g., tracking errors, deviations from set-point values). We restrict attention to the deterministic theory, although the stochastic counterpart is fairly straightforward. In purely linear theory, the admissible controls are of the type u = F x + G r + . . . incorporating state feedback and (possibly) disturbance feedforward. There will be a constraint restricting the control to process only the measured variable y; this translates algebraically as Ker(F,G) 3 Ker(C,e) . One can bring in observers or other dynamic compensators by various standard tricks. What are the basic system concepts? Everything starts from the funda- mental ideas of controllability and observability or, more accurately, the geometric objects "controllable subspace" and "unobservable subspace" of the state space x. Two other related families of subspace that play a basic role are the 'I (A,B)-invariant subspaces" and the 'I (A,B)-controllability sub- spaces. I' To give a brief indication of their meaning, suppose we have simply % = A x + B u , z = D x . Then we say that the subspace V c Ker D is (A,B)-invarimt if, whenever the initial state x(0) E V, there exists a control u(t) (t 2 0) such that the ensuing state trajectory x(t) = x(t;x(O),u) also satisfies x(t) E V (t 1 0); then Dx(t) 5 0. It turns out that there is always a Zargeet (A,B)-invariant subspace, say V", contained in Ker D: roughly, this is the largest'set-of states which can be made unobservable (by suitable 159 c o n t r o l ) a t the ou tput z . F ina l ly , the subspace R c X is an (A,B)- controZZabiZity subspace i f e v e r y x E R can be reached from the origin of x by a ' c o n t r o l l e d state t ra jec tory whol ly conta ined in R: t h u s , i f R c Ker D, you can sneak from zero t o any x E R without being seen from z. There is always a l a r g e s t R, say R*, i n K e r D, and we always have R* c V*. Using j u s t t h e s e n a t u r a l and elementary ideas, i t has been poss ib le to bu i ld up an e labora te set of procedures for s t ruc tura l synthes is . What problems have been solved? H i s t o r i c a l l y ( 1 9 7 0 ) , t h e f i r s t was the longstanding problem of nonin.ter- ac t ing con t ro l o r decoup l ing . In the s imples t ve r s ion , one is g iven the system ? = A x + B u wi th output vec tors zi = Dix , i = l , . . . , k One a s k s f o r a s ta te feedback control with new e x t e r n a l i n p u t s v i , k i= 1 u = Fx + Givi , so tha t each v i con t ro l s t he ou tpu t Z i a lone w i thou t a f f ec t ing t he f o r j # i. Under very reasonable condi t ions on t h e number of ava i lab le controls , one can show t h a t t h i s i s almost a lways possible (a t least by dynamic compensation, omitted here); furthermore, one can achieve more-or-less a rb i t ra ry pole loca t ions for the c losed- loop sys tem. The cond i t ion fo r s o l v a b i l i t y is j u s t t h a t 'j Ri i * + K e r D = x , i = 1 , . . . , k where R i * is the l a rges t con t ro l l ab i l i t y subspace con ta ined i n n Ker D j #i j The numerical checking of this condi t ion i s q u i t e s i m p l e . A t t he Un ive r s i ty of Toronto, w e have developed APL rout ines tha t have been run successfu l ly on decoupling problems of a t least moderate dynamic order (15-20). The second major s t r u c t u r a l problem to be t r ea t ed comple t e ly and suc- c e s s f u l l y by these methods was the gene ra l mu l t iva r i ab le se rvo regu la to r problem (1974). One i s given % = A x + B u , y = C x , z = D x Here t h e dynamics represent the plant together with a dynamic model of t h e d is turbance (and/or re ference) s igna ls tha t must be re jec ted (and/or t racked) . 160 One a s k s f o r a c o n t r o l u = $y so t h a t z is regula ted (z ( t ) + 0 from any i n i t i a l s t a t e ) and so t h a t t h e l o o p is s t a b l e . While these two requirements are not necessar i ly compatible , i t turns out that they almost always are, provided one has a t least as many scalar c o n t r o l s as scalar v a r i a b l e s t o b e regula ted . The p r e c i s e s o l v a b i l i t y c o n d i t i o n s are a l i t t l e too involved to state here , bu t they amount t o q u i t e t r a n s p a r e n t r e l a t i o n s among s u i t a b l y defined V*, R*. Again, i t is st raightforward to develop a synthesis proce- d u r e t h a t t r a n s l a t e s , f o r example, t o APL ( r e f . 9) . L imi t a t ions o f space p roh ib i t go ing in to fu r the r de t a i l : may I j u s t men- t i on tha t t he geomet r i c t heo ry has a l so shed l i gh t on d e c e n t r a l i z e d s t a b i l i z a - t i o n ( r e f . 1 0 ) and t o va r ious syn thes i s problems involving system "zeros" ( r e f . 1 1 ) . I n summary, the "geometr ic approach" has a l ready jus t i f ied i t se l f i n terms of r e s u l t s and shows ample promise for future development. 2.2 Polynomial Matrix Formalism On taking Laplace t ransforms of the equat ions j r = A x + B u z = DX + Cu one is l e d t o t h e poZynomiaZ system matrix: P ( s ) = r;I More generally, pol-ynomial system matrices of form arise when h i g h e r o r d e r d i f f e r e n t i a l o p e r a t o r s a p p e a r i n t h e i n i t i a l dynamic desc r ip t ion . The more f a m i l i a r transfer matrix is now defined as G ( s ) = V ( s ) T " ( s ) U ( s ) + W(s) The r i ch s t ruc tu re theo ry o f P ( s ) and G ( s ) has been explored notably by. Rosenbrock ( r e f s . 5 and 7) and Wolovich ( r e f . 8). A s these au thors adopt somewhat d i f fe ren t v iewpoin ts , we sha l l rev iew the i r approaches separa te ly . Of c o u r s e , i n t h e s p a c e a v a i l a b l e w e cannot do them j u s t i c e . I n terms of P ( s ) , Rosenbrock supp l i e s and j u s t i f i e s t h e a p p r o p r i a t e def ini t ion of system dynamic order (=deg [det T ( s ) ] ) , t oge the r w i th tha t o f system poles and zeros, and relates these concep t s t o the con t ro l l ab i l i t y - o b s e r v a b i l i t y s t r u c t u r e o f a state v a r i a b l e r e a l i z a t i o n . By t h e i n t r o d u c t i o n of " s t r i c t system equivalence" (based on coordinate transformat'ions involving 161 no change in dynamic order), it is shown how P ( s ) can be reduced to various useful standard forms; and then how the invariant structure of P ( s ) relates to that o'f .G(s) (e. g. ; the invariant polynomials of P coincide with the . I Smith-McMillan numerator polynomials of G). (In this review, some accuracy is sacrificed to brevity.) The system matrix is further exploited to discuss . open-loop zero assignment (by output selection) and closed-loop pole assign- ment (by state feedback). The formal theory culminates with an investigation of the key idea of diagonal dominance. A rational matrix Q ( s ) = [qi*(s)] is diagonazzy (rot)) dominant on a contour C in the complex s-plane ii! for all i, and s E C. Rosenbrock now shows that if the forward-loop trans- fer matrix is diagonally dominant (with C a standard Nyquist path), then stability of the closed-loop system can be studied by application of the Nyquist (or inverse Nyquist) criterion to the diagonal elements alone. Coupling effects are accounted for by broadening the usual inverse Nyquist ' loci into "bands," obtained by ingenious exploitation of various inequalities due to Gershgorin and Ostrowski. While something will be said below about the design methods 'that ensue, this summarizes the "synthetic" content of Rosenbrock's approach (refs. 5 and 7). What are the basic system concepts? - again, controllability and observ- ability; next, the polynomial system matrix and its equivalent forms; and, finally, diagonal dominance or quasi-noninteraction. What synthesis problems have been solved? Actually Rosenbrock does not pose synthesis problems in a highly formal way, but proceeds more in the spirit of ''design'' to illustrate how the foregoing concepts can be exploited to satisfy various closed-loop requirements. Heuristics are proposed to achieve open-loop diagonal dominance through simple types of plant precompen- sation. Here interactive computation is essential, with graphical display of inverse Nyquist arrays. The result is to reduce the design problem to decoupled "classical" subproblems for independent single-input/single-output loops. It is reported (convincingly) that criteria of stability margin, bandwidth, structural integrity, and so forth can be effectively brought into Play Turning now to Wolovich's approach (ref. 81, we find a creditable attempt to unify the state and frequency viewpoints. The starting point is a "struc- ture theorem" that represents a transfer matrix G ( s ) in the form G(S) = N(s)D-~ (s) 162 where N ( s ) and D ( s ) are polynomial matrices that satisfy a certain coprime- ness relation. Under the action of linear state-variable feedback, it turns out that the "numerator" matrix N ( s ) remains unaltered, while the "denomina- ' , tor" matrix D ( s ) is almost freely assignable. In this way, the effect of a time-domain compensation scheme can be displayed in the frequency domain. A furtQer result shows that any control scheme that can be implemented by feedforward compensation can also be implemented via an equivalent feedforward/' feedback scheme. , With these as the basic concepts, Wolovich turns to a series of'synthesis problems, formally posed in terms of transfer matrices. The major problems are noninteracting control (decoupling) and exact model matching. The decou- pling problems are resolved by constructive procedures; for model matching, heuristics are proposed. It is worth remarking that, while Wolovich's expressed intent is to unify the state and frequency approaches, his transfer matrices are treated as purely algebraic objects: the usual frequency domain ideas (e.g., bandwidth, gain, and phase margins) never appear at all. ' In effect, his procedures represent disguised linear algebra, with the poly- nomials serving mainly as bookkeeping devices. Of course, this is not to say ' . that the procedures, once mastered, may not be effective; presumably, they are most convenient when system data are supplied initially in transfer matrix . form. . . . . : . 3 . IMPACT ON DESIGN My remarks here must be brief and tentative, as the methods just described are quite recent. One thing is clear: as the methods have been evolved for application to large, complex systems, design will rely on inter- active computing, presumably with graphical display (of Nyquist arrays, pole- zero patterns, time and frequency responses, etc.). For this, development of efficient numerical algorithms and their organization via a high-level proce- dural language is the kind of major advance to be expected in the near-term future. Considerable progress has already been achieved by, for instance, Rosenbrock at the University of Manchester (1J.K.) , Istrgrn at the Lund Insti- tute of Technology (Sweden), and Mansour at the ETH (Zurich). These three groups are sustained in part by substantial interaction with industry. Men- tion should also be made of the work in computer-aided design at Cornell, as reported by Merriam (ref. 12). 4 . CONCLUSION . . Large-system multivariable synthesis has arrived. The outlook for - attacking the further problems of hierarchical and decentralized control is good. The effective exploitation of what is known even now will, neverthe- less, 'demand patience, and commitment of resources. 163 5. REFERENCES 1. Kalman, R. E.; Falb, P. L.; and Arbib, M. A.: Topics in Mathematical System Theory, McGraw-Hill Book Co., 1969. 2. Kamen, E. W.: Applications of 'Modern Algebra to Infinite-Dimensional Systems, to appear. 3. Brockett , R. W. : On the Algebraic' Structure of Bilinear Systems. Theory and Application of Variable Structure Systems, R. R Mohler and A. Ruberti, eds., Academic Press, 1972, pp. 153-168. - , 4. Arbib, M. A. ; and Manes, E. G., eds. : Arrows, Structures and Functors: , .. The Categorical Imperative, Academic Press, 1975. . , , . . . . . . , 1 . . I: ,. 5. Ro'senbrock, H. H. : Computer-Aided Control System Design, Academic Press, ' C . . ' . 1974. . . I 6. Wonham, W. .M.: Linear Multivariable Control: A Geometric App'roach, ' Springer-TTerlag, 1974. , . 7. Rbsenbrock, H: H.: State-Space and Multivariable Theory, JohiWiley and Sons, Inc., 1970. 8. Wolovich, W. A;: Linear Multivariable Systems, Springer-Verlag, 1974. 9. Sebakhy, 0. A . : Synthesis of Linear Multivariable Regulators. Ph.D. . Thesis,' Univ.' of Toronto, 1974. . . . , 10. Corfmat , J. P . ; and Morse, A. S. : Control of Linear Systems Through. ' . Specified'hput Channels, SIAM J. Control, to appear. 11. Bengtsson, G. : A Theory for Control of Linear Multivariable Systems, Rept. 7341'; Lund Institute of Technology, 1973. 12. Merriam, Charles W., 111: Automated Design of Control Systems, Gordon and Breach, 1974. . . 164 A SUMMARY OF NONLINEAR FILTERING Ian B. Rhodes Washington University, Mo. I. INTRODUCTION In this summary paper, we attempt to highlight the main results of modern nonlinear filtering theory and discuss some of the approximations commonly made t o yield suboptimal filter designs (such as extended Kalman filters) 'that avoid the infinite-dimensional on-line computations required of the optimal nonlinear filter. No attempt is made to provide a survey of the field of non- linear filtering or even to begin to approach the breadth and depth of cover- age or the exhaustiveness of referencin.g a survey implies. Nor does this paper provide the self-contained development conventionally thought f as. being implicit in a tutorial introduction. We content ourselves with outlin- ing the main approaches, stating and discussing the main results, and refering to a representative sampling of papers or books in which derivations, proofs, details, and extensions can be found. These considerations plus a de$ire to be brief leave great latitude in the selection and relative emphasis .of topics and, perhaps even more than most papers of a summary or survey type, this paper reflects a personal view of the field of modern nonlinear filtering. This paper is organized as follows: section I1 is devoted to the formu- lation of the prototype nonlinear filtering problem with which the rest of the paper is concerned, along with a discussion of some of the modifications, extensions, and alternative models found in the literature. In section 111, we outline the main approaches used to characterize or find the optimum non- linear filter; in section IV, we recall the enormous simplifications that result when the system is linear. Sections V and VI are concerned with approximations to the optimal nonlinear filter: the former with relinearized estimators and extended Kalman filters, the latter with approximations to the conditional density. Finally, section VI1 summarizes an approach to designing and evaluating the performance of suboptimal filters in terms of upper and lower bounds. 11. FORMULATION OF NONLINEAR FILTERING PROBLEM For the nonlinear filtering problems considered here, attention will be concentrated on the smooth, finite-dimensional, continuous-time dynamic system: dx, = f(xt,t)dt + D(t)dvt dz, = g(xt, t)dt + G(t)dwt 165 where xt is an n-dimensional random vector, vt is r-dimensional, zt and wt are m-dimensional, and v and w are independent.norma1ized Wiener pro- cesses. The initial state x. is independent of v and w . and is Gaussian _. .. with mean x. and covariance Co. Functions f and g are assumed to satis.fr suitable technical properties; G(t) is nonsingular. Everything takes place on the fixed finite interval [O,T]. Less rigorously, we can also write Xt = f (xt, t) + D(t)Gt . . . I (la'). where the formal derivatives and $ of the Wiener processes v and w are (independent) Gaussian white noises. The system equation (1') can be obtained formally be dividing both sides of equation (1) by dt and thinking of y = i as the observed processes. Similar formal manipulations can be used to con- vert, for example, the filters written later in the "differential" notation used in equation (1) to the "derivative" form used in equation (1'). We will also restrict attention to minimum-mean-square-error filtering. Letting Zt be the past of the observation process z, ' that is, Zt = I ( z s , s ) , 0 5 s 5 tl we are thus interested in finding the conditional mean of xt , given Zt, and the corresponding conditional covariance Throughout the paper, prime denotes transposition. The conditional covariance provides, of course, a measure of accuracy or confidence. The filtering problem defined by equations (1) to ( 4 ) has estabiished motivation as a model that arises in a wide variety of control and communica- tion problems, and most of the literature in the systems area is concerned with this model or a modification f it. Some of the many possible modifica- tions and extensions include the following. (a) From a control viewpoint, the most important extension is to replace equation (la) by dx, = f (xt ,ut , t)dt + D(t)dvt (la") where the (r-dimensional) control ut is assumed to be derived causally from the output process z . Most of the results given later have natural counter- parts for this model. 'More precisely, Zt is the o-algebra generated on the .underlying proba-. , bility space by the observation process up to time t. 166 (b) Smoothing and prediction problems, where interest centers E[xt I Z,] ; .prediction problems have s < t, smoothing ones have s generally, one can seek E[h(xt) I Z,] for some suitable function can have s > t, s < t, or s = t. on finding > t. More h, and we (c) The finite-dimensional continuous-time system (1) can be replaced by alternative system models; for example, by a finite-dimensional discrete-time sys tem. o r by an infinite-dimensional system such as a delay-differential equation or a partial differential equation (refs. 1-6). Consideration of the discrete- time finite-dimensional model ( 5 ) avoids a number of technical mathematical . questions associated with the continuous-time system (1); conversely, techni- cal problems are accentuated for infinite-dimensional models, even when they are specialized to being linear. (d) Generalization of noise process models: extensions are available for correlated noise processes v and w (ir and i ) , "colored" (rather than "white") ir and i , and state-dependent noises where the distribution matrices D(t) and G(t) in equation (1) depend also on the system state. When f and g are linear, a "wide-sense" approach can be added: here v and w are taken to be uncorrelated-increment processes (G and wide-sense white) not necessar- ily Gaussian, and one seeks the Zineax minimum-mean-square-error estimator. Problems in which G(t) is singular lead to singular estimation problems: only when f and g are linear has some headway been made (refs. 1, 5 , and 7-11). (e) Different criteria: the minimum-mean-square-error optimality cri- terion can be replaced by other choices, such as maximum a posteriori (MAP) estimation, etc. Other modifications include formulating the problem as one of statistical least-squares estimation; invariant imbedding or the calculus of variations are typical solution techniques (e.g., refs. 12 and 13). (f) Alternative observation models arise naturally in some applications. For example, for certain problems in nuclear medicine and quantum-limited optical communication, the observations take the form of a point (counting) process usually modelled as Poisson process whose intensity (rate) is itself a random process influenced by the system state xt (e.g., refs. 14 and 15). More recently, martingale theory has been used to provide a un fied treatment that includes counting process models and the model (1) as special cases within a more general setting (e.g., refs. 16 and 17). (g) Spec,ial structures: For some classes of systems, a vector space setting is less natural than a formulation in some other mathematical frame- work whose structure is better matched to the characteristics of the class of systems at hand. An example is the formulation of some problems on manifolds 167 and bringing to bear the techniques of differential geometry in attacking them. Included here are estimation problems involving bilinear systems (refs. 18-20). (h) Combination problems: Estimation problems frequently arise in com- bination with additional requirements such as identification or hypothesis- testing (the latter includes detection and some formulations of reliability problems). Superimposed on top of these can also.be a control task., 111. APPROACHES TO SOLVING NONLINEAR FILTERING\ PROBLEM There are essentially two main analytical approaches to finding or char- acterizing the conditional mean and conditional covariance. The first is to find. or characterize the conditional density pt(xt I Zt) of xt given Zt, or the conditional distribution pt(xt 1 Zt) . The conditional mean and covar- iance then follow as kt = xpt(x I z+x = x dvt(x I Zt). ( 6 ) Rn Rn whereb dt on the left side denotes differential with respect to time, W(t) = G(t)G' (t), it = E[g(x,t) I Ztl = I g(x,t)pt(x I Zt)dx (9) Rn and L* is the forward Kolmogorov partial differential operator associated with the system (1): The stochastic partial differential equation ( 8 ) has been called Kushner's equation (refs. 21 and 22). It is also referred to as modified Fokker- Planck equation or a conditional Kolmogorov equation. 168 The practical difficulties in attempting to find pt(xt I Zt) from equa- tion ( 8 ) are the obvious computational problems of solving a partial- differential equation on line. Levieux (ref. 2 3 ) reported recently on his efforts in this direction; indications are that the approach is limited with current computational technology to a state xt of dimension at most tw or three. (B) A second way to characterize pt(xt I Zt) is via the so-called Bucy representation theorem (ref. 1) that relates the conditional density of xt given Zt to the prior density pt(xt) of xt: where (log likelihood ratio) 9t is The representation (11) can be viewed as a form of the Bayes rule suited to the problem under consideration here. Its validity rests on certain condi- tional independence assumptions satisfied for the prototype problem formulated in section 11, but are violated if, for example, v and w are correlated or if equation (la") is substituted for (la). Note that equation (11) is not as it stands an explicit formula from which pt(x I Zt) can be calculated but rather it is a relation that pt(x I Zt) must satisfy since the coefficient of pt(x) on the right side depends implicitly on pt(x I Zt). Note also that the numerator of this coefficient involves a conditional expectation in which attention must be confined to state trajectories that pass through point (x,t). We return in section VI to the use of representation (11) and its variations in constructing approximations to pt(x I 2,). As an alternative to first finding the conditional density and using equations ( 6 ) and (7) to compute the conditional mean and covariance, one can instead write stochastic differential equations for xt and Ct: dCt = -+ [m - d; d;' + D(t)D' (t)dt + [e- G;]'[p(x,,t) &]'w-l(t)[dzt - it dt] where 2, = E[f(xt,t) I Z,], etc. These stochastic differential equations can be derived in a number of ways, such as by combining equations (6) nd ( 8 ) and integrating by parts, by integrating equation (11) by parts, by the innova- tions approach (ref. 7), or by measure transformation approaches (refs. 2 4 - 2 6 ) . 169 However, fhe integration of equation (13) requires the simultaneous computa- tion of f,, it, and M x t , t j and, although stochastic differential equa-, tions.for each of these can also be written down, they, in turn, require ,; simultaneous calculation of even more terms such as = E[g(xt,t)g'(xt,t) I Z,], the stochastic differential equations for which require even more terms, and so ad infinitum. Similar remarks apply to C,. Thus, except in certain special cases where this mushrooming requirement for additional terms terminates at a finite number, we are left with the need to solve an infinite system of stochastic differential equations. Thus, generally all three approaches to computhg the conditional mean and covariance require an infinite-dimensional on-line calculation, whether that is the determination of the conditional density from the partial differ- ential equation (8) or via representation (lo), or the Solution f the infin- ite system of stochastic differential equations in which equations (13) and (14) are imbedded. Each provides its own perspective on the common con- clusion: the exact optimum filter is infinite-dimensional and therefore com- putationally infeasible. Our interest then turns naturally to suboptimum but practically implementable filters and, with it, what each of the three approaches offers in suggesting suboptimum estimator structures and design procedures and in providing a basis for error analysis (performance evalua- tion). Before turning to this, however, we review briefly the special case of- a linear system, for which the optimum filter happens to be finite-dimensional. Apart from its intrinsic interest as a filtering problem with an exact, imple- mentable solution, the linear case serves both as a benchmark and as a source of potential structures for suboptimal candidate designs. IV. LINEAR FILTERING PROBLEMS When the system (1) is linear, f(x,t) = A(t)x , g(x,t) = C(t)x the conditional mean and covariance can be generated from the finite- dimensional system: dGt = A(t)fit dt+ C(t)C'(t)W'(t)[dz - C(t)Gt dt]; x. = x0 - (16) i(t) = A(t)C(t) + C(t)A' (t) - Z(t)C' (t)W-'(t)C(t)Z(t) +D(t)D'(t); C(0) =: Cg The filter defined by equations (16) and (17) is known as the Kalman (or Kalman-Bucy) filter. It can be derived from the general equations (13) and (14) in fit and Et, from the Bucy representation (ll), or directly using any of a variety of techniques such as the projection theorem. The simplifi- cation of equations (13) and (14) to (16) and (17) consists essentially of observing that when f and g are linear they commute with conditional expec- tation. Not only are no additional equations required beyond equations (16) and (17) to completely specify the conditional mean and covariance, but 17 0 equation (17) for the conditional covariance does not depend on the observa- tion process z . Thus C is nonrandom and can be precomputed. To emphasize this, we write C(t) rather than Ct in this case. The linearity of the system and the Gaussianness of the noise processes results in the state and observation processes being jointly Gaussian, so that the conditional density of the state xt given the past observations is also Gaussian. This Gaussian posterior density is thus completely,described by the conditional mean Gt and the conditional covariance Ct. Furthermore, the conditional mean is a linear function of the data Z and, in fact, is the finite-dimensional linear function of the data given by equation (16). For the model equation (la") that includes a control causally derived from the observations with f(x,u,t) = A(t)x + B(t)u and g(x,t) = C(t)x, the state and observation processes are no longer Gaussian, but the conditional density of the state given past observations and controls is still Gaussian with con- ditional mean Gt, the finite-dimensional linear function of past u and z , given by d2t = A(t)Gt dt + B(t)ut dt + C(t)C' (t)W-l(t) [dzt - C(t)Gt dt] (18) The conditional covariance is still given by equation (17) and remains precom- putable and independent of u and z (this fact is of major importance in sto- chastic control situations). Finally, we note the simple structure of the linear filter (18). A s noted, for example, in reference 27, it consists of a model of the determinis- tic part of the system (la") (which generates, in particular, the first two terms on the right side of eq. (8)), a unity gain negative feedback loop that generates the innovations dz - CSt dt, and a gain C C ' W 1 that operates on the innovations to produce the third term on the right side of quation (18). Imitation of this simple structure is one possible starting point in develop- ing approximate filters for nonlinear problems. V. RELINEARIZED ESTIMATORS AND EXTENDED KALMAN FILTERS Because linear estimation problems are so amenable to analysis and have finite-dimensional solutions, while nonlinear problems require infinite- dimensional filters, it is natural to attempt to approximate nonlinear prob- lems by linear ones in the hope that the optimum filter for the linearized problem will suitably approximate the optimum nonlinear filter. In this vein, the most common approach is to continually relinearize the system equa- tions (1) about the current state estimate ;it using a Taylor series expan- sion in which second- and higher-order terms are neglected, namely, f(xt,t) = f(Gt,t) + fx(Gt,t)[xt - ;it] (19a) 171 where fx(Gt,t) denotes the partial' derivative of f with respect to its first argument evaluated at st. The system equations can then be written in the linear form dx, = fx(Gt,t)xt dt '+ ft dt + D(t)dvt (20a) . dzt = gx(Gt,t)xt dt + Et dt + G(t)dwt (20b) where the known (given Gt) quantities ft and Et are given by - Then, with appropriate modifications to reflect that the linear system of interest is now equation (20), the Kalman filter (eqs. (16) and (17)) becomes, after collect-terms, Note that the error covariance CtA now depends on the observation process via the dependence of f, and gx on x. The suboptimum estimator (eqs. (21) and (22)) is usually called an extended Kalman filter. Alternatively, instead of identifying the linearized system (20), one can simply substitute equation (19) into (13) and (14) for the mean and covariance of the nonlinear problem. The resulting equations can be simplified to equa- tions (21) and (22), provided it is assumed that the suboptimum estimator j; satisfies E[xt - Gt I Zt] = 0 so that, for example, we have, from equa- tion (19a), ft = f (xt, t) = f (Gt, t) . By first identifying the linearized sys- tem (20), the need to make this explicit assumption is avoided. Yet another alternative is to take the structure of the optimum linear estimator as a starting point, substituting instead a model of the nonlinear system to obtain dGt = f(Eit,t)dt + Lt[dzt - g(Gt,t)dt] (23) A - where the gain Lt is yet to be determined. Subtracting equation (23) from (I), using the approximations (191, and writing xt for Xt - Gt yields - diit = fX(Gt,t)gt dt + D(t)dvt - Lt dwt-Ltgx(iit,t)jtt dt and, at least for f, and g, regarded simply as time functions, the gain L, that minimizes the covariance of 2 is identified from linear system theory as Lt = Ztg$(%t,t)W1 (t) , thus again leading to the extended Kalman filter (eqs. (21) and (22)). The degree to which the suboptimum estimate 2, given by the extended Kalman filter approximates the optimum (conditional mean) is obviously related 17 2 to the extent to which the second- and higher-order terms in.the Taylor series expansion are negligible. The quality of this latf ?r approximation will be improved wLen the nonlinearities are small and when the deviations of the state and observations (from j; and 2) are small, t5at is, when C o and the noise covariances DD' and GG' = W are small. ' -,wevery almost nothing is known analytically about the performance or quality of approximation of extended Kalman filters (exact evaluation requires an infinite-dimensional calculation) and performance characteristics as mean-square error, sensitiv- ity, etc., are evaluated in specific cases by simulation and Monte Carlo tech- niques. These simulation studies are usually limited to comparisons of com- peting suboptimal designs, rather than relating the suboptimum estimator to the optimum. Bias errors (E% # 0) and divergence problems are not infre- quently encountered. Remedies for these tend to be limited to .ad hoc rules of thumb (e.g., a trial-and-error increase of DD' in the simulations, which has the effect of giving more relative weight to the more recent observations). Various alternatives to continual relinearization about the current state estimate are possible. One is to linearize about a prechosen nominal trajec-, tory; this leads to a precomputable filter gain Lt in equation (23) and pre- computable C y but performance remains difficult o assess, and can be expected to be worse than for continually relinearized estimators. A econd possibility is to retain second-order (or even more) terms in the Taylor series expansion about 2, which on substitution into equations (13) and (14) and simplification yields estimators considerably more complicated than equa- tions (21) and (22). Again, performance evaluation is limited to simulation and Monte Carlo techniques (ref. 28). Other possibilities include statistical linearization (refs. 3 and 13) and a wide-sense approach in which certain nonlinear problems are converted via transformation into a linear problem with non-Gaussian initial state (ref. lo). Like the extended Kalman filter (eqs. (21) and (22)), these estimators all suffer from the absence of analyti- cal methods for design, performance evaluation, error analysis, sensitivity studies, etc., and these characteristics must be evaluated by simulation. Comparisons between competing design procedures are severely limited by the absence of any small collection of accepted benchmark problems that might form a standard of comparison in simulation studies. Finally, any comparisons are almost always between competing suboptimum estimators and not between subopti- mum and optimum. VI. APPROXIMATIONS TO pt (xt I Zt) Another approach to ap roximate nonlinear filtering is to approximate the conditional density pt(zt 7 Zt) in terms of a finite number of basis func- tions. Expansions used here include "point mass," Fourier, Hermite polyno- mials, spline functions, and sums of Gaussian densities (refs. 29-35). These are usually done for discrete-time problems and typically use a version of the Bayes rule derived from the Bucy representation theorem. Because the effects of truncation and other questions related to quality of approximation have not yielded to analytic treatment, Monte Carlo methods are used to assess filter performance, data-processing rates, the number of terms needed, sensitivity, etc. Different expansions yield different data-processing rates and different 173 estimation accuracies; what works well for one problem may not work well for another. Computational effort is typically large. Comparison 'studies have been limited to Monte Carlo comparisons between suboptimal estimators;'almost nothing is known as to how. the performance of the suboptimal filters compares to the optimum achievable performance. . VII. PERFORMANCE BOUNDS The infinite-dimensionality of the optimum nonlinear filter, on the one hand, and the difficulty in evaluating the performance of extended Kalman filters and other suboptimum estimators on the other makes it desirable to seek out an analytical framework within which suboptimum filters can be designed and their performance evaluated with realistic computational effort. One direction along these lines which has received attention is an analysis in terms of precomputable upper and'lower bounds. To establish bounds, some explicit assumptions are needed on the functions f and g in equations (la) or (la") and (lb), and much of the work here considers systems incrementally 8 conic (or cone-bounded) in the sense that f(X + 6 , ~ + y,t) - f(x,u,t) = A(t)x + B(t)u + El(t) + €p(t) g(x + 6,t) - g(x,t) = C(t)x + E&) Thus incrementally conic systems are linear to within a un formly Lipschitz residual. Cone parameters A, B, and C define a nominal linear system, while a, b, and c measure the degree of nonlinearity. Many common modeling func- tions are incrementally conic, including sinusoids, inverse tangents, and saturating nonlinearities; also included are linear systems whose parameters are known only to within specified tolerances. Polynomials and exponentials in x and u are not incrementally conic. Within the framework of incrementally conic systems, upper bounds have been derived in reference 36 on the mean-square error associated with the parametrized (by K) family of easily implemented estimators: dEtt = f(it,ut,t)dt + K(t)[dZt - g(jit,t)dt] ( 2 4 ) These are matrix-ordering bounds of the form where the n x n matrix PK(t) satisfies a linear differential equation. Thus, for any given K(t) in equation ( 2 4 ) , the corresponding mean-square estimation error is guaranteed to be less than PK(t). Furthermore, there existsAa choice K*(t) of K such that the corresponding upper bound P*(t) = PK*(t) is at all times smaller in the matrix ordering sense than that for any other K(t) (i.e., P*(t) ,< PK(t) for all K and all t). Because 17 4 K*(t) depends on the corresponding P*(t), P* satisfies a Riccati-like equation (just as the optimum gain fo,r, linear estimation problems depends on the corresponding error covariance C(t), which results in C(t) satisfying, a Riccati equation). This does not say that the actual mean-square error of the estimator (1) is smaller for K* than for any other K. What it does say is that when K* is used the guaranteed performance is smaller than the per- formance guaranteed using any other K. f j26j where $(t) is the optimum error covariance for the nominal linear estimation. problem (and thus depends only on A , C, C o and the noise covariances DD' and W), while the "shrinkage" factor .r(t) 1 depends on a and c as well. Both upper and lower bounds share most of the properties of the optimum error covariance for linear problems: in particular, they are easily computed bound cfnverges uniformly and with finite ultimate to the optimum error covar- iance P of the nominal linear problem as the nonlinearity vanishes (a,c 3 0 ) ; the upper bound converges similarly for reasonable choice of K, including the bound minimal gain K* and the optimum gain for the nominal linear problem. Together, the upper and lower bounds define a range in which, must lie both the true optimum performance and the actual mean-square error of the suboptimum estimator (eq. ( 2 4 ) ) . In some, specific cases, this range may be sufficiently small that the suboptimum estimator can be deemed sufficiently close to optimum. In others, the guaranteed performance provided by the upper bound might meet acceptable performance specifications with the optimum achievable performance being irrelevant. In such cases, the bounds provide a simple analytical and computational framework for design and evaluation. If the guaranteed performance is inadequate (but still potentially achievable in view of the lower bound), one can then turn to alternative suboptimum designs such as those discussed earlier, with performance assessment by simulation and design modification by trial and error. Similar upper bounds can be found for extended Kalman filters (where K can depend on 2 as well as t), but these upper bounds are larger than those where K depends only on t. The upper and lower bounds can be used to derive performance bounds for stochastic con- trol problems (ref. 38); they have also been extended to smoothing and predic- tion (ref. 1 2 ) . ' by contemporary standards, and they are control-law independent. The lower Other lower bounds of a Cramer-Rao type have been derived in refer- ences 39 to 41. Upper bounds for certain rather restricted classes of. system can be found in reference 42 and refer,ences cited in reference 36 . 175 REFERENCES 1. Bucy, Richard S.; and Joseph, Peter D.: Filtering for Stochastic Pro- cesses with Applications to Guidance, John Wiley & Sons, 1968. 2. Curtain, Ruth: A Survey of Infinite-Dimensional Filtering, SIAM Rev., vol. 17, no. 3, July 1975, pp. 395-411. 3. Gelb, Arthur, ed.: Applied Optimal Estimation, MIT Press, 1974. 4. Kwakernaak, H.: Optimal Filtering for Linear Systems with Time Delays, IEEE Trans. Automatic Control, vol. AC-12, no. 2, April 1967, pp. 169-173. 5 . Sage, Andrew P.; and Melsa, J. L.: Estimation Theory with Applications to Communications and Control, McGraw-Hill Book Co., 1971. 6 . Yu, T. K.; Seinfeld, J. H.; and Ray, W. H.: Optimal Filtering for Sys- tems Governed by Functional Differential Equations, Proc. Joint Automatic Control Conference, Columbus, Ohio, June 20-22, 1973. 7. Frost, Paul A.; and Kailath, Thomas: An Innovations Approach to Least- Squares Estimation - Part 111: Nonlinear Estimation in White Gaussian Noise, IEEE Trans. Automatic Control, vol. AC-16, no. 3, June 1971, pp. 217-226. 8. Jacobson, David H.; and Speyer, J. L.: Necessary and Sufficient Condi- tions for Optimality for Singular Control Problems: A Limit Approach, J. Math. Anal. Applic., vol. 34, no. 2, May 1971, pp. 239-266. , 9 . Jazwinski, Andrew H.: StochasCic Processes and Filtering Theory, Academic Press, 1970. 10. Lo, James Tingo-Ho: Finite-Dimensional Sensor Orbits and Optimal Non- linear Filtering, IEEE Trans. Information Theory, vol. IT-18, no. 5, Sept. 1972, pp. 583-588. 11. Wonham, W. M.: Random Differential Equations in Control Theory, in Probabilistic Methods in Applied Mathematics, vol. 2, Albert T. Bharucha-Reid, ed., Academic Press, 1970, pp. 131-212. 12. Dunn, K. P; and Rhodes, I. B.: Cone-Bounded Nonlinearities and Mean- Square Bounds - Smoothing and Prediction, Automatica, vol. 11, no. 5, Sept. 1975, pp. 517-523. 13. Sunahara, Yoshifumi: An Approximate Method of State Estimation for Non- linear Dynamical Systems, Proc. Joint Automatic Control Conference, Boulder, Colorado, Aug. 5-7, 1969, pp. 161-172. 17 6 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. Segall, Adrian; Davis, Mark H.A.; and Kailath, Thomas: Nonlinear Filter- ing with Counting Observations, IEEE Trans. Information Theory, vol. IT-21, no. 2, March 1975, pp. 143-149. Snyder, Donald L.: Filtering and Detection for Doubly Stochastic Poisson Processes, IEEE Trans. Information Theory, vol. IT-18, no. 1, Jan. 1972, pp. 91-102. van Schuppen, J. H.: Estimation Theory for Continuous-Time Processes: A Martingale Approach, ERL Memo "405, Univ. of Calif., Berkeley, Sept. 1973. Varaiya, Pravin P.: The Martingale Theory of Jump Processes, IEEE Trans. Automatic Control, vol. AC-20, no. 1, Feb. 1975, pp. 34-42. Lo, James Ting-Ho; and Willsky, Alan S.: Estimation for Rotational Pro- cesses with One Degree of Freedom - Parts I, 11, 111, IEEE Trans. Automatic Control, vol. AC-20, no. 1, Feb. 1975, pp. 10-33. Willsky, Alan S.: Fourier Series and Estimation on the Circle with Applications to Synchronous Communication - Parts I, 11, IEEE Trans. Information Theory, vol. IT-20, no. 5, Sept. 1974, pp. 577-590. Willsky, Alan S.; and Marcus, Steven I.: Estimation for Bilinear Sto- chastic Systems, Rept. ESL-R-544, Electronic Systems Lab., M.I.T., May 1974. Kushner, H. J.: Dynamical Equations for Optimal Nonlinear Filtering, J. Differential Equations, vol. 3, no. 2, April 1967, pp. 179-190. Kushner, Harold J.: On the Dynamical Equations of Conditional Probabil- ity Density Functions with Applications to Optimal Stochastic Control Theory, J. Math. Anal. Applic., vol. 8, no. 2, April 1964, pp. 332-334. Levieux, F.: Treatment of Noise in Nonlinear Systems: Conception of New Algorithms and Applications, Rept., France , 19 74. 24. Dunn, Keh-Ping; and Rhodes, Ian B.: A with Applications to Estimation and Allerton Conference on Circuits and Monticello, Ill., pp. 304-314. IRIA-Laboria, Rocquencourt, Generalized Representation Theorem Hypothesis Testing, Proc. 11th System Theory, Oct. 3-8, 1973, 25. Dunn, Keh-Ping: Measure Transformation, Estimation, Detection, and Sto- chastic Control, D.Sc. Dissertation, Washington Univ., May 1974. 26. Wong, Eugene: Stochastic Processes in Information and Dynamical Systems, McGraw-Hill Book Co., 1971. 177 27. 28. 29. 30. 31. 32. 33. 34. 35. 36. 37. 38. Rhodes, Ian B.: A Tutorial Introduction to Estimation and Filtering, ' IEEE Trans. Automatic Control, vol.' AC-16, no. 6, Dec. '1971, pp. 688-706. Athans, Michael; Wzshner, Richard P.; and Bertolini, Anthony: Suboptimal ' State Estimation 'for Continuous-Time Nonlinear Systems from Discrete Noisy Measurements, IEEE Trans. Automatic Control, vol. AC-13, no. 5 , Oct. 1968, pp. '504-514. Bucy,' R. g.; and Senne, K. D. :' Digital Synthesis of Non-Linear Filters, Automatica, vol. . . 7, May 1971, pp. 287-298. Bucy, Richard S.; Hecht, Caivin; and Senne, Kenneth D.: An Engineer's Guide' to Building Nonlinear Filters, Rept . SRL-TR-72-0004, Frank J. Seiler Research Lab., USAF'Academy, May 1972. Bucy, R. A.; Hecht, C.; and Senne, K. D: New Methods for Nonlinear Fil- tering, Revue Francais d'sutomatique Informatique et de Recherche Operationelle, 1973, pp. 3-54. Bucy, B. S . j and Youssef, H. : " Fourier Realization of the Optimal Phase Demodulator, 4th Symposium'on Nonlinear Estimation Theory and Its Applications, San Biego, Calif., Sept. 10-12, 1973. Proceedings. Western Periodicals Co., No. Hollywood, pp. 34-38. Bucy, R. S.; and Youssef, B.: Nonlinear Filtering Representation via Spline Functions, 5th Symposium on Nonlinear Estimation Theory and Its Applications, San Diego, Calif., Sept. 1974.Proceedings. Western Periodicals Co., No. Hollywood, 1975, pp. 51-61. Kushner, H. J.: Approximations to Optimal Nonlinear Filters, IEEE Trans. Automatic Control, vol. AC-12, Oct. 1967, pp. 546-556. Sorenson, H. W . ; and Alspach, D. L.: Gaussian Sum Approximations for Nonlinear Fi l t e r ing , Proc. 1970 IEEE Symposium on Adaptive Processes, Dee. 1970, pp. 1931-1939. Gilman, Alfred S o ; and Rhodes, Ian B.: Cone-Bounded Nonlinearities and Mean-Square Bounds - Estimation Upper Bound, IEEE Trans. Automatic Control, vol. AC-18, no. 3, June 1973, pp. 260-265. Rhodes, Ian B . ; and Gilman, Alfred S.: Cone-Bounded Nonlinearities and Mean-Square Bounds - Estimation Lower Bound, IEEE Trans. Automatic Control, vol. AC-20, no. 5, Oct. 1975, pp. 632-642. G ilm, Alfred S . ; and Rhodes, Ian B.: Cone-Bounded Nonlinearities and Mean-Square Eounds - Quadratic Regulation Bounds, IEEE Trans. Auto- matic Control, vol. AC-21, no. 4, Aug. 1976, pp. 472-483. 178 . . 39. Bobrovsky, Ben Zion; and Zakai, Moshe: A Lower Bound on the Estimation Error for Certain Diffusion Processes, IEEE Trans. Information Theory, vol. IT-22, no. 1, Jan. 1976, pp. 45-52. 40. Snyder, ,Donald L. ; and Rhodes, Ian B. : Filtering and Control Performance Bounds with Implications on Asymptotic Separation, Automatica, vol. 8, no. 6, Nov. 1972, pp. 747-753. 41. Zakai, Moshe; and Ziv, Jacob: Lower and Upper Bounds on the Optimal Filtering Error of Certain Diffusion Processes, IEEE Trans. Informa- tion Theory, vol. IT-18, no. 3, May 1972, pp. 325-331. 42. Licht, B. W.: Approximations in Optimal Nonlinear Filtering, Rept. SRC 70-1, Case Western Reserve Univ., April 1970. BIBLIOGRAPHY Bejczy, A. K.; and Sridhar, R.: Analytical Methods for Performance Evaluation of Nonlinear Filters, J. Math. Anal. Applic., vol. 36, no. 3, Dec. 1971, pp. 477-505. Bellman, R. E.; Kagiwada, H. H.; Kalaba, R. E.; and Sridhar, R.: Invariant Imbedding and Nonlinear Filtering Theory, J. Astron. Sci., vol. 13, May-June 1966, pp. 110-115. Gustafson, D. E.; and Speyer, J. L.: Linear Minimum-Variance Filters Applied to Phase-Lock Loops, National Telecommunications Conference, Atlanta, Georgia, Nov. 26-28, 1973, Conference Record, vol. 2, IEEE Inc., 1973, pp. 26E-1 to 263-8. 17 9 A SURVEY OF DESIGN METHODS FOR FAILURE DETECTION IN DYNAMIC SYSTEMS* Alan S. Willsky Massachusetts Institute of Technology SUMMARY In this paper, we survey a number of methods for detecting abrupt changes (such as failures) in stochastic dynamical systems. We concentrate on the class of linear systems, but the basic concepts, if not the detailed analyses, carry over to other classes of systems. The methods csurveyed range from the design of specific failure-sensitive filters, to the use of statistical tests on filter innovations, to the development of jump process formulations. Trade- offs in complexity versus performance are discussed. I. INTRODUCTION With the increasing availability and decreasing cost of digital hardware and software, there has developed a esire in several disciplines for the development of sophisticated digital system design techniques that can greatly improve overall system performance. A good example of this can be found in the field of digital aircraft control (see, e.g., refs. 1-3), where a great deal of effort is being put into the design of aircraft with reduced static stability, flexible wings, etc. Such vehicles can provide improved performance in terms of drag reduction and decreased fuel consumption, but they also require sophisticated control systems to deal with problems such as active control of unstable aircraft, suppression of flutter, detection of system failures, and management of system redundancy. The demands on such a control system are beyond the capabilities of conventional aircraft control- system design techniques, and the use of digital techniques is essential. Another example can be found in the field of electrocardiography. In 'recent years, a great deal of effort has been devoted to the development of digital techniques for the automatic diagnosis of electrocardiograms (ECG) (see, e.g., ref. 4). Such systems can be for preliminary screening of large numbers of ECG'S, for monitoring patients in a hospital, etc. *This research was supported in part by NASA Ames Research Center under Grant NGL-22-009-124 and in part by NASA Langley Research Center under Grant NSG-1112. 181 In this paper, we review some of the recent work in one area of system theory that is important in both of these examples, as well as in many other system design problems. Specifically, we discuss the problem of detecting abrupt changes in dynamical systems. In the aircraft control problem, one is concerned with the detection of actuator and sensor failures, while in the ECG' analysis problem,, one wants to detect arrhythmias - sudden changes in'the rhythm of the heart. For simplicity in our discussion, we refer to all such abrupt changes as "failures," although, as in the ECG example, the abrupt change need not be a physical failure. Our aim in this survey is to provide " an overview of a number of the basic concepts in failure detection. The pro- blem of system reorganization subsequent to the detection of a failure is con- sidered in several of the references. We will point out these references in the sequel, but we will concentrate primarily on the detection problem. The design of failure detection systems involves the consideration of several issues. One is usually interested in designing a system that will respond rapidly when a failure occurs; however, in high-performance systems, one often cannot tolerate significant degradation in performance during normal system operation. These two considerations are usually in conflict, That is, a system designed to respond quickly to certain abrupt changes must necessarily be sensitive to certain high-frequency effects, and this, in turn, will tend to increase the sensitivity of the system to noise (via the occurrence of false alarms signaled by the failure detection system). The tradeoff between these design issues is best studied in the context of a specific example in which the costs of the various tradeoffs can be assessed. For example, one might be more willing to,tolerate false alarms in a highly redundant system configuration than in a system without substantial backup capabilities. Generally, one would like to design a failure detection system that takes system redundancy into account. For example, in a system containing several backup subsystems, we may be able to devise a simple detection algorithm that is easily implemented but yields only moderate false-alarm rates. On the other hand, by implementing a more complex failure detection algorithm that takes careful account of system dynamics, one may be able to reduce require- ments for costly hardware redundancy. In addition to taking hardware issues into consideration, the designer of failure detection systems should consider the issue of computational complexity. One clearly needs a scheme that has reasonable storage and time requirements. It would also be useful to have a design methodology that admits a range of implementations, allowing a tradeoff study of system com- plexity versus performance. In addition, it would be desirable to have a design that takes advantage of new computer capabilities and structures (e.g., designs amenable to modular or parallel implementations). In this paper, we survey a variety of failure detection methods and, keeping the issues mentioned above in mind, we will comment on the character- istics, advantages, disadvantages, and tradeoffs involved in the various techniques. To provide this survey with some organization and to point out 18 2 some of the key concepts in fa i lure detect ion system design, w e have defined several categories of fa i lure detect ion systems and have placed the designs we have co l lec ted in to these groups. Clearly, such a grouping can only be a rough approximation, and w e caut ion the reader against drawing too much of an inference about individual designs based on our c lassi f icat ion of them (several of t he t echn iques cou ld eas i ly f a l l i n to a number of our c lasses) . I n a d d i t i o n , f o r b r e v i t y , w e have l imi t ed ou r de t a i l ed d i scuss ions to on ly a .f& of t h e many techniques. Our choice of those techniques has been motivated by a desire to span the range of avai lable methods and by ou r f ami l i a r i t y wi th ce r t a in of these algori thms. Final ly , w e have at tempted to col lect a l l of those s tud ie s o f t he f a i lu re de t ec t ion problem of which we are aware, and we apo log ize fo r any oversights. 11. FORMULATIONS OF FAILURE DETECTION PROBLEM In t h i s p a p e r , w e are mo.stly concerned with the analysis of l i n e a r s tochast ic models i n the s tandard state space form: System dynamics: x (k + 1 ) = O(k)x(k) + B(k)u(k) + w(k) (1) Sensor equation: z (k) = H(k)x(k) + J(k)u(k) f v(k) (2) where u is a known input and w and v are zero mean, independent, white Gaussian sequences with covariances defined by where 6kj i s the Kronecker delta. We think of equations (1) to (3 ) as descr ib ing the "normal operat ion" or "no-fai lure" model of the system of i n t e r e s t . I f no fa i lures occur , the opt imal s ta te es t imator is given by the d i s c r e t e Kalman f i l t e r e q u a t i o n s ( r e f . 5 ) : G(k + l ( k ) = Q(k)G(klk) + B(k)u(k) (4) Ei(klk) = G(klk - 1 ) + K(k)y(k) (5) y(k) = z(k) - H(k)&(kIk - 1 ) - J (k)u(k) (6) where y is the zero mean, Gaussian innovation process and gain K is ca l cu la t ed from P(k + I l k ) = O(k)P(k(k)O'(k) + Q (7) V(k) = H(k)P(klk - 1)H' (k) + R ( 8 ) 183 K(k) = P(k P ( k / k ) = P(k Ik - 1)H' ( )V-l (k) ( 9 ) Ik - 1) - K(k)H(k)P(klk - 1) (10) Here P ( i 1 j ) . is the e s t ima t ion e r ro r cova r i ance o f t he estimate s(il j ) and V(k) is the covariance of y(k) . We r e f e r t o e q u a t i o n s ( 4 ) t o (10) as t h e "normal mode f i l t e r " i n t h e s e q u e l . In addi t ion to the above es t imator , one may also have a closed-loop con t ro l l a w , such as t h e l i n e a r l a w , u(k) = G(k)G(klk) We then obta in the normal opera t ion conf igura t ion depic ted in f igure 1. The problem of f a i l u r e d e t e c t i o n i s concerned with detecting abrupt changes i n a system, as modeled by equations (1) t o ( 3 ) . Such abrupt changes can arise i n a number of ways. For example, in aerospace appl icat ions, one i s of ten concerned wi th the fa i lure of cont ro l ac tua tors and sur faces . Such abrupt changes can manifest themselves as s h i f t s i n t h e c o n t r o l g a i n m a t r i x B, increased process noise , o r as a b i a s i n e q u a t i o n (1) (as might arise i f a thruster developed a l e a k ( r e f . 6 ) ) . I n a d d i t i o n , f a i l u r e s o f s e n s o r s may t ake the form of abrupt changes i n H , i n c r e a s e s i n measurement n o i s e , o r as b i a s e s i n e q u a t i o n (2) . For s impl i c i ty , we r e fe r t o ab rup t changes in equation (1) as " a c t u a t o r f a i l u r e s , " and s h i f t s i n e q u a t i o n (2) are c a l l e d "sensor failures." Again, w e p o i n t o u t t h a t , i n many a p p l i c a t i o n s , s h i f t s i n equa t ion (1) o r (2) may be used t o model changes in observed system behav- i o r t h a t h a v e n o t h i n g t o do wi th ac tua to r s o r s enso r s . The main task of a f a i l u r e d e t e c t i o n and compensation design is t o modify t h e normal mode conf igu ra t ion to inc lude the capab i l i t y of de tec t ing abrupt changes and compensating f o r them by a c t i v a t i n g backup systems, adjusting the feedback design appropriately, e tc . Conceptual ly , w e t h ink o f t he de t ec t ion - compensation system as p a r t o f the f i l t e r ing por t ion of the feedback loop . A s i l l u s t r a t e d i n f i g u r e s 2 and 3, t h e r e s u l t i n g f i l t e r d e s i g n can take one of two forms: e i t h e r we perform a comple te redes ign of the f i l t e r , rep lac ing equations ( 4 ) t o (10) wi th a f i l t e r t h a t i s s e n s i t i v e t o f a i l u r e s o r we design a system that monitors the normal system configuration and ad jus ts the sys tem accordingly. We w i l l d i s cuss examples of both structures. A s mentioned earlier, we w i l l concent ra te pr imar i ly on t h e problem of f a i l u r e d e t e c t i o n , which we cons ide r t o cons i s t o f t h ree t a sks - alarm, i s o l a t i o n , and es t imat ion . The alarm task s imply cons is t s of making a b inary dec is ion , e i ther tha t someth ing has gone wrong o r t ha t eve ry th ing is f i n e . The problem of i s o l a t i o n i s t h a t of determining the source of the fai lure (e .g . , which sensor or actuator has fa i led, what type of arrhythmia has occurred, e tc . ) . Final ly , the es t imat ion problem involves the determinat ion of t he ex ten t o f f a i lu re . For example, a sensor may become completely non- operat ional (an ' 'off" or "hard-over" fa i lure) , or i t may simply suffer degrad- a t i o n i n t h e form of a bias or increased inaccuracy, which may be modeled as 184 an increase in the sensor noise covariance. In the latter case, estimates of the bias or the increase in noise may allow continued use of the sensor, albeit in a degraded mode. Clearly, the extent to which we need to perform these various tasks depends on the application. If a human operator is available, we may only be interested in generating an alarm that tells him to perform further tests. In other systems in which backups are available, we might settle for failure isolation without estimation. On the other hand, in the absence of hardware redundancy, we may be interested in using a degraded instrument and thus would need estimation information. Intuitively, we can associate increased software system complexity with the tasks, that is, isolation requires more sophisticated data processing than an alarm, and estimation more than isolation. On the other side, as we increase failure detection capabilities, we may be able to decrease hardware redundancy. Also, in some applications, we may be able to delay isolation and estimation until after an alarm has been sounded. In such a equential structure, one increases detector complexity af t e r a failure has been detected, thereby reducing the computational burden during normal operation. Again the details of such considerations depend on the particular application. Another tradeoff involving failure detection system complexity involves its relation to detection system performance. For example, one might expect that one could achieve better alarm performance by using a priori knowledge concerning likely failure modes. That is, by looking for specific form of system behavior characteristic of certain failures, one should be able to improve detection performance. Thus it seems likely that alarm performance (as measured by the tradeoff between false alarms and missed det,ections) will be improved if we attempt simultaneous detection, isolation, and estima- tion of failures. This tradeoff of complexity versus performance is extremely important in the design of failure detection systems. In the following sections, we discuss several failure detection methods and comment on their characteristics with respect to the issues mentioned in this and the preceding section. We have not provided a general set of fail- ure models to be considered, as the various techniques are based on quite different failure models. These are described as we discuss the various methodologies. III. "FAILURE-SENSITIVE" FILTERS Our first class of failure detection concepts is aimed at overcoming the problem of an "oblivious filter." As has been noted by many authors (refs. 5 and 7-9), the optimal filter defined by equations (4) to (10) performs well if there are no modeling errors. However, it is possible for the filter estimate to diverge if there are substantial unmodeled phenomena. The problem occurs because the filter "learns the state too well," that is, the precomputed error covariance P and filter gain K become small, and the filter relies on old measurements for its estimates and is oblivious to new measurements. Thus, if an abrupt change occurs, the filter will respond 185 quite sluggishly, yielding poor performance. Consequently, one would l i k e to dev i se f i l t e r des igns tha t r ema in sens i t i ve to new d a t a so ' that .abrupt . changes w i l l b e r e f l e c t e d i n t h e f i l t e r b e h a v i o r . Two well-known t e c h n i q u e s f o r k e e p i n g t h e f i l t e r s e n s i t i v e t o new d a t a are 'the exponentially age-weighted filter s t u d i e d by Fagiri (ref. 7) and b y Tarn and Zaborszky (ref. 8) and the l imi t ed memory f i l t e r . proposed by. Jazwinski ( ref . 9) . Others, such as increas ing noise covar iances or s imply f i x i n g t h e f i l t e r g a i n , are discussed by Jazwinski ( re f . 5 ) . These techniques y i e ld on ly ind i r ec t f a i lu re in fo rma t ion ; t ha t is, i f an abrupt change occurs,, t h e s e f i l t e r s w i l l r e spond f a s t e r t han the no rma l f i l t e r , and one can base a f a i l u r e d e t c t i o n d e c i s i o n on sudden changes i n f. . . . . .. It is impor tan t to note a per formance t radeoff ev ident in ' t h i s method. As w e . i nc rease ou r sens i t i v i ty to new da ta (by e f f e c t i v e l y i n c r e a s i n g t h e bandwidth of the Kalman f i l t e r ) , o u r s y s t e m becomes more s e n s i t i v e t o s e n s o r noise, and the performance of the f i l ter under no-failure conditions degrades. I n some cases, t h i s can be rather severe, and one may n o t b e a b l e t o t o l e r a t e the degradat ion in overal l system performance under no-fai lure condi t ions. ' One might then consider a two filter system - the normal mode f i l t e r (eqs. (4)- (10)) as t h e p r i m a r y f i l t e r , w i t h t h i s t y p e o f f a i l u r e - s e n s i t i v e . f i l t e r as an auxiliary monitor, used only to detect abrupt changes. We remark t h a t t h e tradeoff between detection performance and filter behavior under normal condi t ions is a c h a r a c t e r i s t i c o f a l l f a i lu re de t ec t ion sys t ems and is ana logous to the cos ts assoc ia ted wi th false alarms and missed detections i n s tandard detect ion problems (ref . 10) . The techniques mentioned so far are r a t h e r i n d i r e c t f a i l u r e d e t e c t i o n approaches. Several methods have been developed f o r t h e d e s i g n o f f i l t e r s , s e n s i t i v e t o s p e c i f i c f a i l u r e s . One method involves the inclusion of several " f a i l u r e states" i n t h e dynamic model (eqs. (1)-(3)). Kerr ( r e f . 11 ) has considered a procedure i n which f a i l u r e modes, such as the onse t of b iases , are included as state v a r i a b l e s . If t h e estimates of these var iab les vary markedly from t h e i r nominal values, a f a i l u r e is declared . A two-confidence i n t e r v a l o v e r l a p d e c i s i o n r u l e f o r f a i l u r e d e t e c t i o n u s i n g s u c h f a i l u r e states is described and i ts performance i s ana lyzed in re ference 11.. Note t h a t t h i s approach provides fa i lure i so la t ion and es t imat ion a t the expense of increased dimensionality and some performance degradation under no-failure conditions ( inclusion of the added states ef fec t ive ly opens up the bandwidth of the Kalman f i l t e r ) . An a l t e r n a t i v e t o . t h e a d d i t i o n of f a i l u r e states t o t h e dynamic model is t h e class of de tec tor f i l t e rs deve loped by Beard ( r e f . 12 ) and Jones ( r e f . 13 ) . The i r work h a s l e d t o a systematic design procedure for detect ing a wide v a r i e t y of abrupt changes in l inear t ime-invariant systems. They consider the cont inuous- t ime, t ime-invariant , determinis t ic system model: z ( t ) = e x ( t ) 18 6 and design a f i l t e r of t h e form G( t ) &( t ) + D[z(t) - C i ( t ) ] + Bu(t) d The pr imary c r i te r ion in the choice of the ga in mat r ix D is n o t t h a t equation (13) provide a good estimate of x (as i t is wi th observers or op t ima l e s t ima to r s ) , bu t r a the r t ha t t he e f f ec t s o f certain f a i l u r e s are a c c e n t u a t e d i n t h e f i l t e r r e s i d u a l : The b a s i c i d e a is t o choose D so t h a t particular . f a i l u r e modes manifest themselves as r e s idua l s t ha t r ema in in a f i x e d d i r e c t i o n o r i n a f ixed plane. To i l l u s t r a t e t h e Beard-Jones approach, consider a simple example from reference 1 2 . Suppose w e wish t o d e t e c t a f a i l u r e o f t h e i t h a c t u a t o r (i.e., i n t h e ' a c t u a t o r d r i v e n by t h e i t h component of u ) . I f we assume t h e f a i l u r e t a k e s t h e form of a cons tan t b ias , our state equat ion becomes % ( t ) = h ( t ) + B[u(t) + veil = Ax(t) + Bu(t) + vbi , t L t o (15) where e is t h e i t h s t a n d a r d b a s i s v e c t o r , bi is t h e i t h column of B , and to iis t h e (unknown) t i m e o f f a i l u r e . Suppose w e consider the case of f u l l state measurement - l e t C = I. I n t h i s c a s e , we ob ta in a d i f f e r e n t i a l , equa t ion fo r t he r e s idua l q ( t ) = [A - D]y(t) + Vbi I f we choose D = a 1 + A, w e ob ta in Thus, as t h e e f f e c t of t h e i n i t i a l c o n d i t i o n d i e s o u t , y ( t ) m a i n t a i n s a f ixed d i r ec t ion (b i ) w i th magn i tude p ropor t iona l t o f a i lu re s i ze ( V I . Note t h a t as w e i nc rease u ( t h u s i n c r e a s i n g f i l t e r g a i n ) , t h e i n i t i a l condi t ion d ies out fas te r , bu t the magni tude of the s teady-s ta te va lue of y decreases. Thus, i f t h e r e is any noise in the sys tem, we cannot make u a r b i t r a r i l y l a r g e . In t h e i r work, Beard and Jones consider the design of such fi l ters for an ex t remely wide var ie ty of fa i lure modes, including actuator and sensor s h i f t s and s h i f t s i n A and B. The i n i t i a l d e t e r m i n i s t i c a n a l y s i s f o r a l l of these cases w a s considered by Beard ( r e f . 12), while a systematic design procedure is given by Jones ( ref . 13) for the des ign of ga in D t o a l low 187 I1111 I I detection of several failure modes. Jones' approach is quite geometric in" '. nature, and his formulation allows one to gain considerable insight into the detection problem. As pointed out in reference 13, the gain selection problem is-quite similar to the output decoupling problem and requires the introduc- tion of the important concept of "mutually detectable failure modes" to answer the question of whether one can simultaneously .distinguish between several ' '' types of failures. Thus the question of failure isolation is of central importance in the design methodology derived in reference 13. The results in references 12 and 13represent perhaps the most thorough study of the basic concepts underlying failure detection. The tradeoff between detection and filter performance is discussed in depth in reference 13 and an attempt is made in reference 12 to introduce the concept of the level of redundancy in a dynamical system. As mentioned in the example, the basic design procedure is deterministic. However, in this simple example, we can see how one can take noise into account. If the system (eqs. (11) and (12)) contains noise, we have seen that one may not wish to make the scalar u as large as possible. In fact, one could choose Q so as to minimize the mean-square estimation error in the detector filter when there is no failure. In his thesis, Jones (ref. 13) describes a procedure in which one first chooses the structure of D for . . failure detection purposes and then chooses the remaining free parameters to minimize the estimation error covariance. Although this yields a subopti- mal filter design, it may work quite well, as it did in the problem reported I 'in reference 13. The Jones-Beard design methodology is extremely useful conceptually, it can be used to detect a wide variety of failures, and it provides detailed failure isolation information. It is suboptimal as an estimator, and if this presents a serious problem, one might wish to use the detector filter as an auxiliary monitoring system. This appears to be only a minor drawback, and the major limitation of the approach is its applicability only to time- invariant systems. IV. VOTING SYSTEMS Voting techniques are often useful in systems that possess a high degree of parallel hardware redundancy. Memoryless voting methods can work quite well for the detection of "hard" or large failures, and Gilmore and McKern (ref. 14), Pejsa (ref. 15), and Ephgrave (ref. 16) discuss the successful application of voting techniques to the detection of hard gyro failures in inertial navigation systems. In standard voting schemes, one has (at least) three identical instru- ments. Simple logic is then developed to detect failures and eliminate faulty instruments, for example, if one of the three redundant signals differs markedly from the other two, the differing signal is eliminated. Recently, Broen (ref. 17) has developed a class of voter estimators that 188 possesses advantages relative to standard voting techniques. Consider the dynamical system with a triply redundant set of sensors: groen develops a set of recursive filter equations for computing the estimate x(k) that minimizes where Rj is the covariance of the measurement noise vj and yj is the innovations sequence Y j (i> = y. J (i) - Hj@i-ki(k) Here w is a function of yl(i), y2(i), and y3(i) which is large if yj(i) is close to the other two ym(i) terms and is small if yj(i) deviates greatly from the other two. In this way, one obtains a "soft" voting pro- cedure in which faulty sensors are smoothly removed from conside.ration. This greatly alleviates the cost of false alarms, but the price is the on-line computation of the filter gain (which is a function of the w i). Note that in equation (19), Broen appears to allow the yi to be physiJally different sensors (different Hi terms), but the analysis of his paper makes it clear that he requires identical sensors (i.e., H1 = H2 = H3). ji Voting schemes are generally relatively easy to implement and usually provide fast detection of hard failures, but they are applicable only in systems that possess a high level of parallel redundancy. They do not generally take advantage of redundant informat'ion provided by unlike sensors, and thus cannot detect failures in single or even doubly-redundant sensors. In addition, voting techniques can have difficulties in detecting "soft" failures (such as a small bias shift). V. MULTIPLE HYPOTHESIS FILTER DETECTORS A rather large class of adaptive estimation and failure detection schemes involves the use of a "bank" of linear filters based on different hypotheses. concerning the underlying system behavior. In the work of Athans and Willner 189 (ref. 18) and Lainiotis (ref. 19), several different sets of system matrices are hypothesized. Filters for each model are constructed, and the innovations from the various filters are used to compute the conditional probability that each system model is the correct one. In this manner, one can do simultaneous system identification and state estimation. In addition, an abrupt change in the probabilities can be used to detect changes in true system behavior. This technique has been investigated in the context of the adaptive control of the F-8C digital fly-by-wire aircraft by Athans et aZ. (ref. 2 0 ) and also has been applied to the problem of classifying rhythms and detecting rhythm shifts in electrocardiograms. Extremely good results in the latter case are. reported by Gustafson et aZ. (ref. 21). Techniques involving multiple hypotheses have also been used to design failure detection systems. Montgomery et aZ. (refs. 22 and 2 3 ) have used such a technique for digital flight-control systems and have studied its robustness in the presence of nonlinearities via simulations. Recently, a technique involving a bank of observers has been devised (ref. 2 4 ) , and a successful application to a hydrofoil sensor failure problem is reported by Clark et aZ. (ref. 2 4 ) . Also, Willsky et aZ. (refs. 25 and 2 6 ) have applied the methodol- ogy devised by Buxbaum and Haddad (ref. 2 7 ) to study failure detection for an inertial navigation problem. We will briefly describe this technique to illustrate some of the concepts underlying the bank-of-filters approach. We also refer the reader to Wernersson (ref. 28) for a technique similar t o that discussed in reference 2 6 . Consider the system x(k + 1) = @(k)x(k) + w(k) (22) We are interested in detecting sudden shifts in certain of the components of x (e.g., bias states). We model these shifts by choosing the distribu- tion of w appropriately. Let {fl, ..., fr) be the set of hypothesized failure directions. We then assume that w has a high probability of being the usual process noike and a small probability of including a burst of noise in each of the failure directions. Thus the density for w(k) is 2. r pi = 1, p0 >> pi i = 1, ..., r i- 0 Here N(m,P) is a normal density with mean m and covariance P. If we hypothesize such a density at each point in time and if we assume that x(0) is normally distributed, we have the following expression for the conditional density of x(k), given ~(l),. ..,z(k): 190 I Here' i'= (io,...,ik-l) and the density has the following interpretation. Let I-= (joy.. yjk-i) be a random k-tuple where js = i if there is a shift-in the fi direction at time s (i = 0 is used to denote no shift). Then pi = Pr[j = ilz(1) ,...,z(k)] "- and .ni and Pi are the mean and covariance of the Kalman fflter designed assuming j = i-(i. e. , assuming w( s ) has covariance Q + ui f f ' ) . The Pi terms can be computed sequentially as a function of the various filter innovations. We refer the reader to references 25 to 27 for the details of the calculations. - - - s is is Note that the implementation of equation (26) requires an exponentially growing bank of filters (there are (r+l)k terms in eq. (26)). To avoid this problem, a number of approximation techniques have been proposed (refs. 25-27). The one used in reference 26 involves hypothesizing shifts only once every N steps. At the end of each N step period, we "fuse" the (rtl) densities into a single density and begin the procedure again. In this way, we fmple- ment only (r+l) filters at any time. Note that the techniques devised in references 18, 19, and 22 do not involve growing banks of filters (as the number of hypothesized models do not grow in time). However, it is possible for all filters in the bank to become oblivious, and thus shifts between the hypotheses may go undetected (see refs. 2 1 and 26 for examples). The technique of periodic fusing of the densities and initiation of new banks effectively avoids this problem (as would designing the original bank using age-weighted filtering techniques). The technique described above was applied to the problem of detecting gyro and accelerometer bias shifts in a time-varying inertial calibration and alignment system. The results of these tests are extremely impressive. This is not surprising as the multiple-hypothesis method computes precisely the quantities of interest - the probabilities of all types of failures under consideration. The cost associated with such a high level of performance is an extremely complex failure detection system. Note, however, that the paral- lel structure of the system allows one to consider highly efficient parallel processing computer implementations. In addition, the use of reduced-order filters for the various failure hypotheses may increase the practicality of such a scheme, or one might consider the use of a simpler detection-only system to detect failures, with a switch to a multiple hypothesis procedure for failure isolation and estimation after failure has been detected. However, even if such a failure detection scheme cannot be implemented in a particular application, it provides a useful benchmark for comparison 191 with simpler techniques. In addition, by studying the simulation of a multiple hypothesis method, one can gain useful insight into the dynamics of failure propagation and detection (see the discussion in ref. 26). McGarty (ref. 29) has developed a method for rejecting bad measurements which bears some similarity to the approach just discussed. Each measurement has a binary random variable g(k) associated with it. If g(k) = 1, the measurement is "good" (i.e., the measurement contains the signal of interest), while g(k) = 0 denotes a bad data point (measurement is pure noise). McGarty devises a maximum likelihood approach for estimating the values of the exponentially growing set of possibilities [g(i) = 1 or 0, i = 1, ..., k]. He also allows these variables to have a sequential correlation (i.e., knowing that the present measurement is good or bad says something about the next observation). A computationally feasible approximation method is devised and simulation results are described (see ref. 29 for details). Recently, Athans e t aZ. (ref. 30) also considered the problem of designing an estimator that can detect and remove bad or false measurements. Their approach is Bayesian in nature, that is, an estimate is generated of the a posteriori probability that a given measurement is false. The method of calculation of these pseudo-probabilities is quite similar to that used in the other multiple hypothesis methods (see refs. 18, 19, 22, 23, and 27).. (The reader is referred to ref. 30 for details of the analysis and for is- cussion of some successful simulation results.) VI. JUMP PROCESS FORMULATIONS The problem of detecting abrupt changes in dynamical systems suggests the use of jump process techniques in devising system design methodologies (see refs. 31-33 for general results on jump processes). One models potential failures as jumps, characterized by a priori distributions that reflect initial information concerning failure rates. The size of the possible failures are usually taken to be known. One could, however, model failure magnitude as a random variable. This leads to a compound jump process formu- lation that greatly complicates the desired analysis. In any event, taking such a jump process formulation, one can devise failure-sensitive control laws and methods for computing the conditional probability of failure. Control problems of this type have received a great deal of attention in the litera- ture. Sworder and Robinson (refs. 34-38) and Ratner and Luenberger (ref. 39) have considered the design of control laws which consider the possibility of sudden shifts in system matrices. The results they obtained are for the full- state feedback problem with no system randomness other than the jumping of the system matrices among a finite set of possible matrices. Davis (ref. 40) has used nonlinear estimation techniques to solve a fault detection problem. His formulation is as follows: consider the scalar sto- chastic equations, 192 where . w and v are independent Brownian motion processes and where t < T t Z T and T is a random va r i ab le . Here w e i n t e r p r e t a. as the unfailed dynamics, and a'l r e p r e s e n t s t h e f a i l u r e mode. Davis de r ives t he op t ima l , i n f in i t e - dimensional equat ions for the computat ion of the condi t ional mean of x and the cond i t iona l p robab i l i t y i ( t l t ) = P r [ t z T ~ Y ( S ) , o 5 s I t ] (32) An implementable approximation is desc r ibed in r e fe rence 40, but evaluation of i ts performance has not yet been made. Note that Davis ' method l eads to an estimate of x t h a t is suboptimal under no-fai lure condi t ions. Chien (ref , 41) has devised a jump process fo rmula t ion tha t avo ids th i s d i f f i cu l ty fo r t he problem of detecting a jump o r a ramp i n a gyro bias . H e considers the dynamical model: where w is a white noise process. Three hypotheses are conjec tured for the form of the gyro output: Normal mode Ho: z ( t ) = x ( t ) + V(t) , V t (34) Ramp mode Hz: z ( t ) = x ( t ) + n ( t - T)S(t) + v ( t ) , t > T '(36) where n and m are unknown cons tan ts , v is whi te no ise , T is the time of f a i l u r e , and ( ( t ) is as i n equation (31). 193 Chien's approach i s as follows: design a f i l t e r based on Ho (which will thus y i e ld the op t ima l estimate f o r t < T, assuming no false alarms occur) and determine the steady-state ef fec t o f the degrada t ions HI -and Hg on t h e f i l t e r r e s i d u a l s . I f one then hypothesizes a f a i l u r e rate q , tha t is, P(T > t ) = e -qt (37) and i f one f u r t h e r assumes a nominal s i z e f o r t h e b i a s m y one can then com- . , pu te an approx ima te s tochas t i c d i f f e ren t i a l equa t ion fo r Pr(H1 I z ( s ) , s 1 t) , i n which t h e i n p u t t o t h i s e q u a t i o n is t h e r e s i d u a l y of the Ho f i l t e r . The d e t a i l s o f t he ana lys i s are desc r ibed in r e fe rence 41. For h i s problem, Chien is ab le to demonst ra te tha t h i s de tec t ion proce- dure - based on the assumption of a nomina l va lue fo r t he b i a s f a i lu re m - , has the capab i l i t y o f de t ec t ing b i a ses l a rge r t han m and also can be used t o d e t e c t ramps (mode Hz). Of course, the delay times u n t i l d e t e c t i o n i n these cases are g r e a t e r t h a n i f one implemented a f i l t e r based on the proper b i a s s i z e o r i f one were look ing fo r a ramp ( ind ica t ing the po ten t i a l u se fu l - ness of es t imat ing the fai lure magnitude) . The major advantages of Chien's approach are t h e s i m p l i c i t y of the detector (implementation of a scalar s tochas t ic equat ion) and t h e f a c t t h a t one obtains an estimate o f p rec i se ly t h e q u a n t i t y o f i n t e r e s t - t he cond i t iona l p robab i l i t y of f a i l u r e . The s i m p l i c i t y of t h e scheme may, i n f a c t , make it a g r e a t d e a l more r o b u s t i n the face of system modeling errors (such as the use of an extremely simplified gyro e r r o r model) than more sophis t icated approaches. Also, t h i s approach l e a d s t o no degrada t ion in pe r fo rmance p r io r t o de t ec t ion o f t he f a i lu re . In addi t ion , the use of a p robab i l i s t i c desc r ip t ion o f t he time of f a i l u r e a l l o w s one t o avoid the problem of t h e o b l i v i o u s f i l t e r - t h a t is, t h e f a c t t h a t a fa i lure can occur a t any time has been incorporated in the design, which the re fo re w i l l r ema in sens i t i ve to new data. The drawbacks of t h e scheme are the use of a f i x e d b i a s s i z e and t h e u s e o f t h e s t e a d y - s t a t e e f f e c t o f t h e f a i l u r e on t h e f i l t e r r e s i d u a l . The f i r s t of t hese may not be too much of a problem (as Chien pointed out), but the second may c a u s e d i f f i c u l t i e s . S p e c i f i c a l l y , t h i s limits the approach to t ime-invariant systems and f i l ters . In addi t ion, as t h e t r a n s i e n t e f f e c t o f the fa i lure has been ignored , i t may be d i f f i c u l t t o make quick de tec t ions of cer ta in changes ( i .e . , we may have to wait u n t i l t h e t r a n s i e n t d i e s o u t ) . In the nex t s ec t ion , we discuss an approach ( the GLR method) tha t has several concepts in.common with Chien's approach and that allows one to overcome these two drawbacks ( a t t h e c o s t of added computational complexity, of course). Jump p rocess fo rmula t ions appea r t o be qu i t e na tu ra l fo r f a i lu re de t ec - t i o n problems. One usua l ly makes approximat ions in the ana lys i s to obta in implementable solut ions. These s implif icat ions impose some l i m i t a t i o n s on the capab i l i t i e s o f t he des igns , bu t p re sen t ly the re i s no systematic analyt- ical procedure for eva lua t ing these l imi ta t ions or for s tudying t radeoffs between design complexity and system performance. 194 I1 I1 1111 11111 V I I . IMJOVATIONS-BASED DETECTION SYSTEMS Chien 's fa i lure detect ion technique can also be placed in the class of f a i l u r e d e t e c t i o n methods that involve monitor ing the innovat ions of a f i l t e r based on the hypothesis of normal system operation. In such a conf igura t ion , the overa l l sys tem uses the normal f i l t e r un t i l the innovat ion moni tor ing system detects some form of aberrant behavior . The f ac t t ha t t he mon i to r ing system can be attached to a f i l ter-control ler feedback system is p a r t i c u l a r l y appeal ing s ince overal l system behavior i s n o t d i s t u r b e d u n t i l a f t e r t h e monitor s ignals a f a i l u r e and since the monitoring system can be designed to be added to an ex is t ing sys tem. Mehra and Peschon ( r e f . 4 2 ) suggested a number of poss ib l e statist ical tests to be performed on the innovations. One of these is a chi-squared test t h a t w a s appl ied by Willsky et aZ. ( r e f s . 25 and 2 6 ) . L e t y(k) be the p-d imens iona l innovat ions for the f i l t e r def ined by equations ( 4 ) t o (10). I f the system is operat ing normally, the innovat ions is zero mean and white with known covar iance V(k) . In th i s case , the quant i ty is a chi-squared random var iab le wi th Np degrees of freedom (refs. 2 5 , 2 6 , and 4 2 ) . I f a system abnormality occurs, the statistics of y change, and one can consider a d e t e c t i o n r u l e of t h e form Fa i lu re R(k) > E < No f a i l u r e (39) With t h e a i d of chi-squared tables, one can compute the p robab i l i t y PF of f a l s e alarm a s a funct ion of the innovat ions window length N and the deci- s ion th reshold E . The p r o b a b i l i t y PD of correct detection depends on t h e p a r t i c u l a r f a i l u r e mode ( s e e r e f . 26 and the d i scuss ion of t h e GLR approach to fo l low) . We n o t e t h a t , f o r a g iven f a i lu re mode, as N i n c r e a s e s t h e p robab i l i t y of c o r r e c t d e t e c t i o n may decrease - t h a t is , by averaging a l a r g e r number of r e s i d u a l s we smooth ou t t he e f f ec t o f a f a i l u r e on y , and t h e d e t e c t o r may become somewhat ob l iv ious (or a t the very bes t respond qui te s lowly) to new da ta . On the o ther hand , too small a value of N may y i e l d an unacceptably high value of PF. The implementation of the chi-squared test (eqs. (38) and ( 3 9 ) ) is q u i t e s imple, but , as one might expect , one pays for this s implici ty with rather severe l i m i t a t i o n s on performance. A s desc r ibed in r e fe rences 25 and 2 6 , t h i s method w a s a p p l i e d t o t h e same i n e r t i a l c a l i b r a t i o n and alignment problem t o which t h e Buxbaum-Hadded mult iple hypothesis approach (refs . 2 5 - 2 7 ) , described i n s e c t i o n V, w a s appl ied . The performance of the chi-squared test w a s mixed. 195 The method is basically.an alarm method, that is, the system (eqs. (38) and (39)) makes no attempt to isolate failures, and one finds that those failure modes that have dramatic effects on y are detectable by this method. However, more subtle failures are more difficult to detect with this simple. scheme. Comparing the performance of the multiple hypothesis and chi-squared systems, we see that in some cases we can obtain superior alarm capabilities. , if we simultaneously attempt to do failure isolation and estimation. One can obtain some failure isolation information by considering the components of y separately (this may be especially useful for sensor failures), and we refer the reader to references 25 and 26 for a detailed discussion of this and other aspects of the chi-squared method. Another innovation-based approach, developed by Merrill (ref. 4 3 ) , is motivated by a desire to suppress bad sensor data. Merrill devises a modifi- cation of the least-squares criterion to suppress extremely large residuals (which are given a very large weighting in the usual least-squares framework), and he applies his methodology to a p wer system application. A final technique in this category has been studied by several research- ers (refs. 6 and 44-48); we will describe the most. general formulation of the approach, developed in references 44 and 45. This technique, which we call the generalized likelihood ratio (GLR) approach, was in part motivated by the shortcomings of the simpler chi-squared procedure. The GLR approach, which can be applied to a wide range of actuator and sensor failures, attempts to isolate different failures by using knowledge of the different effects such failures have on the system innovations. The method provides an optimum decision rule for failure detection and provides useful failure identifica- tion information for use in system reorganization subsequent to the detection of a failure. In addition, one can devise a number of simplifications of the technique and can study analytically the tradeoff between GLR complexity and GLR performance . Consider the basic dynamical model (eqs. (1) to ( 3 ) ) . The following are four possible modifications of these equations that incorporate certain sudden system changes (see refs. 21, 44, and 45 for the physical motivation for these and other failure modes of the same general type): Dynamic jump : x(k + 1) = cP(k)x(k) + B(k)u(k) + w(k) + ~ 6 ~ + ~ , . (40) Here v is an unknown n-vector, 8 is the unknown time of failure, and 6 is the Kronecker delta. Such a model can be used to model sudden shifts in bias states (as in the inertial problem studied in refs. 25 and 26). ij Dynamic step : x(k + 1) =: @(k)x(k) + B(k)u(k) + w(k) + Here Q is the unit step i j i96 This model can be used to model certain a c t u a t o r f a i l u r e s (compare t o t h e Beard-Jones example i n s e c t i o n 111; see eq. (15)). Sensor jump : . . z(k) = k ( k ) + Ju(k) + v(k) + v6 k, 8 We can .use t h i s t o model bad d a t a p o i n t s . ' Sensor step: z(k) = Hx(k) + Ju(k) + v(k) + vu k, e Sudden changes i n s e n s o r b i a s e s f i t i n t o t h i s model. (43) (44) By the l inear i ty of the sys tem (eqs . (1) to ( 3 ) ) and t h e f i l t e r ( e q s . ( 4 ) t o ( l o ) ) , one can determine the effect of each f a i lu re mode on the innova- t i ons . The general form is - y(k) = G(k;e)v + y(k) (45) where ;(k) is t h e f i l t e r i n n o v a t i o n s i f no f a i l u r e o c c u r s , and the ma t r ix G can be precomputed (see r e f s . 45 and 48). This matrix, which is d i f f e r e n t signature matrix and provides us with an expl ic i t descr ipt ion of how var ious fai lures propagate through the system and f i l t e r . 1 for each of the four cases (eqs. (40) to ( 4 4 ) ) , is c a l l e d t h e failure The full-blown GLR method involves the following: w e assume we are looking for one of the four c lasses of f a i l u r e s and have computed the appro- p r i a t e s igna tu re ma t r ix . Given t h e r e s i d u a l s , we compute t h e maximum l i k e l i - hood estimates of v and 8 and, assuming t h a t t h e s e estimates are c o r r e c t , we compute t h e l o g - l i k e l i h o o d r a t i o f o r f a i l u r e v e r s u s no f a i l u r e ( s e e -.F r e f . 1 0 f o r a general discussion of GLR methods). The implementation of the f u l l GLR r equ i r e s a l i n e a r l y growing bank of matched f i l t e r s , computing t h e b e s t estimates of v assuming a par t icu lar va lue of 8tz11, ..., kl. A number of remarks can be made concerning the GLR system. We n o t e t h a t , as wi th o the r methods such as those of Buxbaum-Haddad o r Chien, the inclusion of var iab le 8 t o i nd ica t e ou r unce r t a in ty as t o t h e time of f a i lu re keeps the de t ec t ion sys t em sens i t i ve to new data . However, it is the e s t ima t ion o f 8 t h a t causes the growing complexity problem. On the o ther hand , even i f the f u l l GLR is not implementable, it can serve as a benchmark f o r o t h e r schemes and can, i n f a c t , b e u s e d as a s t a r t i n g p o i n t f o r t h e d e s i g n of simpler systems. One s i m p l i f i c a t i o n t h a t e l i m i n a t e s t h e growing complexity is t h e r e s t r i c t i o n o f t h e estimate of 8 t o a window 197 .. " ... -. " "_ k - N ' B Z k - M ( 4 5 ) where the lower bound is included to limit complexity and the upper bound is set by failure observability and false-alarm considerations'. Successful ' .'.' simulation runs with N = M (i.e., when we do not optimize 8 at all and have only one matched filter for <) are reported in reference 45. We remark. only that the price one pays for "windowing" the estimate of 8 is a reduction in the accuracy of the estimate of v. For example, for N =. M, we often are able to detect failures extremely quickly, but if 6 = k - N is not the ' : correct time of failure, the estimate of v may be severely degraded ' . ' (e.g., our estimate of the slope f, a ramp changes as we change our estimate of the time at which it started). We note that the estimation of 8 is similar to time-of-arrival estimation problems that arise in various applica- tions (see ref. 49 for a general discussion of several techniques). . . Note that even if the physical system and filter are time-invariant, the GLR monitoring system is time-varying, as the failure signature G includes transient effects. In some cases, one may be able to neglect these and use a simpler steady-state signature. This is quite similar to Chien's use (ref. 41) of the steady-state effect of the failure on the residuals, and the criticisms of that approach (given in section VI) apply here as well. One can also simplify the implementation by either partially or com- pletely specifying the failure magnitude v . Constrained GLR (CGLR) is based on the assumption that v = afi (46') where a is an unknown scalar and fi is one of r possible failure directions. This technique is described in reference 45. If we completely specify v: v = v 0 (4 7) we obtain the simplified GLR (SGLR) algorithm that is extremely simple to implement, as we have completely eliminated the need for the matched filters to estimate v. The use of specified failure sizes is similar to that proposed by Chien (ref. 41), although in SGLR one can use the time-varying failure signature, which should aid in failure detection. As initial results for the detection of electrocardiogram arrhythmias indicate (ref. 21), the estimation of v is not nearly as important for detection as matching the failure signatures. Also, by the use of several values of vo (i.e., by implementing several parallel SGLRs), one can achieve a high level of failure isolation without a great deal of additional software complexity. In addition, one could consider a "dual-mode" procedure in which SGLR is used for alarm and isolation, with full GLR used only afterward t o estimate the magnitude of the failure. 198 The var ious s impl i f ica t ions of GLR, as w e l l as f u l l GLR, are amenable t o c e r t a i n a n a l y s i s , s u c h as t h e c a l c u l a t i o n of PF, PD, and ( a t least f o r SGLR) the expected t i m e de l ay in de t ec t ion . By performing such analyses, one can study i n d e t a i l t h e t r a d e o f f between complexity and performance. A methodology f o r such comparisons is presently being developed and is being applied t o an a i r c r a f t f a i l u r e d e t e c t i o n problem. I n i t i a l r e s u l t s are reported by Chow et aZ. ( r e f . 4 8 ) , and a descr ip t ion of a d e t a i l e d methodology w i l l be repor ted in the near fu ture ( see re f . 50) . In addi t ion to the ca lcu- l a t i o n of PF and PD, the comparison methodology reported in reference 50 includes the computat ion of cross-detect ion probabi l i t ies - t h a t is, t h e p robab i l i t y o f de t ec t ing a f a i l u r e of type A when a f a i l u r e of type B has occurred. Such information can be useful . i n d e s i g n i n g f a i l u r e i s o l a t i o n pro- cedures and also i n determining whether fa i lure detector A can be successful ly used as 'an alarm f o r f a i l u r e s of type B. This can lead to subs tan t ia l s i m p l i f i c a t i o n s i n a f a i l u r e alarm system. (See refs. 21, 45, and 48 f o r successful s imulat ions of the.GLR method.) P resen t ly , t he GLR method is be ing ex tended to o ther fa i lure modes: hard-over actuator, increased process noise, hard-over sensor, and increased sensor no ise . Hard-over actuator fa i lure: x (k + 1 ) = @(k)x(k) + [B + I u W + w(k) ( 4 8 ) With t h i s model, we can take into account complete (or "off") fa i lures of ce r t a in ac tua to r s . Fo r example, an o f f f a i lu re o f t he i t h ac tua to r can be modeled by choosing M a l l ze ro excep t fo r t he i t h column, which i s taken to be the nega t ive o f t he i t h column of B. The GLR d e t e c t o r f o r equa- t i o n ( 4 8 ) is presently under development (refs. 48 and 501, and w e n o t e t h a t t h i s model is more d i f f i c u l t t h a n t h e o t h e r s as the e f f ec t o f t he f a i lu re i s modulated by the input va lues u (k) . Increased process noise fa i lure: Here 5 is addi t ional white process noise . Hard-over s enso r f a i lu re : z(k) = Hx(k) + Ju(k) + v(k) + [Mx(k) + Su(k)]a k,B (50) Here t h e f a i l u r e s are modulated by u and x , and a f a i l u r e o f t h e i t h sensor is modeled by choos ing t he i t h rows of M and S appropr ia te ly . Increased sensor noise fa i lure : 193 The analysis of these failure modes and 50), and it is anticipated that is presently being performed (refs. 48 SGLR algorithms will also be developed. . . 1n.addition to these failure modes, one can develop additional models along these lines for particular applications. In particular, we have devel- oped several additional models similar to those described by equations (40) to (44) for our work on the detection and classification of arrhythmias in electrocardiograms. The results reported in reference 21 are rather striking; as in all the tests performed we observed no false alarms, detected all rhythm changes immediately (with no incorrect estimates of e ) , and classified all rhythm changes correctly. These tests used the full GLR approach and have provided useful insight into the characteristics of the method. For example, the use of maximum likelihood estimates of v and 8 precludes the use of a priori statistics on these variables. In the ECG problem, one is quite : interested in accurate estimates. of v, and one also can come up with reason- able a priori statistics on v based on physical arguments. Thus, it may pay to incorporate such a priori statistics into the GLR system, and this can be done rather easily by proper initialization of the matched filters estimating v . On the other hand, for the ECG problem, one does not want to look for abrupt changes at one point in the record more than at another, and thus it does not make sense to include a priori statistics on 8. In fact, one can argue that inclusion of a priori failure information tends to discount the observed data to avoid false alarms (unless failures are extremely likely), and one should probably avoid the inclusion of such information unless one is especially worried about false alarms. However, if one wishes to use such data, one can use the interpretation of the likelihood ratios as ratios of conditional probabilities of failure times to determine the appropriate modi- fication of GLR (ref. 45). Finally, note that the GLR system provides extremely useful information for system compensation subsequent to the detection of a failure. For example, one can use the GLR-produced estimates of v and 8 to determine an optimal update procedure for the filter estimate and covariance (ref. 45). Once this update has been performed, the GLR system can be used to detect further failures, thus allowing the detection of multiple events. (See refs. 45 and 48 for further discussions of the use of GLR-produced information in the design of failure compensation systems.) VIII. CONCLUSIONS We have discussed a number of the issues involved in the design of fail- ure detection systems. We have also reviewed a variety of existing failure detection methods and have discussed their characteristics and designs trade- offs. The failure detection problem is an extremely complex one, and the choice of an appropriate design depends heavily on the particular application. Issues suc.h as available computational facilities and level of hardware redundancy enter in a crucial way in the design decision. For example, as we have mentioned, the use of a sophisticated failure detection-compensation 200 system may allow one to reduce the level of hardware redundancy without much loss in overall system reliability. The development'of failure detection methods is still a relatively,new subject. At this time, most of the work has been at a theoretical level with .only a few real applications of techniques (refs. 6, 14-17, 21, and 23). Much work remains to be done in the development of implementable systems complete with a variety of design tradeoffs. Work is needed in the develop- ment of efficient techniques for failure compensation and system reorganiza- tion. In addition, there is a great need for the analysis of the robustness of various failure detection systems in the presence of variations in system parameters and in the presence of modeling errors and system nonlinearities. For example, it-is conjectured that SGLR is less sensitive to parameter errors than the more complex full GLR. However, presently there are no, analytlcal results or simulations to support this conjecture. These and other issues, such as the incorporation.of fault-tolerant computer concepts into an overall reliable design methodology (see,ref. 51) await,investigation in the future. Acknowledgments The author is in the debt of many colleagues for their comments during numerous discussions on the subject of failure detection. In particular, special thanks must go to Dr. R. C. Montgomery, Dr. A. K. Caglayan, Dr. D. B. Price, and Mr. J. L. Elliott of Langley Research Center, Mr. B F. Doolin of Ames Research Center, Dr. H. L. Jones, Dr. B. S. Crawford, Dr. T. H. Kerr, Mr. J. H. Fagan, and Mr. W. O'Halloran of The Analytic Sciences Corporation, Dr. J. J. Deyst, Dr. D. E. Gustafson, Mr. J. C. Deckert, Dr. M. Desai, Dr. J. L. Speyer, and Mr. J.-Y. Wang of The Charles Stark Draper Laboratory, Dr. K. P. Dunn of the M.I.T. Lincoln Laboratory, and Prof. M. Athans, Dr. S. B. Gershwin, Mr. E. Chow, and Mr. R. Bueno of M.I.T. 201 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. I . . - REFERENCES ,.,' i / i ! i ' Doolin, B. F. : Reliability Issues for Future Aircraft , MIT-NASA/Ames ! Workshop on System Reliability Issues for Future Aircraft, M.I.T., I . . I Cambridge, Mass,, ,Aug., 18-20, 1975. , . Taylor', L.: Active Control Aircraft Problems, MIT-NASA/Ames Workshop .I. on System Reliability Issues for 'Future Aircraft, M.I.T., Cambridge, ; Mass., Aug'. 18-20,, 1975. i I Meyer, George; and Cicolani, Luigi: A Formal Structure for Advanced . I Automatic Flight-Controi Systems, NASA TN D-7940, 1975. Zyweitz, Chr. ; and Schneider, B.. , eds. : Computer Application on ECG and1 VCG Analysis, North-Holland Pub. Co., Amsterdam, 1973. Jazwinski, Andrew H.: Stochastic Processes and Filtering Theory, ' Academic Press, New York, 1970. Deyst, J. J.; and Deckert, ,J. C.: RCS Jet Failure Identification for the Space Shuttle, Proc..IFAC 75, Cambridge, Mass., Aug. 1975. , - Fagin, S. L.: Recursive Linear Regression Theory, Optimal Filter Theory, and Error Analyses of Optimal Systems, IEEE International Convention Record, March 1964, pp. 216-240. Tarn, Tzyh J.; and Zaborszky, John: A Practical Nondiverging Filter, AIAA J., vol. 8, no. 6, June 1970, pp.'l127-1133. Jazwinski, Andrew H.: Limited Memory Optimal Filtering, IEEE Trans. Automatic Control, vol. 13, no. 5, Oct. 1968, pp. 558-563. Van Trees, Harry L.: Detection, Estimation, and Modulation Theory, Part I: Detection, Estimation, and Linear Modulation Theory, John Wiley and Sons, Inc., New York, 1968. Kerr, T. H.: A Two Ellipsoid Overlap Test for Real Time Failure Detection and Isolation by Confidence Regions, Pittsburgh Conf. on Modeling and Simulation, April 24-26, 1974. Beard, R. V.: Failure Accommodation in Linear Systems Through Self- Reorganization, Rept. MVT-71-1, Ph.D. Thesis, M.I.T., Cambridge, Mass., Feb. 1971. Jones, H. L.: Failure Detection in Linear Systems, Ph.D. Thesis, M.I.T., Cambridge, Mass., Sept. 1973. Gilmore, J. P.; and McKern, R. A.: A Redundant Strapdown Inertial System Mechanization - SIRU, AIAA Paper 70-1027, Aug. 1970. 202 15. Pejsa, a. J.: Optimum Orientation and Accuracy of Redundant Sensor Arrays, AIAA 9th Aerospace Sciences Meeting, New York, Jan. 25-27, 1971. , . . . . , e. 8 ' , , 16. Ephgrave, J. T.: Redundant Adaptive,Strapdown Navigation Systems; Aerospace Rept. TOR-0066(5306)-10, The Aerospace Corp.,,'Oct.';31, 1969. 17. Broen, R. B. : A Nonlinear Voter-Estimator for Redundant Systems'; Proc. . ' 1974 IEEE Conference on Decision and Control, 5th; and" Sympos&w on Adaptive Processes , 13th, Phoenix, Arizona, ' Nov. 20-22', 1974;' . . . r pp. 743-748. . . 18. Athans , M. ; . and Willner', D .: A Practical Scheme'' .for. Adaptive "Aircraft Flight Control Systems, Parameter Estimation Techniques and Applica- , . ' ' tions in Aircraft Flight Testing, NASA.TN D-7647, 1974.' ' . . . . I . > . . I : . , . I j . 19. Lainiotis, D. G.: Joint Detection, Estimation, and System Identification, Information and Control, vol. 19 , no: 1 , Aug. 1971 , 'pp. 75-92.. - r . ' 20. Athans, M.; Dunn, K.-P.; Greene, C. S.; Lee, W. H.; Sandell, N. R., Jr.; : I . '. Segall, I.; and Willsky, A. S.: The Stochastic: Control of'the'F-8C ' ' Aircraft Using the Multiple Model Adaptive Control (MMAC) Method, IEEE Conference on Decision and Control, Houston, Texas, Dec. 10-21, 1975. Proceedings, pp. 217-228. 21. Gustafson, D. E.; Willsky, A. S.; and Wang, J.-Y.: Final Report': ' Cardiac Arrhythmia Detection and Classification Through Si\gnal Analy- sis; Rept. R-920, The Charles Stark Draper Laboratory, Cambridge,. Mass., July 1975. 22. Montgomery, R. C.; and Caglayan, A. K.: A Self-Reorganizing Digital Flight Control System for Aircraft, AIAA Paper 74-21, 1974. 23. Montgomery, R. C. ; and Price, D. B. : Management of Analytical Redundancy in Digital Flight Control Systems for Aircraft, AIAA Paper 74-887, Aug. 1974. 24. Clark, R. N. ; Fosth, D. C.; and Walton, V. M. : Detecting Tnstrumerit ' Malfunctions in Control Systems, IEEE Trans. Aeroseace and Electronic Systems , vol. AES-11, no. 4, July 1975, .pp.' 465-473. ' . . . 25. Willsky, A. S.; Deyst, J. J.'; and Crawford', B.' S.:' Adaptive Filtering and Self-Test Methods for Failure Detection and Compensation,' Proc. . of the 1974 JACC, Austin, Texas, June 19-21, 1974. . . 26. Willsky, 8.' S:.; Deyst', J. J. , Jr. ;' and Crawford, ' B . ' S . : T W Sel,f-Tlst . , Methods Applied to an Inertial System 'Problem, J. 'Spacecraft ,-Rockets, vol. 12, no. 7, July 1975, pp. 434-437. ' 3 . . . . . . , , . . . . . > . " ' i 27. Buxbaum, Paul J.; and Haddad, Richard A.: Recursive.Optima1 Estimation i , /i for a Class of Nongaussian Processes, in Symposium on Computer Process- iqg in Communications, New York, April 8-10, 1969. Proceedings. l r Polytechnic Press of the Polytech Inst. of Brooklyn, 1970, pp. 375-399. , . . . 28. Wernersson, &e:. On Bayesian,Estimators for Discrete Time Linear Systems with Markovian Parameters, Sixth Symp. on Nonlinear Estimation and Its Applications, San Diego, Calif., Sept. 15-17, 975. 29. McGarty, Terrence P. : State Estimation with Faulty Measurements: An Application of Bayesian Outlier Rejection, Proc. Fifth Symposium on Nonlinear Estimation and Its Applications, San Diego, Calif., Sept. 23- 25, 1974, pp. 179-189. , !- 30. Athans, M.; Whiting, R. H.; and Gruber, M.: A Suboptimal Estimation Algorithm with Probabilistic Editing for False Measurements with Applications to Target Tracking with Wake Phenomena, Sixth Symp. on Nonlinear Estimation and Its Applications, San Diego, Calif., Sept. 15- 17, 1975. 31. Boel, R.; Varaiya, R.; and Wong, E.: Martingales on Jump Processes,. I: Representation Results, and 11: Applications, SIAM J. Control, vol. 13, no. 5, Aug. 1975, pp. 999-1061. 32. Segall, Adrien: A Martingale Approach to Modeling, Estimation and Detection of Jump Processes, Ph.D. Dissertation, Stanford University, Aug. 1973. 33. Snyder, D. L.: Random Point Processes, John Wiley and Sons, Inc., 1975. 34. Pierce, Bill D.; and Sworder, David D.: Bayes and Minimax Controllers for a Linear System with Stochastic Jump Parameters, IEEE Trans. Automatic Control, vol. AC-16, no. 4, Aug. 1971, pp. 300-307. 35. Sworder, D. D.: Bayes' Controllers with Memory for a Linear System with Jump Parameters, IEEE Trans. Automatic Control, vol. AC-17, no. 1, Feb. 1972, pp. 119-121. 36. Sworder, David D.; and Robinson, Vernon G.: Feedback Regulators for Jump Parameter Systems with State and Control Dependent Transition Rates, IEEE Trans. Automatic Control, vol. AC-18, no. 4, Aug. 1973, pp. 355-360. 37. Robinson, V. G.; and Sworder, D. D.: A Computational Algorithm for Design of Regulators for Linear Jump Parameter Systems, IEEE Trans. Automatic Control, vol. AC-19, no. 1, Feb. 1974, pp. 47-49. 38. Sworder, David D.: Feedback Control of a Class of Linear Systems with Jump Parameters, IEEE Trans. on Automatic Control, vol. AC-14, no. 1 Feb. 1969, pp. 9-14. 2 04 39. Ratner, Robert S.; and Luenberger, David G.: Performance-Adaptive Renewal - 'Policies for Linear . sys tems , IEEE no. 4, Aug. 1969, pp. 344-351. 40. Davis, M. H. A.: The Application of Detection in Linear Systems, IEEE no. 2, April 1975, pp. 257-259. 41. Chien, T. T.: An Adaptive,Technique Trans. Automatic Control; vol. AC-14, Nonlinear Filtering to Fault Trans. Automatic Control, vol. AC-20, for a Redundant - Sensor Navigation System, Ph.D. Thesis,, Rept. T-566, Draper Labs., Cambridge, Mass., Feb. 1972. I - 42. Mehra, R. K.; and Peschon, J.: An Innovations Approach to Fault Detection and Diagnosis in Dynamic Systems, Automatica, vol. 7, no. 5, Sept. 1971, pp. 637-640. 43. Merrill, H. M.: Bad Data Suppression in State Estimation, with Applica- tions to Problems in Power, Ph.D. Thesis, M.I.T., Cambridge, Mass., June 1972. 44. Willsky, A. S. ; and Jones, H. L. : A Generalized Likelihood Ratio Approach to State Estimation in Linear Systems Subject to Abrupt Changes, Proc. 1974 IEEE Conference on Decision and Control, Phoenix, Arizona, Nov. 1974. 45. Willsky, Alan S.; and Jones, Harold L.: A Generalized Likelihood Ratio Approach to the Detection and Estimation of Jumps in Linear Systems, IEEE Trans. Automatic Control, vol. AC-21, no. 1, Feb. 1976, pp. 108-112. 46. McAulay, R. J.; and Denlinger, E.: A Decision-Directed Adaptive Tracker, IEEE Trans. Aerospace Electronic Sys., vol. AES-9, no. 2, March 1973, pp. 229-236. 47. Sanyal, P.; and Shen, C. N.: Bayes' Decision Rule for Rapid Detection and'Adaptive Estimation Scheme with Space Applications, IEEE Trans. Automatic Control, vol. AC-19, no. 3, June 1974, pp. 228-231. 48. Chow, E.; Dunn, K.-P.; and Willsky, A. S.: Research Status Report to NASA Langley Research Center: A Dual-Mode Generalized Likelihood Ratio Approach to Self-Reorganizing Digital Flight Control System Design, Research Status Report, 15 Oct. 1974, 14 Mar. 1975, M.I.T. Electronic Systems Laboratory, Cambridge, Mass., April 1975, RSR-1. 49. Van Trees, Harry L.: Detection, Estimation, and Modulat,ion. Theory, Part 111: Radar-Sonar Signal Processing and Gaussian Signals in Noise, John Wiley and Sons, Inc., 1971. 205 50. Bueno, R. ; Chow, E.; Gershwin, S. B . ; and Willsky, A. S. : A Dual-Mode Generalized Likelihood Ratio Approach to Self-Reorganizing Digital Flight Control System Design, NASA CR-146,386, 1975. 51. Deyst, J. J . : A Design Approach Co Highly Reliable Flight Control Systems, The Charles Stark Draper Laboratory, Inc., Cambridge, Mass., 1975. . . . . Figure 1.- No-failure system configuration. / conrml low F0i"semitiYc tlllsr 2 Figure 2.- Failure detection system involving failure-sensitive primary filter (6 denotes information concerning detected failures) Figure 3.- Failure detection system involving . a monitoring system for no-failure configuration. 14 I i PARTICIPANTS , .J' Karl Johan ktr6m Robert B. Martindale Lund Institute of Technology, Sweden The Boeing Company Prof. Michael Athans C. Johan Masreliez Massachusetts Institute of Technology The Boeing Company BLAC Raymond A. Ausrotas George Meyer Massachusetts Institute of Technology NASA-Ames Research.Center Frank Bacchialoni Raymond C. Montgomery Massachusetts Institute of Technology NASA Langley Research Center Roger L. Berger McDonnel Douglas Corp. Nick Murray NASA Langley Research Center Douglas Birdwell Robert D. Poyneer Massachusetts Institute of Technology AF Flight Dynamics Laboratory R. N. Clark University of Washington James Deckert C. S. Draper Lab M. N. Desai C. S. Draper Lab Brian F. Doolin NASA Ames Research Center Lt. Col. Walter J. Rabe AFOSR/NM Ian B. Rhodes Washington University George N. Saridis Purdue University Jason L. Speyer C. S. Draper Lab Stanley B. Gershwin Robert F. Stengel Massachusetts Institute of Technology TASC W. Earl Hall, Jr. Systems Control, Inc. Y. C. Ho Harvard University Thomas Kerr TASC David L. Kleinman University of Connecticut Donald T. Makers General Electric Co. Clyde F. Martin Harvard University Lawrence W. Taylor, Jr. NASA Langley Research Center Edison Tse Stanford University W. E. Vander Velde Massachusetts Institute of Technology John S. White NASA Hq. Code REA, Washington, D.C. W. M. Wonham University of Toronto Eugene F. Yore Honeywell . . . . 209 W. M. Wonham University of Toronto Eugene F. Yore Honeywell . . . .. c . - . . 210

Systems reliability issues for future aircraft

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server