Robust Hybrid Systems for Control, Learning, and Optimization in Networked Dynamical Systems

The deployment of advanced real-time control and optimization strategies in socially-integratedengineering systems could significantly improve our quality of life whilecreating jobs and economic opportunity. However, in cyber-physical systems such assmart grids, transportation networks, healthcare, and robotic systems, there still existseveral challenges that prevent the implementation of intelligent control strategies.These challenges include the existence of limited communication networks, dynamicand stochastic environments, multiple decision makers interacting with the system,and complex hybrid dynamics emerging from the feedback interconnection of physicalprocesses and computational devices.In this dissertation, we study the problem of designing robust control and optimizationalgorithms for cyber-physical systems using the framework of hybrid dynamicalsystems. We propose different theoretical frameworks for the design and analysis offeedback mechanisms that optimize the performance of dynamical systems without requiringan explicit characterization of their mathematical model, i.e., in a model-freeway. The closed-loop system that emerges of the interconnection of the plant with thefeedback mechanism describes, in general, a set-valued hybrid dynamical system. Thesetypes of systems combine continuous-time and discrete-time dynamics, and they usuallylack the uniqueness of solutions property. The framework of set-valued hybriddynamical systems allows us to study many complex dynamical systems that emerge indifferent engineering applications, such as networked multi-agent systems with switching graphs, non-smooth mechanical systems, dynamic pricing mechanisms in transportationsystems, autonomous robots with logic-based controllers, etc. We proposea step-by-step approach to the design of different types of discrete-time, continuous-time,hybrid, and stochastic controllers for different types of applications, extendingand generalizing different results in the literature in the area of extremum seeking control,sampled-data extremization, robust synchronization, and stochastic learning innetworked systems. Our theoretical results are illustrated via different simulations andnumerical examples

Data-Driven Nonlinear Control Designs for Constrained Systems

Systems with nonlinear dynamics are theoretically constrained to the realm of nonlinear analysis and design, while explicit constraints are expressed as equalities or inequalities of state, input, and output vectors of differential equations. Few control designs exist for systems with such explicit constraints, and no generalized solution has been provided. This dissertation presents general techniques to design stabilizing controls for a specific class of nonlinear systems with constraints on input and output, and verifies that such designs are straightforward to implement in selected applications. Additionally, a closed-form technique for an open-loop problem with unsolvable dynamic equations is developed. Typical optimal control methods cannot be readily applied to nonlinear systems without heavy modification. However, by embedding a novel control framework based on barrier functions and feedback linearization, well-established optimal control techniques become applicable when constraints are imposed by the design in real-time. Applications in power systems and aircraft control often have safety, performance, and hardware restrictions that are combinations of input and output constraints, while cryogenic memory applications have design restrictions and unknown analytic solutions. Most applications fall into a broad class of systems known as passivity-short, in which certain properties are utilized to form a structural framework for system interconnection with existing general stabilizing control techniques. Previous theoretical contributions are extended to include constraints, which can be readily applied to the development of scalable system networks in practical systems, even in the presence of unknown dynamics. In cases such as these, model identification techniques are used to obtain estimated system models which are guaranteed to be at least passivity-short. With numerous analytic tools accessible, a data-driven nonlinear control design framework is developed using model identification resulting in passivity-short systems which handles input and output saturations. Simulations are presented that prove to effectively control and stabilize example practical systems

Instrumentation and control of anaerobic digestion processes: a review and some research challenges

The final publication is available at Springer via http://dx.doi.org/10.1007/s11157-015-9382-6[EN] To enhance energy production from methane or resource recovery from digestate, anaerobic digestion processes require advanced instrumentation and control tools. Over the years, research on these topics has evolved and followed the main fields of application of anaerobic digestion processes: from municipal sewage sludge to liquid mainly industrial then municipal organic fraction of solid waste and agricultural residues. Time constants of the processes have also changed with respect to the treated waste from minutes or hours to weeks or months. Since fast closed loop control is needed for short time constant processes, human operator is now included in the loop when taking decisions to optimize anaerobic digestion plants dealing with complex solid waste over a long retention time. Control objectives have also moved from the regulation of key variables measured online to the prediction of overall process perfor- mance based on global off-line measurements to optimize the feeding of the processes. Additionally, the need for more accurate prediction of methane production and organic matter biodegradation has impacted the complexity of instrumentation and should include a more detailed characterization of the waste (e.g., biochemical fractions like proteins, lipids and carbohydrates)andtheirbioaccessibility andbiodegradability characteristics. However, even if in the literature several methodologies have been developed to determine biodegradability based on organic matter characterization, only a few papers deal with bioaccessibility assessment. In this review, we emphasize the high potential of some promising techniques, such as spectral analysis, and we discuss issues that could appear in the near future concerning control of AD processes.The authors acknowledge the financial support of INRA (the French National Institute for Agricultural Research), the French National Research Agency (ANR) for the "Phycover" project (project ANR-14-CE04-0011) and ADEME for Inter-laboratory assay financial support.Jimenez, J.; Latrille, E.; Harmand, J.; Robles Martínez, Á.; Ferrer Polo, J.; Gaida, D.; Wolf, C.... (2015). Instrumentation and control of anaerobic digestion processes: a review and some research challenges. Reviews in Environmental Science and Biotechnology. 14(4):615-648. doi:10.1007/s11157-015-9382-6S615648144Aceves-Lara CA, Latrille E, Steyer JP (2010) Optimal control of hydrogen production in a continuous anaerobic fermentation bioreactor. Int J Hydrogen Energ 35:10710–10718Aguado D, Montoya T, Ferrer J, Seco A (2006) Relating ions concentration variations to conductivity variations in a sequencing batch reactor operated for enhanced biological phosphorus removal. Environ Modell Softw 21:845–851Aguilar-Garnica E, Dochain D, Alcaraz-González V, González-Álvarez V (2009) A multivariable control scheme in a two-stage anaerobic digestion system described by partial differential equations. J Process Contr 19:1324–1332Ahring BK, Angelidaki I, Johansen K (1992) Anaerobic treatment of manure together with industrial waste. Water Sci Technol 25:311–318Ajeej A, Thanikal JV, Narayanan CM, Senthil Kumar R (2015) An overview of bio augmentation of methane by anaerobic co-digestion of municipal sludge along with microalgae and waste paper. Renew Sustain Energy Rev 50:270–276Alcaraz-González V, González-Álvarez V (2007) Selected topics in dynamics and control of chemical and biological processes. Springer, BerlinAlcaraz-González V, Harmand J, Rapaport A, Steyer JP, González-Álvarez V, Pelayo-Ortiz C (2005a) Robust interval-based regulation for anaerobic digestion processes. Water Sci Technol 52:449–456Alcaraz-González V, Salazar-Peña R, González-Alvarez V, Gouzé JL, Steyer JP (2005b) A tunable multivariable nonlinear robust observer for biological systems. C R Biol 328:317–325Alferes J, Irizar I (2010) Combination of extremum-seeking algorithms with effective hydraulic handling of equalization tanks to control anaerobic digesters. Water Sci Technol 61:2825–2834Alferes J, García-Heras JL, Roca E, García C, Irizar I (2008) Integration of equalisation tanks within control strategies for anaerobic reactors. Validation based on ADM1 simulations. Water Sci Technol 57:747–752Alimahmoodi M, Mulligan CN (2008) Anaerobic bioconversion of carbon dioxide to biogas in an upflow anaerobic sludge blanket reactor. J Air Waste Manage Assoc 58:95–103Alvarez JA, Otero L, Lema JM (2010) A methodology for optimising feed composition for anaerobic co-digestion of agro-industrial wastes. Bioresour Technol 101:1153–1158Alvarez-Ramirez J, Meraz M, Monroy O, Velasco A (2002) Feedback control design for an anaerobic digestion process. J Chem Technol Biotechnol 77:725–734Anderson GK, Yang G (1992) Determination of bicarbonate and total volatile acid concentration in anaerobic digesters using a simple titration. Water Environ Res 64:53–59Andrews JF, Graef SP (1971) Dynamic modelling and simulation of the AD process. Advances in chemistry series no. 105, Anaerobic Biological Treatment Processes. American Chemical Society, Washington, DC, p 126Andrews JF, Pearson EA (1965) Kinetics and characteristics of volatile acid production in anaerobic fermentation processes. Air Water Pollut 9:439–461Angelidaki I, Sanders W (2004) Assessment of the anaerobic biodegradability of macropllutants. Rev Environ Sci Biotechnol 3:117–129Antila J, Tuohiniemi M, Rissanen A, Kantojärvi U, Lahti M, Viherkanto K, Kaarre M, Malinen J (2014) MEMS- and MOEMS-based near-infrared spectrometers. Encycl Anal Chem 1–36. doi: 10.1002/9780470027318.a9376Antoniades CD, Christofides P (2001) Integrating nonlinear output feedback control and optimal actuator/sensor placement for transport-reaction processes. Chem Eng Sci 56:4517–4535APHA (2005) American Public Health Association/American Water Works Association/Water Environmental Federation, Standard methods for the Examination of Water and Wastewater, 21st edn. Washington, DC, USAAppels L, Baeyens J, Degrève J, Dewil R (2008) Principles and potential of the anaerobic digestion of waste-activated sludge. Prog Energ Combust 34:755–781Appels L, Lauwers J, Gins G, Degreve J, Van Impe J, Dewil R (2011) Parameter identification and modeling of the biochemical methane potential of waste activated sludge. Environ Sci Technol 45:4173–4178Aquino SF, Chernicharo CAL, Soares H, Takemoto SY, Vazoller RF (2008) Methodologies for determining the bioavailability and biodegradability of sludges. Environ Technol 29:855–862Astals S, Esteban-Gutiérrez M, Fernández-Arévalo T, Aymerich E, García-Heras JL, Mata-Alvarez J (2013a) Anaerobic digestion of seven different sewage sludges: a biodegradability and modelling study. Water Res 47:6033–6043Astals S, Nolla-Ardèvol V, Mata-Alvarez J (2013b) Thermophilic co-digestion of pig manure and crude glycerol: process performance and digestate stability. J Biotechnol 166:97–104Babary JP, Julien S, Nihtilä MT et al (1999) New boundary conditions and adaptive control of fixed-bed bioreactors. Chem Eng Process Process Intensif 38:35–44Barat R, Serralta J, Ruano MV, Jiménez E, Ribes J, Seco A, Ferrer J (2012) Biological nutrient removal model No 2 (BNRM2): a general model for wastewater treatment plants. Water Sci Technol 67:1481–1489Bastin G, Dochain D (1990) On-line estimation and adaptive control of bioreactors. Elsevier Science, AmsterdamBatstone DJ (2013) Modelling and control in anaerobic digestion: achievements and challenges. 13th IWA World Congress on Anaerobic Digestion (AD 13), pp 1–6Batstone DJ, Keller J, Angelidaki I et al (2002) Anaerobic digestion model No. 1. (ADM1). IWA Scientific and Technical Report No. 13. IWABatstone DJ, Tait S, Starrenburg D (2009) Estimation of hydrolysis parameters in full-scale anaerobic digesters. Biotechnol Bioeng 102:1513–1520Batstone DJ, Amerlinck Y, Ekama G et al (2012) Towards a generalized physicochemical framework. Water Sci Technol 66:1147–1161Baumann WT, Rugh WJ (1986) Feedback control of nonlinear systems by extended linearization. IEEE Trans Automat Contr AC-31:40–46Benyahia B, Campillo F, Cherki B, Harmand J (2012) Particle filtring for the chemostat. In: MED’12, Barcelone, SpainBernard O (2011) Hurdles and challenges for modelling and control of microalgae for CO2 mitigation and biofuel production. J Process Control 21:1378–1389Bernard O, Gouzé JL (2004) Closed loop observers bundle for uncertain biotechnological models. J Process Control 14:765–774Bernard O, Hadj-Sadok Z, Dochain D et al (2001a) Dynamical model development and parameter identification for an anaerobic wastewater treatment process. Biotechnol Bioeng 75:424–438Bernard O, Polit M, Hadj-Sadok Z, Pengov M, Dochain D, Estaben M, Labat P (2001b) Advanced monitoring and control of anaerobic wastewater treatment plants: software sensors and controllers for an anaerobic digester. Water Sci Technol 43:175–182Bernard O, Chachuat B, Hélias A, Rodriguez J (2005a) Can we assess the model complexity for a bioprocess? Theory and example of the anaerobic digestion process. Water Sci Technol 53:85–92Bernard O, Chachuat B, Hélias A, Le Dantec B, Sialve B, Steyer JP, Lavigne JF (2005b) An integrated system to remote monitor and control anaerobic wastewater treatment plants through the internet. Water Sci Technol 52:457–464Björnsson L, Hörnsten EG, Mattiasson B (2001a) Utilization of a palladium–metal oxide semiconductor (Pd-MOS) sensor for on-line monitoring of dissolved hydrogen in anaerobic digestion. Biotechnol Bioeng 73:35–43Björnsson L, Murto M, Jantsch TG, Mattiasson B (2001b) Evaluation of new methods for the monitoring of alkalinity, dissolved hydrogen and the microbial community in anaerobic digestion. Water Res 35:2833–2840Boe K (2006) Online monitoring and control of the biogas process. Technical University of DenmarkBoe K, Batstone D, Angelidaki I (2007) An innovative online VFA monitoring system for the anerobic process, based on headspace gas chromatography. Biotechnol Bioeng 96:712–721Boe K, Steyer JP, Angelidaki I (2008) Monitoring and control of the biogas process based on propionate concentration using online VFA measurement. Water Sci Technol 57:661–766Boe K, Batstone DJ, Steyer JP, Angelidaki I (2010) State indicators for monitoring the anaerobic digestion process. Water Res 44:5973–5980Bradford MM (1976) A rapid and sensitive method for the quantitation of microgram quantities of protein utilizing the principle of protein-dye binding. Anal Biochem 72:248–254Brinkmann K, Blaschke L, Polle A (2002) Comparison of different methods for lignin determination as a basis for calibration of near-infrared reflectance spectroscopy and implications of lignoproteins. J Chem Ecol 28:2483–2501Buendía IM, Fernández FJ, Villaseñor J, Rodríguez L (2008) Biodegradability of meat industry wastes under anaerobic and aerobic conditions. Water Res 42:3767–3774Buffiere P, Loisel D, Bernet N, Delgenes JP (2006) Towards new indicators for the prediction of solid waste anaerobic digestion properties. Water Sci Technol 53:233–241Cao Y, Pawlowski A (2012) Sewage sludge-to-energy approaches based on anaerobic digestion and pyrolysis: brief overview and energy efficiency assessment. Renew Sust Energ Rev 16:1657–1665Carballa M, Regueiro L, Lema JM (2015) Microbial management of anaerobic digestion: exploiting the microbiome-functionality nexus. Curr Opin Biotechnol 33:103–111Carlos-Hernandez S, Beteau JF, Sanchez EN (2007) Intelligent control strategy for an anaerobic fluidized bed reactor. In: Michel P (ed) Computer applications in biotechnology, vol 1. Cancun, Mexico, pp 73–78Carlos-Hernandez S, Sanchez EN, Bueno JA (2010) Neurofuzzy control strategy for an abattoir wastewater treatment process. In: Banga JR, Bogaerts P, Van Impe J, Dochain D, Smets I (eds) 11th International symposium on computer applications in biotechnology. Leuven, Belgium, pp 84–89Chandler JA, Jewell WJ, Gossett JM (1980) Predicting methane fermentation biodegradability. Biotechnol Bioeng Symp 10:93–107Chen YH (1990) Adaptive robust observers for non-linear uncertain systems. Int J Syst Sci 21:803–814Chen Y, Cheng JJ, Creamer KS (2008) Inhibition of anaerobic digestion process: a review. Bioresour Technol 99:4044–4064Chynoweth DP, Turick CE, Owens JM, Jerger DE, Peck MW (1993) Biochemical methane potential of biomass and waste feedstocks. Biomass Bioenerg 5:95–111Cirne DG, van der Zee FP, Fernandez-Polanco M, Fernandez-Polanco F (2008) Control of sulphide during anaerobic treatment of S-containing wastewaters by adding limited amounts of oxygen or nitrate. Rev Environ Sci Biotechnol 7:93–105Colombié S, Latrille E, Sablayrolles JM (2007) Online estimation of assimilable nitrogen by electrical conductivity measurement during alcoholic fermentation in enological conditions. J Biosci Bioeng 103:229–235Cord-Ruwisch R, Mercz TI, Hoh CY, Strong GE (1997) Dissolved hydrogen concentration as an on-line control parameter for the automated operation and optimization of anaerobic digesters. Biotechnol Bioeng 56:626–634Cossu R, Raga R (2008) Test methods for assessing the biological stability of biodegradable waste. Waste Manage 28:381–388Cresson R, Pommier S, Béline F et al (2014) Etude interlaboratoires pour l’harmonisation des protocoles de mesure du potentiel bio-méthanogène des matrices solides hétérogènes—Final report (in French) ADEMEDalmau J, Comas J, Rodríguez-Roda I, Pagilla K, Steyer JP (2010) Model development and simulation for predicting risk of foaming in anaerobic digestion systems. Bioresour Technol 101:4306–4314Davidsson A, Gruvberger C, Christensen TH, Hansen TL, Jansen J (2007) Methane yield in source-sorted organic fraction of municipal solid waste. Waste Manage 27:406–414De Baere L (2000) Anaerobic digestion of solid waste: state-of-the-art. Water Sci Technol 41:283–290De Baere L (2008) Partial stream digestion of residual municipal solid waste. Water Sci Technol 57:1073–1077De Gracia M, Grau P, Huete E et al (2009) New generic mathematical model for WWTP sludge digesters operating under aerobic and anaerobic conditions: model building and experimental verification. Water Res 43:4626–4642De Vrieze J, Verstraete W, Boon N (2013) Repeated pulse feeding induces functional stability in anaerobic digestion. Microb Biotechnol 6:414–424Delattre C, Dochain D, Winkin J (2004) Observability analysis of nonlinear tubular (bio)reactor models: a case study. J Process Control 14:661–669Di Pinto AC, Limoni N, Passino R, Rozzi A, Tomei MC (1990) Instrumentation, control and automation of water and wastewater treatment and transport systems. In: Proceedings of the 5th IAWPRC workshop, pp 51–58Díaz I, Pérez C, Alfaro N, Fdz-Polanco F (2015) A feasibility study on the bioconversion of CO2 and H2 to biomethane by gas sparging through polymeric membranes. Bioresour Technol 185:246–253Dochain D (2003) State and parameter estimation in chemical and biochemical processes: a tutorial. J Process Control 13:801–818Dochain D, Tali-Maamar N, Babary JP (1997) On modelling, monitoring and control of fixed bed bioreactors. Comput Chem Eng 21:1255–1266Dochain D, Perrier M, Guay M (2011) Extremum seeking control and its application to process and reaction systems: a survey. Math Comput Simulat 82:369–380Donoso-Bravo A, Garcia G, Pérez-Elvira S, Fernandez-Polanco F (2011) Initial rates technique as a procedure to predict the anaerobic digester operation. Biochem Eng J 53(3):275–280Doublet J, Boulanger A, Ponthieux A, Laroche C, Poitrenaud M, Cacho Rivero JA (2013) Predicting the biochemical methane potential of wide range of organic substrates by near infrared spectroscopy. Bioresour Technol 128:252–258Dreywood R (1946) Qualitative test for carbohydrate material. Industrial & Engineering Chemistry Analytical Edition. Am Chem Soc 18:499Dubois M, Gilles KA, Hamilton JK, Rebers PA, Smith F (1956) Colorimetric method for determination of sugars and related substances. Anal Chem 28:350–356Ekama GA, Sotemann SW, Wentzel MC (2007) Biodegradability of activated sludge organics under anaerobic conditions. Water Res 41:244–252Ellison WJ, Pedarros-Caubet F, Caubet R (2007) Automatic and rapid measurement of microbial suspension growth parameters: application to the evaluation of effector agents. J Rapid Meth Aut Mic 15:369–410Fang HHP (2012) Bioenergy production from waste and wastewater in China. In: Technical proceedings of the 2012 NSTI nanotechnology conference and expo, NSTI-nanotech 2012, pp 381–383Fannin KF, Chynoweth DP, Isaacson R (1987) Start-up, operation, stability, and control. Anaerob Dig Biomass 171–196Fdz-Polanco M, Díaz I, Pérez SI, Lopes AC, Fdz-Polanco F (2009a) Hydrogen sulphide removal in the anaerobic digestion of sludge by micro-aerobic processes: pilot plant experience. Water Sci Technol 60:3045–3050Fdz-Polanco M, Pérez-Elvira SI, Díaz I, García L, Torío R, Acevedo AF (2009b) Eliminación de H2S en digestión anaerobia de lodos por procesos microaerofílicos: experiencia en planta piloto. Tecnol del Agua 29:58–64Feitkenhauer H, von Sachs J, Meyer U (2002) On-line titration of volatile fatty acids for the process control of anaerobic digestion plants. Water Res 36:212–218Fernández YB, Soares A, Villa R, Vale P, Cartmell E (2014) Carbon capture and biogas enhancement by carbon dioxide enrichment of anaerobic digesters treating sewage sludge or food waste. Bioresour Technol 159:1–7Fountoulakis MS, Stamatelatou K, Lyberatos G (2008) The effect of pharmaceuticals on the kinetics of methanogenesis and acetogenesis. Bioresour Technol 99:7083–7090Francioso O, Rodriguez-Estrada MT, Montecchio D, Salomoni C, Caputo A, Palenzona D (2010) Chemical characterization of municipal wastewater sludges produced by two-phase anaerobic digestion for biogas production. J Hazard Mater 175:740–746Frigon JC, Roy C, Guiot SR (2012) Anaerobic co-digestion of dairy manure with mulched switchgrass for improvement of the methane yield. Bioprocess Biosyst Eng 35:341–349Frings CS, Dunn RT (1970) A colorimetric method for determination of total serum lipids based on the sulfo-phospho-vanillin reaction. Am J Clin Pathol 53:89–91Frølund B, Palmgren R, Keiding K, Nielsen PH (1996) Extraction of extracellular polymers from activated sludge using a cation exchange resin. Water Res 30:1749–1758Gaida D, Wolf C, Meyer C, Stuhlsatz A, Lippel J, Bäck T, Bongards M, McLoone S (2012) State estimation for anaerobic digesters using the ADM1. Water Sci Technol 66:1088–1095Ganesh R, Torrijos M, Sousbie P et al (2013) Anaerobic co-digestion of solid waste: effect of increasing organic loading rates and characterization of the solubilised organic matter. Bioresource Technol 130:559–569García-Diéguez C, Molina F, Roca E (2011) Multi-objective cascade controller for an anaerobic digester. Process Biochem 46:900–909García-Gen (2015) Modelling, optimisation and control of anaerobic co-digestion processes (2015), Ph.D. Thesis, Universidad de Santiago de Compostela, Departamento de Ingeniería QuímicaGarcía-Gen S, Sousbie P, Rangaraj G et al (2015) Kinetic modelling of anaerobic hydrolysis of solid wastes, including disintegration processes. Waste Manag 35:96–104Gauthier JP, Kupka IAK (1994) Observability and observers for nonlinear systems. SIAM J Control Optim 32:975–994Gauthier JP, Hammouri H, Othman S (1992) A simple observer for nonlinear systems applications to bioreactors. Autom Control IEEE Trans 37:875–880Ge H, Jensen PD, Batstone DJ (2011) Increased temperature in the thermophilic stage in temperature phased anaerobic digestion (TPAD) improves degradability of waste activated sludge. J Hazard Mater 187:355–361Gendron S, Perrier M, Barrett J, Legault N (1993) Adaptive control of brightness: the model weighting approach. Annual meeting—technical section, Canadian Pulp and Paper Association, Preprints. Publ by Canadian Pulp & Paper AssocGhosh S, Conrad JR, Klass DL (1975) Anaerobic acidogenesis of waste activated sludge, WPCF 47Goffaux G, Van de Wouwer A (2005) Bioprocess state estimation: some classical and less classical approaches. Springer, BerlinGornall AG, Bardawill CJ, David MM (1949) Determination of serum proteins by means of the biuret reaction. J Biochem Chem 177:751–766Gouzé JL, Rapaport A, Hadj-Sadok MZ (2000) Interval observers for uncertain biological systems. Ecol Model 133:45–56Grau P, de Gracia M, Vanrolleghem PA, Ayesa E (2007) A new plant-wide modelling methodology for WWTPs. Water Res 41:4357–4372Gregersen KH (2003) Økonomien i biogasfællesanlæg, Udvikling og status medio (2002) Report no. 150. Institute of Food and Resource Economic, Rolighedsvej 25, DK 1958, Frederiksberg C, DenmarkGrepmeier M (2002) Experimentelle Untersuchungen an einer zweistufigen fuzzy-geregelten anaeroben Abwasserreinigungsanlage mit neuartigem Festbettmaterial. TU MunichGuay M, Dochain D, Perrier M (2004) Adaptive extremum seeking control of continuous stirred tank bioreactors with unknown growth kinetics. Automatica 40:881–888Gunaseelan VN (2007) Regression models of ultimate methane yields of fruits and vegetable solid wastes, sorghum and napiergrass on chemical composition. Bioresour Technol 98:1270–1277Gunaseelan VN (2009) Predicting ultimate methane yields of Jatropha curcus and Morus indica from their chemical composition. Bioresour Technol 100:3426–3429Guwy AJ, Hawkes FR, Wilcox SJ, Hawkes DL (1997) Neural network and on-off control of bicarbonate alkalinity in a fluidised-bed anaerobic digester. Water Res 31:2019–2025Guwy AJ, Dinsdale RM, Kim JR et al (2011) Fermentative biohydrogen production systems integration. Bioresour Technol 102:8534–8542Hao OJ (2003) Sulphate-reducing bacteria. In: Mara D, Horan N (eds) Handbook of water and wastewater microbiology. Academic Press Inc, London, pp 459–468Harremoës P, Capodaglio AG, H

Active Flow Control for Bluff Body Drag Reduction Using Reinforcement Learning with Partial Measurements

Active flow control for drag reduction with reinforcement learning (RL) is performed in the wake of a 2D square bluff body at laminar regimes with vortex shedding. Controllers parameterized by neural networks are trained to drive two blowing and suction jets. RL with full observability (sensors in the wake) successfully discovers a control policy which reduces the drag by suppressing the vortex shedding in the wake. However, a non-negligible performance degradation (~50\% less drag reduction) is observed when the controller is trained with partial measurements (sensors on the body). To mitigate this effect, we propose a dynamic, energy-efficient, maximum entropy RL control scheme. First, an energy-efficiency-based reward function is proposed to optimize the energy consumption of the controller while maximising drag reduction. Second, the controller is trained with an augmented state consisting of both current and past observations and actions, which can be formulated as a nonlinear autoregressive exogenous model, to alleviate the partial observability problem. Third, maximum entropy RL algorithms which promote exploration and exploitation in a sample efficient way are used and discover near-optimal policies in the challenging case of partial measurements. Complete stabilisation of the vortex shedding is achieved in the near wake using only surface pressure measurements on the rear of the body, resulting in similar drag reduction as in the case with wake sensors. The proposed approach opens new avenues for dynamic flow control using partial measurements for realistic configurations

Constraint-Aware and Efficiency-Aware Control of Air-Path in Fuel Cell Vehicles

Fuel cell technology offers the potential for clean, efficient, robust energy productionfor both stationary and mobile applications. But without fast and robust control systems, fuel cells cannot hope to maintain real-life efficiencies near enough to their theoretical potential. This work studies control and constraint management techniques to regulate a nonlinear multivariable air-path system for a proton exchange membrane fuel cell (PEMFC). The control objectives are to avoid oxygen starvation, run at the maximum net efficiency, achieve fast tracking of air flow and pressure set-points, and be easy to calibrate. To operate at maximum efficiency, a set-point map is generated for air pressure at the cathode inlet. Considering that the conventional PEMFC system cannot independently control the inlet pressure using only the compressor motor, a new multivariable analysis and control scheme is formulated by considering an electronic throttle body (ETB) valve downstream of the cathode as a new degree of freedom in the control problem. Based on this new configuration of the fuel cell model, an internal model control (IMC) controller is designed with intuitive tuning parameters to simultaneously control airflow and pressure, and achieves a fast and smooth response despite strongly coupled plant dynamics. Further, a reference governor (RG) using a computationally tractable linear prediction model is included with IMC-based Multi-Input Multi-Output (MIMO) controller to satisfy the constraint on oxygen level. Compared with a Single-Input Single-Output (SISO) air-flow control approach, the proposed MIMO control approach demonstrated up to 7.36 percent lower hydrogen fuel consumption