Advances and Future Perspectives

Agharafeie , R., Ramos, J. R. C., Mendes, J. M., & Oliveira, R. M. F. (2023). From Shallow to Deep Bioprocess Hybrid Modeling: Advances and Future Perspectives. Fermentation, 9(10), 1-22. [922]. https://doi.org/10.20944/preprints202310.0107.v1, https://doi.org/10.3390/fermentation9100922--- This work was supported by the Associate Laboratory for Green Chemistry - LAQV which is financed by national funds from FCT/MCTES (UIDB/50006/2020 and UIDP/50006/2020). This work received funding from the European Union’s Horizon 2020 research and innovation program under the grant agreement no. 101099487- BioLaMer-HORIZON-EIC-2022-PATHFINDEROPEN-01 (BioLaMer)Deep learning is emerging in many industrial sectors in hand with big data analytics to streamline production. In the biomanufacturing sector, big data infrastructure is lagging comparatively to other industries. A promising approach is to combine Deep Neural Networks (DNN) with prior knowledge in Hybrid Neural Network (HNN) workflows that are less dependent on the quality and quantity of data. This paper reviews published articles over the past 30 years on the topic of HNN applications to bioprocesses. It revealed that HNNs were applied to various bioprocesses, including microbial cultures, animal cells cultures, mixed microbial cultures, and enzyme biocatalysis. HNNs were mainly applied for process analysis, process monitoring, development of software sensors, open- and closed-loop control, batch-to-batch control, model predictive control, intensified design of experiments, quality-by-design, and recently for the development of digital twins. Most previous HNN studies combined shallow Feedforward Neural Networks (FFNNs) with physical laws, such as macroscopic material balance equations, following the semiparametric design principle. Only recently, deep HNNs based on deep FFNNs, Convolution Neural Networks (CNN), Long Short-Term Memory (LSTM) networks and Physics Informed Neural Networks (PINNs) have been reported. The biopharma sector is currently a major driver but applications to biologics quality attributes, new modalities, and downstream processing are significant research gaps.publishersversionpublishe

Decoding Complexity in Metabolic Networks using Integrated Mechanistic and Machine Learning Approaches

How can we get living cells to do what we want? What do they actually ‘want’? What ‘rules’ do they observe? How can we better understand and manipulate them? Answers to fundamental research questions like these are critical to overcoming bottlenecks in metabolic engineering and optimizing heterologous pathways for synthetic biology applications. Unfortunately, biological systems are too complex to be completely described by physicochemical modeling alone. In this research, I developed and applied integrated mechanistic and data-driven frameworks to help uncover the mysteries of cellular regulation and control. These tools provide a computational framework for seeking answers to pertinent biological questions. Four major tasks were accomplished. First, I developed innovative tools for key areas in the genome-to-phenome mapping pipeline. An efficient gap filling algorithm (called BoostGAPFILL) that integrates mechanistic and machine learning techniques was developed for the refinement of genome-scale metabolic network reconstructions. Genome-scale metabolic network reconstructions are finding ever increasing applications in metabolic engineering for industrial, medical and environmental purposes. Second, I designed a thermodynamics-based framework (called REMEP) for mutant phenotype prediction (integrating metabolomics, fluxomics and thermodynamics data). These tools will go a long way in improving the fidelity of model predictions of microbial cell factories. Third, I designed a data-driven framework for characterizing and predicting the effectiveness of metabolic engineering strategies. This involved building a knowledgebase of historical microbial cell factory performance from published literature. Advanced machine learning concepts, such as ensemble learning and data augmentation, were employed in combination with standard mechanistic models to develop a predictive platform for important industrial biotechnology metrics such as yield, titer, and productivity. Fourth, my modeling tools and skills have been used for case studies on fungal lipid metabolism analyses, E. coli resource allocation balances, reconstruction of the genome-scale metabolic network for a non-model species, R. opacus, as well as the rapid prediction of bacterial heterotrophic fluxomics. In the long run, this integrated modeling approach will significantly shorten the “design-build-test-learn” cycle of metabolic engineering, as well as provide a platform for biological discovery

Scalable and fast heterogeneous molecular simulation with predictive parallelization schemes

Multiscale and inhomogeneous molecular systems are challenging topics in the field of molecular simulation. In particular, modeling biological systems in the context of multiscale simulations and exploring material properties are driving a permanent development of new simulation methods and optimization algorithms. In computational terms, those methods require parallelization schemes that make a productive use of computational resources for each simulation and from its genesis. Here, we introduce the heterogeneous domain decomposition approach which is a combination of an heterogeneity sensitive spatial domain decomposition with an \textit{a priori} rearrangement of subdomain-walls. Within this approach, the theoretical modeling and scaling-laws for the force computation time are proposed and studied as a function of the number of particles and the spatial resolution ratio. We also show the new approach capabilities, by comparing it to both static domain decomposition algorithms and dynamic load balancing schemes. Specifically, two representative molecular systems have been simulated and compared to the heterogeneous domain decomposition proposed in this work. These two systems comprise an adaptive resolution simulation of a biomolecule solvated in water and a phase separated binary Lennard-Jones fluid.Comment: 14 pages, 12 figure

Improved control strategies for the environment within cell culture bioreactors

This paper describes the development of improved control strategies for the standard environmental conditions in a fed-batch bioreactor used for monoclonal antibody cell culture. The consequences of relying on fixed parameter PID based controllers are considered and poor performance is demonstrated as a consequence of non-linearity and loop interactions. The benefits from adopting a more sophisticated control strategy are considered. Model Predictive Control (MPC) relies on a process model that can be identified from small system perturbations. It considers the predicted longer-term response and consequently can deliver improved control and satisfy user defined constraints. Results from experimental trials demonstrate the capability of MPC and the merits are discussed with regards to industrial application

Data-driven Soft Sensors in the Process Industry

In the last two decades Soft Sensors established themselves as a valuable alternative to the traditional means for the acquisition of critical process variables, process monitoring and other tasks which are related to process control. This paper discusses characteristics of the process industry data which are critical for the development of data-driven Soft Sensors. These characteristics are common to a large number of process industry fields, like the chemical industry, bioprocess industry, steel industry, etc. The focus of this work is put on the data-driven Soft Sensors because of their growing popularity, already demonstrated usefulness and huge, though yet not completely realised, potential. A comprehensive selection of case studies covering the three most important Soft Sensor application fields, a general introduction to the most popular Soft Sensor modelling techniques as well as a discussion of some open issues in the Soft Sensor development and maintenance and their possible solutions are the main contributions of this work

Novel analysis and modelling methodologies applied to pultrusion and other processes

Often a manufacturing process may be a bottleneck or critical to a business. This thesis focuses on the analysis and modelling of such processest, to both better understand them, and to support the enhancement of quality or output capability of the process. The main thrusts of this thesis therefore are: To model inter-process physics, inter-relationships, and complex processes in a manner that enables re-exploitation, re-interpretation and reuse of this knowledge and generic elements e.g. using Object Oriented (00) & Qualitative Modelling (QM) techniques. This involves the development of superior process models to capture process complexity and reuse any generic elements; To demonstrate advanced modelling and simulation techniques (e.g. Artificial Neural Networks(ANN), Rule-Based-Systems (RBS), and statistical modelling) on a number of complex manufacturing case studies; To gain a better understanding of the physics and process inter-relationships exhibited in a number of complex manufacturing processes (e.g. pultrusion, bioprocess, and logistics) using analysis and modelling. To these ends, both a novel Object Oriented Qualitative (Problem) Analysis (OOQA) methodology, and a novel Artificial Neural Network Process Modelling (ANNPM) methodology were developed and applied to a number of complex manufacturing case studies- thermoset and thermoplastic pultrusion, bioprocess reactor, and a logistics supply chain. It has been shown that these methodologies and the models developed support capture of complex process inter-relationships, enable reuse of generic elements, support effective variable selection for ANN models, and perform well as a predictor of process properties. In particular the ANN pultrusion models, using laboratory data from IKV, Aachen and Pera, Melton Mowbray, predicted product properties very well

Development of monitoring and control systems for biotechnological processes

The field of biotechnology represents an important research area that has gained increasing success in recent times. Characterized by the involvement of biological organisms in manufacturing processes, its areas of application are broad and include the pharmaceuticals, agri-food, energy, and even waste treatment. The implication of living microorganisms represents the common element in all bioprocesses. Cell cultivations is undoubtedly the key step that requires maintaining environmental conditions in precise and defined ranges, having a significant impact on the process yield and thus on the desired product quality. The apparatus in which this process occurs is the bioreactor. Unfortunately, monitoring and controlling these processes can be a challenging task because of the complexity of the cell growth phenomenon and the limited number of variables can be monitored in real-time. The thesis presented here focuses on the monitoring and control of biotechnological processes, more specifically in the production of bioethanol by fermentation of sugars using yeasts. The study conducted addresses several issues related to the monitoring and control of the bioreactor, in which the fermentation takes place. First, the topic concerning the lack of proper sensors capable of providing online measurements of key variables (biomass, substrate, product) is investigated. For this purpose, nonlinear estimation techniques are analyzed to reconstruct unmeasurable states. In particular, the geometric observer approach is applied to select the best estimation structure and then a comparison with the extended Kalman filter is reported. Both estimators proposed demonstrate good estimation capabilities as input model parameters vary. Guaranteeing the achievement of the desired ethanol composition is the main goal of bioreactor control. To this end, different control strategies, evaluated for three different scenarios, are analzyed. The results show that the MIMO system, together with an estimator for ethanol composition, ensure the compliance with product quality. After analyzing these difficulties through numeric simulations, this research work shifts to testing a specific biotechnological process such as manufacturing bioethanol from brewery’s spent grain (BSG) as renewable waste biomass. Both acid pre-treatment, which is necessary to release sugars, and fermentation are optimized. Results show that a glucose yield of 18.12 per 100 g of dried biomass is obtained when the pre-treatment step is performed under optimized conditions (0.37 M H2SO4, 10% S-L ratio). Regarding the fermentation, T=25°C, pH=4.5, and inoculum volume equal to 12.25% v/v are selected as the best condition, at which an ethanol yield of 82.67% evaluated with respect to theoretical one is obtained. As a final step, the use of Raman spectroscopy combined with chemometric techniques such as Partial Least Square (PLS) analysis is evaluated to develop an online sensor for fermentation process monitoring. The results show that the biomass type involved significantly affects the acquired spectra, making them noisy and difficult to interpret. This represents a nontrivial limitation of the applied methodology, for which more experimental data and more robust statistical techniques could be helpful

Multiphysics simulation of corona discharge induced ionic wind

Ionic wind devices or electrostatic fluid accelerators are becoming of increasing interest as tools for thermal management, in particular for semiconductor devices. In this work, we present a numerical model for predicting the performance of such devices, whose main benefit is the ability to accurately predict the amount of charge injected at the corona electrode. Our multiphysics numerical model consists of a highly nonlinear strongly coupled set of PDEs including the Navier-Stokes equations for fluid flow, Poisson's equation for electrostatic potential, charge continuity and heat transfer equations. To solve this system we employ a staggered solution algorithm that generalizes Gummel's algorithm for charge transport in semiconductors. Predictions of our simulations are validated by comparison with experimental measurements and are shown to closely match. Finally, our simulation tool is used to estimate the effectiveness of the design of an electrohydrodynamic cooling apparatus for power electronics applications.Comment: 24 pages, 17 figure

Investigating biocomplexity through the agent-based paradigm.

Capturing the dynamism that pervades biological systems requires a computational approach that can accommodate both the continuous features of the system environment as well as the flexible and heterogeneous nature of component interactions. This presents a serious challenge for the more traditional mathematical approaches that assume component homogeneity to relate system observables using mathematical equations. While the homogeneity condition does not lead to loss of accuracy while simulating various continua, it fails to offer detailed solutions when applied to systems with dynamically interacting heterogeneous components. As the functionality and architecture of most biological systems is a product of multi-faceted individual interactions at the sub-system level, continuum models rarely offer much beyond qualitative similarity. Agent-based modelling is a class of algorithmic computational approaches that rely on interactions between Turing-complete finite-state machines--or agents--to simulate, from the bottom-up, macroscopic properties of a system. In recognizing the heterogeneity condition, they offer suitable ontologies to the system components being modelled, thereby succeeding where their continuum counterparts tend to struggle. Furthermore, being inherently hierarchical, they are quite amenable to coupling with other computational paradigms. The integration of any agent-based framework with continuum models is arguably the most elegant and precise way of representing biological systems. Although in its nascence, agent-based modelling has been utilized to model biological complexity across a broad range of biological scales (from cells to societies). In this article, we explore the reasons that make agent-based modelling the most precise approach to model biological systems that tend to be non-linear and complex