Multi-tier framework for the inferential measurement and data-driven modeling

A framework for the inferential measurement and data-driven modeling has been proposed and assessed in several real-world application domains. The architecture of the framework has been structured in multiple tiers to facilitate extensibility and the integration of new components. Each of the proposed four tiers has been assessed in an uncoupled way to verify their suitability. The first tier, dealing with exploratory data analysis, has been assessed with the characterization of the chemical space related to the biodegradation of organic chemicals. This analysis has established relationships between physicochemical variables and biodegradation rates that have been used for model development. At the preprocessing level, a novel method for feature selection based on dissimilarity measures between Self-Organizing maps (SOM) has been developed and assessed. The proposed method selected more features than others published in literature but leads to models with improved predictive power. Single and multiple data imputation techniques based on the SOM have also been used to recover missing data in a Waste Water Treatment Plant benchmark. A new dynamic method to adjust the centers and widths of in Radial basis Function networks has been proposed to predict water quality. The proposed method outperformed other neural networks. The proposed modeling components have also been assessed in the development of prediction and classification models for biodegradation rates in different media. The results obtained proved the suitability of this approach to develop data-driven models when the complex dynamics of the process prevents the formulation of mechanistic models. The use of rule generation algorithms and Bayesian dependency models has been preliminary screened to provide the framework with interpretation capabilities. Preliminary results obtained from the classification of Modes of Toxic Action (MOA) indicate that this could be a promising approach to use MOAs as proxy indicators of human health effects of chemicals.Finally, the complete framework has been applied to three different modeling scenarios. A virtual sensor system, capable of inferring product quality indices from primary process variables has been developed and assessed. The system was integrated with the control system in a real chemical plant outperforming multi-linear correlation models usually adopted by chemical manufacturers. A model to predict carcinogenicity from molecular structure for a set of aromatic compounds has been developed and tested. Results obtained after the application of the SOM-dissimilarity feature selection method yielded better results than models published in the literature. Finally, the framework has been used to facilitate a new approach for environmental modeling and risk management within geographical information systems (GIS). The SOM has been successfully used to characterize exposure scenarios and to provide estimations of missing data through geographic interpolation. The combination of SOM and Gaussian Mixture models facilitated the formulation of a new probabilistic risk assessment approach.Aquesta tesi proposa i avalua en diverses aplicacions reals, un marc general de treball per al desenvolupament de sistemes de mesurament inferencial i de modelat basats en dades. L'arquitectura d'aquest marc de treball s'organitza en diverses capes que faciliten la seva extensibilitat així com la integració de nous components. Cadascun dels quatre nivells en que s'estructura la proposta de marc de treball ha estat avaluat de forma independent per a verificar la seva funcionalitat. El primer que nivell s'ocupa de l'anàlisi exploratòria de dades ha esta avaluat a partir de la caracterització de l'espai químic corresponent a la biodegradació de certs compostos orgànics. Fruit d'aquest anàlisi s'han establert relacions entre diverses variables físico-químiques que han estat emprades posteriorment per al desenvolupament de models de biodegradació. A nivell del preprocés de les dades s'ha desenvolupat i avaluat una nova metodologia per a la selecció de variables basada en l'ús del Mapes Autoorganitzats (SOM). Tot i que el mètode proposat selecciona, en general, un major nombre de variables que altres mètodes proposats a la literatura, els models resultants mostren una millor capacitat predictiva. S'han avaluat també tot un conjunt de tècniques d'imputació de dades basades en el SOM amb un conjunt de dades estàndard corresponent als paràmetres d'operació d'una planta de tractament d'aigües residuals. Es proposa i avalua en un problema de predicció de qualitat en aigua un nou model dinàmic per a ajustar el centre i la dispersió en xarxes de funcions de base radial. El mètode proposat millora els resultats obtinguts amb altres arquitectures neuronals. Els components de modelat proposat s'han aplicat també al desenvolupament de models predictius i de classificació de les velocitats de biodegradació de compostos orgànics en diferents medis. Els resultats obtinguts demostren la viabilitat d'aquesta aproximació per a desenvolupar models basats en dades en aquells casos en els que la complexitat de dinàmica del procés impedeix formular models mecanicistes. S'ha dut a terme un estudi preliminar de l'ús de algorismes de generació de regles i de grafs de dependència bayesiana per a introduir una nova capa que faciliti la interpretació dels models. Els resultats preliminars obtinguts a partir de la classificació dels Modes d'acció Tòxica (MOA) apunten a que l'ús dels MOA com a indicadors intermediaris dels efectes dels compostos químics en la salut és una aproximació factible.Finalment, el marc de treball proposat s'ha aplicat en tres escenaris de modelat diferents. En primer lloc, s'ha desenvolupat i avaluat un sensor virtual capaç d'inferir índexs de qualitat a partir de variables primàries de procés. El sensor resultant ha estat implementat en una planta química real millorant els resultats de les correlacions multilineals emprades habitualment. S'ha desenvolupat i avaluat un model per a predir els efectes carcinògens d'un grup de compostos aromàtics a partir de la seva estructura molecular. Els resultats obtinguts desprès d'aplicar el mètode de selecció de variables basat en el SOM milloren els resultats prèviament publicats. Aquest marc de treball s'ha usat també per a proporcionar una nova aproximació al modelat ambiental i l'anàlisi de risc amb sistemes d'informació geogràfica (GIS). S'ha usat el SOM per a caracteritzar escenaris d'exposició i per a desenvolupar un nou mètode d'interpolació geogràfica. La combinació del SOM amb els models de mescla de gaussianes dona una nova formulació al problema de l'anàlisi de risc des d'un punt de vista probabilístic

Video Quality Prediction for Video over Wireless Access Networks (UMTS and WLAN)

Transmission of video content over wireless access networks (in particular, Wireless Local Area Networks (WLAN) and Third Generation Universal Mobile Telecommunication System (3G UMTS)) is growing exponentially and gaining popularity, and is predicted to expose new revenue streams for mobile network operators. However, the success of these video applications over wireless access networks very much depend on meeting the user’s Quality of Service (QoS) requirements. Thus, it is highly desirable to be able to predict and, if appropriate, to control video quality to meet user’s QoS requirements. Video quality is affected by distortions caused by the encoder and the wireless access network. The impact of these distortions is content dependent, but this feature has not been widely used in existing video quality prediction models. The main aim of the project is the development of novel and efficient models for video quality prediction in a non-intrusive way for low bitrate and resolution videos and to demonstrate their application in QoS-driven adaptation schemes for mobile video streaming applications. This led to five main contributions of the thesis as follows:(1) A thorough understanding of the relationships between video quality, wireless access network (UMTS and WLAN) parameters (e.g. packet/block loss, mean burst length and link bandwidth), encoder parameters (e.g. sender bitrate, frame rate) and content type is provided. An understanding of the relationships and interactions between them and their impact on video quality is important as it provides a basis for the development of non-intrusive video quality prediction models.(2) A new content classification method was proposed based on statistical tools as content type was found to be the most important parameter. (3) Efficient regression-based and artificial neural network-based learning models were developed for video quality prediction over WLAN and UMTS access networks. The models are light weight (can be implemented in real time monitoring), provide a measure for user perceived quality, without time consuming subjective tests. The models have potential applications in several other areas, including QoS control and optimization in network planning and content provisioning for network/service providers.(4) The applications of the proposed regression-based models were investigated in (i) optimization of content provisioning and network resource utilization and (ii) A new fuzzy sender bitrate adaptation scheme was presented at the sender side over WLAN and UMTS access networks. (5) Finally, Internet-based subjective tests that captured distortions caused by the encoder and the wireless access network for different types of contents were designed. The database of subjective results has been made available to research community as there is a lack of subjective video quality assessment databases.Partially sponsored by EU FP7 ADAMANTIUM Project (EU Contract 214751

Detection And Classification Of Buried Radioactive Materials

This dissertation develops new approaches for detection and classification of buried radioactive materials. Different spectral transformation methods are proposed to effectively suppress noise and to better distinguish signal features in the transformed space. The contributions of this dissertation are detailed as follows. 1) Propose an unsupervised method for buried radioactive material detection. In the experiments, the original Reed-Xiaoli (RX) algorithm performs similarly as the gross count (GC) method; however, the constrained energy minimization (CEM) method performs better if using feature vectors selected from the RX output. Thus, an unsupervised method is developed by combining the RX and CEM methods, which can efficiently suppress the background noise when applied to the dimensionality-reduced data from principle component analysis (PCA). 2) Propose an approach for buried target detection and classification, which applies spectral transformation followed by noisejusted PCA (NAPCA). To meet the requirement of practical survey mapping, we focus on the circumstance when sensor dwell time is very short. The results show that spectral transformation can alleviate the effects from spectral noisy variation and background clutters, while NAPCA, a better choice than PCA, can extract key features for the following detection and classification. 3) Propose a particle swarm optimization (PSO)-based system to automatically determine the optimal partition for spectral transformation. Two PSOs are incorporated in the system with the outer one being responsible for selecting the optimal number of bins and the inner one for optimal bin-widths. The experimental results demonstrate that using variable bin-widths is better than a fixed bin-width, and PSO can provide better results than the traditional Powell’s method. 4) Develop parallel implementation schemes for the PSO-based spectral partition algorithm. Both cluster and graphics processing units (GPU) implementation are designed. The computational burden of serial version has been greatly reduced. The experimental results also show that GPU algorithm has similar speedup as cluster-based algorithm

Knowledge discovery and data mining to understand and optimise the environmental behavior of wastewater treatment processes

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University LondonDirect nitrous oxide (N2O) emissions during the biological nitrogen removal (BNR) processes can significantly increase the carbon footprint of wastewater treatment plant (WWTP) operations. However, quantifying the emissions and understanding the long-term behaviour of N2O fluxes in WWTPs remains challenging and costly. The aim of the current research is to combine wastewater domain knowledge with data-mining techniques to explain the long-term N2O emissions’ behaviour in full-scale biological reactors. A review of the recent full-scale N2O monitoring campaigns is conducted resulting in the development of an emission factor (EF) database with information on configurations, control strategies and operational conditions. The analysis focused on mechanistic model development, molecular biology methods and on the current data management and analysis practices (i.e. visualization techniques, statistical analysis). Sensor and laboratory data acquired from the N2O monitoring campaigns of mainstream and sidestream wastewater processes were used to develop, test and validate a methodological framework for knowledge discovery in wastewater databases. Abnormal events detection, structural changepoint detection, clustering, classification and regression algorithms are used in order to i) translate data into actionable information, ii) link N2O emissions ranges with specific operational conditions, iii) identify and isolate re-occurring system disturbances that affect performance, iv) predict the range of N2O emissions based on operational and environmental conditions and v) provide feedback to monitoring campaigns for the minimisation of sampling requirements. The analysis showed that the relationship of N2O emissions with the operational variables fluctuates in long-term monitoring campaigns; this should be taken into consideration for the development of mitigation measures and during the investigation of triggering operational conditions. Additionally, findings indicate that structural changepoints of operational variables monitored online can be used to detect changes in the behaviour and range of N2O emissions. Finally, data-driven models can reliably estimate N2O behaviour in wastewater processes under given operational conditions. However, fluctuation of dependencies, system disturbances and process-specific characteristics should be taken into consideration

A review of modeling approaches in activated sludge systems

The feasibility of using models to understand processes, predict and/or simulate, control, monitor and optimize WasteWater Treatment Plants (WWTPs) has been explored by a number of researchers. Mathematical modeling provides a powerful tool for design, operational assistance, forecast future behavior and control. A good model not only elucidates a better understanding of the complicated biological and chemical fundamentals but is also essential for process design, process start-up, dynamics predictions, process control and process optimization. This paper reviews developments and the application of different modeling approaches to wastewater treatment plants, especially activated sludge systems and processes therein in the last decade. In addition, we present an opinion on the wider wastewater treatment related research issues that need to be addressed through modeling.Key words: Mathematical modeling, water, wastewater, wastewater treatment plants, activated sludge systems

Monitoring biological wastewater treatment processes: Recent advances in spectroscopy applications

Biological processes based on aerobic and anaerobic technologies have been continuously developed to wastewater treatment and are currently routinely employed to reduce the contaminants discharge levels in the environment. However, most methodologies commonly applied for monitoring key parameters are labor intensive, time-consuming and just provide a snapshot of the process. Thus, spectroscopy applications in biological processes are, nowadays, considered a rapid and effective alternative technology for real-time monitoring though still lacking implementation in full-scale plants. In this review, the application of spectroscopic techniques to aerobic and anaerobic systems is addressed focusing on UV--Vis, infrared, and fluorescence spectroscopy. Furthermore, chemometric techniques, valuable tools to extract the relevant data, are also referred. To that effect, a detailed analysis is performed for aerobic and anaerobic systems to summarize the findings that have been obtained since 2000. Future prospects for the application of spectroscopic techniques in biological wastewater treatment processes are further discussed.The authors thank the Portuguese Foundation for Science and Technology (FCT) under the scope of the strategic funding of UID/BIO/04469/2013 unit, COMPETE 2020 (POCI-01-0145-FEDER-006684) and the project RECI/BBB-EBI/0179/2012 (FCOMP-01-0124-FEDER-027462) and BioTecNorte operation (NORTE-01-0145-FEDER-000004) funded by the European Regional Development Fund under the scope of Norte2020 - Programa Operacional Regional do Norte. The authors also acknowledge the financial support to Daniela P. Mesquita and Cristina Quintelas through the postdoctoral Grants (SFRH/BPD/82558/2011 and SFRH/BPD/101338/2014) provided by FCT - Portugal.info:eu-repo/semantics/publishedVersio

Fault Detection in Wastewater Treatment Systems Using Multiparametric Programming

In this work, a methodology for fault detection in wastewater treatment systems, based on parameter estimation, using multiparametric programming is presented. The main idea is to detect faults by estimating model parameters, and monitoring the changes in residuals of model parameters. In the proposed methodology, a nonlinear dynamic model of wastewater treatment was discretized to algebraic equations using Euler’s method. A parameter estimation problem was then formulated and transformed into a square system of parametric nonlinear algebraic equations by writing the optimality conditions. The parametric nonlinear algebraic equations were then solved symbolically to obtain the concentration of substrate in the inflow, Scin , inhibition coefficient, Ki , and specific growth rate, µo, as an explicit function of state variables (concentration of biomass, X; concentration of organic matter, Sc; concentration of dissolved oxygen, So; and volume, V). The estimated model parameter values were compared with values from the normal operation. If the residual of model parameters exceeds a certain threshold value, a fault is detected. The application demonstrates the viability of the approach, and highlights its ability to detect faults in wastewater treatment systems by providing quick and accurate parameter estimates using the evaluation of explicit parametric functions