A hybrid, auto-adaptive, and rule-based multi-agent approach using evolutionary algorithms for improved searching

Selecting the most appropriate heuristic for solving a specific problem is not easy, for many reasons. This article focuses on one of these reasons: traditionally, the solution search process has operated in a given manner regardless of the specific problem being solved, and the process has been the same regardless of the size, complexity and domain of the problem. To cope with this situation, search processes should mould the search into areas of the search space that are meaningful for the problem. This article builds on previous work in the development of a multi-agent paradigm using techniques derived from knowledge discovery (data-mining techniques) on databases of so-far visited solutions. The aim is to improve the search mechanisms, increase computational efficiency and use rules to enrich the formulation of optimization problems, while reducing the search space and catering to realistic problems.Izquierdo Sebastián, J.; Montalvo Arango, I.; Campbell, E.; Pérez García, R. (2015). A hybrid, auto-adaptive, and rule-based multi-agent approach using evolutionary algorithms for improved searching. Engineering Optimization. 1-13. doi:10.1080/0305215X.2015.1107434S113Becker, U., & Fahrmeir, L. (2001). Bump Hunting for Risk: a New Data Mining Tool and its Applications. Computational Statistics, 16(3), 373-386. doi:10.1007/s001800100073Bouguessa, M., & Shengrui Wang. (2009). Mining Projected Clusters in High-Dimensional Spaces. IEEE Transactions on Knowledge and Data Engineering, 21(4), 507-522. doi:10.1109/tkde.2008.162Chong, I.-G., & Jun, C.-H. (2005). Performance of some variable selection methods when multicollinearity is present. Chemometrics and Intelligent Laboratory Systems, 78(1-2), 103-112. doi:10.1016/j.chemolab.2004.12.011CHONG, I., & JUN, C. (2008). Flexible patient rule induction method for optimizing process variables in discrete type. Expert Systems with Applications, 34(4), 3014-3020. doi:10.1016/j.eswa.2007.05.047Cole, S. W., Galic, Z., & Zack, J. A. (2003). Controlling false-negative errors in microarray differential expression analysis: a PRIM approach. Bioinformatics, 19(14), 1808-1816. doi:10.1093/bioinformatics/btg242FRIEDMAN, J. H., & FISHER, N. I. (1999). Statistics and Computing, 9(2), 123-143. doi:10.1023/a:1008894516817Geem, Z. W. (2006). Optimal cost design of water distribution networks using harmony search. Engineering Optimization, 38(3), 259-277. doi:10.1080/03052150500467430Goncalves, L. B., Vellasco, M. M. B. R., Pacheco, M. A. C., & Flavio Joaquim de Souza. (2006). Inverted hierarchical neuro-fuzzy BSP system: a novel neuro-fuzzy model for pattern classification and rule extraction in databases. IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews), 36(2), 236-248. doi:10.1109/tsmcc.2004.843220Hastie, T., Friedman, J., & Tibshirani, R. (2001). The Elements of Statistical Learning. Springer Series in Statistics. doi:10.1007/978-0-387-21606-5Chih-Ming Hsu, & Ming-Syan Chen. (2009). On the Design and Applicability of Distance Functions in High-Dimensional Data Space. IEEE Transactions on Knowledge and Data Engineering, 21(4), 523-536. doi:10.1109/tkde.2008.178Hwang, S.-F., & He, R.-S. (2006). A hybrid real-parameter genetic algorithm for function optimization. Advanced Engineering Informatics, 20(1), 7-21. doi:10.1016/j.aei.2005.09.001Izquierdo, J., Montalvo, I., Pérez, R., & Fuertes, V. S. (2008). Design optimization of wastewater collection networks by PSO. Computers & Mathematics with Applications, 56(3), 777-784. doi:10.1016/j.camwa.2008.02.007Javadi, A. A., Farmani, R., & Tan, T. P. (2005). A hybrid intelligent genetic algorithm. Advanced Engineering Informatics, 19(4), 255-262. doi:10.1016/j.aei.2005.07.003Jin, X., Zhang, J., Gao, J., & Wu, W. (2008). Multi-objective optimization of water supply network rehabilitation with non-dominated sorting Genetic Algorithm-II. Journal of Zhejiang University-SCIENCE A, 9(3), 391-400. doi:10.1631/jzus.a071448Johns, M. B., Keedwell, E., & Savic, D. (2014). Adaptive locally constrained genetic algorithm for least-cost water distribution network design. Journal of Hydroinformatics, 16(2), 288-301. doi:10.2166/hydro.2013.218Jourdan, L., Corne, D., Savic, D., & Walters, G. (2005). Preliminary Investigation of the ‘Learnable Evolution Model’ for Faster/Better Multiobjective Water Systems Design. Evolutionary Multi-Criterion Optimization, 841-855. doi:10.1007/978-3-540-31880-4_58Kamwa, I., Samantaray, S. R., & Joos, G. (2009). Development of Rule-Based Classifiers for Rapid Stability Assessment of Wide-Area Post-Disturbance Records. IEEE Transactions on Power Systems, 24(1), 258-270. doi:10.1109/tpwrs.2008.2009430Kang, D., & Lansey, K. (2012). Revisiting Optimal Water-Distribution System Design: Issues and a Heuristic Hierarchical Approach. Journal of Water Resources Planning and Management, 138(3), 208-217. doi:10.1061/(asce)wr.1943-5452.0000165Keedwell, E., & Khu, S.-T. (2005). A hybrid genetic algorithm for the design of water distribution networks. Engineering Applications of Artificial Intelligence, 18(4), 461-472. doi:10.1016/j.engappai.2004.10.001Kehl, V., & Ulm, K. (2006). Responder identification in clinical trials with censored data. Computational Statistics & Data Analysis, 50(5), 1338-1355. doi:10.1016/j.csda.2004.11.015Liu, X., Minin, V., Huang, Y., Seligson, D. B., & Horvath, S. (2004). Statistical Methods for Analyzing Tissue Microarray Data. Journal of Biopharmaceutical Statistics, 14(3), 671-685. doi:10.1081/bip-200025657Marchi, A., Dandy, G., Wilkins, A., & Rohrlach, H. (2014). Methodology for Comparing Evolutionary Algorithms for Optimization of Water Distribution Systems. Journal of Water Resources Planning and Management, 140(1), 22-31. doi:10.1061/(asce)wr.1943-5452.0000321Martínez-Rodríguez, J. B., Montalvo, I., Izquierdo, J., & Pérez-García, R. (2011). Reliability and Tolerance Comparison in Water Supply Networks. Water Resources Management, 25(5), 1437-1448. doi:10.1007/s11269-010-9753-2McClymont, K., Keedwell, E., Savić, D., & Randall-Smith, M. (2013). A general multi-objective hyper-heuristic for water distribution network design with discolouration risk. Journal of Hydroinformatics, 15(3), 700-716. doi:10.2166/hydro.2012.022McClymont, K., Keedwell, E. C., Savić, D., & Randall-Smith, M. (2014). Automated construction of evolutionary algorithm operators for the bi-objective water distribution network design problem using a genetic programming based hyper-heuristic approach. Journal of Hydroinformatics, 16(2), 302-318. doi:10.2166/hydro.2013.226Michalski, R. S. (2000). Machine Learning, 38(1/2), 9-40. doi:10.1023/a:1007677805582Montalvo, I., Izquierdo, J., Pérez-García, R., & Herrera, M. (2014). Water Distribution System Computer-Aided Design by Agent Swarm Optimization. Computer-Aided Civil and Infrastructure Engineering, 29(6), 433-448. doi:10.1111/mice.12062Montalvo, I., Izquierdo, J., Schwarze, S., & Pérez-García, R. (2010). Multi-objective particle swarm optimization applied to water distribution systems design: An approach with human interaction. Mathematical and Computer Modelling, 52(7-8), 1219-1227. doi:10.1016/j.mcm.2010.02.017Nguyen, V. V., Hartmann, D., & König, M. (2012). A distributed agent-based approach for simulation-based optimization. Advanced Engineering Informatics, 26(4), 814-832. doi:10.1016/j.aei.2012.06.001Nicklow, J., Reed, P., Savic, D., Dessalegne, T., Harrell, L., … Chan-Hilton, A. (2010). State of the Art for Genetic Algorithms and Beyond in Water Resources Planning and Management. Journal of Water Resources Planning and Management, 136(4), 412-432. doi:10.1061/(asce)wr.1943-5452.0000053Onwubolu, G. C., & Babu, B. V. (2004). New Optimization Techniques in Engineering. Studies in Fuzziness and Soft Computing. doi:10.1007/978-3-540-39930-8Pelikan, M., Goldberg, D. E., & Lobo, F. G. (2002). Computational Optimization and Applications, 21(1), 5-20. doi:10.1023/a:1013500812258Reed, P. M., Hadka, D., Herman, J. D., Kasprzyk, J. R., & Kollat, J. B. (2013). Evolutionary multiobjective optimization in water resources: The past, present, and future. Advances in Water Resources, 51, 438-456. doi:10.1016/j.advwatres.2012.01.005Shang, W., Zhao, S., & Shen, Y. (2009). A flexible tolerance genetic algorithm for optimal problems with nonlinear equality constraints. Advanced Engineering Informatics, 23(3), 253-264. doi:10.1016/j.aei.2008.09.001Vrugt, J. A., & Robinson, B. A. (2007). Improved evolutionary optimization from genetically adaptive multimethod search. Proceedings of the National Academy of Sciences, 104(3), 708-711. doi:10.1073/pnas.0610471104Vrugt, J. A., Robinson, B. A., & Hyman, J. M. (2009). Self-Adaptive Multimethod Search for Global Optimization in Real-Parameter Spaces. IEEE Transactions on Evolutionary Computation, 13(2), 243-259. doi:10.1109/tevc.2008.924428Xie, X.-F., & Liu, J. (2008). Graph coloring by multiagent fusion search. Journal of Combinatorial Optimization, 18(2), 99-123. doi:10.1007/s10878-008-9140-6Xiao-Feng Xie, & Jiming Liu. (2009). Multiagent Optimization System for Solving the Traveling Salesman Problem (TSP). IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 39(2), 489-502. doi:10.1109/tsmcb.2008.2006910Zheng, F., Simpson, A. R., & Zecchin, A. C. (2013). A decomposition and multistage optimization approach applied to the optimization of water distribution systems with multiple supply sources. Water Resources Research, 49(1), 380-399. doi:10.1029/2012wr013160Zheng, F., Simpson, A. R., & Zecchin, A. C. (2014). Coupled Binary Linear Programming–Differential Evolution Algorithm Approach for Water Distribution System Optimization. Journal of Water Resources Planning and Management, 140(5), 585-597. doi:10.1061/(asce)wr.1943-5452.000036

Clustering Algorithms: Their Application to Gene Expression Data

Gene expression data hide vital information required to understand the biological process that takes place in a particular organism in relation to its environment. Deciphering the hidden patterns in gene expression data proffers a prodigious preference to strengthen the understanding of functional genomics. The complexity of biological networks and the volume of genes present increase the challenges of comprehending and interpretation of the resulting mass of data, which consists of millions of measurements; these data also inhibit vagueness, imprecision, and noise. Therefore, the use of clustering techniques is a first step toward addressing these challenges, which is essential in the data mining process to reveal natural structures and iden-tify interesting patterns in the underlying data. The clustering of gene expression data has been proven to be useful in making known the natural structure inherent in gene expression data, understanding gene functions, cellular processes, and subtypes of cells, mining useful information from noisy data, and understanding gene regulation. The other benefit of clustering gene expression data is the identification of homology, which is very important in vaccine design. This review examines the various clustering algorithms applicable to the gene expression data in order to discover and provide useful knowledge of the appropriate clustering technique that will guarantee stability and high degree of accuracy in its analysis procedure

AVATAR - Machine Learning Pipeline Evaluation Using Surrogate Model

© 2020, The Author(s). The evaluation of machine learning (ML) pipelines is essential during automatic ML pipeline composition and optimisation. The previous methods such as Bayesian-based and genetic-based optimisation, which are implemented in Auto-Weka, Auto-sklearn and TPOT, evaluate pipelines by executing them. Therefore, the pipeline composition and optimisation of these methods requires a tremendous amount of time that prevents them from exploring complex pipelines to find better predictive models. To further explore this research challenge, we have conducted experiments showing that many of the generated pipelines are invalid, and it is unnecessary to execute them to find out whether they are good pipelines. To address this issue, we propose a novel method to evaluate the validity of ML pipelines using a surrogate model (AVATAR). The AVATAR enables to accelerate automatic ML pipeline composition and optimisation by quickly ignoring invalid pipelines. Our experiments show that the AVATAR is more efficient in evaluating complex pipelines in comparison with the traditional evaluation approaches requiring their execution

Multi-objective community detection applied to social and COVID-19 constructed networks

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University LondonCommunity Detection plays an integral part in network analysis, as it facilitates understanding the structures and functional characteristics of the network. Communities organize real-world networks into densely connected groups of nodes. This thesis provides a critical analysis of the Community Detection and highlights the main areas including algorithms, evaluation metrics, applications, and datasets in social networks. After defining the research gap, this thesis proposes two Attribute-Based Label Propagation algorithms that maximizes both Modularity and homogeneity. Homogeneity is considered as an objective function one time, and as a constraint another time. To better capture the homogeneity of real-world networks, a new Penalized Homogeneity degree (PHd) is proposed, that can be easily personalized based on the network characteristics. For the first time, COVID-19 tracing data are utilized to form two dataset networks: one is based on the virus transition between the world countries. While the second dataset is an attributed network based on the virus transition among the contact-tracing in the Kingdom of Bahrain. This type of networks that is concerned in tracking a disease was not formed based on COVID-19 virus and has never been studied as a community detection problem. The proposed datasets are validated and tested in several experiments. The proposed Penalized Homogeneity measure is personalized and used to evaluate the proposed attributed network. Extensive experiments and analysis are carried out to evaluate the proposed methods and benchmark the results with other well-known algorithms. The results are compared in terms of Modularity, proposed PHd, and accuracy measures. The proposed methods have achieved maximum performance among other methods, with 26.6% better performance in Modularity, and 33.96% in PHd on the proposed dataset, as well as noteworthy results on benchmarking datasets with improvement in Modularity measures of 7.24%, and 4.96% respectively, and proposed PHd values 27% and 81.9%