Lipschitzness Effect of a Loss Function on Generalization Performance of Deep Neural Networks Trained by Adam and AdamW Optimizers

The generalization performance of deep neural networks with regard to the optimization algorithm is one of the major concerns in machine learning. This performance can be affected by various factors. In this paper, we theoretically prove that the Lipschitz constant of a loss function is an important factor to diminish the generalization error of the output model obtained by Adam or AdamW. The results can be used as a guideline for choosing the loss function when the optimization algorithm is Adam or AdamW. In addition, to evaluate the theoretical bound in a practical setting, we choose the human age estimation problem in computer vision. For assessing the generalization better, the training and test datasets are drawn from different distributions. Our experimental evaluation shows that the loss function with lower Lipschitz constant and maximum value improves the generalization of the model trained by Adam or AdamW.Comment: 13 pages, 6 figures, 3 table

Reduced Jeffries-Matusita distance: A Novel Loss Function to Improve Generalization Performance of Deep Classification Models

The generalization performance of deep neural networks in classification tasks is a major concern in machine learning research. Despite widespread techniques used to diminish the over-fitting issue such as data augmentation, pseudo-labeling, regularization, and ensemble learning, this performance still needs to be enhanced with other approaches. In recent years, it has been theoretically demonstrated that the loss function characteristics i.e. its Lipschitzness and maximum value affect the generalization performance of deep neural networks which can be utilized as a guidance to propose novel distance measures. In this paper, by analyzing the aforementioned characteristics, we introduce a distance called Reduced Jeffries-Matusita as a loss function for training deep classification models to reduce the over-fitting issue. In our experiments, we evaluate the new loss function in two different problems: image classification in computer vision and node classification in the context of graph learning. The results show that the new distance measure stabilizes the training process significantly, enhances the generalization ability, and improves the performance of the models in the Accuracy and F1-score metrics, even if the training set size is small

A novel AI-based approach for modelling the fate, transportation and prediction of chromium in rivers and agricultural crops: A case study in Iran

Chromium (Cr) pollution caused by the discharge of industrial wastewater into rivers poses a significant threat to the environment, aquatic and human life, as well as agricultural crops irrigated by these rivers. This paper employs artificial intelligence (AI) to introduce a new framework for modeling the fate, transport, and estimation of Cr from its point of discharge into the river until it is absorbed by agricultural products. The framework is demonstrated through its application to the case study River, which serves as the primary water resource for tomato production irrigation in Mashhad city, Iran. Measurements of Cr concentration are taken at three different river depths and in tomato leaves from agricultural lands irrigated by the river, allowing for the identification of bioaccumulation effects. By employing boundary conditions and smart algorithms, various aspects of control systems are evaluated. The concentration of Cr in crops exhibits an accumulative trend, reaching up to 1.29 µg/g by the time of harvest. Using data collected from the case study and exploring different scenarios, AI models are developed to estimate the Cr concentration in tomato leaves. The tested AI models include linear regression (LR), neural network (NN) classifier, and NN regressor, yielding goodness-of-fit values (R2) of 0.931, 0.874, and 0.946, respectively. These results indicate that the NN regressor is the most accurate model, followed by the LR, for estimating Cr levels in tomato leaves

Applications of artificial intelligence for chemical analysis and monitoring of pharmaceutical and personal care products in water and wastewater: A review

© 2024 The Authors. Published by Elsevier Ltd. This is an open access article distributed under the Creative Commons Attribution License, to view a copy of the license, see: https://creativecommons.org/licenses/by/4.0/Specifying and interpreting the occurrence of emerging pollutants is essential for assessing treatment processes and plants, conducting wastewater-based epidemiology, and advancing environmental toxicology research. In recent years, artificial intelligence (AI) has been increasingly applied to enhance chemical analysis and monitoring of contaminants in environmental water and wastewater. However, their specific roles targeting pharmaceuticals and personal care products (PPCPs) have not been reviewed sufficiently. This review aims to narrow the gap by highlighting, scoping, and discussing the incorporation of AI during the detection and quantification of PPCPs when utilising chemical analysis equipment and interpreting their monitoring data for the first time. In the chemical analysis of PPCPs, AI-assisted prediction of chromatographic retention times and collision cross-sections (CCS) in suspect and non-target screenings using high-resolution mass spectrometry (HRMS) enhances detection confidence, reduces analysis time, and lowers costs. AI also aids in interpreting spectroscopic analysis results. However, this approach still cannot be applied in all matrices, as it offers lower sensitivity than liquid chromatography coupled with tandem or HRMS. For the interpretation of monitoring of PPCPs, unsupervised AI methods have recently presented the capacity to survey regional or national community health and socioeconomic factors. Nevertheless, as a challenge, long-term monitoring data sources are not given in the literature, and more comparative AI studies are needed for both chemical analysis and monitoring. Finally, AI assistance anticipates more frequent applications of CCS prediction to enhance detection confidence and the use of AI methods in data processing for wastewater-based epidemiology and community health surveillance.Peer reviewe

One-step hydrothermal synthesis of a green NiCo-LDHs-rGO composite for the treatment of lead ion in aqueous solutions

In this study,, we have synthesised a microspherical nickel-cobalt-layered double hydroxides-reduced graphene oxide composite (NiCo-LDHs-rGO) through a one-step hydrothermal method. We subsequently used this composite as an adsorbent to remove Pb2+ from aqueous solutions. The instruments used for the characterisation of adsorbent included Fourier Transform Infrared Spectrophotometry (FT-IR), Field Emission Scanning Electron Microscopy (FESEM), Mapping Elemental Analysis, Electron Dispersive X-Ray spectroscopy (EDX), X-Ray Diffraction Analysis (XRD) and Brunauer-Emmett-Teller (BET)analysis. We conducted a series of experiments to investigate the factors affecting the adsorption of Pb2+ ions in batch mode such as solution pH, adsorbent dosage, contact time, competing ion and regeneration by NiCo-LDHs-rGO. Under optimised conditions determined using the Taguchi method (pH = 5.0, adsorbent dosage = 20 mg and contact time = 30 min), the best removal rate of 99.7% was achieved for 100 mg L-1 of Pb2+. According to the results, NiCo-LDHs-rGO exhibited a high preference for Pb2+ over Cu2+, Zn2+ and Cd2+. This adsorbent was regenerated for several cycles (using 0.01 M HCl) with no significant deterioration in performance. The analyses of the adsorption isotherm models revealed that the adsorption of Pb2+ followed Freundlich isotherm with a maximum adsorption capacity of 200 mg g-1. The kinetic data also confirmed that pseudo second order kinetic equation is the most accurate model for predicting the adsorption kinetics. Furthermore, the Simulink modelling illustrated that the adsorption kinetics of Pb2+ onto NiCo-LDHs-rGO could be accurately represented in a continuous stirred-tank reactor. Finally, dual interactions of the effective parameters can be modelled by polynomial equations in MATLAB, and according to the Taguchi model, pH emerged as the most influential factor among all the parameters

A novel smart framework for optimal design of green roofs in buildings conforming with energy conservation and thermal comfort

The rise in greenhouse gas emissions in cities and the excessive consumption of fossil energy resources has made the development of green spaces, such as green roofs, an increasingly important focus in urban areas. This study proposes a novel smart energy-comfort system for green roofs in housing estates that utilises integrated machine learning (ML), DesignBuilder (DB) software and Taguchi design computations for optimising green roof design and operation in buildings. The optimisation process maximises energy conservation and thermal comfort of the green roof buildings for effective parameters of green roofs including Leaf Area Index (P1), leaf reflectivity (P2), leaf emissivity (P3), and stomatal resistance (P4). The optimal solutions can result in a 12.8% increase in comfort hours and a 14% reduction in energy consumption compared to the base case. The ML analysis revealed that the adaptive network-based fuzzy inference system is the most appropriate method for predicting Energy-Comfort functions based on effective parameters, with a correlation coefficient greater than 97%. This novel smart framework for the optimal design of green roofs in buildings offers an innovative approach to achieving energy conservation and thermal comfort in urban areas