Image retrieval based on colour and improved NMI texture features

This paper proposes an improved method for extracting NMI features. This method uses Particle Swarm Optimization in advance to optimize the two-dimensional maximum class-to-class variance (2OTSU) in advance. Afterwards, the optimized 2OUSU is introduced into the Pulse Coupled Neural Network (PCNN) to automatically obtain the number of iterations of the loop. We use an improved PCNN method to extract the NMI features of the image. For the problem of low accuracy of single feature, this paper proposes a new method of multi-feature fusion based on image retrieval. It uses HSV colour features and texture features, where, the texture feature extraction methods include: Grey Level Co-occurrence Matrix (GLCM), Local Binary Pattern (LBP) and Improved PCNN. The experimental results show that: on the Corel-1k dataset, compared with similar algorithms, the retrieval accuracy of this method is improved by 13.6%; On the AT&T dataset, the retrieval accuracy is improved by 13.4% compared with the similar algorithm; on the FD-XJ dataset, the retrieval accuracy is improved by 17.7% compared with the similar algorithm. Therefore, the proposed algorithm has better retrieval performance and robustness compared with the existing image retrieval algorithms based on multi-feature fusion

Empower parameterized generative adversarial networks using a novel particle swarm optimizer: algorithms and applications

Copyright © The Author(s) 2021. In this paper, a novel parameterized generative adversarial network (GAN) is proposed where the parameters are introduced to enhance the performance of image segmentation. The developed algorithm is applied to the image-based crack detection problem on the thermal data obtained through the non-destructive testing process. A new regularization term, which contains three tunable hyperparameters, embedded into the objective function of the GAN in order to improve the contrast ratio of certain areas of the image so as to benefit the crack detection process. To automate the selection of the optimal hyperparameters of the GAN, a new particle swarm optimization (PSO) algorithm is put forward where a neighborhood-based velocity updating strategy is developed for the purpose of thoroughly exploring the problem space. The proposed PSO-based GAN algorithm is shown to 1) work well in detecting cracks on the thermal data generated by the eddy current pulsed thermography technique; and 2) outperforms other conventional GAN algorithms.This research work was funded by Institutional Fund Projects under grant no. (IFPIP-221-135-1442). Therefore, the authors gratefully acknowledge technical and fnancial support from the Ministry of Education and King Abdulaziz University, Jeddah, Saudi Arabia. This work was also supported in part by the National Natural Science Foundation of China under Grants 61873148, 61933007 and 61903065, the China Postdoctoral Science Foundation under Grant 2018M643441, the Royal Society of the UK, and the Alexander von Humboldt Foundation of Germany

A review of different deep learning techniques for sperm fertility prediction

Sperm morphology analysis (SMA) is a significant factor in diagnosing male infertility. Therefore, healthy sperm detection is of great significance in this process. However, the traditional manual microscopic sperm detection methods have the disadvantages of a long detection cycle, low detection accuracy in large orders, and very complex fertility prediction. Therefore, it is meaningful to apply computer image analysis technology to the field of fertility prediction. Computer image analysis can give high precision and high efficiency in detecting sperm cells. In this article, first, we analyze the existing sperm detection techniques in chronological order, from traditional image processing and machine learning to deep learning methods in segmentation and classification. Then, we analyze and summarize these existing methods and introduce some potential methods, including visual transformers. Finally, the future development direction and challenges of sperm cell detection are discussed. We have summarized 44 related technical papers from 2012 to the present. This review will help researchers have a more comprehensive understanding of the development process, research status, and future trends in the field of fertility prediction and provide a reference for researchers in other fields

Investigation of Process-Structure Relationship for Additive Manufacturing with Multiphysics Simulation and Physics-Constrained Machine Learning

Metal additive manufacturing (AM) is a group of processes by which metal parts are built layer by layer from powder or wire feedstock with high-energy laser or electron beams. The most well-known metal AM processes include selective laser melting, electron beam melting, and direct energy deposition. Metal AM can significantly improve the manufacturability of products with complex geometries and heterogeneous materials. It has the potential to be widely applied in various industries including automotive, aerospace, biomedical, energy, and other high-value low-volume manufacturing environments. However, the lack of complete and reliable process-structure-property (P-S-P) relationships for metal AM is still the bottleneck to produce defect-free, structurally sound, and reliable AM parts. There are several technical challenges in establishing the P-S-P relationships for process design and optimization. First, there is a lack of fundamental understanding of the rapid solidification process during which microstructures are formed and the properties of solid parts are determined. Second, the curse of dimensionality in the process and structure design space leads to the lack of data to construct reliable P-S-P relationships. Simulation becomes an important tool to enable us to understand rapid solidification given the limitations of experimental techniques for in-situ measurement. In this research, a mesoscale multiphysics simulation model, called phase-field and thermal lattice Boltzmann method (PF-TLBM), is developed with simultaneous considerations of heterogeneous nucleation, solute transport, heat transfer, and phase transition. The simulation can reveal the complex dynamics of rapid solidification in the melt pool, such as the effects of latent heat and cooling rate on dendritic morphology and solute distribution. The microstructure evolution in the complex heating and cooling environment in the layer-by-layer AM process is simulated with the PF-TLBM model. To meet the lack-of-data challenge in constructing P-S-P relationships, a new scheme of multi-fidelity physics-constrained neural network (MF-PCNN) is developed to improve the efficiency of training in neural networks by reducing the required amount of training data and incorporating physical knowledge as constraints. Neural networks with two levels of fidelities are combined to improve prediction accuracy. Low-fidelity networks predict the general trend, whereas high-fidelity networks model local details and fluctuations. The developed MF-PCNN is applied to predict phase transition and dendritic growth. A new physics-constrained neural network with the minimax architecture (PCNN-MM) is also developed, where the training of PCNN-MM is formulated as a minimax problem. A novel training algorithm called Dual-Dimer method is developed to search high-order saddle points. The developed PCNN-MM is also extended to solve multiphysics problems. A new sequential training scheme is developed for PCNN-MMs to ensure the convergence in solving multiphysics problems. A new Dual-Dimer with compressive sampling (DD-CS) algorithm is also developed to alleviate the curse of dimensionality in searching high-order saddle points during the training. A surrogate model of process-structure relationship for AM is constructed based on the PF-TLBM and PCNN-MM. Based on the surrogate model, multi-objective Bayesian optimization is utilized to search the optimal initial temperature and cooling rate to obtain the desired dendritic area and microsegregation level. The developed PF-TLBM and PCNN-MM provide a systematic and efficient approach to construct P-S-P relationships for AM process design.Ph.D

Image Segmentation with Human-in-the-loop in Automated De-caking Process for Powder Bed Additive Manufacturing

Additive manufacturing (AM) becomes a critical technology that increases the speed and flexibility of production and reduces the lead time for high-mix, low-volume manufacturing. One of the major bottlenecks in further increasing its productivity lies around its post-processing procedures. This work focuses on tackling a critical and inevitable step in powder-bed additive manufacturing processes, i.e., powder cleaning or de-caking. Pressing concerns can be raised with human involvement when performing this task manually. Therefore, a robot-driven automatic powder cleaning system could be an alternative to reducing time consumption and increasing safety for AM operators. However, since the color and surface texture of the powder residuals and the sintered parts are similar from a computer vision perspective, it can be challenging for robots to plan their cleaning path. This study proposes a machine learning framework incorporating image segmentation and eye tracking to de-cake the parts printed by a powder bed additive manufacturing process. The proposed framework intends to partially incorporate human biological behaviors to increase the performance of an image segmentation algorithm to assist the path planning for the robot de-caking system. The proposed framework is verified and evaluated by comparing it with the state-of-the-art image segmentation algorithms. Case studies were utilized to validate and verify the proposed human-in-the-loop algorithms. With a mean accuracy, f1-score, precision, and IoU score of 81.2%, 82.3%, 85.8%, and 66.9%, respectively, the suggested HITL eye tracking plus segmentation framework produced the best performance out of all the algorithms evaluated and compared. Regarding computational time, the suggested HITL framework matches the running times of the other test existing models, with a mean time of 0.510655 seconds and a standard deviation of 0.008387. Finally, future works and directions are presented and discussed. A significant portion of this work can be found in (Asare-Manu et al., 2023

A comprehensive review of scab disease detection on Rosaceae family fruits via UAV imagery

Disease detection in plants is essential for food security and economic stability. Unmanned aerial vehicle (UAV) imagery and artificial intelligence (AI) are valuable tools for it. The purpose of this review is to gather several methods used by our peers recently, hoping to provide some knowledge and assistance for researchers and farmers so that they can employ these technologies more advantageously. The studies reviewed in this paper focused on Scab detection in Rosaceae family fruits. Feature extraction, segmentation, and classification methods for processing the UAV-obtained images and detecting the diseases are discussed briefly. The advantages and limitations of diverse kinds of UAVs and imaging sensors are also explained. The widely applied methods for image analysis are machine learning (ML)-based models, and the extensively used UAV platforms are rotary-wing UAVs. Recent technologies that cope with challenges related to disease detection using UAV imagery are also detailed in this paper. Some challenging issues such as higher costs, limited batteries and flying time, huge and complex data, low resolution, and noisy images, etc., still require future consideration. The prime significance of this paper is to promote automation and user-friendly technologies in Scab detection