A review of different deep learning techniques for sperm fertility prediction

Sperm morphology analysis (SMA) is a significant factor in diagnosing male infertility. Therefore, healthy sperm detection is of great significance in this process. However, the traditional manual microscopic sperm detection methods have the disadvantages of a long detection cycle, low detection accuracy in large orders, and very complex fertility prediction. Therefore, it is meaningful to apply computer image analysis technology to the field of fertility prediction. Computer image analysis can give high precision and high efficiency in detecting sperm cells. In this article, first, we analyze the existing sperm detection techniques in chronological order, from traditional image processing and machine learning to deep learning methods in segmentation and classification. Then, we analyze and summarize these existing methods and introduce some potential methods, including visual transformers. Finally, the future development direction and challenges of sperm cell detection are discussed. We have summarized 44 related technical papers from 2012 to the present. This review will help researchers have a more comprehensive understanding of the development process, research status, and future trends in the field of fertility prediction and provide a reference for researchers in other fields

Robust Automatic Multi-Sperm Tracking in Time-Lapse Images

Human sperm cell counting, tracking and motility analysis is of significant interest to biologists studying sperm function and to medical practitioners evaluating male infertility. Today, the prevailing method for analyzing sperm at fertility clinics and research laboratories is laborious and subjective. Namely, the number and quality of sperm are often visually appraised by technicians using a microscope. Although total sperm count and sperm concentration can be reasonably estimated when standard protocols are applied, they have little diagnostic value except in identifying pathologically extreme abnormalities. More dynamic sperm swimming parameters such as curvilinear velocity (VCL), straight-line velocity (VSL), linearity of forward progression (LIN) and amplitude of lateral head displacement (ALH) are increasingly believed to have clinical significance in predicting infertility but are impossible for a human observer to visually discern. Expensive computer-assisted semen analysis (CASA) instruments are also sometimes used but are severely encumbered by crude ad-hoc tracking algorithms which cannot track sperm in close proximity or whose paths intersect and are typically limited to analyzing video clips of < 1 sec duration.In this thesis, we present a robust automatic multi-sperm tracking algorithm that can measure dynamic sperm motility parameters over time in pre-recorded time-lapse images. This effort is informed by progress in signal processing and target tracking technologies over the last three decades. Multi-target tracking algorithms originally developed for radar, sonar and video processing have addressed similar problems in other domains. In this thesis, we demonstrate that their methodologies can be used for sperm tracking and motility analysis. To resolve sperm measurement-to-track association conflicts, we applied and evaluated three multi-target tracking algorithms: the probabilistic data association filter (PDAF), the joint probabilistic data association filter (JPDAF) and the exact nearest neighbor extension to the JPDAF (ENN-JPDAF). We validated the accuracy of our tracking and motility analysis by using simulated sperm trajectories whose ground truth tracks were perfectly known. Using samples collected from five patients at a fertility clinic, we demonstrated automatic sperm detection and tracking even during challenging multi-sperm collision events. Combined analysis, testing and simulation support the use of probabilistic data association techniques robust automatic multi-sperm tracking. This method could provide fertility specialists with new data visualizations and interpretations previously impossible with existing laboratory protocols

Image Processing Methods for Automatic in-vitro Morphology Analysis

The study of male infertility has become a priority for biologists and researchers in the last decades as a consequence of the declining birth rates. This problem has also become a major public health with economic and psychosocial consequences. Analysis of human sperm cells, for instance, is widely used in investigations related to male infertility and assisted conception. Sperm samples are usually analysed by health professionals using microscope devices following a manual process to count and describe their morphology. Nevertheless, this practice is prone to errors and time consuming. This thesis proposes a novel framework based on image processing and machine learning methods to automate the analysis of sperm cells. The proposed method presented an average accuracy performance of 96.4% classify automatically sperm cells in three classes: normal, abnormal and non sperm cell. Performance results have been obtained in challenging conditions: presence of uneven illumination, unwanted noise and blurring caused by the focus drift and occlusion of objects as a result of the overlapping of sperm cells, among others. The object of interest, sperm cells, captured in the images used in this research did not receive any staining or fixation treatment prior to their capture. A novel and robust methodology based on deep neural learning is developed as part of the automatic feature selection prior to the classification. Also, video and image database of sperm samples was produced at the Andrology laboratory of the University of Sheffield as part of this work. The database was used to validate the proposed framework for the segmentation and classification of in-vitro cells

Ros-Drill Automation: Visual Feedback Control And Rotational Motion Tracking

ICSI (intra-cytoplasmic sperm injection) has attracted research interest from both biological and engineering groups. The technology is constantly evolving to perform this procedure with precision and speed. One such development is the contribution of this thesis. We focus on a relatively recent procedure called Ros-Drill© (rotationally oscillating drill), of which the early versions have already been effectively utilized for the mice. In the first part, we present a procedure to automate a critical part of the operation: initiation of the rotational oscillation, Visual feedback is used to track the pipette tip. Predetermined species-specific penetration depth is successfully utilized to initiate the rotational oscillation command. Penetration-depth-based decisions concur with a curvature-based approach. In the second part for the automation we improve the performance of the rotational motion tracking. Ros-Drill© is an inexpensive set-up, which creates high-frequency rotational oscillations at the tip of an injection pipette tracking a harmonic motion profile. These rotational oscillations enable the pipette to drill into cell membranes with minimum biological damage. Such a motion control procedure presents no particular difficulty when it uses sufficiently precise motion sensors. However, size, costs and accessibility of technology on hardware components may severely constrain the sensory capabilities. Then the trajectory tracking is adversely affected. In this thesis we handle such a practical case, and present hardware and software improvements using a commonly available microcontroller and extremely low-resolution position measurements. Biological tests are performed and it is confirmed that the mechanical structure plays a crucial role for success

Automating assessment of human embryo images and time-lapse sequences for IVF treatment

As the number of couples using In Vitro Fertilization (IVF) treatment to give birth increases, so too does the need for robust tools to assist embryologists in selecting the highest quality embryos for implantation. Quality scores assigned to embryonic structures are critical markers for predicting implantation potential of human blastocyst-stage embryos. Timing at which embryos reach certain cell and development stages in vitro also provides valuable information about their development progress and potential to become a positive pregnancy. The current workflow of grading blastocysts by visual assessment is susceptible to subjectivity between embryologists. Visually verifying when embryo cell stage increases is tedious and confirming onset of later development stages is also prone to subjective assessment. This thesis proposes methods to automate embryo image and time-lapse sequence assessment to provide objective evaluation of blastocyst structure quality, cell counting, and timing of development stages

Improving the performance and evaluation of computer-assisted semen analysis

Semen analysis is performed routinely in fertility clinics to analyze the quality of semen and sperm cells of male patients. The analysis is typically performed by trained technicians or by Computer-Assisted Semen Analysis (CASA) systems. Manual semen analysis performed by technicians is subjective, time-consuming, and laborious, and yet most fertility clinics perform semen analysis in this manner. CASA systems, which are designed to perform the same tasks automatically, have a considerable market share, yet many studies still express concerns about their accuracy and consistency. In this dissertation, the focus is on detection, tracking, and classification of sperm cells in semen images, key elements of CASA systems. The objective is to improve existing CASA algorithms and systems by applying validated computer vision, tracking, and computational intelligence algorithms. The first step of the study is the development of simulation models for generating synthetic images of semen samples. The images enable the assessment of CASA systems and their algorithms. Specifically, the simulation models generate time-lapse images of semen samples for various sperm image categories and include ground truth labels. The models exploit standard image processing operations such as point spread functions and 2D convolutions, as well as new models of sperm cell swimming, developed for this study. They embody multiple studies of sperm motility in the form of parameterized motion equations. Use cases are presented to use the swimming models and the simulated images to assess and compare algorithms for sperm cell segmentation, localization, and tracking. Second, a digital washing algorithm is presented for unwashed semen samples. Digital washing has the potential to replace the chemical washing techniques used by fertility clinics at present, which are costly, time-consuming, and unfriendly to the environment. The digital washing algorithm extracts features from moving sperm cells in an image, and uses these features to identify all sperm cells (moving and stationary) within each studied image (simulated or real). The effectiveness of the digital washing algorithm is demonstrated by comparing the performance of the proposed algorithm to other cell segmentation and detection techniques. Third, a classification algorithm for sperm cells is developed, based on their swimming patterns. The classification algorithm uses K-means clustering on a subset of motility parameters of sperm cells selected by the Artificial Bee Colony (ABC) algorithm. Results of classification and clustering are shown, using simulated and real semen images. Swimming pattern classification has the potential to increase understanding of the relationship between the distribution of sperm cell swimming modes in a patient’s semen image and the fertility of that patient. Lastly, a new method is presented to calculate motility parameters from sperm tracks. The movement of sperm cell is modeled as a sinusoidal traveling wave (“traveling sinusoid”). The amplitude and average path of a moving cell are estimated using an extended Kalman filter (EKF). The states estimated by the EKF include position, velocity, amplitude, and frequency of the traveling wave. The motility parameters calculated from this approach are shown to be superior to those calculated by other existing methods in terms of their accuracy and consistency. CASA developers will find in this study (and in the software made available) new tools to improve the performance of their designs, and to compare and contrast different proposed approaches and algorithms

Conference of Advance Research and Innovation (ICARI-2014) 118 ICARI

Abstract With the advent of highly advanced optics and imaging system, currently biological research has reached a stage where scientists can study biological entities and processes at molecular and cellular-level in real time. However, a single experiment consists of hundreds and thousands of parameters to be recorded and a large population of microscopic objects to be tracked. Thus, making manual inspection of such events practically impossible. This calls for an approach to computer-vision based automated tracking and monitoring of cells in biological experiments. This technology promises to revolutionize the research in cellular biology and medical science which includes discovery of diseases by tracking the process in cells, development of therapy and drugs and the study of microscopic biological elements. This article surveys the recent literature in the area of computer vision based automated cell tracking. It discusses the latest trends and successes in the development and introduction of automated cell tracking techniques and systems

Low-cost portable microscopy systems for biomedical imaging and healthcare applications

In recent years, the development of low-cost portable microscopes (LPMs) has opened new possibilities for disease detection and biomedical research, especially in resource-limited areas. Despite these advancements, the majority of existing LPMs are hampered by sophisticated optical and mechanical designs, require extensive post-data analysis, and are often tailored for specific biomedical applications, limiting their broader utility. Furthermore, creating an optical-sectioning microscope that is both compact and cost effective presents a significant challenge. Addressing these critical gaps, this PhD study aims to: (1) develop a universally applicable LPM featuring a simplified mechanical and optical design for real-time biomedical imaging analysis, and (2) design a novel, smartphone-based optical sectioning microscope that is both compact and affordable. These objectives are driven by the need to enhance accessibility to quality diagnostic tools in varied settings, promising a significant leap forward in the democratization of biomedical imaging technologies. With 3D printing, optimised optical design, and AI techniques, we can develop LPM’s real time analysis functionality. I conducted a literature review on LPMs and related applications in my study and implemented two low-cost prototype microscopes and one theoretical study. 1) The first project is a portable AI fluorescence microscope based on a webcam and the NVIDIA Jetson Nano (NJN) with real-time analysis functionality. The system was 3D printed, weighing ~250 grams with a size of 145mm × 172 mm × 144 mm (L×W×H) and costing ~$400. It achieves a physical magnification of ×5 and can resolve 228.1 lp/mm USAF features. The system can recognise and count fluorescent beads and human red blood cells (RBCs). 2) I developed a smartphone-based optical sectioning microscope using the HiLo technique. To our knowledge, it is the first smartphone-based HiLo microscope that offers low-cost optical-sectioned widefield imaging. It has a 571.5 μm telecentric scanning range and an 11.7 μm axial resolution. I successfully used it to realize optical sectioning imaging of fluorescent beads. For this system, I developed a new low-cost HiLo microscopy technique using microlens arrays (MLAs) with incoherent light-emitting diode (LED) light sources. I conducted a numerical simulation study assessing the integration of uncoherent LEDs and MLAs for a low-cost HiLo system. The MLA can generate structured illumination in HiLo. How the MLA’s geometry structure and physical parameters affect the image performance were discussed in detail. This PhD thesis explores the advancement of low-cost portable microscopes (LPMs) through the integration of 3D printing, optimized optical design, and artificial intelligence (AI) techniques to enhance their real-time analysis capabilities. The research involved a comprehensive literature review on LPMs and their applications, leading to the development of two innovative prototype LPMs, alongside a theoretical study. These works contribute significantly to the field by not only addressing the technical and financial barriers associated with advanced microscopy but also by laying the groundwork for future innovations in portable and accessible biomedical imaging. Through its focus on simplification, affordability, and practicality, the research holds promise for substantially expanding the reach and impact of diagnostic imaging technologies, especially in those resource-limited areas.In recent years, the development of low-cost portable microscopes (LPMs) has opened new possibilities for disease detection and biomedical research, especially in resource-limited areas. Despite these advancements, the majority of existing LPMs are hampered by sophisticated optical and mechanical designs, require extensive post-data analysis, and are often tailored for specific biomedical applications, limiting their broader utility. Furthermore, creating an optical-sectioning microscope that is both compact and cost effective presents a significant challenge. Addressing these critical gaps, this PhD study aims to: (1) develop a universally applicable LPM featuring a simplified mechanical and optical design for real-time biomedical imaging analysis, and (2) design a novel, smartphone-based optical sectioning microscope that is both compact and affordable. These objectives are driven by the need to enhance accessibility to quality diagnostic tools in varied settings, promising a significant leap forward in the democratization of biomedical imaging technologies. With 3D printing, optimised optical design, and AI techniques, we can develop LPM’s real time analysis functionality. I conducted a literature review on LPMs and related applications in my study and implemented two low-cost prototype microscopes and one theoretical study. 1) The first project is a portable AI fluorescence microscope based on a webcam and the NVIDIA Jetson Nano (NJN) with real-time analysis functionality. The system was 3D printed, weighing ~250 grams with a size of 145mm × 172 mm × 144 mm (L×W×H) and costing ~$400. It achieves a physical magnification of ×5 and can resolve 228.1 lp/mm USAF features. The system can recognise and count fluorescent beads and human red blood cells (RBCs). 2) I developed a smartphone-based optical sectioning microscope using the HiLo technique. To our knowledge, it is the first smartphone-based HiLo microscope that offers low-cost optical-sectioned widefield imaging. It has a 571.5 μm telecentric scanning range and an 11.7 μm axial resolution. I successfully used it to realize optical sectioning imaging of fluorescent beads. For this system, I developed a new low-cost HiLo microscopy technique using microlens arrays (MLAs) with incoherent light-emitting diode (LED) light sources. I conducted a numerical simulation study assessing the integration of uncoherent LEDs and MLAs for a low-cost HiLo system. The MLA can generate structured illumination in HiLo. How the MLA’s geometry structure and physical parameters affect the image performance were discussed in detail. This PhD thesis explores the advancement of low-cost portable microscopes (LPMs) through the integration of 3D printing, optimized optical design, and artificial intelligence (AI) techniques to enhance their real-time analysis capabilities. The research involved a comprehensive literature review on LPMs and their applications, leading to the development of two innovative prototype LPMs, alongside a theoretical study. These works contribute significantly to the field by not only addressing the technical and financial barriers associated with advanced microscopy but also by laying the groundwork for future innovations in portable and accessible biomedical imaging. Through its focus on simplification, affordability, and practicality, the research holds promise for substantially expanding the reach and impact of diagnostic imaging technologies, especially in those resource-limited areas

Object Detection in medical imaging

A thesis submitted in partial fulfillment of the requirements for the degree of Doctor in Information Management, specialization in Information and Decision SystemsArtificial Intelligence, assisted by deep learning, has emerged in various fields of our society. These systems allow the automation and the improvement of several tasks, even surpassing, in some cases, human capability. Object detection methods are used nowadays in several areas, including medical imaging analysis. However, these methods are susceptible to errors, and there is a lack of a universally accepted method that can be applied across all types of applications with the needed precision in the medical field. Additionally, the application of object detectors in medical imaging analysis has yet to be thoroughly analyzed to achieve a richer understanding of the state of the art. To tackle these shortcomings, we present three studies with distinct goals. First, a quantitative and qualitative analysis of academic research was conducted to gather a perception of which object detectors are employed, the modality of medical imaging used, and the particular body parts under investigation. Secondly, we propose an optimized version of a widely used algorithm to overcome limitations commonly addressed in medical imaging by fine-tuning several hyperparameters. Thirdly, we develop a novel stacking approach to augment the precision of detections on medical imaging analysis. The findings show that despite the late arrival of object detection in medical imaging analysis, the number of publications has increased in recent years, demonstrating the significant potential for growth. Additionally, we establish that it is possible to address some constraints on the data through an exhaustive optimization of the algorithm. Finally, our last study highlights that there is still room for improvement in these advanced techniques, using, as an example, stacking approaches. The contributions of this dissertation are several, as it puts forward a deeper overview of the state-of-the-art applications of object detection algorithms in the medical field and presents strategies for addressing typical constraints in this area.A Inteligência Artificial, auxiliada pelo deep learning, tem emergido em diversas áreas da nossa sociedade. Estes sistemas permitem a automatização e a melhoria de diversas tarefas, superando mesmo, em alguns casos, a capacidade humana. Os métodos de detecção de objetos são utilizados atualmente em diversas áreas, inclusive na análise de imagens médicas. No entanto, esses métodos são suscetíveis a erros e falta um método universalmente aceite que possa ser aplicado em todos os tipos de aplicações com a precisão necessária na área médica. Além disso, a aplicação de detectores de objetos na análise de imagens médicas ainda precisa ser analisada minuciosamente para alcançar uma compreensão mais rica do estado da arte. Para enfrentar essas limitações, apresentamos três estudos com objetivos distintos. Inicialmente, uma análise quantitativa e qualitativa da pesquisa acadêmica foi realizada para obter uma percepção de quais detectores de objetos são empregues, a modalidade de imagem médica usada e as partes específicas do corpo sob investigação. Num segundo estudo, propomos uma versão otimizada de um algoritmo amplamente utilizado para superar limitações comumente abordadas em imagens médicas por meio do ajuste fino de vários hiperparâmetros. Em terceiro lugar, desenvolvemos uma nova abordagem de stacking para aumentar a precisão das detecções na análise de imagens médicas. Os resultados demostram que, apesar da chegada tardia da detecção de objetos na análise de imagens médicas, o número de publicações aumentou nos últimos anos, evidenciando o significativo potencial de crescimento. Adicionalmente, estabelecemos que é possível resolver algumas restrições nos dados por meio de uma otimização exaustiva do algoritmo. Finalmente, o nosso último estudo destaca que ainda há espaço para melhorias nessas técnicas avançadas, usando, como exemplo, abordagens de stacking. As contribuições desta dissertação são várias, apresentando uma visão geral em maior detalhe das aplicações de ponta dos algoritmos de detecção de objetos na área médica e apresenta estratégias para lidar com restrições típicas nesta área