Reconhecimento automático de moedas medievais usando visão por computador

Dissertação de mestrado em Engenharia InformáticaThe use of computer vision for identification and recognition of coins is well studied and of renowned interest. However the focus of research has consistently been on modern coins and the used algorithms present quite disappointing results when applied to ancient coins. This discrepancy is explained by the nature of ancient coins that are manually minted, having plenty variances, failures, ripples and centuries of degradation which further deform the characteristic patterns, making their identification a hard task even for humans. Another noteworthy factor in almost all similar studies is the controlled environments and uniform illumination of all images of the datasets. Though it makes sense to focus on the more problematic variables, this is an impossible premise to find outside the researchers’ laboratory, therefore a problematic that must be approached. This dissertation focuses on medieval and ancient coin recognition in uncontrolled “real world” images, thus trying to pave way to the use of vast repositories of coin images all over the internet that could be used to make our algorithms more robust. The first part of the dissertation proposes a fast and automatic method to segment ancient coins over complex backgrounds using a Histogram Backprojection approach combined with edge detection methods. Results are compared against an automation of GrabCut algorithm. The proposed method achieves a Good or Acceptable rate on 76% of the images, taking an average of 0.29s per image, against 49% in 19.58s for GrabCut. Although this work is oriented to ancient coin segmentation, the method can also be used in other contexts presenting thin objects with uniform colors. In the second part, several state of the art machine learning algorithms are compared in the search for the most promising approach to classify these challenging coins. The best results are achieved using dense SIFT descriptors organized into Bags of Visual Words, and using Support Vector Machine or Naïve Bayes as machine learning strategies.O uso de visão por computador para identificação e reconhecimento de moedas é bastante estudado e de reconhecido interesse. No entanto o foco da investigação tem sido sistematicamente sobre as moedas modernas e os algoritmos usados apresentam resultados bastante desapontantes quando aplicados a moedas antigas. Esta discrepância é justificada pela natureza das moedas antigas que, sendo cunhadas à mão, apresentam bastantes variações, falhas e séculos de degradação que deformam os padrões característicos, tornando a sua identificação dificil mesmo para o ser humano. Adicionalmente, a quase totalidade dos estudos usa ambientes controlados e iluminação uniformizada entre todas as imagens dos datasets. Embora faça sentido focar-se nas variáveis mais problemáticas, esta é uma premissa impossível de encontrar fora do laboratório do investigador e portanto uma problemática que tem que ser estudada. Esta dissertação foca-se no reconhecimento de moedas medievais e clássicas em imagens não controladas, tentando assim abrir caminho ao uso de vastos repositórios de imagens de moedas disponíveis na internet, que poderiam ser usados para tornar os nossos algoritmos mais robustos. Na primeira parte é proposto um método rápido e automático para segmentar moedas antigas sobre fundos complexos, numa abordagem que envolve Histogram Backprojection combinado com deteção de arestas. Os resultados são comparados com uma automação do algoritmo GrabCut. O método proposto obtém uma classificação de Bom ou Aceitável em 76% das imagens, demorando uma média de 0.29s por imagem, contra 49% em 19,58s do GrabCut. Não obstante o foco em segmentação de moedas antigas, este método pode ser usado noutros contextos que incluam objetos planos de cor uniforme. Na segunda parte, o estado da arte de Machine Learning é testado e comparado em busca da abordagem mais promissora para classificar estas moedas. Os melhores resultados são alcançados usando descritores dense SIFT, organizados em Bags of Visual Words e usando Support Vector Machine ou Naive Bayes como estratégias de machine learning

Measurement, optimisation and control of particle properties in pharmaceutical manufacturing processes

Previously held under moratorium from 2 June 2020 until 6 June 2022.The understanding and optimisation of particle properties connected to their structure and morphology is a common objective for particle engineering applications either to improve materialhandling in the manufacturing process or to influence Critical Quality Attributes (CQAs) linked to product performance. This work aims to demonstrate experimental means to support a rational development approach for pharmaceutical particulate systems with a specific focus on droplet drying platforms such as spray drying. Micro-X-ray tomography (micro-XRT) is widely applied in areas such as geo- and biomedical sciences to enable a three dimensional investigation of the specimens. Chapter 4 elaborates on practical aspects of micro-XRT for a quantitative analysis of pharmaceutical solid products with an emphasis on implemented image processing and analysis methodologies. Potential applications of micro-XRT in the pharmaceutical manufacturing process can range from the characterisation of single crystals to fully formulated oral dosage forms. Extracted quantitative information can be utilised to directly inform product design and production for process development or optimisation. The non-destructive nature of the micro-XRT analysis can be further employed to investigate structure-performance relationships which might provide valuable insights for modelling approaches. Chapter 5 further demonstrates the applicability of micro-XRT for the analysis of ibuprofen capsules as a multi-particulate system each with a population of approximately 300 pellets. The in-depth analysis of collected micro-XRT image data allowed the extraction of more than 200 features quantifying aspects of the pellets’ size, shape, porosity, surface and orientation. Employed feature selection and machine learning methods enabled the detection of broken pellets within a classification model. The classification model has an accuracy of more than 99.55% and a minimum precision of 86.20% validated with a test dataset of 886 pellets from three capsules. The combination of single droplet drying (SDD) experiments with a subsequent micro-XRT analysis was used for a quantitative investigation of the particle design space and is described in Chapter 6. The implemented platform was applied to investigate the solidification of formulated metformin hydrochloride particles using D-mannitol and hydroxypropyl methylcellulose within a selected, pragmatic particle design space. The results indicate a significant impact of hydroxypropyl methylcellulose reducing liquid evaporation rates and particle drying kinetics. The morphology and internal structure of the formulated particles after drying are dominated by a crystalline core of D-mannitol partially suppressed with increasing hydroxypropyl methylcellulose additions. The characterisation of formulated metformin hydrochloride particles with increasing polymer content demonstrated the importance of an early-stage quantitative assessment of formulation-related particle properties. A reliable and rational spray drying development approach needs to assess parameters of the compound system as well as of the process itself in order to define a well-controlled and robust operational design space. Chapter 7 presents strategies for process implementation to produce peptide-based formulations via spray drying demonstrated using s-glucagon as a model peptide. The process implementation was supported by an initial characterisation of the lab-scale spray dryer assessing a range of relevant independent process variables including drying temperature and feed rate. The platform response was captured with available and in-house developed Process Analytical Technology. A B-290 Mini-Spray Dryer was used to verify the development approach and to implement the pre-designed spray drying process. Information on the particle formation mechanism observed in SDD experiments were utilised to interpret the characteristics of the spray dried material.The understanding and optimisation of particle properties connected to their structure and morphology is a common objective for particle engineering applications either to improve materialhandling in the manufacturing process or to influence Critical Quality Attributes (CQAs) linked to product performance. This work aims to demonstrate experimental means to support a rational development approach for pharmaceutical particulate systems with a specific focus on droplet drying platforms such as spray drying. Micro-X-ray tomography (micro-XRT) is widely applied in areas such as geo- and biomedical sciences to enable a three dimensional investigation of the specimens. Chapter 4 elaborates on practical aspects of micro-XRT for a quantitative analysis of pharmaceutical solid products with an emphasis on implemented image processing and analysis methodologies. Potential applications of micro-XRT in the pharmaceutical manufacturing process can range from the characterisation of single crystals to fully formulated oral dosage forms. Extracted quantitative information can be utilised to directly inform product design and production for process development or optimisation. The non-destructive nature of the micro-XRT analysis can be further employed to investigate structure-performance relationships which might provide valuable insights for modelling approaches. Chapter 5 further demonstrates the applicability of micro-XRT for the analysis of ibuprofen capsules as a multi-particulate system each with a population of approximately 300 pellets. The in-depth analysis of collected micro-XRT image data allowed the extraction of more than 200 features quantifying aspects of the pellets’ size, shape, porosity, surface and orientation. Employed feature selection and machine learning methods enabled the detection of broken pellets within a classification model. The classification model has an accuracy of more than 99.55% and a minimum precision of 86.20% validated with a test dataset of 886 pellets from three capsules. The combination of single droplet drying (SDD) experiments with a subsequent micro-XRT analysis was used for a quantitative investigation of the particle design space and is described in Chapter 6. The implemented platform was applied to investigate the solidification of formulated metformin hydrochloride particles using D-mannitol and hydroxypropyl methylcellulose within a selected, pragmatic particle design space. The results indicate a significant impact of hydroxypropyl methylcellulose reducing liquid evaporation rates and particle drying kinetics. The morphology and internal structure of the formulated particles after drying are dominated by a crystalline core of D-mannitol partially suppressed with increasing hydroxypropyl methylcellulose additions. The characterisation of formulated metformin hydrochloride particles with increasing polymer content demonstrated the importance of an early-stage quantitative assessment of formulation-related particle properties. A reliable and rational spray drying development approach needs to assess parameters of the compound system as well as of the process itself in order to define a well-controlled and robust operational design space. Chapter 7 presents strategies for process implementation to produce peptide-based formulations via spray drying demonstrated using s-glucagon as a model peptide. The process implementation was supported by an initial characterisation of the lab-scale spray dryer assessing a range of relevant independent process variables including drying temperature and feed rate. The platform response was captured with available and in-house developed Process Analytical Technology. A B-290 Mini-Spray Dryer was used to verify the development approach and to implement the pre-designed spray drying process. Information on the particle formation mechanism observed in SDD experiments were utilised to interpret the characteristics of the spray dried material

LF-Net: Learning Local Features from Images

We present a novel deep architecture and a training strategy to learn a local feature pipeline from scratch, using collections of images without the need for human supervision. To do so we exploit depth and relative camera pose cues to create a virtual target that the network should achieve on one image, provided the outputs of the network for the other image. While this process is inherently non-differentiable, we show that we can optimize the network in a two-branch setup by confining it to one branch, while preserving differentiability in the other. We train our method on both indoor and outdoor datasets, with depth data from 3D sensors for the former, and depth estimates from an off-the-shelf Structure-from-Motion solution for the latter. Our models outperform the state of the art on sparse feature matching on both datasets, while running at 60+ fps for QVGA images.Comment: NIPS 201

Monocular 3d Object Recognition

Object recognition is one of the fundamental tasks of computer vision. Recent advances in the field enable reliable 2D detections from a single cluttered image. However, many challenges still remain. Object detection needs timely response for real world applications. Moreover, we are genuinely interested in estimating the 3D pose and shape of an object or human for the sake of robotic manipulation and human-robot interaction. In this thesis, a suite of solutions to these challenges is presented. First, Active Deformable Part Models (ADPM) is proposed for fast part-based object detection. ADPM dramatically accelerates the detection by dynamically scheduling the part evaluations and efficiently pruning the image locations. Second, we unleash the power of marrying discriminative 2D parts with an explicit 3D geometric representation. Several methods of such scheme are proposed for recovering rich 3D information of both rigid and non-rigid objects from monocular RGB images. (1) The accurate 3D pose of an object instance is recovered from cluttered images using only the CAD model. (2) A global optimal solution for simultaneous 2D part localization, 3D pose and shape estimation is obtained by optimizing a unified convex objective function. Both appearance and geometric compatibility are jointly maximized. (3) 3D human pose estimation from an image sequence is realized via an Expectation-Maximization algorithm. The 2D joint location uncertainties are marginalized out during inference and 3D pose smoothness is enforced across frames. By bridging the gap between 2D and 3D, our methods provide an end-to-end solution to 3D object recognition from images. We demonstrate a range of interesting applications using only a single image or a monocular video, including autonomous robotic grasping with a single image, 3D object image pop-up and a monocular human MoCap system. We also show empirical start-of-art results on a number of benchmarks on 2D detection and 3D pose and shape estimation

Gaussian mixture model classifiers for detection and tracking in UAV video streams.

Masters Degree. University of KwaZulu-Natal, Durban.Manual visual surveillance systems are subject to a high degree of human-error and operator fatigue. The automation of such systems often employs detectors, trackers and classifiers as fundamental building blocks. Detection, tracking and classification are especially useful and challenging in Unmanned Aerial Vehicle (UAV) based surveillance systems. Previous solutions have addressed challenges via complex classification methods. This dissertation proposes less complex Gaussian Mixture Model (GMM) based classifiers that can simplify the process; where data is represented as a reduced set of model parameters, and classification is performed in the low dimensionality parameter-space. The specification and adoption of GMM based classifiers on the UAV visual tracking feature space formed the principal contribution of the work. This methodology can be generalised to other feature spaces. This dissertation presents two main contributions in the form of submissions to ISI accredited journals. In the first paper, objectives are demonstrated with a vehicle detector incorporating a two stage GMM classifier, applied to a single feature space, namely Histogram of Oriented Gradients (HoG). While the second paper demonstrates objectives with a vehicle tracker using colour histograms (in RGB and HSV), with Gaussian Mixture Model (GMM) classifiers and a Kalman filter. The proposed works are comparable to related works with testing performed on benchmark datasets. In the tracking domain for such platforms, tracking alone is insufficient. Adaptive detection and classification can assist in search space reduction, building of knowledge priors and improved target representations. Results show that the proposed approach improves performance and robustness. Findings also indicate potential further enhancements such as a multi-mode tracker with global and local tracking based on a combination of both papers

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

Image Processing and Analysis for Preclinical and Clinical Applications

Radiomics is one of the most successful branches of research in the field of image processing and analysis, as it provides valuable quantitative information for the personalized medicine. It has the potential to discover features of the disease that cannot be appreciated with the naked eye in both preclinical and clinical studies. In general, all quantitative approaches based on biomedical images, such as positron emission tomography (PET), computed tomography (CT) and magnetic resonance imaging (MRI), have a positive clinical impact in the detection of biological processes and diseases as well as in predicting response to treatment. This Special Issue, “Image Processing and Analysis for Preclinical and Clinical Applications”, addresses some gaps in this field to improve the quality of research in the clinical and preclinical environment. It consists of fourteen peer-reviewed papers covering a range of topics and applications related to biomedical image processing and analysis

Change blindness: eradication of gestalt strategies

Arrays of eight, texture-defined rectangles were used as stimuli in a one-shot change blindness (CB) task where there was a 50% chance that one rectangle would change orientation between two successive presentations separated by an interval. CB was eliminated by cueing the target rectangle in the first stimulus, reduced by cueing in the interval and unaffected by cueing in the second presentation. This supports the idea that a representation was formed that persisted through the interval before being 'overwritten' by the second presentation (Landman et al, 2003 Vision Research 43149–164]. Another possibility is that participants used some kind of grouping or Gestalt strategy. To test this we changed the spatial position of the rectangles in the second presentation by shifting them along imaginary spokes (by ±1 degree) emanating from the central fixation point. There was no significant difference seen in performance between this and the standard task [F(1,4)=2.565, p=0.185]. This may suggest two things: (i) Gestalt grouping is not used as a strategy in these tasks, and (ii) it gives further weight to the argument that objects may be stored and retrieved from a pre-attentional store during this task