Machine Learning Methods for Medical and Biological Image Computing

Medical and biological imaging technologies provide valuable visualization information of structure and function for an organ from the level of individual molecules to the whole object. Brain is the most complex organ in body, and it increasingly attracts intense research attentions with the rapid development of medical and bio-logical imaging technologies. A massive amount of high-dimensional brain imaging data being generated makes the design of computational methods for efficient analysis on those images highly demanded. The current study of computational methods using hand-crafted features does not scale with the increasing number of brain images, hindering the pace of scientific discoveries in neuroscience. In this thesis, I propose computational methods using high-level features for automated analysis of brain images at different levels. At the brain function level, I develop a deep learning based framework for completing and integrating multi-modality neuroimaging data, which increases the diagnosis accuracy for Alzheimer’s disease. At the cellular level, I propose to use three dimensional convolutional neural networks (CNNs) for segmenting the volumetric neuronal images, which improves the performance of digital reconstruction of neuron structures. I design a novel CNN architecture such that the model training and testing image prediction can be implemented in an end-to-end manner. At the molecular level, I build a voxel CNN classifier to capture discriminative features of the input along three spatial dimensions, which facilitate the identification of secondary structures of proteins from electron microscopy im-ages. In order to classify genes specifically expressed in different brain cell-type, I propose to use invariant image feature descriptors to capture local gene expression information from cellular-resolution in situ hybridization images. I build image-level representations by applying regularized learning and vector quantization on generated image descriptors. The developed computational methods in this dissertation are evaluated using images from medical and biological experiments in comparison with baseline methods. Experimental results demonstrate that the developed representations, formulations, and algorithms are effective and efficient in learning from brain imaging data

Semi-automatic geometric digital twinning for existing buildings based on images and CAD drawings

Despite the emerging new data capturing technologies and advanced modelling systems, the process of geometric digital twin modelling for existing buildings still lacks a systematic and completed framework to streamline. As-is Building Information Model (BIM) is one of the commonly used geometric digital twin modelling approaches. However, the process of as-is BIM construction is time-consuming and needed to improve. To address this challenge, in this paper, a semi-automatic approach is developed to establish a systematic, accurate and convenient digital twinning system based on images and CAD drawings. With this ultimate goal, this paper summarises the state-of-the-art geometric digital twinning methods and elaborates on the methodological framework of this semi-automatic geometric digital twinning approach. The framework consists of three modules. The Building Framework Construction and Geometry Information Extraction (Module 1) defines the locations of each structural component through recognising special symbols in a floor plan and then extracting data from CAD drawings using the Optical Character Recognition (OCR) technology. Meaningful text information is further filtered based on predefined rules. In order to integrate with completed building information, the Building Information Complementary (Module 2) is developed based on neuro-fuzzy system (NFS) and the image processing procedure to supplement additional building components. Finally, the Information Integration and IFC Creation (Module 3) integrates information from Module 1 and 2 and creates as-is Industry Foundation Classes (IFC) BIM based on IFC schema. A case study using part of an office building and the results of its analysis are provided and discussed from the perspectives of applicability and accuracy. Future works and limitations are also addressed

Multimodal Data Analysis of Dyadic Interactions for an Automated Feedback System Supporting Parent Implementation of Pivotal Response Treatment

abstract: Parents fulfill a pivotal role in early childhood development of social and communication skills. In children with autism, the development of these skills can be delayed. Applied behavioral analysis (ABA) techniques have been created to aid in skill acquisition. Among these, pivotal response treatment (PRT) has been empirically shown to foster improvements. Research into PRT implementation has also shown that parents can be trained to be effective interventionists for their children. The current difficulty in PRT training is how to disseminate training to parents who need it, and how to support and motivate practitioners after training. Evaluation of the parents’ fidelity to implementation is often undertaken using video probes that depict the dyadic interaction occurring between the parent and the child during PRT sessions. These videos are time consuming for clinicians to process, and often result in only minimal feedback for the parents. Current trends in technology could be utilized to alleviate the manual cost of extracting data from the videos, affording greater opportunities for providing clinician created feedback as well as automated assessments. The naturalistic context of the video probes along with the dependence on ubiquitous recording devices creates a difficult scenario for classification tasks. The domain of the PRT video probes can be expected to have high levels of both aleatory and epistemic uncertainty. Addressing these challenges requires examination of the multimodal data along with implementation and evaluation of classification algorithms. This is explored through the use of a new dataset of PRT videos. The relationship between the parent and the clinician is important. The clinician can provide support and help build self-efficacy in addition to providing knowledge and modeling of treatment procedures. Facilitating this relationship along with automated feedback not only provides the opportunity to present expert feedback to the parent, but also allows the clinician to aid in personalizing the classification models. By utilizing a human-in-the-loop framework, clinicians can aid in addressing the uncertainty in the classification models by providing additional labeled samples. This will allow the system to improve classification and provides a person-centered approach to extracting multimodal data from PRT video probes.Dissertation/ThesisDoctoral Dissertation Computer Science 201

Counting the Number of Active Spermatozoa Movements Using Improvement Adaptive Background Learning Algorithm

The most important early stage in sperm infertility research is the detection of sperm objects. The success rate in separating sperm objects from semen fluid has an important role for further analysis. This research performed the detection and calculation of human spermatozoa. The detected sperm was the moving sperm in the video data. An improvement of Adaptive Background Learning was applied to detect the moving sperm. The purpose of this method is to improve the performance of Adaptive Background Learning algorithm in background subtraction process to detect and calculate moving sperm on the microscopic video of sperm fluid. This paper also compared several other background subtraction algorithms to conclude the appropriate background subtraction algorithm for sperm detection and sperm counting. The process done in this research was preprocessing using the Gaussian filter. The next was background subtraction process, followed by morphology operation. To test or validate the detection results of any background subtraction algorithm used, the foreground mask results from the morphological operation were compared to the ground truth of moving sperm image. For visualization purposes, every BLOB area (white object in binary image) on the foreground were given a bounding box to the original frame and the number of BLOB objects present in the foreground mask were counted. This shows that the system had been able to detect and calculate moving sperm. Based on the test results, Adaptive Background Learning method had a value of F-measure of 0.9205 and succeeded in extracting sperm shape close to the original form compared to other methods

Recognising Complex Mental States from Naturalistic Human-Computer Interactions

New advances in computer vision techniques will revolutionize the way we interact with computers, as they, together with other improvements, will help us build machines that understand us better. The face is the main non-verbal channel for human-human communication and contains valuable information about emotion, mood, and mental state. Affective computing researchers have investigated widely how facial expressions can be used for automatically recognizing affect and mental states. Nowadays, physiological signals can be measured by video-based techniques, which can also be utilised for emotion detection. Physiological signals, are an important indicator of internal feelings, and are more robust against social masking. This thesis focuses on computer vision techniques to detect facial expression and physiological changes for recognizing non-basic and natural emotions during human-computer interaction. It covers all stages of the research process from data acquisition, integration and application. Most previous studies focused on acquiring data from prototypic basic emotions acted out under laboratory conditions. To evaluate the proposed method under more practical conditions, two different scenarios were used for data collection. In the first scenario, a set of controlled stimulus was used to trigger the user’s emotion. The second scenario aimed at capturing more naturalistic emotions that might occur during a writing activity. In the second scenario, the engagement level of the participants with other affective states was the target of the system. For the first time this thesis explores how video-based physiological measures can be used in affect detection. Video-based measuring of physiological signals is a new technique that needs more improvement to be used in practical applications. A machine learning approach is proposed and evaluated to improve the accuracy of heart rate (HR) measurement using an ordinary camera during a naturalistic interaction with computer

Web page cleaning for web mining

Ph.DDOCTOR OF PHILOSOPH

Recognising Complex Mental States from Naturalistic Human-Computer Interactions

New advances in computer vision techniques will revolutionize the way we interact with computers, as they, together with other improvements, will help us build machines that understand us better. The face is the main non-verbal channel for human-human communication and contains valuable information about emotion, mood, and mental state. Affective computing researchers have investigated widely how facial expressions can be used for automatically recognizing affect and mental states. Nowadays, physiological signals can be measured by video-based techniques, which can also be utilised for emotion detection. Physiological signals, are an important indicator of internal feelings, and are more robust against social masking. This thesis focuses on computer vision techniques to detect facial expression and physiological changes for recognizing non-basic and natural emotions during human-computer interaction. It covers all stages of the research process from data acquisition, integration and application. Most previous studies focused on acquiring data from prototypic basic emotions acted out under laboratory conditions. To evaluate the proposed method under more practical conditions, two different scenarios were used for data collection. In the first scenario, a set of controlled stimulus was used to trigger the user’s emotion. The second scenario aimed at capturing more naturalistic emotions that might occur during a writing activity. In the second scenario, the engagement level of the participants with other affective states was the target of the system. For the first time this thesis explores how video-based physiological measures can be used in affect detection. Video-based measuring of physiological signals is a new technique that needs more improvement to be used in practical applications. A machine learning approach is proposed and evaluated to improve the accuracy of heart rate (HR) measurement using an ordinary camera during a naturalistic interaction with computer