Digital Oculomotor Biomarkers in Dementia

Dementia is an umbrella term that covers a number of neurodegenerative syndromes featuring gradual disturbance of various cognitive functions that are severe enough to interfere with tasks of daily life. The diagnosis of dementia occurs frequently when pathological changes have been developing for years, symptoms of cognitive impairment are evident and the quality of life of the patients has already been deteriorated significantly. Although brain imaging and fluid biomarkers allow the monitoring of disease progression in vivo, they are expensive, invasive and not necessarily diagnostic in isolation. Recent studies suggest that eye-tracking technology is an innovative tool that holds promise for accelerating early detection of the disease, as well as, supporting the development of strategies that minimise impairment during every day activities. However, the optimal methods for quantitative evaluation of oculomotor behaviour during complex and naturalistic tasks in dementia have yet to be determined. This thesis investigates the development of computational tools and techniques to analyse eye movements of dementia patients and healthy controls under naturalistic and less constrained scenarios to identify novel digital oculomotor biomarkers. Three key contributions are made. First, the evaluation of the role of environment during navigation in patients with typical Alzheimer disease and Posterior Cortical Atrophy compared to a control group using a combination of eye movement and egocentric video analysis. Secondly, the development of a novel method of extracting salient features directly from the raw eye-tracking data of a mixed sample of dementia patients during a novel instruction-less cognitive test to detect oculomotor biomarkers of dementia-related cognitive dysfunction. Third, the application of unsupervised anomaly detection techniques for visualisation of oculomotor anomalies during various cognitive tasks. The work presented in this thesis furthers our understanding of dementia-related oculomotor dysfunction and gives future research direction for the development of computerised cognitive tests and ecological interventions

Deep Learning for Crowd Anomaly Detection

Today, public areas across the globe are monitored by an increasing amount of surveillance cameras. This widespread usage has presented an ever-growing volume of data that cannot realistically be examined in real-time. Therefore, efforts to understand crowd dynamics have brought light to automatic systems for the detection of anomalies in crowds. This thesis explores the methods used across literature for this purpose, with a focus on those fusing dense optical flow in a feature extraction stage to the crowd anomaly detection problem. To this extent, five different deep learning architectures are trained using optical flow maps estimated by three deep learning-based techniques. More specifically, a 2D convolutional network, a 3D convolutional network, and LSTM-based convolutional recurrent network, a pre-trained variant of the latter, and a ConvLSTM-based autoencoder is trained using both regular frames and optical flow maps estimated by LiteFlowNet3, RAFT, and GMA on the UCSD Pedestrian 1 dataset. The experimental results have shown that while prone to overfitting, the use of optical flow maps may improve the performance of supervised spatio-temporal architectures

AI-based framework for automatically extracting high-low features from NDS data to understand driver behavior

Our ability to detect and characterize unsafe driving behaviors in naturalistic driving environments and associate them with road crashes will be a significant step toward developing effective crash countermeasures. Due to some limitations, researchers have not yet fully achieved the stated goal of characterizing unsafe driving behaviors. These limitations include, but are not limited to, the high cost of data collection and the manual processes required to extract information from NDS data. In light of this limitations, the primary objective of this study is to develop an artificial intelligence (AI) framework for automatically extracting high-low features from the NDS dataset to explain driver behavior using a low-cost data collection method. The author proposed three novel objectives for achieving the study's objective in light of the identified research gaps. Initially, the study develops a low-cost data acquisition system for gathering data on naturalistic driving. Second, the study develops a framework that automatically extracts high- to low-level features, such as vehicle density, turning movements, and lane changes, from the data collected by the developed data acquisition system. Thirdly, the study extracted information from the NDS data to gain a better understanding of people's car-following behavior and other driving behaviors in order to develop countermeasures for traffic safety through data collection and analysis. The first objective of this study is to develop a multifunctional smartphone application for collecting NDS data. Three major modules comprised the designed app: a front-end user interface module, a sensor module, and a backend module. The front-end, which is also the application's user interface, was created to provide a streamlined view that exposed the application's key features via a tab bar controller. This allows us to compartmentalize the application's critical components into separate views. The backend module provides computational resources that can be used to accelerate front-end query responses. Google Firebase powered the backend of the developed application. The sensor modules included CoreMotion, CoreLocation, and AVKit. CoreMotion collects motion and environmental data from the onboard hardware of iOS devices, including accelerometers, gyroscopes, pedometers, magnetometers, and barometers. In contrast, CoreLocation determines the altitude, orientation, and geographical location of a device, as well as its position relative to an adjacent iBeacon device. The AVKit finally provides a high-level interface for video content playback. To achieve objective two, we formulated the problem as both a classification and time-series segmentation problem. This is due to the fact that the majority of existing driver maneuver detection methods formulate the problem as a pure classification problem, assuming a discretized input signal with known start and end locations for each event or segment. In practice, however, vehicle telemetry data used for detecting driver maneuvers are continuous; thus, a fully automated driver maneuver detection system should incorporate solutions for both time series segmentation and classification. The five stages of our proposed methodology are as follows: 1) data preprocessing, 2) segmentation of events, 3) machine learning classification, 4) heuristics classification, and 5) frame-by-frame video annotation. The result of the study indicates that the gyroscope reading is an exceptional parameter for extracting driving events, as its accuracy was consistent across all four models developed. The study reveals that the Energy Maximization Algorithm's accuracy ranges from 56.80 percent (left lane change) to 85.20 percent (right lane change) (lane-keeping) All four models developed had comparable accuracies to studies that used similar models. The 1D-CNN model had the highest accuracy (98.99 percent), followed by the LSTM model (97.75 percent), the RF model (97.71 percent), and the SVM model (97.65 percent). To serve as a ground truth, continuous signal data was annotated. In addition, the proposed method outperformed the fixed time window approach. The study analyzed the overall pipeline's accuracy by penalizing the F1 scores of the ML models with the EMA's duration score. The pipeline's accuracy ranged between 56.8 percent and 85.0 percent overall. The ultimate goal of this study was to extract variables from naturalistic driving videos that would facilitate an understanding of driver behavior in a naturalistic driving environment. To achieve this objective, three sub-goals were established. First, we developed a framework for extracting features pertinent to comprehending the behavior of natural-environment drivers. Using the extracted features, we then analyzed the car-following behaviors of various demographic groups. Thirdly, using a machine learning algorithm, we modeled the acceleration of both the ego-vehicle and the leading vehicle. Younger drivers are more likely to be aggressive, according to the findings of this study. In addition, the study revealed that drivers tend to accelerate when the distance between them and the vehicle in front of them is substantial. Lastly, compared to younger drivers, elderly motorists maintain a significantly larger following distance. This study's results have numerous safety implications. First, the analysis of the driving behavior of different demographic groups will enable safety engineers to develop the most effective crash countermeasures by enhancing their understanding of the driving styles of different demographic groups and the causes of collisions. Second, the models developed to predict the acceleration of both the ego-vehicle and the leading vehicle will provide enough information to explain the behavior of the ego-driver.Includes bibliographical references

Improving neural network representations using human similarity judgments

Deep neural networks have reached human-level performance on many computer vision tasks. However, the objectives used to train these networks enforce only that similar images are embedded at similar locations in the representation space, and do not directly constrain the global structure of the resulting space. Here, we explore the impact of supervising this global structure by linearly aligning it with human similarity judgments. We find that a naive approach leads to large changes in local representational structure that harm downstream performance. Thus, we propose a novel method that aligns the global structure of representations while preserving their local structure. This global-local transform considerably improves accuracy across a variety of few-shot learning and anomaly detection tasks. Our results indicate that human visual representations are globally organized in a way that facilitates learning from few examples, and incorporating this global structure into neural network representations improves performance on downstream tasks.Comment: Published as a conference paper at NeurIPS 202

I expect, therefore I see: individual differences in visual awareness

Predictive processing theories posit that awareness of the visual world emerges as the brain engages in predictive inference about the causes of its sensory input. At each level of the processing hierarchy top-down predictions are corrected by bottom-up sensory prediction error to form behaviourally optimal inferences about the state of the visual world. Research suggests there may be individual differences in predictive processing mechanisms such that some individuals are more reliant on prior knowledge, whereas others assign more weight to sensory evidence. Predictive processing biases are thought to manifest in a range of typical and atypical perceptual experiences including proneness to perceptual illusions, sensory sensitivity in autism, and hallucinations in psychosis. The overarching aim of this thesis was to investigate whether in the general population predictive processing biases predict individual differences in visual awareness. Change blindness was selected as the central paradigm of investigation, as it can be conceptualised as a failure to incorporate a novel change into the current prediction about the state of the visual world. The empirical work in Chapter 2 aimed to characterise individual differences in visual change detection using naturalistic scenes and to identify the perceptual and cognitive measures that predict noticing ability. There were reliable individual differences in change detection that generalised to ecologically valid displays. The ability to notice visual changes was predicted by the strength and stability of perceptual predictions, as measured by the accuracy of visual short-term memory and attentional control in the face of distractors. In Chapter 3 I used voxel-based-morphometry to investigate whether inter-individual variability in brain structure predicts individual differences in visual awareness. The latter was assessed by the change blindness task as well as its strongest predictor measures (visual short-term memory, attentional capture, and perceptual rivalry). Regions of interest (ROIs) were selected in the parietal and visual cortices based on previous evidence that these areas are causally involved in the awareness of visual stimuli. This study aimed to discover whether the average grey matter density in the ROIs predict susceptibility to CB. The ROI-based analyses revealed the average grey matter density in left posterior parietal cortex predicted visual short-term memory accuracy but none of the other hypothesised relationships were significant. Chapter 4 aimed to measure individual differences in the reliance on prior knowledge by employing the Mooney face detection task. In this task participants disambiguated faces in two-tone degraded images before and after the presentation of the original versions of the images. Better change detection was predicted by Mooney face detection without any prior knowledge of the images, a measure of ‘perceptual closure’ or an ability to generate a gestalt of a scene. The attention to detail subscale of the autism spectrum also predicted superior change detection. Reliance on prior knowledge in visual perception (assessed by improvement in Mooney face detection after seeing original images) did not consistently predict atypical perceptual experiences associated with the autism spectrum or schizotypy. Chapter 5 was an investigation into, firstly, whether there is a general predictive processing bias, which manifests across different methods of inducing prior knowledge, or whether such a bias is paradigm-specific and, secondly, whether reliance on priors predicts perceptual experiences and traits. All prior manipulations in this study lead to an increased tendency to see the expected stimulus in a binocular rivalry display, except adaptation, which lead to a suppression of visual awareness. Attentional control, perceptual priming, expectancy, and imagery loaded onto a common factor, suggesting that the strength of selective attention is closely linked with the facilitatory effect of expectation. The strength of adaptation predicted superior change detection and perceptual priming predicted the propensity to experience perceptual illusions. Taken together, these findings suggest that there are reliable individual differences in visual change detection, and these are predicted by the strength of visual short-term memory representations, attentional control, perceptual closure ability, as well as the strength of low-level adaptation. Possessing expectations facilitates the entry of the corresponding percept into awareness, irrespective of the method of prior induction. The facilitatory effect that priors exert on visual awareness across different methods is closely linked with the ability to exert attentional control. This suggests that the effects of expectations on awareness may be attentional. However, predictive processing biases were method-specific in that a facilitatory effect using one prior induction method will not necessarily predict the magnitude of the effect using a different method. Some prior effects (e.g., perceptual priming, imagery, and adaptation) yielded correlations with perceptual experiences and traits in the general population. As the research in this thesis is correlational, future studies will need to delineate the effects of expectation, attention, and adaptation on visual awareness and explore the neural representations of these mechanisms