Recurrent Connections Aid Occluded Object Recognition by Discounting Occluders

Recurrent connections in the visual cortex are thought to aid object recognition when part of the stimulus is occluded. Here we investigate if and how recurrent connections in artificial neural networks similarly aid object recognition. We systematically test and compare architectures comprised of bottom-up (B), lateral (L) and top-down (T) connections. Performance is evaluated on a novel stereoscopic occluded object recognition dataset. The task consists of recognizing one target digit occluded by multiple occluder digits in a pseudo-3D environment. We find that recurrent models perform significantly better than their feedforward counterparts, which were matched in parametric complexity. Furthermore, we analyze how the network's representation of the stimuli evolves over time due to recurrent connections. We show that the recurrent connections tend to move the network's representation of an occluded digit towards its un-occluded version. Our results suggest that both the brain and artificial neural networks can exploit recurrent connectivity to aid occluded object recognition.Comment: 13 pages, 5 figures, accepted at the 28th International Conference on Artificial Neural Networks, published in Springer Lecture Notes in Computer Science vol 1172

Decoding the Recognition of Occluded Objects in the Human Brain

The dynamics of object recognition are intricate, particularly when under challenging visual conditions, such as occlusion. Current models of vision often fall short in explaining the human visual system's remarkable ability to represent occluded objects. Previous studies have predominantly employed simple shapes as occluders, limiting the understanding of real-world occlusion scenarios. Chapter 2 delves into neural representations by investigating occlusion with realistic stimuli—objects occluding other objects. Using event-related fMRI, participants engaged in a one-back task while viewing objects in isolation, occluded by other objects, or cut out by silhouettes. Decoding analyses in early visual cortex (EVC) revealed a reliance on visible features, while inferotemporal cortex (IT) exhibited robust representations, incorporating both visible and inferred features. Competition effects across multiple objects were evident in EVC but notably weaker in IT, highlighting IT's capacity to disentangle neural responses amidst competing stimuli. Chapter 3 expands the exploration to behavioural aspects, unveiling the impact of occlusion magnitude on recognition difficulty. IT displayed a linear increase of beta weights in processing allocation with recognition difficulty. Behaviourally, unoccluded conditions showed enhanced accuracy and faster response times, with unique recognition patterns emerging when objects served as both occluders and occluded objects. Chapter 4 uses fMRI to examine the theoretical perspective of predictive processing, employing expectation suppression in EVC during occlusion, motivated by high-level occlusion responses found previously. Multivariate pattern analysis indicated an expectation suppression effect aligning with the sharpening account of predictive processing. The concluding chapter synthesises these findings, emphasising the practical and theoretical implications. Notably, the thesis underscores the importance of utilising ecological visual information in visual neuroscience studies and highlights the differing capabilities of EVC and IT. Collectively, our research contributes valuable insights into the neural mechanisms underlying object recognition in challenging visual conditions, paving the way for future research avenues

Replacing Backpropagation with Biological Plausible Top-down Credit Assignment in Deep Neural Networks Training

Top-down connections in the biological brain has been shown to be important in high cognitive functions. However, the function of this mechanism in machine learning has not been defined clearly. In this study, we propose to lay out a framework constituted by a bottom-up and a top-down network. Here, we use a Top-down Credit Assignment Network (TDCA-network) to replace the loss function and back propagation (BP) which serve as the feedback mechanism in traditional bottom-up network training paradigm. Our results show that the credit given by well-trained TDCA-network outperforms the gradient from backpropagation in classification task under different settings on multiple datasets. In addition, we successfully use a credit diffusing trick, which can keep training and testing performance remain unchanged, to reduce parameter complexity of the TDCA-network. More importantly, by comparing their trajectories in the parameter landscape, we find that TDCA-network directly achieved a global optimum, in contrast to that backpropagation only can gain a localized optimum. Thus, our results demonstrate that TDCA-network not only provide a biological plausible learning mechanism, but also has the potential to directly achieve global optimum, indicating that top-down credit assignment can substitute backpropagation, and provide a better learning framework for Deep Neural Networks

Recurrence is required to capture the representational dynamics of the human visual system.

The human visual system is an intricate network of brain regions that enables us to recognize the world around us. Despite its abundant lateral and feedback connections, object processing is commonly viewed and studied as a feedforward process. Here, we measure and model the rapid representational dynamics across multiple stages of the human ventral stream using time-resolved brain imaging and deep learning. We observe substantial representational transformations during the first 300 ms of processing within and across ventral-stream regions. Categorical divisions emerge in sequence, cascading forward and in reverse across regions, and Granger causality analysis suggests bidirectional information flow between regions. Finally, recurrent deep neural network models clearly outperform parameter-matched feedforward models in terms of their ability to capture the multiregion cortical dynamics. Targeted virtual cooling experiments on the recurrent deep network models further substantiate the importance of their lateral and top-down connections. These results establish that recurrent models are required to understand information processing in the human ventral stream

Automated identification and behaviour classification for modelling social dynamics in group-housed mice

Mice are often used in biology as exploratory models of human conditions, due to their similar genetics and physiology. Unfortunately, research on behaviour has traditionally been limited to studying individuals in isolated environments and over short periods of time. This can miss critical time-effects, and, since mice are social creatures, bias results. This work addresses this gap in research by developing tools to analyse the individual behaviour of group-housed mice in the home-cage over several days and with minimal disruption. Using data provided by the Mary Lyon Centre at MRC Harwell we designed an end-to-end system that (a) tracks and identifies mice in a cage, (b) infers their behaviour, and subsequently (c) models the group dynamics as functions of individual activities. In support of the above, we also curated and made available a large dataset of mouse localisation and behaviour classifications (IMADGE), as well as two smaller annotated datasets for training/evaluating the identification (TIDe) and behaviour inference (ABODe) systems. This research constitutes the first of its kind in terms of the scale and challenges addressed. The data source (side-view single-channel video with clutter and no identification markers for mice) presents challenging conditions for analysis, but has the potential to give richer information while using industry standard housing. A Tracking and Identification module was developed to automatically detect, track and identify the (visually similar) mice in the cluttered home-cage using only single-channel IR video and coarse position from RFID readings. Existing detectors and trackers were combined with a novel Integer Linear Programming formulation to assign anonymous tracks to mouse identities. This utilised a probabilistic weight model of affinity between detections and RFID pickups. The next task necessitated the implementation of the Activity Labelling module that classifies the behaviour of each mouse, handling occlusion to avoid giving unreliable classifications when the mice cannot be observed. Two key aspects of this were (a) careful feature-selection, and (b) judicious balancing of the errors of the system in line with the repercussions for our setup. Given these sequences of individual behaviours, we analysed the interaction dynamics between mice in the same cage by collapsing the group behaviour into a sequence of interpretable latent regimes using both static and temporal (Markov) models. Using a permutation matrix, we were able to automatically assign mice to roles in the HMM, fit a global model to a group of cages and analyse abnormalities in data from a different demographic