Effect of top-down connections in Hierarchical Sparse Coding

Hierarchical Sparse Coding (HSC) is a powerful model to efficiently represent multi-dimensional, structured data such as images. The simplest solution to solve this computationally hard problem is to decompose it into independent layer-wise subproblems. However, neuroscientific evidence would suggest inter-connecting these subproblems as in the Predictive Coding (PC) theory, which adds top-down connections between consecutive layers. In this study, a new model called 2-Layers Sparse Predictive Coding (2L-SPC) is introduced to assess the impact of this inter-layer feedback connection. In particular, the 2L-SPC is compared with a Hierarchical Lasso (Hi-La) network made out of a sequence of independent Lasso layers. The 2L-SPC and the 2-layers Hi-La networks are trained on 4 different databases and with different sparsity parameters on each layer. First, we show that the overall prediction error generated by 2L-SPC is lower thanks to the feedback mechanism as it transfers prediction error between layers. Second, we demonstrate that the inference stage of the 2L-SPC is faster to converge than for the Hi-La model. Third, we show that the 2L-SPC also accelerates the learning process. Finally, the qualitative analysis of both models dictionaries, supported by their activation probability, show that the 2L-SPC features are more generic and informative

Localization of anatomical changes in patients during proton therapy with in-beam PET monitoring: a voxel-based morphometry approach exploiting Monte Carlo simulations

Purpose: In-beam positron emission tomography (PET) is one of the modalities that can be used for in vivo noninvasive treatment monitoring in proton therapy. Although PET monitoring has been frequently applied for this purpose, there is still no straightforward method to translate the information obtained from the PET images into easy-to-interpret information for clinical personnel. The purpose of this work is to propose a statistical method for analyzing in-beam PET monitoring images that can be used to locate, quantify, and visualize regions with possible morphological changes occurring over the course of treatment. Methods: We selected a patient treated for squamous cell carcinoma (SCC) with proton therapy, to perform multiple Monte Carlo (MC) simulations of the expected PET signal at the start of treatment, and to study how the PET signal may change along the treatment course due to morphological changes. We performed voxel-wise two-tailed statistical tests of the simulated PET images, resembling the voxel-based morphometry (VBM) method commonly used in neuroimaging data analysis, to locate regions with significant morphological changes and to quantify the change. Results: The VBM resembling method has been successfully applied to the simulated in-beam PET images, despite the fact that such images suffer from image artifacts and limited statistics. Three dimensional probability maps were obtained, that allowed to identify interfractional morphological changes and to visualize them superimposed on the computed tomography (CT) scan. In particular, the characteristic color patterns resulting from the two-tailed statistical tests lend themselves to trigger alarms in case of morphological changes along the course of treatment. Conclusions: The statistical method presented in this work is a promising method to apply to PET monitoring data to reveal interfractional morphological changes in patients, occurring over the course of treatment. Based on simulated in-beam PET treatment monitoring images, we showed that with our method it was possible to correctly identify the regions that changed. Moreover we could quantify the changes, and visualize them superimposed on the CT scan. The proposed method can possibly help clinical personnel in the replanning procedure in adaptive proton therapy treatments

In-vivo range verification analysis with in-beam PET data for patients treated with proton therapy at CNAO

Morphological changes that may arise through a treatment course are probably one of the most significant sources of range uncertainty in proton therapy. Non-invasive in-vivo treatment monitoring is useful to increase treatment quality. The INSIDE in-beam Positron Emission Tomography (PET) scanner performs in-vivo range monitoring in proton and carbon therapy treatments at the National Center of Oncological Hadrontherapy (CNAO). It is currently in a clinical trial (ID: NCT03662373) and has acquired in-beam PET data during the treatment of various patients. In this work we analyze the in-beam PET (IB-PET) data of eight patients treated with proton therapy at CNAO. The goal of the analysis is twofold. First, we assess the level of experimental fluctuations in inter-fractional range differences (sensitivity) of the INSIDE PET system by studying patients without morphological changes. Second, we use the obtained results to see whether we can observe anomalously large range variations in patients where morphological changes have occurred. The sensitivity of the INSIDE IB-PET scanner was quantified as the standard deviation of the range difference distributions observed for six patients that did not show morphological changes. Inter-fractional range variations with respect to a reference distribution were estimated using the Most-Likely-Shift (MLS) method. To establish the efficacy of this method, we made a comparison with the Beam's Eye View (BEV) method. For patients showing no morphological changes in the control CT the average range variation standard deviation was found to be 2.5 mm with the MLS method and 2.3 mm with the BEV method. On the other hand, for patients where some small anatomical changes occurred, we found larger standard deviation values. In these patients we evaluated where anomalous range differences were found and compared them with the CT. We found that the identified regions were mostly in agreement with the morphological changes seen in the CT scan

Monitoring Carbon Ion Beams Transverse Position Detecting Charged Secondary Fragments: Results From Patient Treatment Performed at CNAO

Particle therapy in which deep seated tumours are treated using 12C ions (Carbon Ions RadioTherapy or CIRT) exploits the high conformity in the dose release, the high relative biological effectiveness and low oxygen enhancement ratio of such projectiles. The advantages of CIRT are driving a rapid increase in the number of centres that are trying to implement such technique. To fully profit from the ballistic precision achievable in delivering the dose to the target volume an online range verification system would be needed, but currently missing. The 12C ions beams range could only be monitored by looking at the secondary radiation emitted by the primary beam interaction with the patient tissues and no technical solution capable of the needed precision has been adopted in the clinical centres yet. The detection of charged secondary fragments, mainly protons, emitted by the patient is a promising approach, and is currently being explored in clinical trials at CNAO. Charged particles are easy to detect and can be back-tracked to the emission point with high efficiency in an almost background-free environment. These fragments are the product of projectiles fragmentation, and are hence mainly produced along the beam path inside the patient. This experimental signature can be used to monitor the beam position in the plane orthogonal to its flight direction, providing an online feedback to the beam transverse position monitor chambers used in the clinical centres. This information could be used to cross-check, validate and calibrate, whenever needed, the information provided by the ion chambers already implemented in most clinical centres as beam control detectors. In this paper we study the feasibility of such strategy in the clinical routine, analysing the data collected during the clinical trial performed at the CNAO facility on patients treated using 12C ions and monitored using the Dose Profiler (DP) detector developed within the INSIDE project. On the basis of the data collected monitoring three patients, the technique potential and limitations will be discussed

SDPC : un modèle parcimonieux et prédictif du cortex visuel primaire

Un des objectifs des neurosciences visuelles est de comprendre comment le cerveau interprète les informations sensorielles et de décrire les représentations corticales grâce à un modèle computationnel. Dans cette thèse, nous décrivons comment un modèle de perception visuelle, le Codage Prédictif, peut être étendu pour rendre compte des opérations non linéaires dans le cortex visuel primaire des mammifères (V1). Dans cette thèse, nous généralisons le Codage Prédictif dans un réseau convolutif pour créer un modèle appelé Sparse Deep Predictive Coding (SDPC). Nous présentons le SDPC dans deux articles scientifiques : dans le premier, nous utilisons notre réseau pour modéliser les interactions locales dans le système visuel précoce (V1/V2) et nous montrons comment la connectivité de rétroaction permet au système visuel de s'adapter aux statistiques des images naturelles. Dans un second article, nous montrons que le SDPC peut prédire l'émergence de réponses non linéaires dans V1 (cellules complexes) et expliquer le lien entre cellules complexes et des structures de plus haut niveau comme les cartes d'orientation corticales, et ceci pour différentes espèces. Enfin, nous proposerons quelques extensions qui permettront au SDPC de servir de modèle général du système visuel.One goal of visual neuroscience is to understand how the brain interprets sensory information and to describe cortical representations according to a specific computational model. In this thesis, we describe how a successful model for visual perception, Predictive Coding (PC), can be extended to account for highly nonlinear operations in the primary visual cortex of mammals (V1). In this thesis, we generalize PC in a convolutional network and propose an algorithm called Sparse Deep Predictive Coding (SDPC), which models the properties of the early visual cortex. We present the SDPC framework in two scientific articles: in the first, we use our network to model local interactions in the early visual system (V1/V2) and we show how feedback connectivity allows the visual system to adapt to the statistics of natural images. In a second article, we show that the SDPC can predict the emergence of nonlinear responses in V1 (complex cells) and explain the link between complex cells and higher-level structures like cortical orientation maps, across species. Finally, we will propose some extensions that will allow the SDPC to serve as a general model of the visual system

Effect of Top-Down Connections in Hierarchical Sparse Coding

International audienceHierarchical sparse coding (HSC) is a powerful model to efficiently represent multidimensional, structured data such as images. The simplest solution to solve this computationally hard problem is to decompose it into independent layer-wise subproblems. However, neuroscientific evidence would suggest interconnecting these subproblems as in predictive coding (PC) theory, which adds top-down connections between consecutive layers. In this study, we introduce a new model, 2-layer sparse predictive coding (2L-SPC), to assess the impact of this interlayer feedback connection. In particular, the 2L-SPC is compared with a hierarchical Lasso (Hi-La) network made out of a sequence of independent Lasso layers. The 2L-SPC and a 2-layer Hi-La networks are trained on four different databases and with different sparsity parameters on each layer. First, we show that the overall prediction error generated by 2L-SPC is lower thanks to the feedback mechanism as it transfers prediction error between layers. Second, we demonstrate that the inference stage of the 2L-SPC is faster to converge and generates a refined representation in the second layer compared to the Hi-La model. Third, we show that the 2L-SPC top-down connection accelerates the learning process of the HSC problem. Finally, the analysis of the emerging dictionaries shows that the 2L-SPC features are more generic and present a larger spatial extension

Pooling in a predictive model of V1 explains functional and structural diversity across species

Abstract Neurons in the primary visual cortex are selective to orientation with various degrees of selectivity to the spatial phase, from high selectivity in simple cells to low selectivity in complex cells. Various computational models have suggested a possible link between the presence of phase invariant cells and the existence of cortical orientation maps in higher mammals’ V1. These models, however, do not explain the emergence of complex cells in animals that do not show orientation maps. In this study, we build a model of V1 based on a convolutional network called Sparse Deep Predictive Coding (SDPC) and show that a single computational mechanism, pooling, allows the SDPC model to account for the emergence of complex cells as well as cortical orientation maps in V1, as observed in distinct species of mammals. By using different pooling functions, our model developed complex cells in networks that exhibit orientation maps (e.g., like in carnivores and primates) or not (e.g., rodents and lagomorphs). The SDPC can therefore be viewed as a unifying framework that explains the diversity of structural and functional phenomena observed in V1. In particular, we show that orientation maps emerge naturally as the most cost-efficient structure to generate complex cells under the predictive coding principle. Significance Cortical orientation maps are among the most fascinating structures observed in higher mammals brains: In such maps, similar orientations in the input image activate neighboring cells in the cortical surface. However, the computational advantage brought by these structures remains unclear, as some species (rodents and lagomorphs) completely lack orientation maps. In this study, we introduce a computational model that links the presence of orientation maps to a class of nonlinear neurons called complex cells. In particular, we propose that the presence or absence orientation maps correspond to different strategies employed by different species to generate invariance to complex stimuli

Sparse deep predictive coding captures contour integration capabilities of the early visual system

International audienceBoth neurophysiological and psychophysical experiments have pointed out the crucial role of recurrent and feedback connections to process context-dependent information in the early visual cortex. While numerous models have accounted for feedback effects at either neural or representational level, none of them were able to bind those two levels of analysis. Is it possible to describe feedback effects at both levels using the same model? We answer this question by combining Predictive Coding (PC) and Sparse Coding (SC) into a hierarchical and convolutional framework applied to realistic problems. In the Sparse Deep Predictive Coding (SDPC) model, the SC component models the internal recurrent processing within each layer, and the PC component describes the interactions between layers using feedforward and feedback connections. Here, we train a 2-layered SDPC on two different databases of images, and we interpret it as a model of the early visual system (V1 & V2). We first demonstrate that once the training has converged, SDPC exhibits oriented and localized receptive fields in V1 and more complex features in V2. Second, we analyze the effects of feedback on the neural organization beyond the classical receptive field of V1 neurons using interaction maps. These maps are similar to association fields and reflect the Gestalt principle of good continuation. We demonstrate that feedback signals reorganize interaction maps and modulate neural activity to promote contour integration. Third, we demonstrate at the representational level that the SDPC feedback connections are able to overcome noise in input images. Therefore, the SDPC captures the association field principle at the neural level which results in a better reconstruction of blurred images at the representational level