A Biologically Plausible Learning Rule for Deep Learning in the Brain

Researchers have proposed that deep learning, which is providing important progress in a wide range of high complexity tasks, might inspire new insights into learning in the brain. However, the methods used for deep learning by artificial neural networks are biologically unrealistic and would need to be replaced by biologically realistic counterparts. Previous biologically plausible reinforcement learning rules, like AGREL and AuGMEnT, showed promising results but focused on shallow networks with three layers. Will these learning rules also generalize to networks with more layers and can they handle tasks of higher complexity? We demonstrate the learning scheme on classical and hard image-classification benchmarks, namely MNIST, CIFAR10 and CIFAR100, cast as direct reward tasks, both for fully connected, convolutional and locally connected architectures. We show that our learning rule - Q-AGREL - performs comparably to supervised learning via error-backpropagation, with this type of trial-and-error reinforcement learning requiring only 1.5-2.5 times more epochs, even when classifying 100 different classes as in CIFAR100. Our results provide new insights into how deep learning may be implemented in the brain

Developing a spiking neural model of Long Short-Term Memory architectures

Current advances in Deep Learning have shown significant improvements in common Machine Learning applications such as image, speech and text recognition. Specifically, in order to process time series, deep Neural Networks (NNs) with Long Short-Term Memory (LSTM) units are widely used in sequence recognition problems to store recent information and to use it for future predictions. The efficiency in data analysis, especially when big-sized data sets are involved, can be greatly improved thanks to the advancement of the ongoing research on Neural Networks (NNs) and Machine Learning for many applications in Physics. However, whenever acquisition and processing of data at different time resolutions is required, a synchronization problem for which the same piece of information is processed multiple times arises, and the advantageous efficiency of NNs, which lack the natural notion of time, ceases to exist. Spiking Neural Networks (SNNs) are the next generation of NNs that allow efficient information coding and processing by means of spikes, i.e. binary pulses propagating between neurons. In this way, information can be encoded in time, and the communication of information is activated only when the input to the neurons change, thus giving higher efficiency. In the present work, analog neurons are used for training and then they are substituted with spiking neurons in order to perform tasks. The aim for this project is to find a transfer function which allows a simple and accurate switching between analog and spiking neurons, and then to prove that the obtained network performs well in different tasks. At first, an analytical transfer function for more biologically plausible values for some neuronal parameters is derived and tested. Subsequently, the stochastic nature of the biological neurons is implemented in the neuronal model used. A new transfer function is then approximated by studying the stochastic behavior of artificial neurons, allowing to implement a simplified description for the gates and the input cell in the LSTM units. The stochastic LSTM networks are then tested on Sequence Prediction and T-Maze, i.e. typical memory-involving Machine Learning tasks, showing that almost all the resulting spiking networks correctly compute the original tasks. The main conclusion drawn from this project is that by means of a neuronal model comprising of a stochastic description of the neuron it is possible to obtain an accurate mapping from analog to spiking memory networks, which gives good results on Machine Learning tasks.Spiking neurons communicate with each other by means of a telegraph-like mechanism: a message is encoded by a neuron in binary events–called spikes–that are sent to another neuron, which decodes the incoming spikes of signal by means of the same coding originally used by the sending neuron. The problem addressed in this project then was: is it possible to make a group of such neurons remember things in the short-term but for long-enough time that they are able to solve tasks that require memory? Imagine you are driving to work through a road you have never taken before, and your task is to turn right at the next traffic light. The memory-tasks we wanted the neural networks to learn and solve are of this sort, and no spiking networks exist that can do this. With regards to this goal, the approach we opted for was to train a network of standard artificial neurons and then, once the network learned how to perform the task, we would switch the standard neurons with our modeled spiking neurons. In order to do this, of course, there are some constraints, in particular the two types of neurons (standard and spiking) have to encode signals in the same way, meaning that they need to have the same coding policy. In this project, I had to find an adequate coding policy for the spiking neurons, in order to give the same policy to a network of standard neurons and to test this superposition. Turned out, after the standard networks had learned the tasks and then switched with spiking units, the spiking neurons were indeed able to remember short-term information (such as looking for a traffic light before turning right) and to perform well in such memory tasks, allowing useful computation over time. One of the scientific fields in need of improvement is, in fact, signal processing over time. Nowadays most of the detection instruments collect signal during a time window, meaning that the signal collected in a small time range is considered as a whole, instead of detecting in continuous time. In the first case, a buffer of the history of the time windows (the information gathered before meeting the traffic light) is stored, while when information is processed in continuous time only relevant information (the time at which the traffic light is encountered) is needed. Being able to classify signals as soon as they are detected is a characteristic of asynchronous detection, an example of which is our sight, or hearing. The brain, in fact, is one of the most efficient and powerful systems existent. So why not studying a computation method inspired by the brain? Spiking neurons are exactly that: artificial units performing brain-like computation. Hence, these neurons potentially offer efficient computation and an advantageous method for continuous-time signal processing, which hopefully will be implemented in many research fields in the future

A biologically plausible learning rule for deep learning in the brain

Researchers have proposed that deep learning, which is providing important progress in a wide range of high complexity tasks, might inspire new insights into learning in the brain. However, the methods used for deep learning by artificial neural networks are biologically unrealistic and would need to be replaced by biologically realistic counterparts. Previous biologically plausible reinforcement learning rules, like AGREL and AuGMEnT, showed promising results but focused on shallow networks with three layers. Will these learning rules also generalize to networks with more layers and can they handle tasks of higher complexity? Here, we demonstrate that these learning schemes indeed generalize to deep networks, if we include an attention network that propagates information about the selected action to lower network levels. The resulting learning rule, called Q-AGREL, is equivalent to a particular form of error-backpropagation that trains one output unit at any one time. To demonstrate the utility of the learning scheme for larger problems, we trained networks with two hidden layers on the MNIST dataset, a standard and interesting Machine Learning task. Our results demonstrate that the capability of Q-AGREL is comparable to that of error backpropagation, although the learning rate is 1.5-2 times slower because the network has to learn by trial-and-error and updates the action value of only one output unit at a time. Our results provide new insights into how deep learning can be implemented in the brain

Gating sensory noise in a spiking subtractive LSTM

Spiking neural networks are being investigated both as biologically plausible models of neural computation and also as a potentially more efficient type of neural network. Recurrent neural networks in the form of networks of gating memory cells have been central in state-of-the-art solutions in problem domains that involve sequence recognition or generation. Here, we design an analog Long Short-Term Memory (LSTM) cell where its neurons can be substituted with efficient spiking neurons, where we use subtractive gating (following the subLSTM in [1]) instead of multiplicative gating. Subtractive gating allows for a less sensitive gating mechanism, critical when using spiking neurons. By using fast adapting spiking neurons with a smoothed Rectified Linear Unit (ReLU)-like effective activation function, we show that then an accurate conversion from an analog subLSTM to a continuous-time spiking subLSTM is possible. This architecture results in memory networks that compute very efficiently, with low average firing rates comparable to those in biological neurons, while operating in continuous time

Attention-Gated Brain Propagation: How the brain can implement reward-based error backpropagation

Much recent work has focused on biologically plausible variants of supervised learning algorithms. However, there is no teacher in the motor cortex that instructs the motor neurons and learning in the brain depends on reward and punishment. We demonstrate a biologically plausible reinforcement learning scheme for deep networks with an arbitrary number of layers. The network chooses an action by selecting a unit in the output layer and uses feedback connections to assign credit to the units in successively lower layers that are responsible for this action. After the choice, the network receives reinforcement and there is no teacher correcting the errors. We show how the new learning scheme – Attention-Gated Brain Propagation (BrainProp) – is mathematically equivalent to error backpropagation, for one output unit at a time. We demonstrate successful learning of deep fully connected, convolutional and locally connected networks on classical and hard image-classification benchmarks; MNIST, CIFAR10, CIFAR100 andTiny ImageNet. BrainProp achieves an accuracy that is equivalent to that of standard error-backpropagation, and better than state-of-the-art biologically inspired learning schemes. Additionally, the trial-and-error nature of learning is associated with limited additional training time so that BrainProp is a factor of 1-3.5 times slower. Our results thereby provide new insights into how deep learning may be implemented in the brain

Exploiting the bile acid binding protein as transporter of a Cholic Acid/Mirin bioconjugate for potential applications in liver cancer therapy

Bioconjugation is one of the most promising strategies to improve drug delivery, especially in cancer therapy. Biomolecules such as bile acids (BAs) have been intensively explored as carriers, due to their peculiar physicochemical properties and biocompatibility. BAs trafficking is regulated by intracellular lipid-binding proteins and their transport in the liver can be studied using chicken liver Bile Acid-Binding Proteins (cL-BABPs) as a reference model. Therefore, we conceived the idea of developing a BA-conjugate with Mirin, an exonuclease inhibitor of Mre11 endowed with different anticancer activities, to direct its transport to the liver. Following computational analysis of various BAs in complex with cL-BABP, we identified cholic acid (CA) as the most promising candidate as carrier, leading to the synthesis of a novel bioconjugate named CA-M11. As predicted by computational data and confirmed by X-ray crystallographic studies, CA-M11 was able to accommodate into the binding pocket of BABP. Hence, it can enter BAs trafficking in the hepatic compartment and here release Mirin. The effect of CA-M11, evaluated in combination with varying concentrations of Doxorubicin on HepG2 cell line, demonstrated a significant increase in cell mortality compared to the use of the cytotoxic drug or Mirin alone, thus highlighting chemo-sensitizing properties. The promising results regarding plasma stability for CA-M11 validate its potential as a valuable agent or adjuvant for hepatic cancer therapy

Opsonin-deficient nucleoproteic corona endows unPEGylated liposomes with stealth properties in vivo

For several decades, surface grafted polyethylene glycol (PEG) has been a go-to strategy for preserving the synthetic identity of liposomes in physiological milieu and preventing clearance by immune cells. However, the limited clinical translation of PEGylated liposomes is mainly due to the protein corona formation and the subsequent modification of liposomes’ synthetic identity, which affects their interactions with immune cells and blood residency. Here we exploit the electric charge of DNA to generate unPEGylated liposome/DNA complexes that, upon exposure to human plasma, gets covered with an opsonin-deficient protein corona. The final product of the synthetic process is a biomimetic nanoparticle type covered by a proteonucleotidic corona, or “proteoDNAsome”, which maintains its synthetic identity in vivo and is able to slip past the immune system more efficiently than PEGylated liposomes. Accumulation of proteoDNAsomes in the spleen and the liver was lower than that of PEGylated systems. Our work highlights the importance of generating stable biomolecular coronas in the development of stealth unPEGylated particles, thus providing a connection between the biological behavior of particles in vivo and their synthetic identity

Adjuvant capecitabine in triple negative breast cancer patients with residual disease after neoadjuvant treatment: real-world evidence from CaRe, a multicentric, observational study

Background: In triple negative breast cancer patients treated with neoadjuvant chemotherapy, residual disease at surgery is the most relevant unfavorable prognostic factor. Current guidelines consider the use of adjuvant capecitabine, based on the results of the randomized CREATE-X study, carried out in Asian patients and including a small subset of triple negative tumors. Thus far, evidence on Caucasian patients is limited, and no real-world data are available. Methods: We carried out a multicenter, observational study, involving 44 oncologic centres. Triple negative breast cancer patients with residual disease, treated with adjuvant capecitabine from January 2017 through June 2021, were recruited. We primarily focused on treatment tolerability, with toxicity being reported as potential cause of treatment discontinuation. Secondarily, we assessed effectiveness in the overall study population and in a subset having a minimum follow-up of 2 years. Results: Overall, 270 patients were retrospectively identified. The 50.4% of the patients had residual node positive disease, 7.8% and 81.9% had large or G3 residual tumor, respectively, and 80.4% a Ki-67 >20%. Toxicity-related treatment discontinuation was observed only in 10.4% of the patients. In the whole population, at a median follow-up of 15 months, 2-year disease-free survival was 62%, 2 and 3-year overall survival 84.0% and 76.2%, respectively. In 129 patients with a median follow-up of 25 months, 2-year disease-free survival was 43.4%, 2 and 3-year overall survival 78.0% and 70.8%, respectively. Six or more cycles of capecitabine were associated with more favourable outcomes compared with less than six cycles. Conclusion: The CaRe study shows an unexpectedly good tolerance of adjuvant capecitabine in a real-world setting, although effectiveness appears to be lower than that observed in the CREATE-X study. Methodological differences between the two studies impose significant limits to comparability concerning effectiveness, and strongly invite further research