Insights into reward-based decisions using computational models and ultra-high field MRI

The ability to integrate past and current feedback associated with di↵erent environmental stimuli is crucial for adaptive and goal-directed behavior. The field of reinforcement learning (RL) focuses on understanding such ability: Computational models propose algorithms for the updating of beliefs based on the received feedback, while neural models describe how these algorithms are implemented in the brain. In this dissertation, I investigate learning- by-feedback both at the algorithmic level (in the first part) and at the implementation level (in the second part). In the first part, in particular, I focus on human behavior during learning-by- feedback in the presence of both monetary gains and losses (first manuscript) or of di↵erent magnitudes of monetary gains (second manuscript). To date, most computational RL mod- els focused on trial-by-trial dynamics and have not explained how response times (RTs) change during learning or are a↵ected by di↵erent learning contexts (e.g., in the presence of gains vs. losses). In both manuscripts, I argue that RTs are crucial for the understand- ing of reward-based decisions: In both studies, participants’ RTs were a↵ected by di↵erent learning contexts, while their choice preferences not always were. To jointly explain the e↵ects on both preferences and RTs, I used a sequential sampling model, the di↵usion decision model (DDM). In the first manuscript, I propose a meta-analytical approach to simultaneously analyze the e↵ects of di↵erent learning contexts on choice preferences and RTs from four independent experiments. In the second manuscript, I propose a new model that incorporates an RL algorithm into the DDM. In the second part of my thesis, I investigated the coding of losses and gains by the dopaminergic nuclei in the human brain. Since these nuclei are situated deep in the brain, their signal is hard to study using non-invasive imaging techniques such as mag- netic resonance imaging (MRI): To date, human studies have provided incomplete and partially contradicting findings about the reward signals in dopaminergic nuclei. In the third manuscript, I provide evidence that clarifies the role of the dopaminergic nuclei when receiving more or less surprising gains and losses, as well as when expecting higher or lower outcome risk. To do so, I capitalize on ultra-high field MRI and on the use of multimodal images to delineate the dopaminergic nuclei on a participant level

Analysis of Horse Myostatin Gene and Identification of Single Nucleotide Polymorphisms in Breeds of Different Morphological Types

Myostatin (MSTN) is a negative modulator of muscle mass. We characterized the horse (Equus caballus) MSTN gene and identified and analysed single nucleotide polymorphisms (SNPs) in breeds of different morphological types. Sequencing of coding, untranslated, intronic, and regulatory regions of MSTN gene in 12 horses from 10 breeds revealed seven SNPs: two in the promoter, four in intron 1, and one in intron 2. The SNPs of the promoter (GQ183900:g.26T>C and GQ183900:g.156T>C, the latter located within a conserved TATA-box like motif) were screened in 396 horses from 16 breeds. The g.26C and the g.156C alleles presented higher frequency in heavy (brachymorphic type) than in light breeds (dolichomorphic type such as Italian Trotter breed). The significant difference of allele frequencies for the SNPs at the promoter and analysis of molecular variance (AMOVA) on haplotypes indicates that these polymorphisms could be associated with variability of morphology traits in horse breeds

A reinforcement learning diffusion decision model for value-based decisions

Psychological models of value-based decision-making describe how subjective values are formed and mapped to single choices. Recently, additional efforts have been made to describe the temporal dynamics of these processes by adopting sequential sampling models from the perceptual decision-making tradition, such as the diffusion decision model (DDM). These models, when applied to value-based decision-making, allow mapping of subjective values not only to choices but also to response times. However, very few attempts have been made to adapt these models to situations in which decisions are followed by rewards, thereby producing learning effects. In this study, we propose a new combined reinforcement learning diffusion decision model (RLDDM) and test it on a learning task in which pairs of options differ with respect to both value difference and overall value. We found that participants became more accurate and faster with learning, responded faster and more accurately when options had more dissimilar values, and decided faster when confronted with more attractive (i.e., overall more valuable) pairs of options. We demonstrate that the suggested RLDDM can accommodate these effects and does so better than previously proposed models. To gain a better understanding of the model dynamics, we also compare it to standard DDMs and reinforcement learning models. Our work is a step forward towards bridging the gap between two traditions of decision-making research

Exploiting phenotype diversity in a local animal genetic resource: identification of a single nucleotide polymorphism associated with the tail shape phenotype in the autochthonous Casertana pig breed

Casertana is a local pig breed mainly raised in Central-South regions of Italy. Pigs of this breed are considered the descendants of the ancient Neapolitan population that largely influenced the constitution of the modern commercial pigs. The pigs of this breed are usually curly-tailed, like several other domestic pig populations. However, Casertana population shows some variability for this trait, including animals having straight tail as observed in wild boars. In this study, we run, for the first time, a genome wide association study (GWAS) comparing the curly tailed (no. = 53) and straight tailed (no. = 19) Casertana pigs to identify genomic regions associated with the tail shape phenotype in Sus scrofa. All animals were genotyped with the Illumina PorcineSNP60 BeadChip v.2. GEMMA software was used in the GWAS for which we were able to correct for stratification in the analysed cohort. A single nucleotide polymorphism (rs81439488), located on porcine chromosome 12, was significantly associated with the investigated trait. This marker is close to the SRY-box 9 (SOX9) gene that encodes for a transcription factor that is required during sequential steps of the chondrocyte differentiation pathway, notochord maintenance and skeletogenesis. As the shape of the tail could be important in relation to the problem of tail biting in pigs, the obtained results might open new perspectives for defining selection programs answering indirectly animal welfare issues. This work demonstrated that autochthonous animal genetic resources might be used to disclose genetic factors affecting peculiar traits by exploiting segregating phenotypes and genetic variability

Effect of sex and RYR1 gene mutation on the muscle proteomic profile and main physiological biomarkers in pigs at slaughter

Gender and RYR1 gene mutation might have an effect on the muscle metabolic characteristics and on the animal's stress at slaughter, which could influence the process of muscle-to-meat conversion. Forty-eight pigs were distributed in a design including two factors: sex (male/female) and RYR1 genotype (NN/Nn). At slaughter, physiological blood biomarkers and muscle proteome were analyzed and carcass and meat quality traits were registered. Females had higher serum levels of glucose, urea, C-reactive protein "CRP", Pig-MAP and glutation-peroxidase "GPx" and lower levels of lactate, showed faster muscle pH decline and higher meat exudation. RYR1 mutation increased serum creatinine, creatine kinase and CRP and decreased GPx. The proteomic study highlighted significant effects of gender and RYR1 genotype on proteins related to fibre composition, antioxidant defense and post mortem glycolytic pathway, which correlate to differences of meat quality. This study provides interesting information on muscle biomarkers of the ultimate meat quality that are modulated by the animal's individual susceptibility to stress at slaughter.info:eu-repo/semantics/acceptedVersio

Mitochondrial physiology

As the knowledge base and importance of mitochondrial physiology to evolution, health and disease expands, the necessity for harmonizing the terminology concerning mitochondrial respiratory states and rates has become increasingly apparent. The chemiosmotic theory establishes the mechanism of energy transformation and coupling in oxidative phosphorylation. The unifying concept of the protonmotive force provides the framework for developing a consistent theoretical foundation of mitochondrial physiology and bioenergetics. We follow the latest SI guidelines and those of the International Union of Pure and Applied Chemistry (IUPAC) on terminology in physical chemistry, extended by considerations of open systems and thermodynamics of irreversible processes. The concept-driven constructive terminology incorporates the meaning of each quantity and aligns concepts and symbols with the nomenclature of classical bioenergetics. We endeavour to provide a balanced view of mitochondrial respiratory control and a critical discussion on reporting data of mitochondrial respiration in terms of metabolic flows and fluxes. Uniform standards for evaluation of respiratory states and rates will ultimately contribute to reproducibility between laboratories and thus support the development of data repositories of mitochondrial respiratory function in species, tissues, and cells. Clarity of concept and consistency of nomenclature facilitate effective transdisciplinary communication, education, and ultimately further discovery

Mitochondrial physiology

As the knowledge base and importance of mitochondrial physiology to evolution, health and disease expands, the necessity for harmonizing the terminology concerning mitochondrial respiratory states and rates has become increasingly apparent. The chemiosmotic theory establishes the mechanism of energy transformation and coupling in oxidative phosphorylation. The unifying concept of the protonmotive force provides the framework for developing a consistent theoretical foundation of mitochondrial physiology and bioenergetics. We follow the latest SI guidelines and those of the International Union of Pure and Applied Chemistry (IUPAC) on terminology in physical chemistry, extended by considerations of open systems and thermodynamics of irreversible processes. The concept-driven constructive terminology incorporates the meaning of each quantity and aligns concepts and symbols with the nomenclature of classical bioenergetics. We endeavour to provide a balanced view of mitochondrial respiratory control and a critical discussion on reporting data of mitochondrial respiration in terms of metabolic flows and fluxes. Uniform standards for evaluation of respiratory states and rates will ultimately contribute to reproducibility between laboratories and thus support the development of data repositories of mitochondrial respiratory function in species, tissues, and cells. Clarity of concept and consistency of nomenclature facilitate effective transdisciplinary communication, education, and ultimately further discovery

Measurement of t(t)over-bar normalised multi-differential cross sections in pp collisions at root s=13 TeV, and simultaneous determination of the strong coupling strength, top quark pole mass, and parton distribution functions

Peer reviewe

An embedding technique to determine ττ backgrounds in proton-proton collision data

An embedding technique is presented to estimate standard model tau tau backgrounds from data with minimal simulation input. In the data, the muons are removed from reconstructed mu mu events and replaced with simulated tau leptons with the same kinematic properties. In this way, a set of hybrid events is obtained that does not rely on simulation except for the decay of the tau leptons. The challenges in describing the underlying event or the production of associated jets in the simulation are avoided. The technique described in this paper was developed for CMS. Its validation and the inherent uncertainties are also discussed. The demonstration of the performance of the technique is based on a sample of proton-proton collisions collected by CMS in 2017 at root s = 13 TeV corresponding to an integrated luminosity of 41.5 fb(-1).Peer reviewe