On the neural computation of utility

The rewarding effect produced by electrical stimulation of the lateral hypothalamus can compete and summate with gustatory rewards. However, physiological manipulations, such as sodium depletion and the accumulation of an energy-rich solution in the gut, can alter the rewarding impact of the gustatory stimuli without producing substantial changes in the rewarding effect of the electrical stimulation. On the basis of their competition and summation, it is argued that the artificial and natural rewards are evaluated in a common currency, represented in an aggregate firing-rate code. Such a code would make it possible for the synchronous, spatially contiguous pattern of neural firing induced by the electrode to simulate a signal normally produced by asynchronous, spatially distributed activity. It is suggested that a unidimensional code of this sort is employed to represent the utility of a goal object. In order for physiological feedback to alter the utility of one natural reward, such as sucrose, without changing the utility of a second natural reward, such as a salt solution, the physiological feedback signals must enter into the computation of utility at a stage of processing in which the representations of the two natural rewards are distinct. However, orderly choice between such rewards implies that their utilities are expressed ultimately in a common neural currency. That physiological feedback alters the rewarding effects of the gustatory stimuli suggests that the physiological feedback signals modulate the value of such natural stimuli at a stage of processing prior to their translation into a common currency. In contrast, physiological feedback would fail to alter the rewarding effect of the electrical stimulation if the electrically evoked signal is injected at a later stage processing, a stage in which different rewards are represented in a common currency. In this view, the signal injected by the electrical stimulation mimics the utility of a natural stimulus but not its sensory quality

Analysis of the Reaction Rate Coefficients for Slow Bimolecular Chemical Reactions

Simple bimolecular reactions $A_1+A_2\rightleftharpoons A_3+A_4$ are analyzed within the framework of the Boltzmann equation in the initial stage of a chemical reaction with the system far from chemical equilibrium. The Chapman-Enskog methodology is applied to determine the coefficients of the expansion of the distribution functions in terms of Sonine polynomials for peculiar molecular velocities. The results are applied to the reaction $H_2+Cl\rightleftharpoons HCl+H$, and the influence of the non-Maxwellian distribution and of the activation-energy dependent reactive cross sections upon the forward and reverse reaction rate coefficients are discussed.Comment: 11 pages, 5 figures, to appear in vol.42 of the Brazilian Journal of Physic

Scarce Means with Alternative Uses: Robbins’ Definition of Economics and Its Extension to the Behavioral and Neurobiological Study of Animal Decision Making

Almost 80 years ago, Lionel Robbins proposed a highly influential definition of the subject matter of economics: the allocation of scarce means that have alternative ends. Robbins confined his definition to human behavior, and he strove to separate economics from the natural sciences in general and from psychology in particular. Nonetheless, I extend his definition to the behavior of non-human animals, rooting my account in psychological processes and their neural underpinnings. Some historical developments are reviewed that render such a view more plausible today than would have been the case in Robbins’ time. To illustrate a neuroeconomic perspective on decision making in non-human animals, I discuss research on the rewarding effect of electrical brain stimulation. Central to this discussion is an empirically based, functional/computational model of how the subjective intensity of the electrical reward is computed and combined with subjective costs so as to determine the allocation of time to the pursuit of reward. Some successes achieved by applying the model are discussed, along with limitations, and evidence is presented regarding the roles played by several different neural populations in processes posited by the model. I present a rationale for marshaling convergent experimental methods to ground psychological and computational processes in the activity of identified neural populations, and I discuss the strengths, weaknesses, and complementarity of the individual approaches. I then sketch some recent developments that hold great promise for advancing our understanding of structure–function relationships in neuroscience in general and in the neuroeconomic study of decision making in particular

On the neural computation of utility: implications from studies of brain stimulation reward

1. Like other vertebrates, from goldfish to humans, rats will work in order to deliver electrical stimulation to certain brain sites. Although the stimulation produces no evident physiological benefit, it is sought out avidly, as if it were a biologically significant resource. Thus, it has long been thought that the rewarding stimulation activates neural circuitry involved in the evaluation and selection of goals. 2. Computing the utility of goal objects involves a tightly integrated set of perceptual, cognitive, and motivational mechanisms. I argue that rewarding electrical brain stimulation engages only a subset of these mechanisms. If so, comparison of the ways in which the utility of electrical brain stimulation and natural reinforcers are computed may highlight operating principles and isolate components of the computational mechanisms. 3. In the view proposed here, information about goal objects and consummatory acts is processed, in parallel, in three different channels. 3.1. Perceptual processing indicates what and where the goal object is. 3.2. A stopwatch-like interval timer predicts when or how often the goal object will be available. 3.3. Under the influence of information about the current physiological state, an evaluative channel returns a subjective weighting of strength variables such as the concentration of a sucrose solution or the temperature of an air current. 3.4. The output of these channels is recorded in multidimensional records that include 3.4.1. information of perceptual origin about amount and kind (e.g., food, water,or salt) 3.4.2. information from the timer about rate and delay 3.4.3. a subjective assessment of intensity provided by the evaluative channel 4. This chapter addresses the relationships between brain stimulation reward (BSR), the perceptual, interval timing, and evaluative channels, and the variants of utility proposed by Kahneman and his coworkers on the basis of their studies of evaluation and choice in human subjects. 4.1. It is argued that the output of the evaluative channel can be manifested in experience as pleasure or suffering but that awareness is not necessary in order for this signal to influence action. 5. The neural signal injected by rewarding electrical stimulation is portrayed as providing meaningful information about rate, delay and intensity but not about amount or kind. This proposal is used to account for 5.1. competition and summation between BSR and natural rewards 5.2. differential effects of physiological feedback on the utility of BSR and natural rewards 5.3. matching of behavioral allocation to the relative rates and intensities of BSR 5.4. differences in the elasticity of demand for BSR and food in a closed economy 5.5. the high substitutability of BSR for food and water in an open economy 6. The powerful aftereffect of BSR that potentiates efforts to obtain additional stimulation is related to expectancy

Neural Basis of Utility Estimation

The allocation of behavior among competing activities and goal objects depends on the payoffs they provide.Payoff is evaluated among multiple dimensions including intensity, rate, delay, and kind. Recent findings suggest that by triggering a stream of action potentials in myelinated, medial forebrain bundle axons, rewarding electrical brain stimulation delivers a meaningful intensity signal to the process that computes payoff

Brain Stimulation Reward

In 1953, Olds and Milner discovered that rats would readily learn to work for electrical stimulation of certain brain sites. Their findings inspired a large body of research on the neural basis of reward, motivation, and learning. Unlike consummatory behaviors, which satiate as a result of ingestion of and contact with the goal object, performance for rewarding brain stimulation is remarkably stable and persistent. Pursuit of the stimulation is enhanced by different classes of dependence-inducing drugs, suggesting that common neural mechanisms underlie the rewarding effects of drugs and electrical brain stimulation. Indeed, dopamine-containing neurons in the midbrain are implicated in both phenomena. Major schools of thought that have addressed brain stimulation reward differ with regards to the roles played by hedonic experience and craving, although there is substantial overlap between the different viewpoints. A tradition that arose in the study of machine learning has been brought to bear on the role of dopamine neurons in reward-related learning in animals and on the phenomenon of intracranial self-stimulation. Neuroeconomic perspectives strive to integrate the processing of benefits, costs, and risks into an account of decision making grounded in brain circuitry. Adjudication of the differences between the various viewpoints and progress towards identifying the relevant neural circuitry has been hindered by the lack of specificity inherent in the use of electrical stimulation to study central nervous system function. Many neurons in addition to those targeted are likely activated by such stimulation. Recently developed optogenetic methods may overcome this obstacle, providing much more specific means for stimulating or silencing populations of nerve cells selected on the basis of their gene expression, cell-body location, and projections. Coupled with behavioral methods of increasing sophistication and specificity and with quantitative modeling of signal flow in the relevant neural circuitry, the new optogenetic methods promise to bring us much closer to fulfillment of the hopes engendered long ago by the discovery of brain stimulation reward

Travelling waves in a mixture of gases with bimolecular reversible reactions

Starting from the kinetic approach for a mixture of reacting gases whose particles interact through elastic scattering and a bimolecular reversible chemical reaction, the equations that govern the dynamics of the system are obtained by means of the relevant Boltzmann-like equation. Conservation laws are considered. Fluid dynamic approximations are used at the Euler level to obtain a close set of PDEs for six unknown macroscopic fields. The dispersion relation of the mixture of reacting gases is explicitly derived in the homogeneous equilibrium state. A set of ODE that governs the propagation of a plane travelling wave is obtained using the Galilei invariance. After numerical integration some solutions, including the well-known Maxwellian and the hard spheres cases, are found for various meaningful interaction laws. The main macroscopic observables for the gas mixture such as the drift velocity, temperature, total density, pressure and its chemical composition are shown.Comment: 13 pages, 2 figures, accepted on Physica

Video illustrating the reward-mountain model

This video reveals a fundamental source of ambiguity in two-dimensional measurements of operant performance for reward, such as those obtained in the curve-shift and progressive-ratio paradigms. We show how the three-dimensional portrayal provided by the reward-mountain model resolves this ambiguity