Monte Carlo Planning method estimates planning horizons during interactive social exchange

Reciprocating interactions represent a central feature of all human exchanges. They have been the target of various recent experiments, with healthy participants and psychiatric populations engaging as dyads in multi-round exchanges such as a repeated trust task. Behaviour in such exchanges involves complexities related to each agent's preference for equity with their partner, beliefs about the partner's appetite for equity, beliefs about the partner's model of their partner, and so on. Agents may also plan different numbers of steps into the future. Providing a computationally precise account of the behaviour is an essential step towards understanding what underlies choices. A natural framework for this is that of an interactive partially observable Markov decision process (IPOMDP). However, the various complexities make IPOMDPs inordinately computationally challenging. Here, we show how to approximate the solution for the multi-round trust task using a variant of the Monte-Carlo tree search algorithm. We demonstrate that the algorithm is efficient and effective, and therefore can be used to invert observations of behavioural choices. We use generated behaviour to elucidate the richness and sophistication of interactive inference

Conceptual model of salt marsh management on Merritt Island National Wildlife Refuge, Florida: final report

Diking and holding water on salt marshes ("impounding" the marsh) is a management technique used on Merritt Island National Wildlife Refuge (MINWR) and elsewhere in the Southeast to: a) prevent the reproduction of saltmarsh mosquitos, and b) attract wintertering waterfowl and other marsh, shore, and wading birds. Because of concern that diking and holding water may interfere with the production of estuarine fish and shellfish, impoundment managers are being asked to consider altering management protocol to reduce or eliminate any such negative influence. How to change protocol and preserve effective mosquito control and wildlife management is a decision of great complexity because: a) the relationships between estuarine organisms and the fringing salt marshes at the land-water interface are complex, and b) impounded marshes are currently good habitat for a variety of species of fish and wildlife. Most data collection by scientists and managers in the area has not been focused on this particular problem. Furthermore, collection of needed data may not be possible before changes in protocol are demanded. Therefore, the purpose of this document is two-fold: 1) to suggest management alternatives, given existing information, and 2) to help identify research needs that have a high probability of leading to improved simultaneous management of mosquitos, waterfowl, other wildlife, freshwater fish, and estuarine fish and shellfish on the marshland of the Merritt Island National Wildlife Refuge. (92 page document

Electromechanical Simulation of Actively Controlled Rotordynamic Systems with Piezoelectric Actuators

Theories and tests for incorporating piezoelectric pushers as actuator devices for active vibration control are discussed. It started from a simple model with the assumption of ideal pusher characteristics and progressed to electromechanical models with nonideal pushers. Effects on system stability due to the nonideal characteristics of piezoelectric pushers and other elements in the control loop were investigated

Categorized bibliography for a conceptual model of salt marsh management on Merritt Island, Florida

Enclosed is a bibliography of 556 published articles, technical reports, theses, dissertations, and books that form the basis for a conceptual model of salt marsh management on Merritt Island, Florida (Section 1). A copy of each item is available on file at the Florida Cooperative Fish and Wildlife Research Unit, Gainesville. Some relevant proprietary items and unpublished drafts have not been included pending permission of the authors. We will continue to add pertinent references to our bibliography and files. Currently, some topics are represented by very few items. As our synthesis develops, we will be able to indicate a subset of papers most pertinent to an understanding of the ecology and management of Merritt Island salt marshes. (98 page document

FGF/heparin differentially regulates Schwann cell and olfactory ensheathing cell interactions with astrocytes: a role in astrocytosis

After injury, the CNS undergoes an astrocyte stress response characterized by reactive astrocytosis/proliferation, boundary formation, and increased glial fibrillary acidic protein (GFAP) and chondroitin sulfate proteoglycan (CSPG) expression. Previously, we showed that in vitro astrocytes exhibit this stress response when in contact with Schwann cells but not olfactory ensheathing cells (OECs). In this study, we confirm this finding in vivo by demonstrating that astrocytes mingle with OECs but not Schwann cells after injection into normal spinal cord. We show that Schwann cell-conditioned media (SCM) induces proliferation in monocultures of astrocytes and increases CSPG expression in a fibroblast growth factor receptor 1 (FGFR1)-independent manner. However, SCM added to OEC/astrocyte cocultures induces reactive astrocytosis and boundary formation, which, although sensitive to FGFR1 inhibition, was not induced by FGF2 alone. Addition of heparin to OEC/astrocyte cultures induces boundary formation, whereas heparinase or chlorate treatment of Schwann cell/astrocyte cultures reduces it, suggesting that heparan sulfate proteoglycans (HSPGs) are modulating this activity. In vivo, FGF2 and FGFR1 immunoreactivity was increased over grafted OECs and Schwann cells compared with the surrounding tissue, and HSPG immunoreactivity is increased over reactive astrocytes bordering the Schwann cell graft. These data suggest that components of the astrocyte stress response, including boundary formation, astrocyte hypertrophy, and GFAP expression, are mediated by an FGF family member, whereas proliferation and CSPG expression are not. Furthermore, after cell transplantation, HSPGs may be important for mediating the stress response in astrocytes via FGF2. Identification of factors secreted by Schwann cells that induce this negative response in astrocytes would further our ability to manipulate the inhibitory environment induced after injury to promote regeneration

Simulating future value in intertemporal choice

The laboratory study of how humans and other animals trade-off value and time has a long and storied history, and is the subject of a vast literature. However, despite a long history of study, there is no agreed upon mechanistic explanation of how intertemporal choice preferences arise. Several theorists have recently proposed model-based reinforcement learning as a candidate framework. This framework describes a suite of algorithms by which a model of the environment, in the form of a state transition function and reward function, can be converted on-line into a decision. The state transition function allows the model-based system to make decisions based on projected future states, while the reward function assigns value to each state, together capturing the necessary components for successful intertemporal choice. Empirical work has also pointed to a possible relationship between increased prospection and reduced discounting. In the current paper, we look for direct evidence of a relationship between temporal discounting and model-based control in a large new data set (n = 168). However, testing the relationship under several different modeling formulations revealed no indication that the two quantities are related

Loss Aversion Correlates With the Propensity to Deploy Model-Based Control

Reward-based decision making is thought to be driven by at least two different types of decision systems: a simple stimulus–response cache-based system which embodies the common-sense notion of “habit,” for which model-free reinforcement learning serves as a computational substrate, and a more deliberate, prospective, model-based planning system. Previous work has shown that loss aversion, a well-studied measure of how much more on average individuals weigh losses relative to gains during decision making, is reduced when participants take all possible decisions and outcomes into account including future ones, relative to when they myopically focus on the current decision. Model-based control offers a putative mechanism for implementing such foresight. Using a well-powered data set (N = 117) in which participants completed two different tasks designed to measure each of the two quantities of interest, and four models of choice data for these tasks, we found consistent evidence of a relationship between loss aversion and model-based control but in the direction opposite to that expected based on previous work: loss aversion had a positive relationship with model-based control. We did not find evidence for a relationship between either decision system and risk aversion, a related aspect of subjective utility