2,346 research outputs found
Endogenous growth and property rights over renewable resources
We study how different regimes of access rights to renewable natural resources - namely, open access versus full property rights – affect sustainability, growth and welfare in the context of modern endogenous growth theory. Resource exhaustion may occur under both regimes but is more likely to arise under open access. Moreover, under full property rights, positive resource rents increase expenditures on manufacturing goods and temporarily accelerate productivity growth, but also yield a higher resource price at least in the short-to-medium run. We characterize analytically and quantitatively the model’s dynamics to assess the welfare implications of differences in property rights enforcement
Electrophysiological responses of medial prefrontal cortex to feedback at different levels of hierarchy
Recent advances in computational reinforcement learning suggest that humans and animals can learn from different types of reinforcers in a hierarchically organised fashion. According to this theoretical framework, while humans learn to coordinate subroutines based on external reinforcers such as food rewards, simple actions within those subroutines are reinforced by an internal reinforcer called a pseudo-reward. Although the neural mechanisms underlying these processes are unknown, recent empirical evidence suggests that the medial prefrontal cortex (MPFC) is involved. To elucidate this issue, we measured a component of the human event-related brain potential, called the reward positivity, that is said to reflect a reward prediction error signal generated in the MPFC. Using a task paradigm involving reinforcers at two levels of hierarchy, we show that reward positivity amplitude is sensitive to the valence of low-level pseudo-rewards but, contrary to our expectation, is not modulated by high-level rewards. Further, reward positivity amplitude to low-level feedback is modulated by the goals of the higher level. These results, which were further replicated in a control experiment, suggest that the MPFC is involved in the processing of rewards at multiple levels of hierarchy
Goal impact influences the evaluative component of performance monitoring : evidence from ERPs
Successful performance monitoring (PM) requires continuous assessment of context and action outcomes. Electrophysiological studies have reliably identified event-related potential (ERP) markers for evaluative feedback processing during PM: the Feedback-Related Negativity (FRN) and P3 components. The functional significance of FRN remains debated in the literature, with recent research suggesting that feedback's goal relevance can account for FRN (amplitude) modulation, apart from its valence or expectedness alone. Extending this account, the present study assessed whether graded differentiations in feedback's relevance or importance to one's goal (referred to as goal impact) would influence PM at the FRN (and P3) level. To this end, we ran a within-subject crossover design experiment in which 40 participants completed two standard cognitive control tasks (Go/No Go and Simon), while 64-channel electroencephalography was recorded. Critically, both tasks entailed similar reward processing but systematically varied in goal impact assignment (high vs. low), manipulated through their supposed diagnosticity for daily life functioning and activation of social comparison. ERP results showed that goal impact reliably modulated FRN in a general manner. Irrespective of feedback valence, it was overall less negative in the high compared to the low impact condition, suggesting a general decrease in feedback monitoring in the former compared to the latter condition. These findings lend support to the idea that PM is best conceived operating not solely based on motor cues, but is shaped by motivational demands
Digital supply chain through dynamic inventory and smart contracts
This paper develops a digital supply chain game, modeling marketing and operation interactions between members. The main novelty of the paper concerns a comparison between static and dynamic solutions of the supply chain game achieved when moving from traditional to digital platforms. Therefore, this study proposes centralized and decentralized versions of the game, comparing their solutions under static and dynamic settings. Moreover, it investigates the decentralized supply chain by evaluating two smart contracts: Revenue sharing and wholesale price contracts. In both cases, the firms use an artificial intelligence system to determine the optimal contract parameters. Numerical and qualitative analyses are used for comparing configurations (centralized, decentralized), settings (static, dynamic), and contract schemes (revenue sharing contract, wholesale price contract). The findings identify the conditions under which smart revenue sharing mechanisms are worth applying
The potential of error-related potentials. Analysis and decoding for control, neuro-rehabilitation and motor substitution
Las interfaces cerebro-máquina (BMIs, por sus siglas en inglĂ©s) permiten la decodificaciĂłn de patrones de activaciĂłn neuronal del cerebro de los usuarios para proporcionar a personas con movilidad severamente limitada, ya sea debido a un accidente o a una enfermedad neurodegenerativa, una forma de establecer una conexiĂłn directa entre su cerebro y un dispositivo. En este sentido, las BMIs basadas en tĂ©cnicas no invasivas, como el electroencefalograma (EEG) han ofrecido a estos usuarios nuevas oportunidades para recuperar el control sobre las actividades de su vida diaria que de otro modo no podrĂan realizar, especialmente en las áreas de comunicaciĂłn y control de su entorno.En los Ăşltimos años, la tecnologĂa está avanzando a grandes pasos y con ella la complejidad de dispositivos ha incrementado significativamente, ampliando el nĂşmero de posibilidades para controlar sofisticados dispositivos robĂłticos, prĂłtesis con numerosos grados de libertad o incluso para la aplicaciĂłn de complejos patrones de estimulaciĂłn elĂ©ctrica en las propias extremidades paralizadas de un usuario, que le permitan ejecutar movimientos precisos. Sin embargo, la cantidad de informaciĂłn que se puede transmitir entre el cerebro y estos dispositivos sigue siendo muy limitada, tanto por el nĂşmero como por la velocidad a la que se pueden decodificar los comandos neuronales. Por lo tanto, depender Ăşnicamente de las señales neuronales no garantiza un control Ăłptimo y preciso.Para poder sacar el máximo partido de estas tecnologĂas, el campo de las BMIs adoptĂł el conocido enfoque de “control-compartido". Esta estrategia de control pretende crear un sistema de cooperaciĂłn entre el usuario y un dispositivo inteligente, liberando al usuario de las tareas más pesadas requeridas para ejecutar la tarea sin llegar a perder la sensaciĂłn de estar en control. De esta manera, los usuarios solo necesitan centrar su atenciĂłn en los comandos de alto nivel (por ejemplo, elegir un elemento especĂfico que agarrar, o elegir el destino final donde moverse) mientras el agente inteligente resuelve problemas de bajo nivel (como planificaciĂłn de trayectorias, esquivar obstáculos, etc.) que permitan realizar la tarea designada de la manera Ăłptima.En particular, esta tesis gira en torno a una señal neuronal cognitiva de alto nivel originada como la falta de coincidencia entre las expectativas del usuario y las acciones reales ejecutadas por los dispositivos inteligentes. Estas señales, denominadas potenciales de error (ErrPs), se consideran una forma natural de intercomunicar nuestro cerebro con máquinas y, por lo tanto, los usuarios solo requieren monitorizar las acciones de un dispositivo y evaluar mentalmente si este Ăşltimo se comporta correctamente o no. Esto puede verse como una forma de supervisar el comportamiento del dispositivo, en el que la decodificaciĂłn de estas evaluaciones mentales se utiliza para proporcionar a estos dispositivos retroalimentaciĂłn directamente relacionada con la ejecuciĂłn de una tarea determinada para que puedan aprender y adaptarse a las preferencias del usuario.Dado que la respuesta neuronal de ErrP está asociada a un evento exĂłgeno (dispositivo que comete una acciĂłn errĂłnea), la mayorĂa de los trabajos desarrollados han intentado distinguir si una acciĂłn es correcta o errĂłnea mediante la explotaciĂłn de eventos discretos en escenarios bien controlados. Esta tesis presenta el primer intento de cambiar hacia configuraciones asĂncronas que se centran en tareas relacionadas con el aumento de las capacidades motoras, con el objetivo de desarrollar interfaces para usuarios con movilidad limitada. En este tipo de configuraciones, dos desafĂos importantes son que los eventos correctos o errĂłneos no están claramente definidos y los usuarios tienen que evaluar continuamente la tarea ejecutada, mientras que la clasificaciĂłn de las señales EEG debe realizarse de forma asĂncrona. Como resultado, los decodificadores tienen que lidiar constantemente con la actividad EEG de fondo, que tĂpicamente conduce a una gran cantidad de errores de detecciĂłn de firmas de error. Para superar estos desafĂos, esta tesis aborda dos lĂneas principales de trabajo.Primero, explora la neurofisiologĂa de las señales neuronales evocadas asociadas con la percepciĂłn de errores durante el uso interactivo de un BMI en escenarios continuos y más realistas.Se realizaron dos estudios para encontrar caracterĂsticas alternativas basadas en el dominio de la frecuencia como una forma de lidiar con la alta variabilidad de las señales del EEG. Resultados, revelaron que existe un patrĂłn estable representado como oscilaciones "theta" que mejoran la generalizaciĂłn durante la clasificaciĂłn. Además, se utilizaron tĂ©cnicas de aprendizaje automático de Ăşltima generaciĂłn para aplicar el aprendizaje de transferencia para discriminar asincrĂłnicamente los errores cuando se introdujeron de forma gradual y no se conoce presumiblemente el inicio que desencadena los ErrPs. Además, los análisis de neurofisiologĂa arrojan algo de luz sobre los mecanismos cognitivos subyacentes que provocan ErrP durante las tareas continuas, lo que sugiere la existencia de modelos neuronales en nuestro cerebro que acumulan evidencia y solo toman una decisiĂłn al alcanzar un cierto umbral. En segundo lugar, esta tesis evalĂşa la implementaciĂłn de estos potenciales relacionados con errores en tres aplicaciones orientadas al usuario. Estos estudios no solo exploran cĂłmo maximizar el rendimiento de decodificaciĂłn de las firmas ErrP, sino que tambiĂ©n investigan los mecanismos neuronales subyacentes y cĂłmo los diferentes factores afectan las señales provocadas.La primera aplicaciĂłn de esta tesis presenta una nueva forma de guiar a un robot mĂłvil que se mueve en un entorno continuo utilizando solo potenciales de error como retroalimentaciĂłn que podrĂan usarse para el control directo de dispositivos de asistencia. Con este propĂłsito, proponemos un algoritmo basado en el emparejamiento de polĂticas para el aprendizaje de refuerzo inverso para inferir el objetivo del usuario a partir de señales cerebrales.La segunda aplicaciĂłn presentada en esta tesis contempla los primeros pasos hacia un BCI hĂbrido para ejecutar distintos tipos de agarre de objetos, con el objetivo de ayudar a las personas que han perdido la funcionalidad motora de su extremidad superior. Este BMI combina la decodificaciĂłn del tipo de agarre a partir de señales de EEG obtenidas del espectro de baja frecuencia con los potenciales de error provocados como resultado de la monitorizaciĂłn de movimientos de agarre errĂłneos. Los resultados muestran que, en efecto los ErrP aparecen en combinaciones de señales motoras originadas a partir de movimientos de agarre consistentes en una Ăşnica repeticiĂłn. Además, la evaluaciĂłn de los diferentes factores involucrados en el diseño de la interfaz hĂbrida (como la velocidad de los estĂmulos, el tipo de agarre o la tarea mental) muestra cĂłmo dichos factores afectan la morfologĂa del subsiguiente potencial de error evocado.La tercera aplicaciĂłn investiga los correlatos neuronales y los procesos cognitivos subyacentes asociados con desajustes somatosensoriales producidos por perturbaciones inesperadas durante la estimulaciĂłn elĂ©ctrica neuromuscular en el brazo de un usuario. Este estudio simula los posibles errores que ocurren durante la terapia de neuro-rehabilitaciĂłn, en la que la activaciĂłn simultánea de la estimulaciĂłn aferente mientras los sujetos se concentran en la realizaciĂłn de una tarea motora es crucial para una recuperaciĂłn Ăłptima. Los resultados muestran que los errores pueden aumentar la atenciĂłn del sujeto en la tarea y desencadenar mecanismos de aprendizaje que al mismo tiempo podrĂan promover la neuroplasticidad motora.En resumen, a lo largo de esta tesis, se han diseñado varios paradigmas experimentales para mejorar la comprensiĂłn de cĂłmo se generan los potenciales relacionados con errores durante el uso interactivo de BMI en aplicaciones orientadas al usuario. Se han propuesto diferentes mĂ©todos para pasar de la configuraciĂłn bloqueada en el tiempo a la asĂncrona, tanto en tĂ©rminos de decodificaciĂłn como de percepciĂłn de los eventos errĂłneos; y ha explorado tres aplicaciones relacionadas con el aumento de las capacidades motoras, en las cuales los ErrPs se pueden usar para el control de dispositivos, la sustituciĂłn de motores y la neuro-rehabilitaciĂłn.Brain-machine interfaces (BMIs) allow the decoding of cortical activation patterns from the users brain to provide people with severely limited mobility, due to an accident or disease, a way to establish a direct connection between their brain and a device. In this sense, BMIs based in noninvasive recordings, such as the electroencephalogram (EEG) have o↵ered these users new opportunities to regain control over activities of their daily life that they could not perform otherwise, especially in the areas of communication and control of their environment. Over the past years and with the latest technological advancements, devices have significantly grown on complexity expanding the number of possibilities to control complex robotic devices, prosthesis with numerous degrees of freedom or even to apply compound patterns of electrical stimulation on the subjects own paralyzed extremities to execute precise movements. However, the band-with of communication between brain and devices is still very limited, both in terms of the number and the speed at which neural commands can be decoded, and thus solely relying on neural signals do not guarantee accurate control them. In order to benefit of these technologies, the field of BMIs adopted the well-known approach of shared-control. This strategy intends to create a cooperation system between the user and an intelligent device, liberating the user from the burdensome parts of the task without losing the feeling of being in control. Here, users only need to focus their attention on high-level commands (e.g. choose the final destination to reach, or a specific item to grab) while the intelligent agent resolve low-level problems (e.g. trajectory planning, obstacle avoidance, etc) to perform the designated task in the optimal way. In particular, this thesis revolves around a high-level cognitive neural signal originated as the mismatch between the expectations of the user and the actual actions executed by the intelligent devices. These signals, denoted as error-related potentials (ErrPs), are thought as a natural way to intercommunicate our brain with machines and thus users only require to monitor the actions of a device and mentally assess whether the latter is behaving correctly or not. This can be seen as a way to supervise the device’s behavior, in which the decoding of these mental assessments is used to provide these devices with feedback directly related with the performance of a given task so they can learn and adapt to the user’s preferences. Since the ErrP’s neural response is associated to an exogenous event (device committing an erroneous action), most of the developed works have attempted to distinguish whether an action is correct or erroneous by exploiting discrete events under well-controlled scenarios. This thesis presents the first attempt to shift towards asynchronous settings that focus on tasks related with the augmentation of motor capabilities, with the objective of developing interfaces for users with limited mobility. In this type of setups, two important challenges are that correct or erroneous events are not clearly defined and users have to continuously evaluate the executed task, while classification of EEG signals has to be performed asynchronously. As a result, the decoders have to constantly deal with background EEG activity, which typically leads to a large number of missdetection of error signatures. To overcome these challenges, this thesis addresses two main lines of work. First, it explores the neurophysiology of the evoked neural signatures associated with the perception of errors during the interactive use of a BMI in continuous and more realistic scenarios. Two studies were performed to find alternative features based on the frequency domain as a way of dealing with the high variability of EEG signals. Results, revealed that there exists a stable pattern represented as theta oscillations that enhance generalization during classification. Also, state-of-the-art machine learning techniques were used to apply transfer learning to asynchronously discriminate errors when they were introduced in a gradual fashion and the onset that triggers the ErrPs is not presumably known. Furthermore, neurophsysiology analyses shed some light about the underlying cognitive mechanisms that elicit ErrP during continuous tasks, suggesting the existence of neural models in our brain that accumulate evidence and only take a decision upon reaching a certain threshold. Secondly, this thesis evaluates the implementation of these error-related potentials in three user-oriented applications. These studies not only explore how to maximize the decoding performance of ErrP signatures but also investigate the underlying neural mechanisms and how di↵erent factors a↵ect the elicited signals. The first application of this thesis presents a new way to guide a mobile robot moving in a continuous environment using only error potentials as feedback which could be used for the direct control of assistive devices. With this purpose, we propose an algorithm based on policy matching for inverse reinforcement learning to infer the user goal from brain signals. The second application presented in this thesis contemplates the first steps towards a hybrid BMI for grasping oriented to assist people who have lost motor functionality of their upper-limb. This BMI combines the decoding of the type of grasp from low-frequency EEG signals with error-related potentials elicited as the result of monitoring an erroneous grasping. The results show that ErrPs are elicited in combination of motor signatures from the low-frequency spectrum originated from single repetition grasping tasks and evaluates how di↵erent design factors (such as the speed of the stimuli, type of grasp or mental task) impact the morphology of the subsequent evoked ErrP. The third application investigates the neural correlates and the underlying cognitive processes associated with somatosensory mismatches produced by unexpected disturbances during neuromsucular electrical stimulation on a user’s arm. This study simulates possible errors that occur during neurorehabilitation therapy, in which the simultaneous activation of a↵erent stimulation while the subjects are concentrated in performing a motor task is crucial for optimal recovery. The results showed that errors may increase subject’s attention on the task and trigger learning mechanisms that at the same time could promote motor neuroplasticity. In summary, throughout this thesis, several experimental paradigms have been designed to improve the understanding of how error-related potentials are generated during the interactive use of BMIs in user-oriented applications. Di↵erent methods have been proposed to shift from time-locked to asynchronous settings, both in terms of decoding and perception of the erroneous events; and it has explored three applications related with the augmentation of motor capabilities, in which ErrPs can be used for control of devices, motor substitution and neurorehabilitation.<br /
The brain as a generative model: information-theoretic surprise in learning and action
Our environment is rich with statistical regularities, such as a sudden cold gust of wind indicating a potential change in weather. A combination of theoretical work and empirical evidence suggests that humans embed this information in an internal representation of the world. This generative model is used to perform probabilistic inference, which may be approximated through surprise minimization. This process rests on current beliefs enabling predictions, with expectation violation amounting to surprise. Through repeated interaction with the world, beliefs become more accurate and grow more certain over time. Perception and learning may be accounted for by minimizing surprise of current observations, while action is proposed to minimize expected surprise of future events. This framework thus shows promise as a common formulation for different brain functions.
The work presented here adopts information-theoretic quantities of surprise to investigate both perceptual learning and action. We recorded electroencephalography (EEG) of participants in a somatosensory roving-stimulus paradigm and performed trial-by-trial modeling of cortical dynamics. Bayesian model selection suggests early processing in somatosensory cortices to encode confidence-corrected surprise and subsequently Bayesian surprise. This suggests the somatosensory system to signal surprise of observations and update a probabilistic model learning transition probabilities. We also extended this framework to include audition and vision in a multi-modal roving-stimulus study. Next, we studied action by investigating a sensitivity to expected Bayesian surprise. Interestingly, this quantity is also known as information gain and arises as an incentive to reduce uncertainty in the active inference framework, which can correspond to surprise minimization. In comparing active inference to a classical reinforcement learning model on the two-step decision-making task, we provided initial evidence for active inference to better account for human model-based behaviour. This appeared to relate to participants’ sensitivity to expected Bayesian surprise and contributed to explaining exploration behaviour not accounted for by the reinforcement learning model. Overall, our findings provide evidence for information-theoretic surprise as a model for perceptual learning signals while also guiding human action.Unsere Umwelt ist reich an statistischen Regelmäßigkeiten, wie z. B. ein plötzlicher kalter Windstoß, der einen möglichen Wetterumschwung ankündigt. Eine Kombination aus theoretischen Arbeiten und empirischen Erkenntnissen legt nahe, dass der Mensch diese Informationen in eine interne Darstellung der Welt einbettet. Dieses generative Modell wird verwendet, um probabilistische Inferenz durchzuführen, die durch Minimierung von Überraschungen angenähert werden kann. Der Prozess beruht auf aktuellen Annahmen, die Vorhersagen ermöglichen, wobei eine Verletzung der Erwartungen einer Überraschung gleichkommt. Durch wiederholte Interaktion mit der Welt nehmen die Annahmen mit der Zeit an Genauigkeit und Gewissheit zu. Es wird angenommen, dass Wahrnehmung und Lernen durch die Minimierung von Überraschungen bei aktuellen Beobachtungen erklärt werden können, während Handlung erwartete Überraschungen für zukünftige Beobachtungen minimiert. Dieser Rahmen ist daher als gemeinsame Bezeichnung für verschiedene Gehirnfunktionen vielversprechend.
In der hier vorgestellten Arbeit werden informationstheoretische Größen der Überraschung verwendet, um sowohl Wahrnehmungslernen als auch Handeln zu untersuchen. Wir haben die Elektroenzephalographie (EEG) von Teilnehmern in einem somatosensorischen Paradigma aufgezeichnet und eine trial-by-trial Modellierung der kortikalen Dynamik durchgeführt. Die Bayes'sche Modellauswahl deutet darauf hin, dass frühe Verarbeitung in den somatosensorischen Kortizes confidence corrected surprise und Bayesian surprise kodiert. Dies legt nahe, dass das somatosensorische System die Überraschung über Beobachtungen signalisiert und ein probabilistisches Modell aktualisiert, welches wiederum Wahrscheinlichkeiten in Bezug auf Übergänge zwischen Reizen lernt. In einer weiteren multimodalen Roving-Stimulus-Studie haben wir diesen Rahmen auch auf die auditorische und visuelle Modalität ausgeweitet. Als Nächstes untersuchten wir Handlungen, indem wir die Empfindlichkeit gegenüber der erwarteten Bayesian surprise betrachteten. Interessanterweise ist diese informationstheoretische Größe auch als Informationsgewinn bekannt und stellt, im Rahmen von active inference, einen Anreiz dar, Unsicherheit zu reduzieren. Dies wiederum kann einer Minimierung der Überraschung entsprechen. Durch den Vergleich von active inference mit einem klassischen Modell des Verstärkungslernens (reinforcement learning) bei der zweistufigen Entscheidungsaufgabe konnten wir erste Belege dafür liefern, dass active inference menschliches modellbasiertes Verhalten besser abbildet. Dies scheint mit der Sensibilität der Teilnehmer gegenüber der erwarteten Bayesian surprise zusammenzuhängen und trägt zur Erklärung des Explorationsverhaltens bei, das jedoch nicht vom reinforcement learning-Modell erklärt werden kann. Insgesamt liefern unsere Ergebnisse Hinweise für Formulierungen der informationstheoretischen Überraschung als Modell für Signale wahrnehmungsbasierten Lernens, die auch menschliches Handeln steuern
Prediction error dependent changes in brain connectivity during associative learning
One of the fundaments of associative learning theories is that surprising events drive
learning by signalling the need to update one’s beliefs. It has long been suggested
that plasticity of connection strengths between neurons underlies the learning of
predictive associations: Neural units encoding associated entities change their
connectivity to encode the learned associative strength. Surprisingly, previous
imaging studies have focused on correlations between regional brain activity and
variables of learning models, but neglected how these variables changes in interregional
connectivity. Dynamic Causal Models (DCMs) of neuronal populations and
their effective connectivity form a novel technique to investigate such learning
dependent changes in connection strengths.
In the work presented here, I embedded computational learning models into DCMs to
investigate how computational processes are reflected by changes in connectivity.
These novel models were then used to explain fMRI data from three associative
learning studies. The first study integrated a Rescorla-Wagner model into a DCM
using an incidental learning paradigm where auditory cues predicted the
presence/absence of visual stimuli. Results showed that even for behaviourally
irrelevant probabilistic associations, prediction errors drove the consolidation of
connection strengths between the auditory and visual areas. In the second study I
combined a Bayesian observer model and a nonlinear DCM, using an fMRI
paradigm where auditory cues differentially predicted visual stimuli, to investigate
how predictions about sensory stimuli influence motor responses. Here, the degree of
striatal prediction error activity controlled the plasticity of visuo-motor connections.
In a third study, I used a nonlinear DCM and data from a fear learning study to
demonstrate that prediction error activity in the amygdala exerts a modulatory
influence on visuo-striatal connections.
Though postulated by many models and theories about learning, to our knowledge
the work presented in this thesis constitutes the first direct report that prediction
errors can modulate connection strength
- …