On the computational and neural characterisation of reward learning behaviour

Do we learn differently from better- or worse-than-expected decision outcomes? Over the past decades, converging evidence emerged about the crucial role of the dopaminergic system in guiding learning through signalling reward prediction errors. However, a complete characterisation of how this learning process is influenced by feedback valence, surprise, and uncertainty is still lacking. The current thesis focuses on exploring the differential behavioural and neural mechanisms related to learning from positive versus negative decision outcomes whilst examining the influence of uncertainty on these processes. In our first experiment, we collected simultaneous EEG and eye-tracking data during a probabilistic reversal learning task. Using multivariate EEG analysis, we replicated the two distinct spatiotemporal reward learning systems reported by Fouragnan and colleagues (2015). Given that locus-coeruleus-noradrenaline (LC-NA) activity is difficult to directly measure non-invasively in humans, we used the pupil response as a proxy for LC-NA activity. We showed that the increased feedback-related pupil response to negative compared to positive outcomes is exclusively driven by increased negative feedback processing in the early and the late system. Additionally, a stronger coupling between early, but not late, system activity and the feedback-evoked pupil response was linked to reduced performance, increased uncertainty as well as exploration propensity. In line with existing research indicating the LC-NA network in uncertainty signalling and network resets, we propose that when internal estimates of environmental uncertainty surge in response to negative feedback, the early system, regulated by noradrenergic activity, interrupts processing in structures of the late system. Such network resets may aid flexible adaptation to changing environments by simultaneously reducing the influence of learned value representations and increasing the neural gain of new information. Our second experimental chapter extended the above study by examining post-feedback response adaptation as a function of early and late system activity. Specifically, we utilised hierarchical drift diffusion modelling, in which the drift rate and boundary separation were constrained by trial-wise and valence-specific early and late system activity. We hypothesised that an LC-NA-induced interruption in reward learning structures would reduce subsequent evidence accumulation as learned value representations become less influential and participants consider a reversal in reward contingencies more likely. Consistent with this hypothesis, we found that increased negative feedback processing by the early and late system reduced evidence accumulation in the next trial. Furthermore, a stronger association between the feedback-locked pupil response and early system activity following negative outcomes was significantly associated with the degree of drift rate reduction prompted by the early system. This result implies that LC-NA mediated network resets may be primarily associated with the early system, which in turn may down-regulate late system activity. Our final study explored differential value learning in the Balloon Analogue Risk Task (BART) under varying levels of uncertainty. By deriving differential learning rates from the newly developed Scaled Target Learning model, we showed that participants preferentially learn from positive compared to negative feedback under increased levels of uncertainty. Furthermore, the degree of this learning bias was negatively related to performance under the highest level of uncertainty. These results provide further evidence for differential mechanisms implicated in positive and negative feedback processing and indicate the important modulatory role of uncertainty in reward learning. Together, this thesis provides novel insights on the valence-specific neural and behavioural characteristics associated with feedback processing. Our results also highlight the important modulatory role uncertainty and noradrenaline play in reward learning and thus provide a more complete depiction of reward learning behaviour

Do informal caregivers of people with dementia mirror the cognitive deficits of their demented patients?:A pilot study

Recent research suggests that informal caregivers of people with dementia (ICs) experience more cognitive deficits than noncaregivers. The reason for this is not yet clear. Objective: to test the hypothesis that ICs ‘mirror' the cognitive deficits of the demented people they care for. Participants and methods: 105 adult ICs were asked to complete three neuropsychological tests: letter fluency, category fluency, and the logical memory test from the WMS-III. The ICs were grouped according to the diagnosis of their demented patients. One-sample ttests were conducted to investigate if the standardized mean scores (t-scores) of the ICs were different from normative data. A Bonferroni correction was used to correct for multiple comparisons. Results: 82 ICs cared for people with Alzheimer's dementia and 23 ICs cared for people with vascular dementia. Mean letter fluency score of the ICs of people with Alzheimer's dementia was significantly lower than the normative mean letter fluency score, p = .002. The other tests yielded no significant results. Conclusion: our data shows that ICs of Alzheimer patients have cognitive deficits on the letter fluency test. This test primarily measures executive functioning and it has been found to be sensitive to mild cognitive impairment in recent research. Our data tentatively suggests that ICs who care for Alzheimer patients also show signs of cognitive impairment but that it is too early to tell if this is cause for concern or not

Individual differences in risk preference: insights from self-report, behavioral and neural measures, and their convergence

From the time of conception until the time of death, the path of the human organism is created and shaped by decisions. Some decisions we make ourselves, some are made for us; some will make us, some will break us. What most decisions have in common, however, is that they are made under risk, that is, without complete information regarding the potential decision outcomes. One interesting feature about decisions under risk is variability: different individuals make different choices, and even the same individual may, given repeated occasions, make different choices. This doctoral thesis aims to address the issue of individual differences by looking at several specific variables which may impact inter- and intra-individual differences in risk taking, namely age, the measures used to assess risk-taking, neural function and neural structure. In a set of four studies, the following questions were addressed: (1) To what extent do life span trajectories of risk taking change as a function of whether self-report or behavioral measures are used to assess risk taking? (2) Do younger and older individuals differ in the neural functional representation of risk and reward? (3) Do the neural representations of described and experienced risk converge, both at group and individual level? To what extent is neural function predictive of risky choice? (4) To what extent do individual differences in neural structure explain variance in psychometrically derived risk preference factors? The main findings are: (1) Self-report and behavioral measures of risk taking do not converge and lead to different life span trajectories. (2) The ventromedial prefrontal cortex is differentially activated in younger and older adults, with activation differences possessing differential explanatory power for choice in the two age groups. (3) Described and experienced risks show convergence at group level, divergence at the individual level, and are differentially predictive of risky choice. (4) Neural structural indices explain variance in the general risk preference factor, but not domain-specific risk preference factors. Based on the findings from all four studies, this thesis provides corroborating evidence for the argument that not all risk-taking measures are created equal and that a taxonomy of risk-taking measures and their respective cognitive and affective demands is required to understand individual differences in risk taking

ICS Materials. Towards a re-Interpretation of material qualities through interactive, connected, and smart materials.

The domain of materials for design is changing under the influence of an increased technological advancement, miniaturization and democratization. Materials are becoming connected, augmented, computational, interactive, active, responsive, and dynamic. These are ICS Materials, an acronym that stands for Interactive, Connected and Smart. While labs around the world are experimenting with these new materials, there is the need to reflect on their potentials and impact on design. This paper is a first step in this direction: to interpret and describe the qualities of ICS materials, considering their experiential pattern, their expressive sensorial dimension, and their aesthetic of interaction. Through case studies, we analyse and classify these emerging ICS Materials and identified common characteristics, and challenges, e.g. the ability to change over time or their programmability by the designers and users. On that basis, we argue there is the need to reframe and redesign existing models to describe ICS materials, making their qualities emerge

Because It is Wrong : An Essay on the Immorality and Illegality of the Online Service Contracts of Google and Facebook

This essay argues that the behavioral-advertising business model under which an internet platform, such as Google or Facebook, provides free services in exchange for the user’s personal data is immoral and illegal. It is immoral because it relies on addiction, surveillance, and manipulation of the user to deplete the user’s autonomy. The contract between the company and the user is immoral. It can also be plausibly argued that the contract is illegal under California law because it is contrary to good morals, is unconscionable, and is against public policy. As society becomes more aware of these moral and legal defects, courts in the future should be more willing to find these contracts illegal and thus void. In such case, the user’s consent to the contract would be nullified and the company would have no legal right to gather and monetize the personal data of the user. The companies should then be forced to convert to a subscription model with a fiduciary duty to users to restrict the gathering and monetizing of personal data. This essay employs perspectives not only from morality and law, but also from philosophy, history, political theory, and neuroscience. Part One covers morality, Part Two legality

Contributions of Computational Cognitive Modeling to the Understanding of the Financial Markets

Tese de doutoramento em Ciências e Tecnologias da Informação, apresentada ao Departamento de Engenharia Informática da Faculdade de Ciências e Tecnologia da Universidade de CoimbraOs mercados financeiros são sistemas socioeconómicos complexos, dinâmicos e estratégicos nos quais um grande número de heterogéneos participantes interagem por meio da compra e venda de diferentes tipos de ativos. Os mercados financeiros tais como os mercados de ações (e.g., S&P500) possuem múltiplas funções. Por exemplo, os mercados financeiros propiciam meios para que os participantes e as companhias façam um melhor processo de alocação de capital. Para além disso, o comportamento dos mercados financeiros são geralmente considerados como importantes medidas/sinais para a compreensão do estado atual e futuro das companhias e, em última análise, de todo o sistema económico e financeiro. Entretanto, a importância dos mercados financeiros poderá ser melhor compreendida quando os mercados não cumprem suas funções primordias, mais especificamente e de forma dramática quando ocorre um crash nos mercados financeiros. Em Setembro de 2008, por exemplo, uma série de eventos ameaçou a estabilidade do sistema financeiro mundial. Gigantescas empresas dos mercados financeiros inesperadamente falharam e tiveram de ser resgatadas pelos seus respectivos governos, enquanto que outras simplesmente entraram com pedido de falência. O sistema financeiro mundial esteve próximo do colapso. Embora o desastre tenha, de certa forma, sido evitado, o Crash de 2008 teve consequências imediatas, profundas, e duradouras para a economia mundial. A verdadeira compreensão dos mercados financeiros é de fato difícil. Diferentes hipóteses têm sido propostas para explicar o comportamento dos participantes dos mercados financeiros de forma individual bem como o comportamento dos mercados de forma global. Por um lado, teorias económicas tradicionais como, por exemplo, a Hipótese do Mercado Eficiente, tendem a considerar que os participantes dos mercados financeiros são racionais e que os mercados são eficientes. Entretanto, pesquisas na área de economia comportamental têm fornecido extensa e vasta evidência que demonstra que os participantes dos mercados financeiros possuem desvios comportamentais do chamado comportamento racional. Para além das evidências da economia comportamental, disciplinas tais como a neurociência cognitiva e a neuroeconomia têm clarificado a função das emoções (e.g., felicidade, tristeza, surpresa) no processo de raciocínio e tomada de decisão, aprendizado, bem como a importância das emoções no âmbito da memória humana, particularmente para os processos de codificação e recuperação de memórias. Por outro lado, teorias recentes como a Hipótese do Mercado Adaptativo tentam reconciliar a ideia de mercados eficientes com a economia comportamental, ao reconhecer a importância das emoções, a existência de desvios comportamentais, e a ocorrência de fenómenos e anomalias como as bolhas. O Crash de 2008 conjuntamente com novas evidências fornecidas por diferentes áreas têm salientado a necessidade de novas e interdisciplinares abordagens para o estudo de sistemas e problemas económicos e financeiros. Uma destas abordagens é o uso de Agent-based Financial Markets. Esta abordagem permite aos investigadores se distanciarem das tradicionais crenças a fim de testar novas hipóteses, conceitos, ideias, etc, tornando possível o projeto e realização de experimentos mais realistas e mais plausíveis em termos comportamentais. Esta tese está em linha com este contexto. Neste trabalho exploratório, nosso objetivo é investigar quais contribuições a aplicação de uma abordagem de modelação cognitiva pode trazer para a compreensão dos mercados financeiros, especificamente para o comportamento dos participantes dos mercados financeiros (individualmente) e dos mercados financeiros (globalmente). O ponto de partida é a criação de agentes artificiais com mecanismos similares aos ou inspirados nos usados pelos seres humanos de modo que seja possível conceber agentes artificiais cognitivos, i.e., agentes artificiais com diferentes sistemas de memórias e processos, com a capacidade de reconhecer, simular, e expressar emoções, diferentes processos de tomada de decisão, com a habilidade de receber e processar diferentes tipos de informação, e com a habilidade de aprender. Para este fim, nós primeiro concebemos um modelo cognitivo genérico individual dos participantes dos mercados financeiros (agentes humanos) intitulado TribeCA (Trading and investing with behavioral-emotional Cognitive Agents). O modelo cognitivo proposto é baseado na Belief-Desire Theory of Emotions (BDTE), no modelo cognitive-psychoevolutionary de surpresa proposto por Myer e colegas, e no modelo de surpresa artificial proposto por Macedo e Cardoso. De seguida nós fornecemos uma implementação do modelo proposto a qual foi posteriormente integrada a duas ferramentas utilizadas no contexto dos agent-based financial markets. A plataforma resultante permite o projeto e realização de uma variedade de experimentos económicos e financeiros com agentes artificiais cognitivos. Nós realizamos neste trabalho três experimentos com agentes e multi-agentes a fim de endereçar alguns aspectos fundamentais dos mercados financeiros tais como eficiência e racionalidade. Adicionalmente, nós realizamos dois estudos de casos a fim de comparar a perspectiva tradicional (económica e financeira) com a perspectiva da ciência cognitiva na modelação e computação da surpresa na economia e finança. Esta tese fornece contribuições para o avanço no projeto e realização de abordagens interdisciplinares para o estudo de sistemas ou problemas económicos e financeiros. Nosso modelo cognitivo genérico e sua implementação podem ser utilizados a fim de que sejam explorados outros aspectos dos mercados financeiros, para além dos que foram endereçados nest trabalho, e em outros modelos baseados em agentes. Nós consideramos que este trabalho abre um novo conjunto de possibilidades para investigações quer na academia quer na indústria. Ao final, nós poderemos obter uma melhor compreensão e entendimento sobre o comportamento dos participantes dos mercados financeiros (individualmente) bem como dos mercados financeiros (globalmente). Estas investigações poderão resultar, por exemplo, no desenvolvimento de novos (potencialmente melhores e altamente lucrativos) serviços financeiros para suportar o processo de tomada de decisão dos participantes dos mercados financeiros baseados nas suas emoções, comportamentos, etc...........................................................The financial markets are complex, dynamic, and strategic socio-economic systems in which a great number of heterogeneous market participants interact by essentially buying and selling assets of di ff erent types. The financial markets such as stock markets (e.g., S&P500) serve many functions. For instance, the financial markets help market participants and companies in improving the capital allocation process. Additionally, the behavior of the financial markets is assumed to be an important gauge for helping the understanding of the current and future state of companies and, ultimately, of the whole economic and financial system. However, the importance of the financial markets might be better noticed when they do not fulfill their primary functions, specifically and most dramatically when the financial markets crash. On September 2008, for example, a series of events threated the stability of the world’s financial system. Some gigantic financial services companies had unexpectedly failed and had to be rescued by governments while others simply filled for bankruptcy. The world’s financial system came close to a meltdown. Although disaster had somehow been averted due to a series of actions, the Crash of 2008 had immediate, profound, vast and long-lasting consequences for the world economy. The true understanding of the financial markets are indeed quite di ffi cult. Several hy- potheses have been proposed to try to explain the behavior of the market participants individually as well as the behavior of markets as a whole. On the one hand, tradi- tional economic theories such as the E ffi cient Market Hypothesis tend to assume that market participants are rational as well as that markets are e ffi cient. However, beha- vioral economics research has been providing extensive and vast evidence that market participants have what is known as behavioral biases, i.e., deviations from the so-called rational behavior. In addition to the behavioral economics evidence, disciplines such as cognitive neuroscience and neuroeconomics have been clarifying the role of emotions (e.g., happiness, unhappiness, surprise) for the human reasoning, memory system and processes, and decision-making process. For instance, emotions play a very important role in the memory processes of encoding and retrieving as well as are the basis of a sort of learning system. On the other hand, recent theories such as the Adaptive Market Hypothesis tries to reconciliate market e ffi ciency with behavioral economics by acknowledging the importance of emotions, the existence of behavioral biases, and the occurrence of interesting phenomena and anomalies such as bubbles. The Crash of 2008 together with new evidence provided by di ff erent research areas have been stressing the need for novel and interdisciplinary approaches for the study of economic and financial systems and problems. One of these approaches is the use of Agent-based Financial Markets. Agent-based Financial Markets allows researchers to depart from classical assumptions in order to test di ff erent hypotheses, concepts, ideas, etc, making it possible the design and realization of more realistic and behavioral plausible experiments. This thesis is in line with this context. In this exploratory work we aim to investigate which contributions the application of a cognitive modeling approach might bring to the understanding of the financial markets, specifically to the behavior of both market participants (individually) and the financial markets (globally). The starting point is to empower artificial agents with mechanisms similar to or inspired in those used by humans so that we have artificial cognitive agents, i.e., artificial agents with di ff erent memory systems and processes, the capacity of recognizing, simulating and expressing emotions, decision-making processes, the ability to receive and process di ff erent kinds of information, and the ability to learn. To this end, we first conceive a generic novel cognitive model of individual market participants (human agents) named TribeCA (Trading and investing with behavioral- emotional Cognitive Agents). TribeCA is based on the Belief-Desire Theory of Emo- tions (BDTE), on the cognitive-psychoevolutionary model of surprise proposed by Meyer and colleagues, and on the artificial surprise model proposed by Macedo and Cardoso. Then we provide an implementation of the proposed model which is later integrated into two tools used in the context of agent-based financial markets. The re- sulting platform allows the design and realization of a variety of economic and financial experiments with artificial cognitive agents. We carried out three agent and multi- agent based experiments to address some fundamental aspects regarding the financial markets such as e ffi ciency and rationality. Additionally, we carried out two case studies on comparing the traditional (economics and finance) perspective with the cognitive science perspective on modeling and computing surprise in economics and finance. This thesis provides contributions to the advance in the design and realization of in- terdisciplinary approaches to the study of economic and financial systems or problems. Our generic conceptual cognitive model and implementation might be used both to explore other aspects of the financial markets in addition to those addressed in this work and to other agent-based models. We consider this work opens up a set of novel possibilities for investigations in the academia and in the industry. In the end, we may have a better understanding of the behavior of market participants individually as well as of the financial markets globally. It has the potential to result in, for instance, the development of novel (potentially better and highly lucrative) financial services to support the market participants’ decision-making process based on his/her emotions, behavior, etc.FEDER - project TribeCA (Trading and investing with behavioralemotional Cognitive Agents

Cross-National Policy Diffusion in States and Provinces

Cross-national policy sharing between legislatures of states in different countries is not well studied, possibly due to the existence of many domestic peer states whose laws provide the most frequently studied examples of cross-state borrowing. But in addition to these domestic examples, instances of cross-national policy transfer continue to arise. The theories that explain state-level policy diffusion among actors in the same country have not yet been extended to instances of cross-national policy proliferation. Therefore, this phenomenon of cross-national sub-national policy diffusion continues to play a role in legislative outcomes yet remains largely unexplained. This dissertation builds on the policy diffusion literature to investigate why cross-national learning occurs and under what conditions is it expected. It theorizes that legislators are motivated to study policies from states in other countries when lawmakers in those foreign states produce novel or innovative policies, or when legislators choose to undertake more thorough research to improve a policy that is not performing well at home. It further theorizes that state-level institutions and attributes associated with legislative professionalism affect capacity to research policy in states in other countries and synthesize best practices into new legislation in the home state. Hypotheses are tested using network analysis, generalized linear mixed models, and text analysis. Results suggest that many states of varying levels of professionalism and economic size are included in cross-national policy networks and that the state-level attributes of legislative professionalism, particularly staff levels, are important to providing the capacity to research foreign policies. However, these attributes are negatively associated with levels of textual similarity between the foreign policy originator and the domestic policy borrower. This indicates that professionalism attributes enable state policymakers to collect more best practices to synthesize into final policy documents and laws. Text analysis detects a reduced but meaningful level of textual similarity between the text of foreign policy originators and subsequent domestic borrowers, as compared to the level of policy similarity detected between two states in the same country. These findings propose answers to why legislators might opt to learn from states in other countries in addition to peers at home and provide insight into the conditions under which cross-national sub-national policy diffusion is more likely to occur. Examples of cross-national policy diffusion such as public bike sharing programs and primary seatbelt legislation suggest that many policies that lawmakers borrow from across international borders provide quantifiable benefits to the jurisdictions that adopt them. As public servants in states and provinces around the world continue to tackle similar policy issues, networks that foster the sharing of best practices have the potential to enhance cross-national learning and improve citizen quality of life more rapidly than when sub-national policymakers work in isolation.PHDPublic Policy & Political ScienceUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/162970/1/azbeatty_1.pd

"The authority of the steam" : power dynamics of digital production in the Bitcoin blockchain

This thesis offers a critical investigation of the Bitcoin currency and the operation of its technical structure, i.e. blockchain technology. The main objective of the research is to identify and describe the specific power dynamics performed by and through this digital phenomenon. “Power dynamics” are framed in this work largely in terms of authority and sovereignty. To structure an exploration of such dynamics, the narrative is overarched by four different notions of “utopia” —as paradox, ideal, no-place, and imagined governance— that address the following main questions always underpinned by the general inquiry on power: What is the Bitcoin Blockchain? Where is it located? How are power relations performed in it? And how are power relations modified in relation with previous institutional systems? The thesis addresses distinct notions of authority in Bitcoin through the observation of its historical, spatial, and organizational characteristics. It maps the techno-political emergence of the blockchain system, the geographical distribution of Bitcoin’s infrastructural network, and the strategies for governance involved in its development as software. Based on the observation of these settings, this thesis argues that Bitcoin posits a restructuration of power dynamics through the automation of code, in particular, through its process of production. In order to develop this restructuration, the power dynamics of the Bitcoin blockchain are weighted against authority models of the state’s institutions. The thesis builds upon existing political theories of Empire (Hardt and Negri), protocol (Galloway), and the Stack (Bratton) to develop a critical account of Bitcoin’s power dynamics. The work sits in between the disciplines of Media Theory, Software Studies, Political Theory, and Digital Methods, and makes use of qualitative and quantitative methods to empirically support the former argument