my Human Brain Project (mHBP)

How can we make an agent that thinks like us humans? An agent that can have proprioception, intrinsic motivation, identify deception, use small amounts of energy, transfer knowledge between tasks and evolve? This is the problem that this thesis is focusing on. Being able to create a piece of software that can perform tasks like a human being, is a goal that, if achieved, will allow us to extend our own capabilities to a very high level, and have more tasks performed in a predictable fashion. This is one of the motivations for this thesis. To address this problem, we have proposed a modular architecture for Reinforcement Learning computation and developed an implementation to have this architecture exercised. This software, that we call mHBP, is created in Python using Webots as an environment for the agent, and Neo4J, a graph database, as memory. mHBP takes the sensory data or other inputs, and produces, based on the body parts / tools that the agent has available, an output consisting of actions to perform. This thesis involves experimental design with several iterations, exploring a theoretical approach to RL based on graph databases. We conclude, with our work in this thesis, that it is possible to represent episodic data in a graph, and is also possible to interconnect Webots, Python and Neo4J to support a stable architecture for Reinforcement Learning. In this work we also find a way to search for policies using the Neo4J querying language: Cypher. Another key conclusion of this work is that state representation needs to have further research to find a state definition that enables policy search to produce more useful policies. The article “REINFORCEMENT LEARNING: A LITERATURE REVIEW (2020)” at Research Gate with doi 10.13140/RG.2.2.30323.76327 is an outcome of this thesis.Como podemos criar um agente que pense como nós humanos? Um agente que tenha propriocepção, motivação intrínseca, seja capaz de identificar ilusão, usar pequenas quantidades de energia, transferir conhecimento entre tarefas e evoluir? Este é o problema em que se foca esta tese. Ser capaz de criar uma peça de software que desempenhe tarefas como um ser humano é um objectivo que, se conseguido, nos permitirá estender as nossas capacidades a um nível muito alto, e conseguir realizar mais tarefas de uma forma previsível. Esta é uma das motivações desta tese. Para endereçar este problema, propomos uma arquitectura modular para computação de aprendizagem por reforço e desenvolvemos uma implementação para exercitar esta arquitetura. Este software, ao qual chamamos mHBP, foi criado em Python usando o Webots como um ambiente para o agente, e o Neo4J, uma base de dados de grafos, como memória. O mHBP recebe dados sensoriais ou outros inputs, e produz, baseado nas partes do corpo / ferramentas que o agente tem disponíveis, um output que consiste em ações a desempenhar. Uma boa parte desta tese envolve desenho experimental com diversas iterações, explorando uma abordagem teórica assente em bases de dados de grafos. Concluímos, com o trabalho nesta tese, que é possível representar episódios em um grafo, e que é, também, possível interligar o Webots, com o Python e o Neo4J para suportar uma arquitetura estável para a aprendizagem por reforço. Neste trabalho, também, encontramos uma forma de procurar políticas usando a linguagem de pesquisa do Neo4J: Cypher. Outra conclusão chave deste trabalho é que a representação de estados necessita de mais investigação para encontrar uma definição de estado que permita à pesquisa de políticas produzir políticas que sejam mais úteis. O artigo “REINFORCEMENT LEARNING: A LITERATURE REVIEW (2020)” no Research Gate com o doi 10.13140/RG.2.2.30323.76327 é um sub-produto desta tese

Balancing exploration and exploitation: task-targeted exploration for scientific decision-making

Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy at the Massachusetts Institute of Technology and the Woods Hole Oceanographic Institution September 2022.How do we collect observational data that reveal fundamental properties of scientific phenomena? This is a key challenge in modern scientific discovery. Scientific phenomena are complex—they have high-dimensional and continuous state, exhibit chaotic dynamics, and generate noisy sensor observations. Additionally, scientific experimentation often requires significant time, money, and human effort. In the face of these challenges, we propose to leverage autonomous decision-making to augment and accelerate human scientific discovery. Autonomous decision-making in scientific domains faces an important and classical challenge: balancing exploration and exploitation when making decisions under uncertainty. This thesis argues that efficient decision-making in real-world, scientific domains requires task-targeted exploration—exploration strategies that are tuned to a specific task. By quantifying the change in task performance due to exploratory actions, we enable decision-makers that can contend with highly uncertain real-world environments, performing exploration parsimoniously to improve task performance. The thesis presents three novel paradigms for task-targeted exploration that are motivated by and applied to real-world scientific problems. We first consider exploration in partially observable Markov decision processes (POMDPs) and present two novel planners that leverage task-driven information measures to balance exploration and exploitation. These planners drive robots in simulation and oceanographic field trials to robustly identify plume sources and track targets with stochastic dynamics. We next consider the exploration- exploitation trade-off in online learning paradigms, a robust alternative to POMDPs when the environment is adversarial or difficult to model. We present novel online learning algorithms that balance exploitative and exploratory plays optimally under real-world constraints, including delayed feedback, partial predictability, and short regret horizons. We use these algorithms to perform model selection for subseasonal temperature and precipitation forecasting, achieving state-of-the-art forecasting accuracy. The human scientific endeavor is poised to benefit from our emerging capacity to integrate observational data into the process of model development and validation. Realizing the full potential of these data requires autonomous decision-makers that can contend with the inherent uncertainty of real-world scientific domains. This thesis highlights the critical role that task-targeted exploration plays in efficient scientific decision-making and proposes three novel methods to achieve task-targeted exploration in real-world oceanographic and climate science applications.This material is based upon work supported by the NSF Graduate Research Fellowship Program and a Microsoft Research PhD Fellowship, as well as the Department of Energy / National Nuclear Security Administration under Award Number DE-NA0003921, the Office of Naval Research under Award Number N00014-17-1-2072, and DARPA under Award Number HR001120C0033

Human and Artificial Intelligence

Although tremendous advances have been made in recent years, many real-world problems still cannot be solved by machines alone. Hence, the integration between Human Intelligence and Artificial Intelligence is needed. However, several challenges make this integration complex. The aim of this Special Issue was to provide a large and varied collection of high-level contributions presenting novel approaches and solutions to address the above issues. This Special Issue contains 14 papers (13 research papers and 1 review paper) that deal with various topics related to human–machine interactions and cooperation. Most of these works concern different aspects of recommender systems, which are among the most widespread decision support systems. The domains covered range from healthcare to movies and from biometrics to cultural heritage. However, there are also contributions on vocal assistants and smart interactive technologies. In summary, each paper included in this Special Issue represents a step towards a future with human–machine interactions and cooperation. We hope the readers enjoy reading these articles and may find inspiration for their research activities

Linguistic Representation of Problem Solving Processes in Unaided Object Assembly

This thesis investigates the linguistic representation of problem solving processes in data recorded during unaided object assembly. It combines traditional approaches of analyzing verbal protocols with the recent approach of Cognitive Discourse Analysis

Massive modularity : an ontological hypothesis or an adaptationist discovery heuristic?

Cognitive modules are internal mental structures. Some theorists and empirical researchers hypothesize that the human mind is either partially or massively comprised of structures that are modular in nature. Modules are also invoked to explain cognitive capacities associated with the performance of specific functional tasks. Jerry Fodor (1983) considered that modules are useful only for explaining relatively low-level systems (input systems). These are the systems involved in capacities like perception and language. For Fodor, the central (high-level) systems of mind — those involved in capacities like judgment (the “fixation of belief” in Fodor’s jargon), planning (practical reasoning), decision-making, and so forth — are not explainable in terms of modular mechanisms. However, some other philosophers as well as proponents and practitioners of evolutionary psychology consider that it is sensible — and even necessary — to invoke modules for explaining these systems, too. Indeed, the debate over modularity is mainly a debate over the modularity of central systems (high-level cognitive capacities). Admittedly, it is also possible to raise doubts about the modularity of peripheral systems (Prinz, 2006), but this kind of skepticism is not widespread in the literature. In contrast, both the empirical and the theoretical (a priori) cases in favor of central modularity are usually contested on multiple fronts and in tough terms. The case for the modularity of central systems is the core of the case for the massive modularity of mind hypothesis (viz. the hypothesis that consists in asserting that both the peripheral and the central systems of human mind are largely composed of modules). Granted, the massive modularity of mind hypothesis is an empirical statement and, as such, its truth should be ultimately decided in an empirical way, not a priori (Sperber, 1994, 2001). Nonetheless, much of the debate over massive modularity has taken place on theoretical grounds. This is due to an alleged underdetermination of the hypotheses regarding the modularity of central systems by data. In such conditions, the remaining open option has been to advance theoretical considerations in favor of the plausibility — and not directly in favor of the empirical truth or truthlikeness — of the massive modularity of mind generally, and of the modularity of some central systems in particular. These theoretical considerations are mostly based on an adaptationist view of evolution cum a classical computationalist approach to mind. They include arguments about the nature and evolution of hierarchically ordered complex systems (the evolvability of complexity argument), a presumption of optimality expressed in the apparent design of phenotypic traits shaped by natural selection (the task-specificity argument) and dismissing nonmodular mental architectures as computationally intractable (the tractability argument). Is the massive modularity of mind hypothesis a cogent view about the ontological nature of human mind or is it, rather, an effective/ineffective adaptationist discovery heuristic for generating predictively successful hypothesis about both heretofore unknown psychological traits and unknown properties of already identified psychological traits? Considering the inadequacies of the case in favor of massive modularity as an ontological hypothesis, I suggest approaching and valuing massive modularity as an adaptationist discovery heuristic

Reflecting on Performance Feedback: The Effect of Counterfactual Thinking on Subsequent Leader Performance

Performance feedback is an integral aspect of facilitating employee learning. Despite its importance, research suggests that when that feedback conveys a performance discrepancy, subsequent performance does not improve. Researchers have advanced reflection as a strategy for increasing feedback effectiveness and have established its value for learning and performance improvement. However, these studies have not accounted for the effects of specific types of reflection on performance. To this point, the current research examines the role of one form of reflection, counterfactual thinking, for learning after performance discrepancies. I explored boundary conditions that might influence self-focused upward counterfactual thinking—a form of reflection particularly important for learning and performance improvement—and examined whether and when such thinking influences the relationship between a baseline performance discrepancy and subsequent performance. To investigate these issues, I designed, developed, and validated a computer simulated leadership skills task and administered it to graduate and undergraduate students (N= 169) in a web-based research setting. I tested the proposed relationships using conditional process analysis. The results of this study demonstrated that when individuals encounter performance discrepancies they might attempt to reconcile such through self-focused upward counterfactual thinking. This research represents a step toward an improved understanding of reflection, performance discrepancy feedback processing, and subsequent performance effects

Emotions and Sensemaking: How Anger, Guilt, and Emotion Regulation Impact Ethical Decision Making

Affect plays an important role in cognition and behavior, but how discrete emotions influence decision making is still unclear. To contribute to the understanding of this process, this study investigated the impact of guilt and anger in ethical decision making. By taking a sensemaking approach, the findings demonstrated that experiencing integral guilt and anger have important and differential impacts on sensemaking processes and resultant ethical decision making. Overall, guilt was more beneficial than anger, but both emotions drew participants’ attention to particular aspects of the situation. Specifically, anger prompted reflection on the past and the causes of the situation while guilt helped participants focus on the future and the outcomes of the current situation. The moderating impact of two emotion regulation strategies, cognitive reappraisal and suppression, was also investigated, and results indicated that emotion regulation may be markedly difficult in ethical decision-making situations. In addition, suppressing anger may be particularly harmful for making ethical decisions via its impact on selfishness. Implications and future research directions are also discussed