On Reinforcement Learning, Nurturing, and the Evolution of Risk Neutral

Reinforcement learning depends on agents being learning individuals, and when agents rely on their instincts rather than gathering data and acting accordingly, the population tends to be less successful than a true RL population. ÒRiskinessÓ is the elementary metric for determining how willing to rely on learning an individual or a population is. With a high learning parameter, as we denote riskiness in this paper, agents find the safest option and seldom deviate from it, essentially using learning to become a non-learning individual. With a low learning rate, agents ignore recency entirely and seek out the highest reward, regardless of the risk. We attempt in this paper to evolve this Òrisk neutralityÓ in a population by adding a safe exploration nurturing period during which agents are free to explore without consequence. We discovered the environmental conditions necessary for our hypotheses to be mostly satisfied and found that nurturing enables agents to distinguish between two different risky options to evolve risk neutrality. Too long of a nurturing period causes the evolution to waver before settling on a path with essentially random results, while a short nurturing period causes a successful evolution of risk neutrality. The non-nurturing case evolves risk aversion by default as we expected from a reinforcement learning system, because agents are unable to distinguish between the good risk and bad risk, so they decide to avoid risks altogether.Noundergraduat

NURTURING PROMOTES THE EVOLUTION OF LEARNING IN CHANGING ENVIRONMENTS

An agent may interact with its environment and learn complex tasks based on evaluative feedback through a process known as reinforcement learning. Reinforcement learning requires exploration of unfamiliar situations, which necessarily involves unknown and potentially dangerous or costly outcomes. Supervising agents in these situations can be seen as a type of nurturing and requires an investment of time usually by humans. Nurturing, one individual investing in the development of another individual with which it has an ongoing relationship, is widely seen in the biological world, often with parents nurturing their o spring. There are many types of nurturing, including helping an individual to carry out a task by doing part of the task for it. In arti cial intelligence, nurturing can be seen as an opportunity to develop both better machine learning algorithms and robots that assist or supervise other robots. Although the area of nurturing robotics is at a very early stage, the hope is that this approach can result in more sophisticated learning systems. This dissertation demonstrates the e ectiveness of nurturing through experiments involving the evolution of the parameters of a reinforcement learning algorithm that is capable of nding good policies in a changing environment in which the agent must learn an episodic task in which there is discrete input with perceptual aliasing, continuous output, and delayed reward. The results show that nurturing is capable of promoting the evolution of learning in such environments

Nurturing as Safe Exploration Promotes the Evolution of Generalized Supervised Learning

The ability to learn is often a desirable property of intelligent systems which can make them more adaptive. However, it is difficult to develop sophisticated learning algorithms that are effective. One approach to the development of learning algorithms is to evolve them using evolutionary algorithms. The evolution of learning is interesting as a practical matter because harnessing it may allow us to develop better artificial intelligence; it is interesting also from a more theoretical perspective of understanding how the sophisticated learning seen in nature---including that of humans---could have arisen. A potential obstacle to the evolution of learning when alternative behavioral strategies (e.g., instincts) can evolve is that learning individuals tend to exhibit ineffective behavior before effective behavior is learned. Nurturing, defined as one individual investing in the development of another individual with which it has an ongoing relationship, is often seen in nature in species that exhibit sophisticated learning behavior. It is hypothesized that nurturing may be able to increase the competitiveness of learning in an evolutionary environment by ameliorating the consequences of incorrect initial behavior. The approach taken is to expand upon a foundational work in the evolution of learning to enable also the evolution of instincts and then examining the strategies evolved with and without a nurturing condition in which individuals are not penalized for mistakes made during a learning period. It is found that nurturing promotes the evolution of learning in these environments

FinGPT: Open-Source Financial Large Language Models

Large language models (LLMs) have shown the potential of revolutionizing natural language processing tasks in diverse domains, sparking great interest in finance. Accessing high-quality financial data is the first challenge for financial LLMs (FinLLMs). While proprietary models like BloombergGPT have taken advantage of their unique data accumulation, such privileged access calls for an open-source alternative to democratize Internet-scale financial data. In this paper, we present an open-source large language model, FinGPT, for the finance sector. Unlike proprietary models, FinGPT takes a data-centric approach, providing researchers and practitioners with accessible and transparent resources to develop their FinLLMs. We highlight the importance of an automatic data curation pipeline and the lightweight low-rank adaptation technique in building FinGPT. Furthermore, we showcase several potential applications as stepping stones for users, such as robo-advising, algorithmic trading, and low-code development. Through collaborative efforts within the open-source AI4Finance community, FinGPT aims to stimulate innovation, democratize FinLLMs, and unlock new opportunities in open finance. Two associated code repos are \url{https://github.com/AI4Finance-Foundation/FinGPT} and \url{https://github.com/AI4Finance-Foundation/FinNLP

Born to learn: The inspiration, progress, and future of evolved plastic artificial neural networks

Biological plastic neural networks are systems of extraordinary computational capabilities shaped by evolution, development, and lifetime learning. The interplay of these elements leads to the emergence of adaptive behavior and intelligence. Inspired by such intricate natural phenomena, Evolved Plastic Artificial Neural Networks (EPANNs) use simulated evolution in-silico to breed plastic neural networks with a large variety of dynamics, architectures, and plasticity rules: these artificial systems are composed of inputs, outputs, and plastic components that change in response to experiences in an environment. These systems may autonomously discover novel adaptive algorithms, and lead to hypotheses on the emergence of biological adaptation. EPANNs have seen considerable progress over the last two decades. Current scientific and technological advances in artificial neural networks are now setting the conditions for radically new approaches and results. In particular, the limitations of hand-designed networks could be overcome by more flexible and innovative solutions. This paper brings together a variety of inspiring ideas that define the field of EPANNs. The main methods and results are reviewed. Finally, new opportunities and developments are presented

Grounding Artificial Intelligence in the Origins of Human Behavior

Recent advances in Artificial Intelligence (AI) have revived the quest for agents able to acquire an open-ended repertoire of skills. However, although this ability is fundamentally related to the characteristics of human intelligence, research in this field rarely considers the processes that may have guided the emergence of complex cognitive capacities during the evolution of the species. Research in Human Behavioral Ecology (HBE) seeks to understand how the behaviors characterizing human nature can be conceived as adaptive responses to major changes in the structure of our ecological niche. In this paper, we propose a framework highlighting the role of environmental complexity in open-ended skill acquisition, grounded in major hypotheses from HBE and recent contributions in Reinforcement learning (RL). We use this framework to highlight fundamental links between the two disciplines, as well as to identify feedback loops that bootstrap ecological complexity and create promising research directions for AI researchers

Role and Discipline Relationships in a Transdisciplinary Biomedical Team: Structuration, Values Override and Context Scaffolding

Though accepted that "team science" is needed to tackle and conquer the health problems that are plaguing our society significant empirical evidence of team mechanisms and functional dynamics is still lacking in abundance. Through grounded methods the relationship between scientific disciplines and team roles was observed in a United States National Institutes of Health-funded (NIH) research consortium. Interviews and the Organizational Culture Assessment Instrument (OCAI) were employed.. Findings show strong role and discipline idiosyncrasies that when viewed separately provide different insights into team functioning and change receptivity. When considered simultaneously, value-latent characteristics emerged showing self-perceived contributions to the team. This micro/meso analysis suggests that individual participation in team level interactions can inform the structuration of roles and disciplines in an attempt to tackle macro level problems.Comment: Presented at COINs13 Conference, Chile, 2013 (arxiv:1308.1028

Towards a more sustainable future: simple recommendations to integrate planetary health into education

This Personal View presents recommendations aimed at integrating planetary health into various stages of education, which are simple but effective, and designed with teachers in training and those who have not yet considered how to incorporate UNESCO's Education for Sustainable Development into their teaching practice. However, the constantly evolving nature of the Education for Sustainable Development programme must be recognised, and the importance of being able to adapt teaching methods to meet the changing needs of students as they progress through their educational journey should be highlighted. Therefore, this Personal View considers the cognitive, social, and ethical evolution of students and offers specific recommendations for preschool, primary, secondary, and university education levels. We recommend that educators should focus on teaching students to critically evaluate data on sustainability and to develop innovative solutions to environmental challenges. We also highlight the importance of incorporating practical projects, using active methods that promote skills related to caring for the planet, or the importance of situated learning that attends to the particularities of each context. In this way, students can develop skills and values that contribute to a more sustainable future. The recommendations made here aim to provide educators and researchers with simple but effective ways to integrate planetary health into education

Enabling creativity in learning environments: lessons from the CREANOVA project

The paper employs data from a European Union funded project to outline the dif- ferent contexts and factors that enable creativity and innovation. It suggests that creativity and innovation are supported by flexible work settings, adaptable learning environments, collaborative design processes, determined effort, and liberating in- novative relationships. It concludes that learning environments that seek to enable creativity and innovation should encourage collaborative working, offer flexibility for both learners and educators, enable learner-led innovative processes, and recognize that creativity occurs in curriculum areas beyond the creative arts