A conceptual framework for externally-influenced agents: an assisted reinforcement learning review

A long-term goal of reinforcement learning agents is to be able to perform tasks in complex real-world scenarios. The use of external information is one way of scaling agents to more complex problems. However, there is a general lack of collaboration or interoperability between different approaches using external information. In this work, while reviewing externally-influenced methods, we propose a conceptual framework and taxonomy for assisted reinforcement learning, aimed at fostering collaboration by classifying and comparing various methods that use external information in the learning process. The proposed taxonomy details the relationship between the external information source and the learner agent, highlighting the process of information decomposition, structure, retention, and how it can be used to influence agent learning. As well as reviewing state-of-the-art methods, we identify current streams of reinforcement learning that use external information in order to improve the agent’s performance and its decision-making process. These include heuristic reinforcement learning, interactive reinforcement learning, learning from demonstration, transfer learning, and learning from multiple sources, among others. These streams of reinforcement learning operate with the shared objective of scaffolding the learner agent. Lastly, we discuss further possibilities for future work in the field of assisted reinforcement learning systems. © 2021, The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature

Client-Focused Management of Expectations for Legal Fees in Large Chapter 11 Cases

Large chapter 11 cases can have fees that run into the hundreds of millions of dollars. That\u27s one of the reasons that, in 2013, the Executive Office of the United States Trustee promulgated additional guidelines that affect legal fees in large chapter 11 cases. Bankruptcy courts have been appointing fee examiners and fee committees in large cases to aid the courts in their duty to ensure that the fees and expenses of estate-paid professionals are reasonable. I\u27ve been one of those people charged with helping bankruptcy courts review fees. As such, I\u27ve seen first-hand what happens when the professionals involved in high-stakes, bet-the-company litigation serve as the actual decisionmakers, rather than involving their clients deeply in their decisions. This article will discuss the dynamics that create a disincentive for most parties to monitor fees in large chapter 11 cases and will then provide suggestions to inside counsel whose organizations find themselves involved in those cases-as the debtor, as a member of the creditors\u27 committee, or as a secured creditor whose collateral is being tapped for the carve-out to pay the professionals\u27 fees

Investment Games

Popular zero-commission stock trading apps like Robinhood innovate in user-experience design, featuring “gamification” practices—flashy graphics, leaderboards, and the like—that make it attractive, easy, and fun to trade stocks. Regulators are increasingly scrutinizing gamification and other digital engagement practices, with efforts underway at the SEC to adopt rules in broker-dealer and investment-advisor regulation. This attention reflects considerable skepticism about gamification in securities markets. At best, these practices encourage motivation and engagement, and democratize access to financial markets. But at worst, these practices encourage people to trade habitually and unreflectively, and more than they might want. This can lead to undesirable market-wide effects, like distorting the process by which markets allocate investment capital to firms and projects that will grow the real economy, as well as socially wasteful (and individually harmful) excessive trading. And given that interventions in retail investor choice have significant implications for market quality and wealth inequality, regulatory responses here are a high stakes matter for society broadly. Calls to regulate gamification highlight a tension at the core of securities markets. Securities law has largely ceded the field of market structure to the interests of sophisticated financial intermediaries in producing liquidity and price discovery. By permitting gamification practices that encourage active trading for the primary benefit of financial intermediaries, securities law subordinates its investor protection function to encourage wasteful investment in achieving eversmaller improvements in liquidity and price discovery. Regulatory intervention would be socially desirable, I argue, not just given what we know about retail trader behavior and its effects on personal finance and markets—but because it is an opportunity for securities law to recalibrate away from an all-out arms race in arbitrage. This Article takes up the problem of gamification and related digital engagement practices. It considers how gamification is the nearly inevitable consequence of the rise of retail investors who trade without superior information about a stock’s fundamental value, competition on brokerage commissions, and a fragmented market structure. Yet calls for regulatory interventions often elide important distinctions between how securities law should treat active traders who prefer risk, and those with preferences distorted by gamification. This Article explains how we got here; examines the social-welfare case for regulating gamification and related digital engagement practices; offers a typology of techniques that securities regulators can adopt in response; and assesses these interventions against existing securities law doctrine and policy. This Article also considers how the securities laws’ tenuous relationship with innovative stock-market technology shapes how retail investors engage with financial markets

Education Reform for the Digital Era

Will the digital-learning movement repeat the mistakes of the charter-school movement? How much more successful might today's charter universe look if yesterday's proponents had focused on the policies and practices needed to ensure its quality, freedom, and resources over the long term? What mistakes might have been avoided? Damaging scandals forestalled? Missed opportunities seized

Open Problems in DAOs

Decentralized autonomous organizations (DAOs) are a new, rapidly-growing class of organizations governed by smart contracts. Here we describe how researchers can contribute to the emerging science of DAOs and other digitally-constituted organizations. From granular privacy primitives to mechanism designs to model laws, we identify high-impact problems in the DAO ecosystem where existing gaps might be tackled through a new data set or by applying tools and ideas from existing research fields such as political science, computer science, economics, law, and organizational science. Our recommendations encompass exciting research questions as well as promising business opportunities. We call on the wider research community to join the global effort to invent the next generation of organizations

Travels along the hype cycle: a set of blockchain applications and the economic processes they impact

Some commentators refer to blockchain as a potential General Purpose Technology. Yet despite a plethora of cryptoassets and projects, it has struggled to gain traction beyond payments and price discovery. This thesis explores how the technology is being applied to better understand the potential and risks of deploying blockchain. It examines four different use cases with econometric and case study methods: (1) Bitcoin mining as the token incentivized processing of records, (2) Initial Coin Offering tokens as a form of venture financing, (3) Uniswap the decentralized exchange and (4) Kompany improving the data integrity of compliance records via notarization to a public blockchain. It finds that blockchain enables capabilities that did not exist before, but that these capabilities are bounded by trade offs and developer priorities. Ultimately this research expands the literature on blockchain applications and argues that blockchain does not build better systems, but different systems that can achieve different objectives. It provides evidence that firms and society are gradually traversing the hype cycle, deploying blockchain, solving real world economic problems and creating value