Quantum-enhanced reinforcement learning for finite-episode games with discrete state spaces

Quantum annealing algorithms belong to the class of metaheuristic tools, applicable for solving binary optimization problems. Hardware implementations of quantum annealing, such as the quantum annealing machines produced by D-Wave Systems, have been subject to multiple analyses in research, with the aim of characterizing the technology's usefulness for optimization and sampling tasks. Here, we present a way to partially embed both Monte Carlo policy iteration for finding an optimal policy on random observations, as well as how to embed (n) sub-optimal state-value functions for approximating an improved state-value function given a policy for finite horizon games with discrete state spaces on a D-Wave 2000Q quantum processing unit (QPU). We explain how both problems can be expressed as a quadratic unconstrained binary optimization (QUBO) problem, and show that quantum-enhanced Monte Carlo policy evaluation allows for finding equivalent or better state-value functions for a given policy with the same number episodes compared to a purely classical Monte Carlo algorithm. Additionally, we describe a quantum-classical policy learning algorithm. Our first and foremost aim is to explain how to represent and solve parts of these problems with the help of the QPU, and not to prove supremacy over every existing classical policy evaluation algorithm.Comment: 17 pages, 7 figure

Regional Implications of Financial Market Development: Credit Rationing, Trade, and Location

We develop a heterogeneous-fi rms model with trade in goods, labor mobility and credit constraints due to moral hazard. Mitigating fi nancial frictions reduces the incentive of high-skilled workers to migrate to one region such that an unequal distribution of industrial activity becomes less likely. Hence, financial market development has opposite regional implications as trade liberalization. While the former leads to more dispersion of economic activity across space, the latter tends to drive clustering. We provide empirical evidence for this hypothesis by combining industry-region variation in the spatial concentration of economic activity with information on the access to credit and the dependence on external finance. Our estimates for 20 European countries and eleven industries con firm that fi nancial market development mitigates the clustering of economic activity

The persistent effects of placed-based policy - Evidence from the West-German Zonenrandgebiet

Using a natural experiment from Germany, we show that temporary place-based subsidies generate persistent effects on economic density. We identify employment and capital formation as main channels for higher income per square kilometer. As the spatial regression discontinuity design allows us to control for all spatially-continuous determinants of agglomeration (e.g. home-market effects, knowledge spillovers), we attribute an important role to capital formation in explaining persistent spatial patterns of economic activity. However, estimates of externalities at the treatment border point to small net effects of the policy. We find strong evidence that pre-treatment land owners have benefitted substantially from the program and that transfers have shown larger effects in high-density places. Finally, accounting for regional subsidies raises the causal effect of market access for economic development as identified in Redding and Sturm (2008) by about 45 percent

The persistent effects of regional policy - Evidence from the West-German Zonenrandgebiet

In this paper, we provide novel evidence on the contemporaneous and persistent effects of regional policy. We apply a quasi-experimental identification strategy exploiting the fact that municipalities in the West-German Zonenrandgebiet (ZRG) were eligible for substantial regional transfers between 1971 and 1994. The ZRG was an approximately 40km-band adjacent to the Iron Curtain during the Cold War in West Germany. Apart from determining transfer eligibility, this pure geographic threshold did not have any institutional, cultural or economic relevance. We use regression discontinuity to estimate the causal effects of regional policy on economic activity. Using disaggregated data on the municipality level and satellite night light data (as a proxy for GDP), we find that the ZRG treatment led to an increase of income per square kilometer of about 50 percent in 1986. Importantly, economic density remained high in 2010 although the transfers had phased out in 1994. This speaks against unique equilibria determined by locational advantage, but rather strengthens market externalities as an important explanation for the spatial distribution of economic activity. We also examine several potential channels finding strong contemporaneous and persistent effects of transfers on population density and the business tax base

Regional implications of financial market development: Credit rationing, trade, and location

We develop a heterogeneous-firms model with trade in goods, labor mobility and credit constraints due to moral hazard. Mitigating financial frictions reduces the incentive of high-skilled workers to migrate to one region such that an unequal distribution of industrial activity becomes less likely. Hence, financial market development has opposite regional implications as trade liberalization. While the former leads to more dispersion of economic activity across space, the latter tends to drive clustering. We provide empirical evidence for this hypothesis by combining industry-region variation in the spatial concentration of economic activity with information on the access to credit and the dependence on external finance. Our estimates for 20 European countries and eleven industries confirm that financial market development mitigates the clustering of economic activity

The persistent effects of place-based policy: Evidence from the West-German Zonenrandgebiet

We show that temporary place-based subsidies generate persistent effects on economic density. As our design allows us to control for agglomeration economies, we attribute an important role to policy-induced locational advantage (e.g. capital structures) in explaining persistent spatial patterns of economic activity. With regard to distributional implications, we show that subsidies have capitalized in land rents, so pre-treatment land owners have benefitted predominantly from the program