MQLV: Optimal Policy of Money Management in Retail Banking with Q-Learning

Reinforcement learning has become one of the best approach to train a computer game emulator capable of human level performance. In a reinforcement learning approach, an optimal value function is learned across a set of actions, or decisions, that leads to a set of states giving different rewards, with the objective to maximize the overall reward. A policy assigns to each state-action pairs an expected return. We call an optimal policy a policy for which the value function is optimal. QLBS, Q-Learner in the Black-Scholes(-Merton) Worlds, applies the reinforcement learning concepts, and noticeably, the popular Q-learning algorithm, to the financial stochastic model of Black, Scholes and Merton. It is, however, specifically optimized for the geometric Brownian motion and the vanilla options. Its range of application is, therefore, limited to vanilla option pricing within financial markets. We propose MQLV, Modified Q-Learner for the Vasicek model, a new reinforcement learning approach that determines the optimal policy of money management based on the aggregated financial transactions of the clients. It unlocks new frontiers to establish personalized credit card limits or to fulfill bank loan applications, targeting the retail banking industry. MQLV extends the simulation to mean reverting stochastic diffusion processes and it uses a digital function, a Heaviside step function expressed in its discrete form, to estimate the probability of a future event such as a payment default. In our experiments, we first show the similarities between a set of historical financial transactions and Vasicek generated transactions and, then, we underline the potential of MQLV on generated Monte Carlo simulations. Finally, MQLV is the first Q-learning Vasicek-based methodology addressing transparent decision making processes in retail banking

Visualization of AE's Training on Credit Card Transactions with Persistent Homology

Auto-encoders are among the most popular neural network architecture for dimension reduction. They are composed of two parts: the encoder which maps the model distribution to a latent manifold and the decoder which maps the latent manifold to a reconstructed distribution. However, auto-encoders are known to provoke chaotically scattered data distribution in the latent manifold resulting in an incomplete reconstructed distribution. Current distance measures fail to detect this problem because they are not able to acknowledge the shape of the data manifolds, i.e. their topological features, and the scale at which the manifolds should be analyzed. We propose Persistent Homology for Wasserstein Auto-Encoders, called PHom-WAE, a new methodology to assess and measure the data distribution of a generative model. PHom-WAE minimizes the Wasserstein distance between the true distribution and the reconstructed distribution and uses persistent homology, the study of the topological features of a space at different spatial resolutions, to compare the nature of the latent manifold and the reconstructed distribution. Our experiments underline the potential of persistent homology for Wasserstein Auto-Encoders in comparison to Variational Auto-Encoders, another type of generative model. The experiments are conducted on a real-world data set particularly challenging for traditional distance measures and auto-encoders. PHom-WAE is the first methodology to propose a topological distance measure, the bottleneck distance, for Wasserstein Auto-Encoders used to compare decoded samples of high quality in the context of credit card transactions.Comment: arXiv admin note: substantial text overlap with arXiv:1905.0989

Alteration and release of aliphatic compounds by the polychaete Nereis virens (Sars) experimentally fed with hydrocarbons

In the laboratory, marine worms were fed with a mixture of algae and several aliphatic hydrocarbons for 15 days. After ingestion by the worms, 34.9% of hydrocarbons are found in the faeces and only 3.1% accumulated in the gut. The comparison between the initial mixture and the faeces shows that the worm’s digestive process lead to changes in the distribution of the n-alkane mixture. These changes are different from those only due to physical processes in the experimental conditions. In our experiment, no variation in the distribution of hydrocarbons in faeces with time and no microbial hydrocarbon biodegradation were evidenced. Our results suggest that marine worm feeding can substantially affect the fate of hydrocarbons in the sedimentary marine ecosystem by predominantly stimulating dissolution processes

Pharmaceutische Waarenkunde

Die Waarenkunden steht in vielfacher Beziehung mit der Naturgeschichte, darf aber nicht ganz mit derselben verwechselt werden. ..

Effects of temperature on in vitro sediment reworking processes by a gallery biodiffusor, the polychaete Neanthes virens

Temperature-induced variations in bioturbation could affect sediment mixing processes in the marine benthic environment. In this study, sediment reworking by Neanthes virens (Sars), a widely distributed polychaete in muddy sand communities of northern temperate latitudes, was studied under different temperature conditions representing winter (1°C), spring and fall (6°C), summer(13°C), and tide pool (18°C) temperatures in the lower St. Lawrence Estuary, Québec, Canada. Sediment reworking was quantified using inert fluorescent particles (luminophores) deposited at the sediment surface. Based on the 1-D luminophore distributions obtained after 5 and 30 d, the use of the specific ‘gallery-biodiffusor’ model allowed us to quantify both biodiffusion (Db) and biotransport (Vb) due to the organisms. Our results showed temperature effects on sediment transport. The lowest biotransport and biodiffusion coefficients were measured at 1 and 6°C and did not change with time. The highest biodiffusion occurred at 13°C for both sampling periods. At 18°C, biodiffusion was intermediate while biotransport was maximal. Differences between the 13°C biodiffusive transport and the other temperatures increased with time. Low transport values at 1 and 6°C suggest that a quiescent stage exists for this species at these temperatures, with sediment mixing occurring mostly during burrow construction. On the other hand, sediment mixing resulted from both the burrow construction and maintenance phases at higher temperatures (13 and 18°C)

Monte Carlo Tree Search Guided by Symbolic Advice for MDPs

In this paper, we consider the online computation of a strategy that aims at optimizing the expected average reward in a Markov decision process. The strategy is computed with a receding horizon and using Monte Carlo tree search (MCTS). We augment the MCTS algorithm with the notion of symbolic advice, and show that its classical theoretical guarantees are maintained. Symbolic advice are used to bias the selection and simulation strategies of MCTS. We describe how to use QBF and SAT solvers to implement symbolic advice in an efficient way. We illustrate our new algorithm using the popular game Pac-Man and show that the performances of our algorithm exceed those of plain MCTS as well as the performances of human players