Learning a Partitioning Advisor with Deep Reinforcement Learning

Commercial data analytics products such as Microsoft Azure SQL Data Warehouse or Amazon Redshift provide ready-to-use scale-out database solutions for OLAP-style workloads in the cloud. While the provisioning of a database cluster is usually fully automated by cloud providers, customers typically still have to make important design decisions which were traditionally made by the database administrator such as selecting the partitioning schemes. In this paper we introduce a learned partitioning advisor for analytical OLAP-style workloads based on Deep Reinforcement Learning (DRL). The main idea is that a DRL agent learns its decisions based on experience by monitoring the rewards for different workloads and partitioning schemes. We evaluate our learned partitioning advisor in an experimental evaluation with different databases schemata and workloads of varying complexity. In the evaluation, we show that our advisor is not only able to find partitionings that outperform existing approaches for automated partitioning design but that it also can easily adjust to different deployments. This is especially important in cloud setups where customers can easily migrate their cluster to a new set of (virtual) machines

Practice Makes Perfect: On Professional Standards

Practicing is a matter of increasing the reliability of ones skills rather than relying on a tool or a strike of genius to get it right. Once perfection has been achieved the individual will aim for higher quality since the effort is more likely to be worthwhile. Furthermore because the returns to achieving perfection are higher the harder it is to achieve, the perfectionist equilibrium only arises in situations where genius is rare and reliability is low. From this follows that as tools improve, even though perfection then has become easier to achieve, professional standards may nonetheless decline. This mechanism is captured in an oligopoly model, where the failure rate and the quality are endogenously determined