Learning a Partitioning Advisor with Deep Reinforcement Learning

Commercial data analytics products such as Microsoft Azure SQL Data Warehouse or Amazon Redshift provide ready-to-use scale-out database solutions for OLAP-style workloads in the cloud. While the provisioning of a database cluster is usually fully automated by cloud providers, customers typically still have to make important design decisions which were traditionally made by the database administrator such as selecting the partitioning schemes. In this paper we introduce a learned partitioning advisor for analytical OLAP-style workloads based on Deep Reinforcement Learning (DRL). The main idea is that a DRL agent learns its decisions based on experience by monitoring the rewards for different workloads and partitioning schemes. We evaluate our learned partitioning advisor in an experimental evaluation with different databases schemata and workloads of varying complexity. In the evaluation, we show that our advisor is not only able to find partitionings that outperform existing approaches for automated partitioning design but that it also can easily adjust to different deployments. This is especially important in cloud setups where customers can easily migrate their cluster to a new set of (virtual) machines

A requirements specification for a software design support system

Most existing software design systems (SDSS) support the use of only a single design methodology. A good SDSS should support a wide variety of design methods and languages including structured design, object-oriented design, and finite state machines. It might seem that a multiparadigm SDSS would be expensive in both time and money to construct. However, it is proposed that instead an extensible SDSS that directly implements only minimal database and graphical facilities be constructed. In particular, it should not directly implement tools to faciliate language definition and analysis. It is believed that such a system could be rapidly developed and put into limited production use, with the experience gained used to refine and evolve the systems over time

The crustal dynamics intelligent user interface anthology

The National Space Science Data Center (NSSDC) has initiated an Intelligent Data Management (IDM) research effort which has, as one of its components, the development of an Intelligent User Interface (IUI). The intent of the IUI is to develop a friendly and intelligent user interface service based on expert systems and natural language processing technologies. The purpose of such a service is to support the large number of potential scientific and engineering users that have need of space and land-related research and technical data, but have little or no experience in query languages or understanding of the information content or architecture of the databases of interest. This document presents the design concepts, development approach and evaluation of the performance of a prototype IUI system for the Crustal Dynamics Project Database, which was developed using a microcomputer-based expert system tool (M. 1), the natural language query processor THEMIS, and the graphics software system GSS. The IUI design is based on a multiple view representation of a database from both the user and database perspective, with intelligent processes to translate between the views

Forensic flavour

Databases often receive an uninspired and uninterested response. The curriculum content of a database module generally involves the design of entity-relationship models, SQL programming, application development and advanced database applications such as data warehousing and data mining. These are often taught within the tired and relatively worn case studies of purchase order systems, retail or health care systems. However the current trend for crime scene investigation drama and the frequent stories in the news of personal tragedies involving incorrect data, missing data or data mix-up capture the attention of many. The truth is that crimes require data investigation and expert database witnesses to provide evidence and this requires database knowledge and skill. This project involved the introduction of a ‘forensic flavour’ to the teaching of databases as part of an undergraduate Computing Degree to students. The ‘forensic flavour’ involved introducing investigative and enquiry based learning techniques as well as selecting case studies based around real-life crimes and crime data. The learning objectives remained unchanged for the modules as did the curriculum content. The initial findings are that the students engaged on average 40% better and enjoyed the experience more

Cleaning Data Helps Clean the Air

In this project, students use a real-world, complex database and experience firsthand the consequences of inadequate data modeling. The U.S. Environmental Protection Agency created the database as part of a multimillion dollar data collection effort undertaken in order to set limits on air pollutants from electric power plants. First, students explore the database to identify design limitations from the perspective of a data analyst with a specific goal. Second, students create a new database design which overcomes identified problems. Through this case study, students develop the skill to infer usage implications by studying the design of an existing database. This is important since developers often inherit databases designed by others. Students also learn how to prepare data stored in a relational database for a data analysis project. By experiencing the consequences of an inadequate design from a user perspective, students can better appreciate the importance of relational database design principles and become more committed to using them

Collaborative Engineering Environments. Two Examples of Process Improvement

Companies are recognising that innovative processes are determining factors in competitiveness. Two examples from projects in aircraft development describe the introduction of collaborative engineering environments as a way to improve engineering processes. A multi-disciplinary simulation environment integrates models from all disciplines involved in a common functional structure. Quick configuration for specific design problems and powerful feedback / visualisation capabilities enable engineering teams to concentrate on the integrated behaviour of the design. An engineering process management system allows engineering teams to work concurrently in tasks, following a defined flow of activities, applying tools on a shared database. Automated management of workspaces including data consistency enables engineering teams to concentrate on the design activities. The huge amount of experience in companies must be transformed for effective application in engineering processes. Compatible concepts, notations and implementation platforms make tangible knowledge like models and algorithms accessible. Computer-based design management makes knowledge on engineering processes and methods explicit

Working with graphic design students to promote `Land of Lost Content¿ at Leeds Met.

yesAs Leeds Metropolitan is a university of festivals and partnerships, the library was given the opportunity this year to host a festival to promote the library and all its services to the university. The library festival was a week of events and activities to promote the library and its space in new ways. It was a good way to remind staff and students of the value of libraries, not only as places to learn but also as places to enhance our leisure and working lives ... we chose to promote the database `Land of Lost Content¿. This was because graphic design and art students are based at our campus and we also thought this database would attract a wide range of students who might otherwise think that electronic databases contain little of interest for them. Our promotion has been so successful in many ways that we would like to share our experience with other librarians

Film as database: a visual analysis of 2001: A Space Odyssey

This paper reports on research that explores new possibilities for experiencing film as a digital database. The way we access, interact with and experience film is changed with new digital tools and initial visual experiments towards the design of a graphical user interface respond to this change. The film 2001:A Space Odyssey by Stanley Kubrick is the focus for the research. It is essentially a visual film with an expanded context of critical writing and archived material. Two visual outcomes that map the use of red within scenes of the film will be demonstrated in order to critique and review the film in its new context