The EXTraS project: Exploring the X-ray transient and variable sky

Temporal variability in flux and spectral shape is ubiquitous in the X-ray sky and carries crucial information about the nature and emission physics of the sources. The EPIC instrument on board the XMM-Newton observatory is the most powerful tool for studying variability even in faint sources. Each day, it collects a large amount of information about hundreds of new serendipitous sources, but the resulting huge (and growing) dataset is largely unexplored in the time domain. The project called Exploring the X-ray transient and variable sky (EXTraS) systematically extracted all temporal domain information in the XMM-Newton archive. This included a search and characterisation of variability, both periodic and aperiodic, in hundreds of thousands of sources spanning more than eight orders of magnitude in timescale and six orders of magnitude in flux, and a search for fast transients that were missed by standard image analysis. All results, products, and software tools have been released to the community in a public archive. A science gateway has also been implemented to allow users to run the EXTraS analysis remotely on recent XMM datasets. We give details on the new algorithms that were designed and implemented to perform all steps of EPIC data analysis, including data preparation, source and background modelling, generation of time series and power spectra, and search for and characterisation of different types of variabilities. We describe our results and products and give information about their basic statistical properties and advice on their usage. We also describe available online resources. The EXTraS database of results and its ancillary products is a rich resource for any kind of investigation in almost all fields of astrophysics. Algorithms and lessons learnt from our project are also a very useful reference for any current and future experiment in the time domain