The Zwicky Transient Facility Alert Distribution System

The Zwicky Transient Facility (ZTF) survey generates real-time alerts for optical transients, variables, and moving objects discovered in its wide-field survey. We describe the ZTF alert stream distribution and processing (filtering) system. The system uses existing open-source technologies developed in industry: Kafka, a real-time streaming platform, and Avro, a binary serialization format. The technologies used in this system provide a number of advantages for the ZTF use case, including (1) built-in replication, scalability, and stream rewind for the distribution mechanism; (2) structured messages with strictly enforced schemas and dynamic typing for fast parsing; and (3) a Python-based stream processing interface that is similar to batch for a familiar and user-friendly plug-in filter system, all in a modular, primarily containerized system. The production deployment has successfully supported streaming up to 1.2 million alerts or roughly 70 GB of data per night, with each alert available to a consumer within about 10 s of alert candidate production. Data transfer rates of about 80,000 alerts/minute have been observed. In this paper, we discuss this alert distribution and processing system, the design motivations for the technology choices for the framework, performance in production, and how this system may be generally suitable for other alert stream use cases, including the upcoming Large Synoptic Survey Telescope.Comment: Published in PASP Focus Issue on the Zwicky Transient Facility (doi: 10.1088/1538-3873/aae904). 9 Pages, 2 Figure

A Strategy for LSST to Unveil a Population of Kilonovae without Gravitational-wave Triggers

We present a cadence optimization strategy to unveil a large population of kilonovae using optical imaging alone. These transients are generated during binary neutron star and potentially neutron star–black hole mergers and are electromagnetic counterparts to gravitational-wave signals detectable in nearby events with Advanced LIGO, Advanced Virgo, and other interferometers that will be online in the near future. Discovering a large population of kilonovae will allow us to determine how heavy-element production varies with the intrinsic parameters of the merger and across cosmic time. The rate of binary neutron star mergers is still uncertain, but only few (≾ 15) events with associated kilonovae may be detectable per year within the horizon of next-generation ground-based interferometers. The rapid evolution (~days) at optical/infrared wavelengths, relatively low luminosity, and the low volumetric rate of kilonovae makes their discovery difficult, especially during blind surveys of the sky. We propose future large surveys to adopt a rolling cadence in which g-i observations are taken nightly for blocks of 10 consecutive nights. With the current baseline2018a cadence designed for the Large Synoptic Survey Telescope (LSST), l≾ 7.5 poorly sampled kilonovae are expected to be detected in both the Wide Fast Deep (WFD) and Deep Drilling Fields (DDF) surveys per year, under optimistic assumptions on their rate, duration, and luminosity. We estimate the proposed strategy to return up to ~272 GW170817-like kilonovae throughout the LSST WFD survey, discovered independently from gravitational-wave triggers