Perspects in astrophysical databases

Astrophysics has become a domain extremely rich of scientific data. Data mining tools are needed for information extraction from such large datasets. This asks for an approach to data management emphasizing the efficiency and simplicity of data access; efficiency is obtained using multidimensional access methods and simplicity is achieved by properly handling metadata. Moreover, clustering and classification techniques on large datasets pose additional requirements in terms of computation and memory scalability and interpretability of results. In this study we review some possible solutions

DAS: a data management system for instrument tests and operations

The Data Access System (DAS) is a metadata and data management software system, providing a reusable solution for the storage of data acquired both from telescopes and auxiliary data sources during the instrument development phases and operations. It is part of the Customizable Instrument WorkStation system (CIWS-FW), a framework for the storage, processing and quick-look at the data acquired from scientific instruments. The DAS provides a data access layer mainly targeted to software applications: quick-look displays, pre-processing pipelines and scientific workflows. It is logically organized in three main components: an intuitive and compact Data Definition Language (DAS DDL) in XML format, aimed for user-defined data types; an Application Programming Interface (DAS API), automatically adding classes and methods supporting the DDL data types, and providing an object-oriented query language; a data management component, which maps the metadata of the DDL data types in a relational Data Base Management System (DBMS), and stores the data in a shared (network) file system. With the DAS DDL, developers define the data model for a particular project, specifying for each data type the metadata attributes, the data format and layout (if applicable), and named references to related or aggregated data types. Together with the DDL user-defined data types, the DAS API acts as the only interface to store, query and retrieve the metadata and data in the DAS system, providing both an abstract interface and a data model specific one in C, C++ and Python. The mapping of metadata in the back-end database is automatic and supports several relational DBMSs, including MySQL, Oracle and PostgreSQL.Comment: Accepted for pubblication on ADASS Conference Serie

CIWS-FW: a Customizable InstrumentWorkstation Software Framework for instrument-independent data handling

The CIWS-FW is aimed at providing a common and standard solution for the storage, processing and quick look at the data acquired from scientific instruments for astrophysics. The target system is the instrument workstation either in the context of the Electrical Ground Support Equipment for space-borne experiments, or in the context of the data acquisition system for instrumentation. The CIWS-FW core includes software developed by team members for previous experiments and provides new components and tools that improve the software reusability, configurability and extensibility attributes. The CIWS-FW mainly consists of two packages: the data processing system and the data access system. The former provides the software components and libraries to support the data acquisition, transformation, display and storage in near real time of either a data packet stream and/or a sequence of data files generated by the instrument. The latter is a meta-data and data management system, providing a reusable solution for the archiving and retrieval of the acquired data. A built-in operator GUI allows to control and configure the IW. In addition, the framework provides mechanisms for system error and logging handling. A web portal provides the access to the CIWS-FW documentation, software repository and bug tracking tools for CIWS-FW developers. We will describe the CIWS-FW architecture and summarize the project status.Comment: Accepted for pubblication on ADASS Conference Serie

Planck LFI – FM database Release Note

The present document contains the release note for the LFI FM database version 4.3.2 based on the official release (LFI FM 4.2) with the following NCRs applied: NCR 13323 - a check of the telemetry validity based, for instance, on the switch on/off of the equipment; NCR 13333 – even if the Memory Dump Absolute Address decommutation is a variable packet, for two fixed memory areas (with identical structures) of the DPU ASW and the SPU SUSW, an explicit decommutation is needed to automatize the UFT procedure; analogously, memory Check Report Absolute Address decommutation requires an explicit decommutation to automatize the UFT procedure

In-flight calibration and verification of the Planck-LFI instrument

In this paper we discuss the Planck-LFI in-flight calibration campaign. After a brief overview of the ground test campaigns, we describe in detail the calibration and performance verification (CPV) phase, carried out in space during and just after the cool-down of LFI. We discuss in detail the functionality verification, the tuning of the front-end and warm electronics, the preliminary performance assessment and the thermal susceptibility tests. The logic, sequence, goals and results of the in-flight tests are discussed. All the calibration activities were successfully carried out and the instrument response was comparable to the one observed on ground. For some channels the in-flight tuning activity allowed us to improve significantly the noise performance.Comment: Long technical paper on Planck LFI in flight calibration campaign: 109 pages in this (not final) version, 100 page in the final JINST versio

Planck LFI – AVM database Release Note

The present document contains the release note for the LFI AVM database version 1.1.0 based on the first official release (LFI AVM 1.0.1) with the following NCs applied: NC 13323 - periodic packet outdated; NC13333 - change of parameters type from 7.2 to 3.12; NC LFI-0001 – Change of way to display the LM207342 parameter inside the LA110369 monitor display from binary to Hexadecimal; NC LFI-0002 - the LM553350 parameter inside the LA110369 was incorrectly named and it was changed in the AVM 1.1.0 to LM553340

Planck LFI – FM database Release Note

This document contains the release note for the LFI FM database version 4.4.2 based on the official release (LFI FM 4.3.2) with the following NCRs applied: NCR 14328 - Temperatures and currents validity, Currents OOL and Alphanumeric display layout; NCR 11635 - not possible to add monitoring on telemetry event 5.1, 5.2, 5.4; NCR LFI-0004 - Numerical calibration format; NCR LFI-0005 - Numerical calibration decimal separator; NCR LFI-0006 - CDF_INTER for uncalibrated values; NCR LFI-0007 - Wrong parameter used in TM(5,1); NCR LFI-0009 - Unreferenced parameters and empty graphical display; NCR LFI-0013 - Soft instead of hard OOL; NCR LFI-0014 - Redundant parameters in display LA009350; NCR LFI-0015 - FEM drain currents warm OOLs; NCR LFI-0016 - REBA internal temperatures; NCR LFI-0017 - DAE Fast Essential and REBA Essential packets; NCR LFI-0018 - Missing parameter in the DAE Actual configuration packet; NCR LFI-0019 - Missing nominal scientific packet uncompressed; NCR LFI-0020 - Alarm and warning declaration for event packets; NCR LFI-0021 - Missing Command Verification Stages for LFI telecommands

Planck LFI – FM database Release Note

This document contains the release note for the LFI FM database version 4.5.0, based on the official release (LFI FM 4.4.2) with the following NCRs applied: NCR LFI-0022: New REBA diagnostic HK parameters; NCR LFI-0023 - Changes in TM(8,6); NCR LFI-0024 - Drain currents OOL (Out Of Limit) changes; NCR LFI-0025 - Changes in TM(1,2); NCR LFI-0026 - Changes in TM(1,8)

Planck LFI – AVM database Release Note

This document contains the release note for the LFI AVM database version 1.3.0 based on the official release (LFI AVM 1.2.2) with the following NCRs applied: NCR 14328: Temperatures and currents validity; NCR 11635 - not possible to add monitoring on telemetry event 5.1, 5.2, 5.4; NCR LFI-0006 - CDF_INTER for uncalibrated values; NCR LFI-0009 - Unreferenced parameters; NCR LFI-0014 - Redundant parameters in display LA009350

Planck LFI – FM database Release Note

The present document contains the release note for the LFI FM database version 4.2 based on the first official release (LFI FM 4.1) with the following NCs applied: NC 13273 - Telemetry monitoring (OOL) Action:#8 from MN-8861; NC 13323 (periodic packet outdated); NC13333 (change of parameters type from 7.2 to 3.12 to avoid issues with the MOIS system); NC LFI-0001 – Change of way to display the LM207342 parameter inside the LA110369 monitor display from binary to Hexadecimal