23,064 research outputs found
An Advanced, Three-Dimensional Plotting Library for Astronomy
We present a new, three-dimensional (3D) plotting library with advanced
features, and support for standard and enhanced display devices. The library -
S2PLOT - is written in C and can be used by C, C++ and FORTRAN programs on
GNU/Linux and Apple/OSX systems. S2PLOT draws objects in a 3D (x,y,z) Cartesian
space and the user interactively controls how this space is rendered at run
time. With a PGPLOT inspired interface, S2PLOT provides astronomers with
elegant techniques for displaying and exploring 3D data sets directly from
their program code, and the potential to use stereoscopic and dome display
devices. The S2PLOT architecture supports dynamic geometry and can be used to
plot time-evolving data sets, such as might be produced by simulation codes. In
this paper, we introduce S2PLOT to the astronomical community, describe its
potential applications, and present some example uses of the library.Comment: 12 pages, 10 eps figures (higher resolution versions available from
http://astronomy.swin.edu.au/s2plot/paperfigures). The S2PLOT library is
available for download from http://astronomy.swin.edu.au/s2plo
Vaex: Big Data exploration in the era of Gaia
We present a new Python library called vaex, to handle extremely large
tabular datasets, such as astronomical catalogues like the Gaia catalogue,
N-body simulations or any other regular datasets which can be structured in
rows and columns. Fast computations of statistics on regular N-dimensional
grids allows analysis and visualization in the order of a billion rows per
second. We use streaming algorithms, memory mapped files and a zero memory copy
policy to allow exploration of datasets larger than memory, e.g. out-of-core
algorithms. Vaex allows arbitrary (mathematical) transformations using normal
Python expressions and (a subset of) numpy functions which are lazily evaluated
and computed when needed in small chunks, which avoids wasting of RAM. Boolean
expressions (which are also lazily evaluated) can be used to explore subsets of
the data, which we call selections. Vaex uses a similar DataFrame API as
Pandas, a very popular library, which helps migration from Pandas.
Visualization is one of the key points of vaex, and is done using binned
statistics in 1d (e.g. histogram), in 2d (e.g. 2d histograms with colormapping)
and 3d (using volume rendering). Vaex is split in in several packages:
vaex-core for the computational part, vaex-viz for visualization mostly based
on matplotlib, vaex-jupyter for visualization in the Jupyter notebook/lab based
in IPyWidgets, vaex-server for the (optional) client-server communication,
vaex-ui for the Qt based interface, vaex-hdf5 for hdf5 based memory mapped
storage, vaex-astro for astronomy related selections, transformations and
memory mapped (column based) fits storage. Vaex is open source and available
under MIT license on github, documentation and other information can be found
on the main website: https://vaex.io, https://docs.vaex.io or
https://github.com/maartenbreddels/vaexComment: 14 pages, 8 figures, Submitted to A&A, interactive version of Fig 4:
https://vaex.io/paper/fig
The space physics environment data analysis system (SPEDAS)
With the advent of the Heliophysics/Geospace System Observatory (H/GSO), a complement of multi-spacecraft missions and ground-based observatories to study the space environment, data retrieval, analysis, and visualization of space physics data can be daunting. The Space Physics Environment Data Analysis System (SPEDAS), a grass-roots software development platform (www.spedas.org), is now officially supported by NASA Heliophysics as part of its data environment infrastructure. It serves more than a dozen space missions and ground observatories and can integrate the full complement of past and upcoming space physics missions with minimal resources, following clear, simple, and well-proven guidelines. Free, modular and configurable to the needs of individual missions, it works in both command-line (ideal for experienced users) and Graphical User Interface (GUI) mode (reducing the learning curve for first-time users). Both options have “crib-sheets,” user-command sequences in ASCII format that can facilitate record-and-repeat actions, especially for complex operations and plotting. Crib-sheets enhance scientific interactions, as users can move rapidly and accurately from exchanges of technical information on data processing to efficient discussions regarding data interpretation and science. SPEDAS can readily query and ingest all International Solar Terrestrial Physics (ISTP)-compatible products from the Space Physics Data Facility (SPDF), enabling access to a vast collection of historic and current mission data. The planned incorporation of Heliophysics Application Programmer’s Interface (HAPI) standards will facilitate data ingestion from distributed datasets that adhere to these standards. Although SPEDAS is currently Interactive Data Language (IDL)-based (and interfaces to Java-based tools such as Autoplot), efforts are under-way to expand it further to work with python (first as an interface tool and potentially even receiving an under-the-hood replacement). We review the SPEDAS development history, goals, and current implementation. We explain its “modes of use” with examples geared for users and outline its technical implementation and requirements with software developers in mind. We also describe SPEDAS personnel and software management, interfaces with other organizations, resources and support structure available to the community, and future development plans.Published versio
ROOT - A C++ Framework for Petabyte Data Storage, Statistical Analysis and Visualization
ROOT is an object-oriented C++ framework conceived in the high-energy physics
(HEP) community, designed for storing and analyzing petabytes of data in an
efficient way. Any instance of a C++ class can be stored into a ROOT file in a
machine-independent compressed binary format. In ROOT the TTree object
container is optimized for statistical data analysis over very large data sets
by using vertical data storage techniques. These containers can span a large
number of files on local disks, the web, or a number of different shared file
systems. In order to analyze this data, the user can chose out of a wide set of
mathematical and statistical functions, including linear algebra classes,
numerical algorithms such as integration and minimization, and various methods
for performing regression analysis (fitting). In particular, ROOT offers
packages for complex data modeling and fitting, as well as multivariate
classification based on machine learning techniques. A central piece in these
analysis tools are the histogram classes which provide binning of one- and
multi-dimensional data. Results can be saved in high-quality graphical formats
like Postscript and PDF or in bitmap formats like JPG or GIF. The result can
also be stored into ROOT macros that allow a full recreation and rework of the
graphics. Users typically create their analysis macros step by step, making use
of the interactive C++ interpreter CINT, while running over small data samples.
Once the development is finished, they can run these macros at full compiled
speed over large data sets, using on-the-fly compilation, or by creating a
stand-alone batch program. Finally, if processing farms are available, the user
can reduce the execution time of intrinsically parallel tasks - e.g. data
mining in HEP - by using PROOF, which will take care of optimally distributing
the work over the available resources in a transparent way
Analysis of radial velocity variations in multiple planetary systems
The study of multiple extrasolar planetary systems has the opportunity to
obtain constraints for the planetary masses and orbital inclinations via the
detection of mutual perturbations. The analysis of precise radial velocity
measurements might reveal these planet-planet interactions and yields a more
accurate view of such planetary systems. Like in the generic data modelling
problems, a fit to radial velocity data series has a set of unknown parameters
of which parametric derivatives have to be known by both the regression methods
and the estimations for the uncertainties. In this paper an algorithm is
described that aids the computation of such derivatives in case of when
planetary perturbations are not neglected. The application of the algorithm is
demonstrated on the planetary systems of HD 73526, HD 128311 and HD 155358. In
addition to the functions related to radial velocity analysis, the actual
implementation of the algorithm contains functions that computes spatial
coordinates, velocities and barycentric coordinates for each planet. These
functions aid the joint analysis of multiple transiting planetary systems,
transit timing and/or duration variations or systems where the proper motion of
the host star is also measured involving high precision astrometry. The
practical implementation related to the above mentioned problems features
functions that makethese kind of investigations rather simple and effective.Comment: Accepted for publication in MNRAS, 11 pages, 1 figure, 3 table
Diffusion Maps Kalman Filter for a Class of Systems with Gradient Flows
In this paper, we propose a non-parametric method for state estimation of
high-dimensional nonlinear stochastic dynamical systems, which evolve according
to gradient flows with isotropic diffusion. We combine diffusion maps, a
manifold learning technique, with a linear Kalman filter and with concepts from
Koopman operator theory. More concretely, using diffusion maps, we construct
data-driven virtual state coordinates, which linearize the system model. Based
on these coordinates, we devise a data-driven framework for state estimation
using the Kalman filter. We demonstrate the strengths of our method with
respect to both parametric and non-parametric algorithms in three tracking
problems. In particular, applying the approach to actual recordings of
hippocampal neural activity in rodents directly yields a representation of the
position of the animals. We show that the proposed method outperforms competing
non-parametric algorithms in the examined stochastic problem formulations.
Additionally, we obtain results comparable to classical parametric algorithms,
which, in contrast to our method, are equipped with model knowledge.Comment: 15 pages, 12 figures, submitted to IEEE TS
- …