Next: ISAS Data Archive and Transmission System (DARTS)
Up: Archiving
Previous: Object Oriented Near Real Time Packet Distribution as Part of the FIRST Integrated Network and Archive System
Table of Contents -
Subject Index -
Author Index -
PS reprint -
Werger, M., Bennett, K., Hazell, A., O'Mullane, W., Oosterbroek, T., & Parmar, A. 2000, in ASP Conf. Ser., Vol. 216, Astronomical Data
Analysis Software and Systems IX, eds. N. Manset, C. Veillet, D. Crabtree (San Francisco: ASP), 176
The BeppoSAX Low-Energy Concentrator
Spectrometer Archive at the Space Science
Department of ESA
M. Werger, K. Bennett, A. Hazell, W. O'Mullane, T. Oosterbroek,
A. Parmar
Astrophysics Division of ESA, ESTEC, Noordwijk, The Netherlands
Abstract:
The
Astrophysics
Division
of the European Space Agency (ESA)
has developed a data analysis and archive system in order to provide
scientific products of a consistent quality for the BeppoSAX
Mission. The archive is principally used as a tool to support the
calibration of the
Low-Energy Concentrator
Spectrometer (LECS)
instrument - supplied by the Astrophysics Division - onboard
BeppoSAX. This is an imaging X-ray detector sensitive in the energy
range 0.1-10 keV with an energy resolution of 9% at 6 keV, and a 40'
diameter circular field of view. The spatial resolution is around 1.5'
full-width at half-maximum (FWHM) and the finest time resolution
available is 16 microseconds.
With minimal operator intervention, all data handling and reduction
tasks for the archive are performed in an integrated manner on a daily
basis. We show the user interface, the set-up of the pipeline, and the
data archive. All three components contain elements common to other
scientific missions of the Astrophysics Division. While some parts are
specific to the particular database system used, or to the LECS energy
range, the overall design may serve as a template for future data
archives.
The satellite BeppoSAX, a
major program of the Italian Space Agency (ASI) with participation of
the Netherlands Agency for Aerospace Programs (NIVR), was launched on
April 30, 1996 from Cape Canaveral. The satellite payload consists of
four narrow-field instruments (NFIs) and two wide-field cameras
(WFCs) (Boella et al. 1997).
The NFIs are the Low Energy Concentrator Spectrometer (LECS, energy
range 0.1 - 10 keV), the Medium Energy Concentrator Spectrometer (MECS,
1.3 - 10 keV), a collimated High Pressure Gas Scintillation Proportional
Counter (HPGSPC, 4 - 120 keV), and a collimated Phoswich Detector System
(PDS, 15-300 keV). Thus the NFIs cover more than three decades of energy
- from 0.1 to 300 keV - with a relatively large area, a medium energy
resolution and imaging capabilities in the range of 0.1 - 10 keV with an
approximate resolution of 1.5'.
Both WFCs are oriented perpendicular to the NFIs looking in opposite
directions. They cover an energy range from 2 - 30 keV with an angular
resolution about 5'. While the NFIs image only a field of 40' diameter
for LECS and 60' diameter for the MECS to 1.3
for PDS, the WFCs
viewing field measures 20 by 20 FWHM. BeppoSAX orbits
the earth in a circular orbit with an altitude of about 600 km and an
inclination of 3.9. During the orbital period of 96 minutes,
there is only a single 10 minute contact with the ground station in
Malindi, Kenya, during which all data is downloaded and forthcoming
commands are uploaded.
Figure 1:
The SAXbrowse User Interface using Oracle FORMS.
 |
The archive software may be taken as a typical example of rapid
application development. Data-types and handling as well as the archive
structure are constrained by user requirements. The full archive
software is implemented in a pre-existing environment, determined by the
database, the physical data archive, and the computer
network. Consistency and arbitrary usability of data is guaranteed by
using a commercial database product (Oracle 7.3) to store all mission
science related data. Data access is significantly constrained in
methodological terms: database transactions are performed using only the
Oracle tools, e.g. sqlloader and sqlplus, or the Perl DBI module. Data
relationships and descriptions are written in database independent
schema files. This enables easy adaption of the archive software and
database tables during the development and maintenance
process. Simultaneous prototyping resulted in a software suite which
encompasses the full range of functions including creation of the
archive database, population of the tables, and performing other routine
tasks. This software suite is running on several clustered UNIX
workstations with Sun Solaris 2.5 or higher. Software development is
facilitated by use of the Concurrent Versions System (CVS) which allows
easy tracking and merging of changes. Accordingly, a central repository
holds all related files. While in principal CVS is command line driven,
queries about source code may be made using CVSWEB, an optionally
available CGI script which has been adapted to local requirements.
The Final Observation Tapes (FOTs) are received from the
SAX Scientific Data Centre
(SDC)
in Rome, Italy. Prior to any actual
scientific analysis with popular X-ray software packages (e.g. Xspec,
Ximage), the FOT data have to be pre-processed using a system of
dedicated instrument-specific programs. This software is referred to as
the SAX Data Analysis System (SAXDAS). The first step is ingesting the
FOT to produce the raw FITS event files. The following steps are:
creation of the linearized event lists from the raw event files,
instrument response generation, and creation of standard background
files. The other instruments (i.e. MECS, HPGSPC, and PDS) data are
treated similarly. The data files are then scientifically exploited in
several steps. First of all, data from the MECS detectors are merged to
improve statistics. Then images are created from these products. Source
detection is then performed with Ximage. Finally, creation of spectra
and lightcurves for detected sources follows. Spectral fitting is
performed separately for the LECS and MECS, and also on a combination of
both data sets. The response matrix appropriate for the position in the
LECS field of view is used. The LECS and MECS images are checked for
spatial coincidence of sources. Thus, the database contains data of all
scientific products, i.e. LECS and MECS data. This allows the use of the
database for several tasks:
- Selection of suitable sources as calibration objects,
- Long-term monitoring, e.g. of instrumental background,
- Long-term checks, e.g. on temperature-gain relation.
The first of the available user interfaces has been built using the
Oracle FORMS interface (see Figure 1). Here the user has full
access to all project related tables and contents. Through the top level
database menu data from the observation log, the list of received FOTs,
the cleaned event lists, spectra, and spectral fits are shown. Simple
queries may be entered directly. Complex SQL queries may be designed
with the Oracle Browser. The second user interface provides a quick-look
facility by means of a set of HTML pages using pre-formatted text and
frames for fast access and simple navigation. The generation of these
pages is done by UNIX cron jobs. These HTML pages summarize performed
observations, available FOTs, related products of the pipeline, and the
relations to each other. In addition, a simple HTML form interface
enables primitive queries which formats the search result in the same
style as the other HTML pages.
A cone search of cleaned event lists is available through a Perl/Tk or
an Oracle FORMS interface.
All project related files are stored in a hierarchically organized
archive. This consists of two Hewlett-Packard juke-boxes with
magneto-optical platters (1.3 or 2.6 GB for each medium) and one
tape-robot manufactured by ADIC-Grau handling Sony AIT tapes (25 GB
uncompressed for each tape). The optical juke-boxes are used like
external drives attached to the SCSI bus of the server, while the
tape-robot communicates via TCP/IP on the LAN and via a direct SCSI
connection with the server. Special server software (AMASS, DataMgr, and
DAS by ADIC, formerly EMASS) makes accessing this archive fully
transparent to the user. Files are cached both on NFS exported
directories and on hard-disks in the server for improved access of
frequently and/or recently used files. Loading/Unloading media from the
drives results in a non-negligible response time which is typically less
than 30 seconds; maximum read throughput is of the order of 2
MB/s. For performance issues, small files are mainly held on optical
platters, while files larger than a certain limit (several 10 MB)
are generally kept on tapes. The LECS archive currently holds about 1300
FOTs of approximately 400 unique observation targets. Additionally, 3000
product files are available: about 1040 images and 1950 spectra for all
available observations. Mean file sizes for the raw FOTs are about 64
MB (which may exceed 200 MB for single FOTs), for cleaned event
lists about 27 MB, and for housekeeping files about 21 MB. All
other files sizes are typically less than 1 MB. Currently, in total
23000 files are stored on optical platters or tape and the total used
space is about 222 GB. The archive is used also by other ESA
projects, e.g. COMPTEL, Hipparcos, EXOSAT, and Planck. The total
available space is 12 TB - if all possible media (more than 800)
would be in use.
It is possible to quickly build useful systems like the one described
using available tools and resources. Using a commercial database
management system (DMBS) and/or a repository supports the implementation
of such systems however a well engineered design of the final layout is
still needed.
References
Boella, R. et al. 1997, BeppoSAX, the wide band mission for X-Ray Astronomy, A&AS, 122, 299, and references therein
© Copyright 2000 Astronomical Society of the Pacific, 390 Ashton Avenue, San Francisco, California 94112, USA
Next: ISAS Data Archive and Transmission System (DARTS)
Up: Archiving
Previous: Object Oriented Near Real Time Packet Distribution as Part of the FIRST Integrated Network and Archive System
Table of Contents -
Subject Index -
Author Index -
PS reprint -
adass@cfht.hawaii.edu