Next: Remote Access to the Tycho Catalogue and the Tycho Photometric Annex
Previous: FITS BoF Session
Up: Data Archives
Table of Contents - Index - PS reprint - PDF reprint

---

Astronomical Data Analysis Software and Systems VI
ASP Conference Series, Vol. 125, 1997
Editors: Gareth Hunt and H. E. Payne

---

The XTE Data Finder (XDF)

A. H. Rots
Universities Space Research Association, XTE Guest Observer Facility, Code 660.2, Goddard Space Flight Center, Greenbelt, MD 20771

K. C. Hilldrup
Hughes STX, XTE Guest Observer Facility, Code 660.2, Goddard Space Flight Center, Greenbelt, MD 20771

Abstract:

1. Introduction

The Rossi X-ray Timing Explorer ( RXTE) is a High Energy Astrophysics mission, launched in December 1995. It carries two pointed instruments (PCA and HEXTE) that together cover the range 2-250keV with µs time resolution and moderate spectral resolution. A third instrument (ASM) will monitor most of the X-ray sky every 90minutes in the 2-10keV range.

RXTE's on-board science data systems provide considerable processing power and unprecedented flexibility in telemetry data modes. Events are processed on-board in several simultaneous data modes, chosen from a large repertoire. New data modes may be added during the mission. Consequently, it is a challenge to keep track of the database contents, and to provide a mechanism to select data that satisfy selection criteria expressed in physical terms.

That challenge is met by the design of a database that exclusively uses FITS binary tables (for storage of observational data as well as metadata) and of a tool for navigating that database.

2. Design of the FITS Database

The RXTE FITS Database (XFDB) is described in more detail bye Rots (1996). Here we will only provide a brief description of design features that are essential for understanding the data finder.

Our basic design contains a three level hierarchy: Master Index, Subsystem Index, and Data Table. Table 1 outlines this structure. In all tables, the ``vertical'' (or ``row'') axis is time (or observation-which amounts to the same), while the ``horizontal'' (or ``column'') axis distinguishes spacecraft subsystem units, or data sources.

Starting at the top of the hierarchy, each row in the Master Index contains, for a single observation, references to all the Subsystem Indices that, in turn, contain references to Data Tables belonging, or pertaining, to that observation. There are Subsystem Index tables for the science instruments, for the spacecraft attitude control system, for the clock corrections, for the orbit ephemerides, for the system of calibration files, etc. In addition, the Master Index contains columns with observational parameters, such as Observation ID, start and stop times, and source information.

The Subsystem Index tables contain rows that correspond to segments of the observation during which all telemetry data for that subsystem were deposited in the same set of Data Tables. Each row contains references to those Data Tables, as well as data mode and configuration information when relevant.

3. Database Navigation

Given a set of selection constraints, such as selected source(s) and time range(s), it becomes fairly simple to navigate through the system and find the data items one is looking for. The Master Index acts as an observation catalog with references to Subsystem Indices, while the latter contain configuration information and references to the Data Tables.

We have written an XTE Data Finder (XDF) as an itcl script with some C code. It allows the user to navigate the database interactively, using the Master Index and Subsystem Indices, producing a list of data files that satisfy the user's selection criteria. As such, it is the data selection interface to subsequent data analysis tools. A sample of XDF's user interface is shown in Figure 1.

  
Figure: XDF graphical user interface. Original PostScript figure (36kB).

4. Data Retrieval

Even though XDF was designed purely as a database navigation tool, it quickly became clear to us that its power would be greatly enhanced if we would incorporate data retrieval capabilities-something that was very easy to do, as a matter of fact. For its navigation functions XDF only needs the information from the index tables; its final product is a list of data tables. It is trivial to incorporate the actual retrieval of these tables (by ftp) to this function. The next step is that not even all index tables need to be resident locally; XDF can decide which ones are required and, again, download them as needed. One might even imagine that, if it cannot find the Master Index Table, it will retrieve that as well.

The result is that XDF allows the user unlimited access to the entire RXTE mission archive (at least, the part that is public), while requiring mass storage space only for those data that are actually selected. We would like to emphasize that XDF is primarily the data selection interface to analysis software, and hence the data retrieval capabilities are made transparent to the user, without the need for a separate Web browsing or Internet searching tool. The issue of needing only to store data that are actually going to be used is an important one since the RXTE (compressed) archive data volume is of the order of 1GB/day.

Finally, one more implementation detail. To save space, and to increase data transfer speeds, all data files in the archive are compressed by the utility gzip. XDF will automatically take care of the uncompressing of the data files, too.

References:

Rots, A. H. 1996, in Astronomical Data Analysis Software and Systems V, ASP Conf. Ser., Vol. 101, eds. G. H. Jacoby and J. Barnes (San Francisco, ASP), 540

Next: Remote Access to the Tycho Catalogue and the Tycho Photometric Annex
Previous: FITS BoF Session
Up: Data Archives
Table of Contents - Index - PS reprint - PDF reprint