Документ взят из кэша поисковой машины. Адрес
оригинального документа
: http://www.stecf.org/conferences/adass/adassVII/wangz.html
Дата изменения: Mon Jun 12 18:51:57 2006 Дата индексирования: Mon Oct 1 23:31:12 2012 Кодировка: Поисковые слова: mdi |
Next: The IDL Wavelet Workbench
Up: Applications
Previous: The Mosaic Data Capture Agent
Table of Contents -- Index -- PS reprint -- PDF reprint
Zhong Wang
Smithsonian Astrophysical Observatory,
Cambridge, MA 02138 U.S.A. Email: zwang@cfa.harvard.edu
Historically, spectral data were taken roughly one spectrum at a time, and most existing data formats reflect such a methodology. The traditional ways in which FITS can be used for spectral data are less than adequate in meeting the needs of a new generation of instruments that produce large amount of relatively homogeneous data, which nevertheless need to be reduced individually. A set of more efficient, better streamlined reduction procedures are necessary, which in turn require more careful considerations in data packaging. This is particularly important for projects that adopt non-interactive, pipeline-style data processing as their primary mode of operation.
The details of the SWAS mission can be found on the internet in SAO's SWAS homepage at SWAS (Submillimeter Wave Astronomy Satellite) mission is essentially a small orbiting radio telescope working at relatively high (about 500 GHz) frequencies (Wang 1995). It is designed to make simultaneous spectroscopic measurements at several sub-mm line wavelengths that are inaccessible from the ground. The spectral data have some special characteristics:
The data reduction plan for the mission calls for a highly efficient pipeline processing scheme in which a large amount of spectroscopic data can be sorted, selected, calibrated and co-added based on some user-changeable criteria. All processing operations have to be recorded and traceable, and are preferably done with minimal human intervention.
We have selected IRAF as our primary programming platform for the SWAS pipeline software, and use FITS as the main format for our data product. However, to store individual scan data in the conventional one- (single spectrum) or two-dimensional (``multispec'' or echelle) format would be very inefficient and even confusing.
Our approach is to make use of the BINTABLE and IMAGE extensions of FITS and store spectral header and pixel data in these two types of extensions in a single FITS data file. The basic rules we adopted are:
In case of using variable-length, vector (array) columns to store pixel data, a table row can have more than one array element, each representing data of a separate spectral band. The length of an array element is the number of pixels in a spectrum of that band.
In case of images, each image row corresponds to a row in the header parameter table. The number of columns of the image is the same as the number of pixels in a spectrum. There can be multiple image extensions in each data file, with different extensions representing data of different spectral bands.
The choice between using the table extension of FITS alone versus using table plus image extensions depends on the actual application. In principle, saving everything in a single table is structurally more appealing and efficient, but not many existing software tools can accept variable-length vector column FITS tables -- which means more development work for basic tools. On the other hand, making use of 2-D images can facilitate data inspection procedures with various tools that examine FITS images. Its shortcoming is that information about a single spectrum is saved in two or more extensions, requiring care in managing them.
Given the lack of a widely-accepted standard in formatting a largely homogeneous spectroscopic dataset, we find the packaging using FITS extensions to be a viable alternative to the conventional ways in which spectral data are saved:
Many of the existing software tools that deal with FITS table and image data are readily available for use with these new data files, sometimes with minor modifications. This can significantly save the programming effort and shorten the software development cycle.
Despite the considerable merits of this approach, some problems remain to be addressed, and we are working on future enhancement of our software.
One of the main problems is the interface with existing software tools (such as visualization tools from certain existing tools packages). In practice, we were able to partly circumvent this shortcoming by writing customized interfaces, but the solutions are not always satisfactory. It appears that if a tabular style file format is deemed sufficiently important, a more fundamental (kernel-level) interface for spectral data in FITS should be developed.
The approach described in this paper is an explorative step in enhancing the use of FITS, especially in taking advantage of the multiple extension development and the use of vector columns in FITS tables. With the rapid advancement in database and data warehousing technology, one would ideally like to have a more rigorous database-like framework to deal with the management of the observational data. This, however, needs to be balanced with the requirement of cost-effective and quick prototyping of many projects' practical applications. The latter often means maximizing the use of existing software tools and adopting conventional approaches. The multiple extension FITS adopted by the community and being incorporated by NOAO and STScI's IRAF teams (e.g., Zarate & Greenfield 1996, Zarate 1998) and other software groups is an important and innovative development, which preserves the usefulness of many old tools while allowing new methods to be tested for astronomical data applications. We believe that more efforts are needed in exploring ways to take full advantage of this very useful data format.
Wang, Z. 1995, in Astronomical Data Analysis Software and Systems IV, ASP Conf. Ser., Vol. 77, eds. R. A. Shaw, H. E. Payne & J. J. E. Hayes (San Francisco, ASP), 402
Zarate, N. & Greenfield, P. 1996, in Astronomical Data Analysis Software and Systems V, ASP Conf. Ser., Vol. 101, eds. G. H. Jacoby and J. Barnes (San Francisco, ASP), 331
Zarate, N. 1998, this volume
We are grateful to Phil Hodge and Nelson Zarate for their help in the software development discussed in this paper. Dr. Matthew Ashby has participated in implementation and testing of the SWAS software.
Next: The IDL Wavelet Workbench
Up: Applications
Previous: The Mosaic Data Capture Agent
Table of Contents -- Index -- PS reprint -- PDF reprint