|
Документ взят из кэша поисковой машины. Адрес
оригинального документа
: http://www.adass.org/adass/proceedings/adass02/P6-4/
Дата изменения: Thu Mar 13 03:15:35 2003 Дата индексирования: Tue Oct 2 04:40:40 2012 Кодировка: Поисковые слова: п п п п п п п п п п п п п п п |
k
Place,
University Park,
Hilo, HI
96720
Malcolm J. Currie
ORAC-DR is the data reduction pipeline for the Observatory Reduction and Acquisition Control project (ORAC1, Wright et al 2001). The ORAC-DR pipeline design was presented previously by Economou et al (1998) and Jenness & Economou (1999) and makes extensive reuse of existing data reduction packages (Economou et al 1999). Allan et al (2002) described groundwork for turning ORAC-DR into a truly generic data reduction pipeline. For the past three years it has been used at the United Kingdom Infrared Telescope (UKIRT) and for SCUBA observations on the James Clerk Maxwell Telescope (JCMT).
When developing the ORAC-DR pipeline, we recognized that many of the tasks performed by instrument-specific pipelines are shared. File handling, calibration systems, and header translations are tasks that are often duplicated. Also, the actual data reduction tasks are similar from one instrument to the next. Repeating the implementation of these tasks by developing a specific pipeline for each new instrument was considered to be a waste of time and resources, so we sought a generic solution.
To accomplish this, we created a generic, and yet advanced, object-oriented ORAC-DR infrastructure which handles file conversions, header translations, calibration information, and other generic tasks. It also runs on a `primitive'-based system, where one primitive (Perl script) does the work of one astronomically meaningful step. These primitives also do a bare minimum of computing; processing of bulk data is done by CCDPACK (Draper, Taylor & Allan 2000), KAPPA (Currie & Berry 2000), and Figaro (Shortridge et al 1999; Shortridge 1993), all supported by Starlink.
As ORAC-DR is a data-driven data reduction pipeline, it obtains most of the information required to reduce data from the data itself. For astronomical data, this means that FITS headers need to be complete and correct. Simple instrument-specific Perl modules translate FITS headers into internal headers that are used by any code relying on headers. These translations are not limited to changes of name, and may involve combinations of headers. Through inheritance one instrument can acquire header translations from another, further reducing the number of header-translation modifications necessary to use ORAC-DR with a new instrument.
This same philosophy applies to file formats. As the actual data reduction calculations are performed by Starlink tasks whose internal data format is the Extensible N-Dimensional Data Format (NDF, Warren-Smith 2000), ORAC-DR minimizes the number of conversions from a different format (FITS, for example) by doing a conversion to NDF when it first sees a data file. Which conversion ORAC-DR does is instrument-specific, and this is programmed in such a way that adding a new instrument and new data conversions requires minimal effort.
Inheritance is also used for recipes and primitives. A large collection of generic recipes and primitives exists. Should an instrument require a new or modified recipe or primitive then this new code is placed in an instrument-specific subdirectory. When ORAC-DR searches for a primitive, it looks in the instrument-specific directory, then in the instrument-agnostic directory.
Occasionally new observing modes are needed, or instruments with different capabilities are developed. An example of this is the imaging spectrometer UIST, which also has an integral field unit. ORAC-DR had no previous IFU support, so it was necessary to develop a new set of primitives and recipes to reduce IFU data.
Modifying ORAC-DR to be used on an instrument similar to currently supported instruments takes approximately one to two weeks of programming time. If new observing modes are needed, then this time is increased to one to four months, depending on the number of recipes needed and the instrument's complexity.
IRIS2 is an infrared imager and spectrograph currently being commissioned for the 3.9-meter Anglo-Australian Telescope. In addition to wide-field imaging and long-slit spectroscopy, it also offers multi-object spectroscopy. It writes data in FITS format, not the NDF that UKIRT instruments use.
In July 2002 ORAC-DR was commissioned for imaging with IRIS2, needing only two weeks of programming effort to become fully usable. As ORAC-DR relies on FITS headers to process data, a side-effect of porting it to other instruments is an ability to find omissions and deficiencies in header information. During the commissioning of ORAC-DR it was noticed that an incorrect value for the array read noise of IRIS2 was being reported. This was traced to a bug in the data acquisition software and was immediately fixed. Had ORAC-DR not been running at the AAT at the time, this bug would have been found much later and after much more effort.
During the commissioning of ORAC-DR, IRIS2 made a deep image of a section of sky that would later be observed with the Very Large Telescope. As ORAC-DR was able to calculate a reduced mosaic of the region in real-time, it was possible to send this mosaic to the principle investigator. He then used this mosaic to select out high- z galaxy candidates from low-mass stars for follow-up VLT multi-object spectroscopy a few days later. ORAC-DR was able to save the principle investigator the time and effort necessary to reduce their data and did so needing only minimal changes.
UIST is an imaging spectrometer very similar to IRIS2 in capabilities. Adapting ORAC-DR to work with imaging and spectroscopy modes with UIST was thus trivial, requiring approximately two weeks of effort to run correctly.
In addition to imaging and spectroscopy modes, UIST has an integral field unit which allows it to simultaneously obtain spectra at a number of points on a field. This mode is one that had not been previously supported by ORAC-DR, so programming effort required to get ORAC-DR properly reducing IFU data was approximately three months by a programmer with no previous experience with ORAC-DR.
GMOS is a multi-object spectrograph with imaging, long-slit spectroscopy, and IFU modes. Basic System Verification data were released for imaging and multi-object spectrograph modes, and imaging data was used for ORAC-DR. The FITS headers for this SV data are incomplete and raw calibration data are not available, so ORAC-DR support for GMOS is minimal. However, ORAC-DR can successfully reduce publicly available GMOS data. The programming time needed to attain this level of support was eight hours.
We would like to thank STARLINK for distributing ORAC-DR and supporting many packages ORAC-DR uses. We thank Gemini for releasing GMOS System Verification data for its use with ORAC-DR.
Allan, A., Jenness, T., Economou, F., Currie, M. J., & Bly, M. 2002, in ASP Conf. Ser., Vol. 281, Astronomical Data Analysis Software and Systems XI, ed. David A. Bohlender, Daniel Durand and T. H. Handley (San Francisco: ASP), 311
Currie, M. J., Berry, D. S. 2000, Starlink User Note 95, Starlink Project, CCLRC
Draper, P. W., Taylor, M., & Allan, A. 2000, Starlink User Note 139, Starlink Project, CCLRC
Economou, F., Bridger, A., Wright, G. S., Rees, N. P., & Jenness, T. 1998, in ASP Conf. Ser., Vol. 145, Astronomical Data Analysis Software and Systems VII, ed. R. Albrecht, R. N. Hook, & H. A. Bushouse (San Francisco: ASP), 196
Jenness, T., & Economou, F. 1999, in ASP Conf. Ser., Vol. 172, Astronomical Data Analysis Software and Systems VIII, ed. David M. Mehringer, Raymond L. Plante, & Douglas A. Roberts (San Francisco: ASP), 171
Shortridge, K. 1993, in ASP Conf. Ser., Vol. 52, Astronomical Data Analysis Software and Systems II, ed. R. J. Hanisch, R. J. V. Brissenden, & J. Barnes (San Francisco: ASP), 219
Shortridge, K., et al. 1999, Starlink User Note 86, Starlink Project, CCLRC
Warren-Smith, R. F. 2000, Starlink User Note 33, Starlink Project, CCLRC
Wright, G. S. et al. 2001, in ASP Conf. Ser., Vol. 238, Astronomical Data Analysis Software and Systems X, ed. F. R. Harnden, Jr., Francis A. Primini, & Harry E. Payne (San Francisco: ASP), 137