Next: Variable-Pixel Linear Combination
Previous: The Portable-CGS4DR Graphical User Interface
Up: Science Software Applications
Table of Contents - Index - PS reprint - PDF reprint

---

Astronomical Data Analysis Software and Systems VI
ASP Conference Series, Vol. 125, 1997
Editors: Gareth Hunt and H. E. Payne

---

RoadRunner: An Automated Reduction System for Long Slit Spectroscopic Data

Susan P. Tokarz and John Roll
Smithsonian Astrophysical Observatory, Cambridge, MA 02138

Abstract:

Driven by a dramatic, almost fivefold, increase in data flow, we developed a system, RoadRunner for the automatic and rapid reduction of CCD two-dimensional long slit data taken at the 1.5 meter telescope at the Fred L. Whipple Observatory on Mt. Hopkins. The typical 70 to 100 observations made in an average night are now routinely reduced in a few hours the following day. The reduction process takes the data from raw two-dimensional images to one-dimensional wavelength calibrated spectra with measured redshifts which have been checked for accuracy. The reduced spectra are stored in an on-line archive and are entered into a relational database. Standardizing observing protocols proved to be a requirement for designing an automated reduction system. RoadRunner, which is based on IRAF data reduction routines, has many internal checks for errors and anomalies and produces a data set of high quality.

1. Introduction

In January 1994 a new spectrograph, FAST (Fabricant et al. 1997), was mounted on the 1.5 meter telescope on Mt. Hopkins. This two-dimensional longslit CCD spectrograph replaced the Z-machine, a one-dimensional Reticon detector which had been in operation for almost 16 years and was used primarily to measure galaxy redshifts (e.g., the First CFA Redshift Survey). In 2.75 years of operation, 22,889 FAST spectra have been reduced. By comparison, over the nearly 16 year lifetime of the Z-machine, 27,169 spectra were reduced.

The huge increase in the amount of data obtained from the 1.5 meter, combined with the more complex reduction procedure required by two-dimensional data made it necessary to automate our reduction. A further complication for the reduction is that the higher efficiency and greater flexibility of FAST allows a vastly wider range of observing programs; there are now 55 programs using FAST and they require different combinations of gratings, CCD pixel binnings, slits, and grating tilts. In addition to the redshift surveys, these programs include observations of symbiotic stars, ISO quasars, AGNs for monitoring, dwarf novae, etc.

2. Observing Protocols

As a first and essential step in automating reduction we set observing protocols. The FAST Observing Protocols are a list of instructions about how and when to take biases, flats, darks, comps, skys and standards; how to determine object integration times; and how to handle setup changes. These protocols ensure uniformity and completeness of the data and make it possible to reduce data automatically from queued observations and for a variety of observers. Although our remote observers do at least 80% of the observing, FAST is also used by graduate students, postdoctoral fellows, and staff scientists. Without protocols, automated reduction would not be possible.

3. Automated Reduction

The data are transferred automatically at the end of a night's observing from Mt. Hopkins to SAO in Cambridge, Mass. Data from a night can be reduced and entered into a database in just a few hours of processing. A recent typical night had 287 raw data files: 40 biases, 40 flats, four darks, five skys, 83 comps, and 115 object exposures. There were two binning modes: in some cases, the CCD pixels were binned by two in the spatial direction, in others by four.

The automated reduction system, RoadRunner, consists of a series of IRAF routines combined in processing scripts. Many of the routines used, such as CCDPROC, RESPONSE, REIDENTIFY, APALL, etc., are widely available. Additional routines were written as necessary for automated processing. Some of the new routines are fairly straightforward: for example, there is code to pass the calculated shift between two comparison lamps from FXCOR, where the calculation is made, to REIDENTIFY, where it is used. Thus we are able to use an old wavelength solution to effortlessly compute good wavelength solutions for our current comparison frames. Other programs are more complex: FINDALL, written by Doug Mink, adjusts size and background parameters used in APALL enabling us to extract a variety of galaxy and stellar spectra automatically. FINDALL is able to extract the correct object and a satisfactory background even when there is more than one object on the slit.

3.1. RoadRunner

RoadRunner, the original script, is completely automated and performs all the preprocessing necessary for reduction. It accepts raw data from one night of observing and produces two-dimensional, wavelength calibrated spectra ready for extraction. Because the data may be taken using a variety of setups at the telescope, the script first sorts the data into sets (e.g., 3´´ slit, binned by four, 300l/mm grating, grating tilt = 607 is a set) and each set is reduced separately.

Processing steps included in RoadRunner are:

 
		  overscan and bias subtraction

		  trimming

		  normalization

		  illumination correction

		  wavelength calibration

		  transformation

There are many checks in RoadRunner to guarantee the accuracy and integrity of the data. For example, there is a plot of the means of the bias frames that can show at a glance when there is something awry in the biases for a night. Most of the other checks are arithmetic rather than visual. Limits are set for computed values and a log is maintained and checked to make certain that all is going as expected. A high residual in a comparison can mean lower signal or some other problem in the lamp. An unusual value in the chip gain/read noise calculation can mean an undocumented change in the grating tilt. The program will also notify the user when something is wrong as when, for example, there is no comparison file for an object.

3.2. BeepBeep

A second major script, BeepBeep, combines automated and interactive utilities which are necessary to complete the reduction. This script accepts the two-dimensional, wavelength calibrated images from RoadRunner and extracts one-dimensional, wavelength calibrated spectra with measured redshifts ready to be archived and entered in our database.

The steps in this script include:

 
 FINDALL 	extract spectra 		 automatic

 QSPEC  	check and correct FINDALL	 interactive

 PFAST  	remove cosmic rays 		 interactive

 XCSAO  	compute redshifts 		 automatic

 QPLOT  	check redshifts 		 interactive

Although it may seem somewhat strange to switch back and forth between automated and interactive programs in one script, doing so has allowed us to efficiently use programs which must be interactive while automating where possible.

4. Output

The reduced one-dimensional, wavelength calibrated spectra with measured redshifts are transferred to a permanent archive on disk. The data are maintained in IRAF format in the archive. From the archive, the data are entered into a relational database which consists of IRAF keywords pulled from the headers. The database consists of flat ASCII files in the starbase database format (Roll 1995). This format permits easy and fast retrieval of information for data analysis.

5. Summary

Automated reduction has become necessary due to the large increase in the amount of data we now collect as well as the increasingly complex data that new instruments provide. The system we have built allows us to handle this data in an efficient way. Automation increases the reliability of the reduced data by ensuring greater consistency in data reduction. An added bonus of automation has been rapid feedback to the observer allowing problems to be addressed at an early stage.

Acknowledgments:

We wish to thank Doug Mink, who over the years has written routines which made automating our reduction easier and who is always generous with his time and energy. Mike Kurtz has participated in many discussions about reduction technique and his sage counsel has been much appreciated. We thank Dan Fabricant for his thoughtful and valuable critique of this paper. Margaret Geller was the guiding force behind automating reduction and we are grateful for her generative spirit which stimulates, encourages and fosters growth.

References:

Fabricant, D., Cheimets, P., Caldwell, N., & Geary, J. 1997, PASP, in press

Roll, J. 1996, in Astronomical Data Analysis Software and Systems V, ASP Conf. Ser., Vol. 101, eds. G. H. Jacoby and J. Barnes (San Francisco, ASP), 536

Next: Variable-Pixel Linear Combination
Previous: The Portable-CGS4DR Graphical User Interface
Up: Science Software Applications
Table of Contents - Index - PS reprint - PDF reprint