Next: A Broadband Wide-Field Survey on the Isaac Newton Telescope
Up: Data Pipelines
Previous: The SCUBA Data Reduction Pipeline: ORAC-DR at the JCMT
Table of Contents -
Subject Index -
Author Index -
Search -
PS reprint -
PDF reprint
Currie, M. J., Wright, G. S., Bridger, A., & Economou, F. 1999, in ASP Conf. Ser., Vol. 172, Astronomical Data Analysis Software and Systems VIII, eds. D. M. Mehringer, R. L. Plante, & D. A. Roberts (San Francisco: ASP), 175
Data Reduction of Jittered Infrared Images Using the ORAC Pipeline
Malcolm Currie, Gillian Wright, Alan Bridger
UKATC, Royal Observatory, Blackford Hill, Edinburgh, UK
Frossie Economou
Joint Astronomy Centre, 660, N`ohk Place, Hilo, HI
96720
Abstract:
We relate our experiences using the ORAC data reduction pipeline for
jittered images of stars and galaxies. The reduction recipes
currently combine applications from several Starlink packages with
intelligent Perl recipes to cater to UKIRT data. We describe the
recipes and some of the algorithms used, and compare the quality of
the resultant mosaics and photometry with the existing facilities.
The UKIRT ORAC (Observatory Reduction and Acquisition Control) project
(Bridger et al. 1998) aims to improve the efficiency of observing at
UKIRT by providing a modern software suite that will be capable of
fully automated observing. ORAC will also make the telescope and
instruments easier to use for traditional observing, by, for example,
reducing the amount of manual intervention that is needed.
One of the key components is the ORAC Data Reduction system
(ORAC-DR). Its goal is to offer observers near
real-time data reduction, and hence let observers accurately assess
the quality of their data while they are still at the telescope. Some
of the benefits are more efficient use of the telescope, a higher
publication rate, and better quality data. ORAC-DR
uses a modular pipeline with the intent of producing close to
publication quality images and spectra.
This paper concentrates on the imaging, describing some of the
data reduction recipes, and lists some pros and cons of using other
people's software.
We have written a number of common data reduction recipes and
matching observing sequences which cover a range of photometry and
imaging techniques in use at UKIRT.
A recipe is a series of high level instructions such as ``make a
flat'' or ``generate offsets''. The implementation of each of these
instructions is through a Perl script, called a primitive, which
calls particular data reduction packages, currently from Starlink, to
actually do the work.
Many primitives are generic, so new recipes can be created fairly
quickly. Most recipes contain a steering primitive to instruct the
other primitives what operations should be performed on that image or
usually the group of images to which it belongs, and to classify
blank-sky from target frames. The recipes run initialisation scripts
to enable history recording and specify detector characteristics and
subsets, and all apply a bad pixel mask.
The pipeline uses calibration indices for dark, flat, and sky frames.
When a calibration frame is required, the recipe accesses the index to
find a suitable image, i.e., one which satisfies a set of criteria stored
in a rules file. Typical rules are that the filters match and pick
the most recent calibration.
In the summary of the jitter-pattern recipes below, the letters in
parentheses following the names are codes for processing steps invoked
in the recipe. The codes are D=dark subtracted, F=self-flat, M=make
mosaic, O=objects masked, R=automatic registration, S=sky subtracted.
Section 3 describes these steps.
- BRIGHT_POINT_SOURCE
- (DORM)
It assumes that a SKY_FLAT flat-field is sufficiently
stable for a significant fraction of a night not to affect photometry.
- EXTENDED_nxm
- (DSORM) This is experimental for
extended objects where there may be little sky in close proximity for
background subtraction. It alternates between jittered blank
sky and the extended source, mapping the source in a n column by m
row grid with 50% overlap of adjacent target frames. The mosaic is built
in rows. At the completion of a row, the interleaved blank-sky frames are
combined to make a flat, which is then applied to the source frames in
that row. The order within rows and the order the rows are observed
is such that adjacent frames are not observed sequentially. This
randomising aims to reduce systematic errors from the sky subtraction.
The interpolated mode of the bracketing sky frames is subtracted prior
to flat-fielding. This method of sky subtraction has worked in
practice achieving uniformity to 0.15% in good sky conditions.
This is a major improvement over the
formerly used IRCAMDR
algorithms applied to the same galaxy.
- JITTER_SELF_FLAT
- (DFORM) This is a 5- or 9-point
jitter for point sources.
- QUADRANT_JITTER
- (DFORM) The original was
devised by Lance Miller to search for faint extensions around bright
objects, without the need to take separate flat observations, or rely
upon the flat's long-term stability. The jitter pattern moves the
target source to approximately the centre of each quadrant, and
the cycle is repeated to the desired integration time. A preliminary
flat is derived from a median of the target frames with the quadrant
containing the source excluded. The improved version uses this as
a trial flat and masks all the prominent objects to make a superior flat
(see the Object Masking description in §3.). The refinement
reduces the noise around the central
target by 40% and removes artifacts. Sky regions are typically
flat to better than 0.01% even in crowded fields.
- REDUCE_DARK
- This just applies the bad pixel mask
and files the calibration.
- SKY+JITTERn
- (SRM) It is similar to
BRIGHT_POINT_SOURCE, except that it subtracts the sky
rather than a dark frame, and is intended for moderately
faint sources. n = 5, 9. It files the sky calibration.
- SKY_FLAT
- (D) This forms and files a flat from a
5 (or more) point jittered mosaic of blank sky. There is a variant which
masks sources within the frame.
Primitives are the Perl scripts which actually call the application
tasks to do most of the work. Some tasks were designed with CCDs in
mind, so a degree of tuning is sometimes needed to allow for
infrared instruments, pixel scale, seeing, or filter, for example.
- Flat creation.
- Frames are optionally cleaned to remove extreme outliers
(3 or 6 about the mean in 15 x 15-pixel neighbourhood),
normalised, combined pixel by pixel using a median, and the combined
array normalised.
- Object Masking.
- After the application of an approximate flat field, an
inventory is made of objects having at least 12 connected pixels above
one sigma above sky. (The optimum thresholds are still being
determined.) The locations, shapes, orientations, and sizes are used to
make a mask. The mask is applied to the dark-subtracted frames and a
new flat created. The improved flat typically shows a uniformity at
0.02% of the sky. Systematic errors in the sky, which are a major uncertainty
in infrared point source photometry, are also reduced significantly
by this algorithm.
- Automatic Registration.
- This makes an inventory of the sources above
a threshold in each frame. It then performs a pattern recognition
(Draper 1998) to identify common features in jittered frames. If the
fraction of common objects is under half or the total is fewer than 3,
the registration fails, and so the script resorts to reading the
offsets stored in the FITS headers or matching a central bright
object. Using telescope offsets can lead to trailed sources, as
occurred with the IRCAMDR package. The improved
registration leads to the detection or more accurate measurement of
faint sources.
- Mosaicking.
- This is fully automated. Observers do
not have to
painstakingly measure centroids and manually tile to form mosaics.
In addition to Cartesian shifts derived from the automatic
registration, the primitive applies a preset rotation matrix to align
the image to the cardinal directions with only one resampling.
Experiments are underway to use jitter offsets aligned with the chip
and to put the rotation information into the World Coordinate System
to minimise artifacts in the data and to save CPU cycles.
The mosaicking uses the CCDPACK algorithm (Warren-Smith 1993). Only
zero-point shifts of intensity are applied to the resampled frames to
create the mosaic. Depending on the recipe, the mosaic may be trimmed
to the dimensions of a raw frame. Mosaicking removes virtually all
the bad pixels.
- Automatic Photometry.
- This is adaptive. It locates
the source, measures its point spread function, selects a suitable
aperture size
scaled in terms of the FWHM and a concentric sky annulus, integrates
the flux within the object aperture, and subtracts the modal sky
value. The previous software used a fixed aperture and the
median sky from a narrow annulus, and hence could generate less accurate
photometry. The script also evaluates a correction for light outside
the aperture. The results presented include a standard extinction
coefficient and zero-point and a formal measurement error.
Rather than consisting of large packages dedicated to each instrument,
ORAC-DR uses other people's general purpose,
data reduction software. This approach reduces maintenance costs,
provides greater flexibility (such as to reflect quickly a change in
the performance or understanding of an instrument), and has the ability to
take advantage of new astronomical software and techniques.
It has its downside too. Sometimes the packages either do not provide what
we need for a recipe, or we have to invoke several atomic tasks at the
cost of efficiency and intermediate files. For example, the
point spread function of UKIRT images are markedly non-Gaussian. The
KAPPA/PSF (Currie 1997) only fits Gaussian-like
functions, whereas a two-component model is needed. Looking at an
extreme case
reveals a
double hexagonal pattern, for which a general psf-fitting task would
never be expected to model. Thus, we expect some recipes will require
the odd bespoke application.
See Economou et al. (1999) for further discussion on this topic.
Acknowledgments
We thank the Starlink programmers, particularly Peter Draper,
for implementing enhancements to packages.
We also thank the various astronomers whose data have been used to test
ORAC-DR.
References
Bridger, A., Economou, F., Wright, G. S., & Currie, M. J. 1998,
in SPIE Proc., Vol. 3349, Observatory Operations to Optimize Scientific Return,
ed. P. J. Quinn, (Bellingham: SPIE), 184
Currie, M. J. 1997, KAPPA - Kernel
Application Package, (Starlink User Note 95, Rutherford Appleton
Laboratory)
Draper, P. W. 1998, CCDPACK - CCD
Data Reduction Package,
(Starlink User Note 139, Rutherford Appleton Laboratory)
Economou, F., Bridger, A., Wright, G. S., Jenness, T.,
Currie, M. J., & Adamson, A. 1999 this volume, 11
Warren-Smith, R. F. 1993, in Proc. of the 5th
ESO/ST-ECF Data Analysis Workshop, ed. P. Grosbøl & R. Ruijsscher (Garching: ESO), 39
© Copyright 1999 Astronomical Society of the Pacific, 390 Ashton Avenue, San Francisco, California 94112, USA
Next: A Broadband Wide-Field Survey on the Isaac Newton Telescope
Up: Data Pipelines
Previous: The SCUBA Data Reduction Pipeline: ORAC-DR at the JCMT
Table of Contents -
Subject Index -
Author Index -
Search -
PS reprint -
PDF reprint
adass@ncsa.uiuc.edu