Документ взят из кэша поисковой машины. Адрес оригинального документа : http://www.stsci.edu/hst/HST_overview/documents/calworkshop/workshop2005/papers/mutchler.pdf Дата изменения: Wed Apr 5 00:02:25 2006 Дата индексирования: Sun Dec 23 23:30:07 2007 Кодировка: Поисковые слова: http www.stsci.edu science goods

The 2005 HST Calibration Workshop Space Telescope Science Institute, 2005 A. M. Koekemoer, P. Goudfrooij, and L. L. Dressel, eds.

Pipeline Calibrations of ACS Data
Max Mutchler, Marco Sirianni1 , & Ray A. Lucas Space Telescope Science Institute, 3700 San Martin Drive, Baltimore, MD 21218 E-mail: mutchler@stsci.edu, sirianni@stsci.edu, lucas@stsci.edu Abstract. Our strategies for collecting ACS calibration data, and converting it into reference files for use in the calibration pip eline, have b een continually evolving since the instrument was installed during Hubble Servicing Mission 3B in March of 2002. We provide an historical overview of basic ACS pip eline calibrations, including the impact that some of the more recent changes have on downstream data processing (e.g. drizzling). This pap er emphasizes bias and dark calibrations for the ACS CCDs, and it represents a significant up date to previous documentation of this sub ject (Mutchler et al. 2004). We describ e the exp ected detector degradations that these calibrations are designed to track and correct, and also some unexp ected anomalies we've encountered and the steps we've taken to ameliorate them.

1.

Introduction: Philosophy and Practices

We have b een continually improving the quality and degree of automation in producing and delivering calibration reference files for use in the ACS pip eline (CALACS, see Pavlovsky et al. 2005). The detector characteristics we are striving to calibrate are describ ed in great detail in Sirianni et al. (2004). The emphasis of this pap er is on bias and dark calibrations for the Wide Field Channel (WFC) and High Resolution Channel (HRC) of the Advanced Camera for Surveys (ACS). In Octob er 2004, we implemented several improvements which we document here. Our goal has always b een to strike the b est balance b etween quality and timeliness. We strive to produce the b est calibrations p ossible within 2-3 weeks following any given ACS observation. For this reason, ACS users should retrieve (or re-retrieve) their data via on-the-fly-reprocessing (OTFR) several weeks after they occur, mainly to ensure that the b est sup erbias and sup erdark reference files have b een applied. In August 2004, we migrated our reference file production system to the native pip eline environment (SunFire), with more streamlined production and overall quality control (Lucas et al. 2006, these Proceedings). The delivery of reference files to the Calibration Database System (CDBS) for use in the pip eline has also b ecome more consolidated and efficient (Diaz-Miller 2005). We also deliver our reference files to the Multimission Archive at STScI (MAST) and make them available for downloading from our `jref ' directory (see Section 8). 2. Bias features and calibration

For the WFC and HRC, the bias level is measured from each frame's overscan region, and subtracted from each amplifier quadrant indep endently. Bias features are subtracted by the `sup erbias' reference file, which is identified in the BIASFILE keyword in image headers.

1

Affiliated to the Space Telescop e Division, Europ ean Space Agency

51 c Copyright 2005 Space Telescop e Science Institute. All rights reserved.


52

Mutchler, Sirianni, & Lucas

Before Octob er 2004, we were obtaining one bias frame every day (for b oth WFC and HRC), and producing a WFC sup erbias every week, and an HRC sup erbias every two weeks. Bias (mainly bad CCD columns) structure was identified (with data quality flag 128) only in a bad pixel table (BPIXTAB) produced in July 2002. Since Octob er 2004, we have b een obtaining one bias frame every other day, and producing sup erbiases nominally every two weeks, for b oth WFC and HRC. The sup erbiases are simple cleaned combinations of the 8 bias frames obtained during each bi-week. Bias structure is now identified with flag 128 in each sup erbias data quality [DQ] array, which propagates to the [DQ] array of science data. Therefore, the growth of bad CCD columns is now b eing tracked and flagged much b etter, typically at two week intervals. Figure 2 illustrates the WFC bias structure. 3. Dark features and calibration

We conduct a `monthly' (roughly every 26 days) CCD annealing which restores many hot pixels to the nominal dark current, although the p opulation of non-annealing hot pixels grows continually (Sirianni, these Proceedings). The annealing cycle also sets the cadence for our reference file production and delivery: we make a batch of sup erbias and sup erdark reference files for every half of one anneal p eriod (roughly every 2 weeks). Before Octob er 2004, we were obtaining four 1000-second dark frames every day, and producing a sup erdark reference file for every day. We were flagging only hot pixels in sup erdark data quality or [DQ] arrays, using flag 16. Since Octob er 2004, we b egan collecting 4 dark frames every other day, and therefore we now produce a sup erdark reference file for every other day. We reduced the numb er of bias and dark frames we obtain to lower the profile of this (the largest) ACS calibration program, with what we felt was minimal impact on the quality of the resulting calibrations. In addition to hot pixels, we also b egan flagging several other dark features: warm pixels, CTE tails of hot pixels, and saturated pixels, which we describ e in detail in the next section. Our dark reference files are actually a hybrid combination of two typ es of sup erdarks. First, we make a 2-week `basedark', which is a high signal-to-noise combination of 32 dark frames. Then we also make a 4-frame `daydark' for each day in the bi-week, which gives the b est snapshot of the warm and hot pixel p opulations on that date. The reference sup erdark is a copy of the basedark, with the warm and hot pixels inserted from the relevant daydark. Figure 3 illustrates the more subtle WFC dark structures. The histogram in Figure 4 illustrate how the reference files get the ma jority of their pixels (normal dark current pixels, which exhibit only Poisson noise) from the `basedark', and get their warm and hot pixels from the `daydark'. Due to a small bias level variation (describ ed in Section 5), we also b egan equalizing any residual bias level variations seen in the sup erdarks, so that at least the sup erdarks will not propagate this problem to science data. 4. Data quality flagging and monitoring

In Table 1, we define all the flags used in the data quality arrays of ACS data (i.e. the [DQ] image extension) to identify good, bad, and corrected pixels (see Figures 5 and 6). Some of these flags emanate from the `p ermanent' bad pixel tables (identified in the BPIXTAB image header keyword), although as of Octob er 2004 we b egan using the bad pixel tables much less. Currently, most of these flags emanate from the generic conversion of the science data, or they propagate from the data quality arrays of the various reference files used to calibrate and combine data in the pip eline (CALACS and MultiDrizzle). We have also re-defined some flags for new uses ­ warm pixels (flag 64) and the CTE tails of hot pixel (flag 32) ­ although in the pip eline we set parameters (bits=96) to ignore


Pip eline Calibrations of ACS Data these new stand-alon is bits=0, flags (i.e., 5.

53

flags. So for now, they are provided only as optional data processing leverage for e processing. Note that in the stand-alone environment, the MultiDrizzle default so the user would need to manually set bits=96 to similarly ignore these new to include these pixels) during image combination.

Unexp ected anomalies

Most of the detector calibrations were b egun during pre-flight thermal vacuum testing, so their characteristics were reasonably well understood b efore launch. But inflight exp erience has revealed some unexp ected anomalies. We briefly mention some notable anomalies here, and the steps we have taken to ameliorate them. 1. Random bias level variations can occur b etween the `real' pixels in an image and their own overscans (Sirianni et al. 2003), where the bias level is measured, to b e subtracted from the rest of the image. There is nothing we can do to correct this when it occurs in science exp osures. But as of Octob er 2004, we have b een correcting this effect when seen in our sup erdarks (i.e. when it occurs in any of our input dark frames), so that at least our sup erdarks do not imp ose this effect on science data. 2. We have sometimes seen faint diffuse scattered light in dark frames, which would survive cosmic ray rejection and app ear in the corresp onding sup erdarks, and propagate to science data. After this problem was discovered, reference files which excluded the affected dark frames were delivered. We noticed that this problem seemed to occur following CCD annealings, when we had b een leaving the filter wheels op en (with CLEAR,CLEAR filters). We have not seen this problem after modifying our annealing program by rotating crossed filters into the optical path following each anneal. 3. In the first science data taken with ACS in April 2002, very faint negative imprints of bright ob jects were evident as mirror-image `ghosts' replicated in each amplifier quadrant (see Figure 1). This is an electronic effect, caused by amplifier crosstalk, which shouldn't have much impact on science data. Nonetheless, we recently changed the default WFC gain (from gain=1 to gain=2) which minimizes this effect. 6. Pip eline drizzling

Although the most significant recent change to the ACS pip eline was the addition of MultiDrizzle for combining and cleaning associated datasets, this topic is b eing covered more completely elsewhere (Koekemoer 2006, these Proceedings), so we only briefly mention here some details of its implementation in the pip eline environment. 1. Pre-defined drizzling parameter sets (determined mainly by the numb er of images b eing combined) are applied to associated datasets via a FITS table, which is identified in image headers by the MDRIZTAB keyword. While this table define reasonably good default parameters for combining images in the pip eline, the user can exp eriment considerably with these parameters in the stand-alone environment, where trial-anderror iterations will often lead to more optimal parameters for a sp ecific dataset. 2. As of mid-2005, p ointing patterns defined with POS TARGs are now recognized as associated datasets, along with patterns defined with `convenience' pattern forms (Mutchler & Cox 2001). Only associated datasets are automatically combined by MultiDrizzle in the pip eline. Large datasets involving multiple visits/ep ochs or large mosaics are not automatically or fully associated, and therefore must b e combined by the user in the stand-alone environment.


54

Mutchler, Sirianni, & Lucas 3. In early 2005, filter-dep endent distortion solutions (4th-order p olynomials) were introduced, along with residual distortion correction images. These are identified in image headers in the IDCTAB and DGEOFILE keywords, resp ectively.

7.

Max's wish list

The following items are some remaining issues, and thoughts on how they could b e addressed. They do not necessarily represent issues for which the ACS group at STScI has given high priority. Rather, they are included here for consideration and discussion. 1. We now have two eras of data quality flagging. It would b e desirable to somehow make the new flagging scheme retroactive to ACS launch (March 2002). Creating all new sup erbiases and sup erdarks for 2002-2004 and delivering them is impractical at this p oint. Perhaps monthly bad pixel tables could b e created for 2002-2004, which reflect the new flagging scheme (contain warm pixels, CTE tails, saturated pixels). With such bad pixel tables in place in the pip eline, subsequent data retrievals (via OTFR) would have flagging much more like we have had since Octob er 2004. 2. AS CTE worsens, it will b ecome increasingly desirable to reject the growing CTE tails of bright artifacts such as cosmic rays and hot pixels. More complete and unique flagging of hot pixel CTE tails in sup erdark reference files would help with the latter. As of Octob er 2004, the first trailing pixel is flagged uniquely (data quality flag 32), while any subsequent pixels in the CTE tail are flagged along with (i.e. not distinguished from) the warm pixels (flag 64). Note that MultiDrizzle allows the user to `grow' the rejection of artifacts, preferentially along the trailing detector y-axis, so that their CTE tails are also rejected. However, this feature is currently not utilized in the pip eline. 3. At present, only a small fraction of ACS data is associated, so most ACS datasets are unable to take advantage of MultiDrizzle b eing in the pip eline. For existing archival data, logical and complete association tables could b e generated offline, and ingested into the archive, such that subsequent data retrievals (via OTFR) would produce clean drizzle-combined products. 4. Add image registration (e.g. tweakshifts) to the pip eline, to allow the combination of data taken with different guide stars (e.g. data from different visits, ep ochs, and/or observing programs). Another way to approach this problem is to measure shifts offline, and in conjunction with the ab ove item, ingest association tabs with the shifts and rotations emb edded in them. 5. Improved handling of moving target (planetary) observations in the pip eline. Associated moving target datasets are improp erly combined by MultiDrizzle, which uses the World Coordinate System (WCS) to register images. Hubble's moving target tracking is generally accurate to within a fraction of a pixel, so a combination which simply ignores the WCS would produce good results. Also, since moving targets often exhibit additional complex motions (e.g. planets rotate b etween exp osures), singleimage drizzled output is always desirable, for associated or unassociated datasets (i.e. single exp osures which have only b een distortion corrected). 8. More information

More information ab out the basic ACS pip eline calibrations describ ed in this pap er can b e found on the web at: http://www.stsci.edu/hst/acs/analysis/reference files. ACS reference files in FITS format can b e directly downloaded from: ftp://ftp.stsci.edu/cdbs/cdbs7/jref/.


Pip eline Calibrations of ACS Data

55

Figure 1: Amplifier crosstalk evident in the first ACS science data. The signal from the Tadp ole Galaxy (in the amplifier A quadrant) generates faint negative `ghosts' in the other amplifier quadrants. The ACS Data Handb ook and Instrument Science Rep orts referenced in this document are available online at: http://www.stsci.edu/hst/acs/documents. Acknowledgments. We thank the following p eople for essential contributions to the overall success of ACS pip eline calibrations and drizzling: Warren Hack, Anton Koekemoer, Chris Hanley, Mike Swam, and Rossy Diaz-Miller. References Diaz-Miller, R. I., 2005, Technical Instrument Report CDBS-2005-01 (Baltimore: STScI), available through http://www.stsci.edu/hst/observatory/cdbs Lucas, R. A., Swam, M., Mutchler, M., & Sirianni, M., 2006, The 2005 HST Calibration Workshop. Eds. A. M. Koekemoer, P. Goudfrooij, & L. L. Dressel, this volume, 61 Koekemoer A., et al., 2006, The 2005 HST Calibration Workshop. Eds. A. M. Koekemoer, P. Goudfrooij, & L. L. Dressel, this volume, 423 Mutchler, M., & Cox C., 2001, Instrument Science Report ACS 2001-07 (Baltimore: STScI), available through http://www.stsci.edu/hst/acs Mutchler, M., Sirianni, M., van Orsow, D., & Riess, A., 2004, Instrument Science Report ACS 2004-07, (Baltimore: STScI) Pavlovsky, C., et al., 2005, ACS Data Handbook, Version 4.0 (Baltimore: STScI) Sirianni, M., et al., 2003, in Proc. 2002 HST Calibration Workshop, ed. S. Arribas, A. Koekemoer, & B. Whitmore (Baltimore: STScI), p. 82 Sirianni, M., Mutchler, M., Clampin, M., Ford, H., Illingworth, G., Hartig, G., van Orsow, D., & Wheeler, T., 2004, Optical and Infrared Detectors for Astronomy, Proc. SPIE, Vol. 5499 Sirianni, M., et al., 2006, The 2005 HST Calibration Workshop. Eds. A. M. Koekemoer, P. Goudfrooij, & L. L. Dressel, this volume, 45


56

Mutchler, Sirianni, & Lucas Table 1: ACS data quality flag definitions

Flag 0 1 2 4 8 16 32
a

64

b

128

256

512

1024 2048 4096

8192

Definition Good pixels Reed-Solomon decoding error; e.g. data lost during compression. Data replaced by fill value; e.g. neighb oring cosmic ray contaminated pixels. Bad detector pixel, or b eyond ap erture. In the HRC BPIXTAB, this identifies a small detector defect in the upp er right corner. Pixels masked by ap erture feature, e.g. the HRC occulting finger. Hot pixels with dark current greater than 0.08 e- /sec; flagged in sup erdark data quality [DQ] arrays. CTE tails of hot pixels; flagged in sup erdark [DQ] arrays. For now, we only flag the first pixel trailing each hot pixel, but this flagging may b ecome more sophisticated and complete as CTE worsens. Note that these CTE tails are flagged more completely as warm pixels (flag 64). Warm pixels with dark current b etween 0.02 and 0.08 e- /sec; flagged in sup erdark [DQ] arrays. Bias structure (mostly bad columns). Since 8 Oct 2004, bias structure has b een flagged in the bi-weekly sup erbias [DQ] arrays. Before 8 Oct 2004, bias structure was only flagged in an increasingly outdated bad pixel table (BPIXTAB), which was created in July 2002. Both full-well (useable at higher gain setting) and A-to-D (never useable) saturated pixels are flagged by ATODCORR, based on the CCDTAB. But A-to-D saturation is also flagged 2048, so it can b e distinguished from full-well saturation. Also, since 8 Oct 2004, full-well saturated pixels in sup erdarks are flagged in their corresp onding [DQ] arrays (note that they are also flagged as hot pixels with flag 16). Before 8 Oct 2004, saturated pixels in the sup erdarks were flagged only in a BPIXTAB created in July 2002. Bad pixel in reference file. Used in the flatfield [DQ] arrays to indicate a p ortion of the flat which is not defined or not calibrated with the same accuracy as the other regions, often around the detector edges. Used for F892N and WFC p olarizer observations, where the filter only subtends a p ortion of the chip. Used to identify dust mote replacement patches. Charge traps; flagged in the bad pixel table (BPIXTAB). A-to-D saturated pixels which are never useable, even at higher gain settings; flagged by ATODCORR, using thresholds in the CCD table (CCDTAB). Cosmic rays and detector artifacts rejected during MultiDrizzle image combination. These flags are not present in the drizzled output images (*drz.fits). Rather, they are propagated back to the [DQ] arrays of the input images (*flt.fits). Only data from p ointing patterns (using pattern forms or POS TARGs) and CR-SPLITs are automatically associated and combined by MultiDrizzle in the pip eline. Cosmic rays rejected during the combination of CR-SPLIT images in the pip eline (ACSREJ); flagged in the [DQ] arrays of the output (*crj.fits) images.

a

b

Re-defined as of 8 Octob er 2004. Was previously used to flag `blobs' of bleeding around saturated pixels (at the end of bad columns). These blobs are also flagged as hot or saturated pixels, so this was redundant. Re-defined as of 8 Octob er 2004. Was previously used to flag 'p ermanent' (non-annealing) hot pixels which had p ersisted through the first four CCD annealing cycles after launch in 2002. These flags existed only in an increasingly outdated bad pixel table (BPIXTAB), and they were redundant with flag 16.


Pip eline Calibrations of ACS Data

57

Figure 2: Bias structure in a typical WFC sup erbias. This image has b een binned and smoothed to enhance subtle features. The two WFC chips are mosaicked: chip 1 or [sci,2] is on top, and chip 2 or [sci,1] is on b ottom, with the interchip gap b etween them. Many bad columns are evident (their numb ers are increasing with time, due to radiation damage), and each amplifier quadrant exhibits its own distinct structure. The HRC bias structure is relatively featureless, and is not displayed in this pap er.


58

Mutchler, Sirianni, & Lucas

Figure 3: Dark structure in a typical WFC sup erdark. This image has b een binned and smoothed to enhance the more subtle dark features (unobscured by the many hot pixels), and the two WFC chips are mosaicked, with the interchip gap b etween them. The growth of hot pixels has b een well-documented (Sirianni, these Proceedings), but the more subtle structures seen here have not changed significantly since launch, and are mostly the result of CCD manufacturing processes. The HRC dark structure is relatively featureless, and is not displayed in this pap er.


Pip eline Calibrations of ACS Data

59

Figure 4: Overlayed HRC sup erdark histograms (numb er of pixels vs dark current in e- /sec): a daydark, a basedark, and the resulting hybrid sup erdark. Our sup erdarks get most of their pixels from a high signal-to-noise 2-week basedark. A sup erdark for a given observation date is a copy of the basedark, with the warm pixels (0.2 to 0.8 e- /sec, identified with data quality flag 64) and hot pixels (ab ove 0.8 e- /sec, identified with data quality flag 16) added from the corresp onding 4-frame daydark.


60

Mutchler, Sirianni, & Lucas

Figure 5: A closeup of some typical WFC dark structures: Scattered warm and hot pixels (some with prominent CTE tails), and columns of saturated pixels. The corresp onding data quality flagging is displayed in Figure 6.

Figure 6: Data quality [DQ] flagging corresp onding to Figure chosen here, to give every flag value (or combination of flag can b e seen in the on-line version). Hot pixels always have CTE, but more of the CTE tails are also flagged as warm pixels are also flagged uniquely.

5. A colorful lookup table was values) a unique color (which one trailing pixel flagged for pixels. Saturated columns of