Документ взят из кэша поисковой машины. Адрес оригинального документа : http://www.stsci.edu/science/share/time_history_swade_011502.txt Дата изменения: Wed Feb 6 17:25:10 2002 Дата индексирования: Sat Dec 22 04:35:03 2007 Кодировка:

==========================================
Processing Data Sets Based on Time History
==========================================

Description:

Use the time history of an instrument's observing program and calibration state as an additional factor in reprocessing data.

Scientific Case:

Some aspects of data processing are time-dependent. In one case the optimal data processing may depend on characteristics of the instrument that change over time. In addition, optimal processing can be related to the spacecraft's environment at the time of the exposure or related to the previous science exposures in a series.

The HST data processing system already carries out some simple time-dependent procedures. The 'best' reference files are selected based on the date on which an observation is taken. This may also include "super" or "delta" dark reference files.

Other, more sophisticated examples can be identified. NICMOS data are subject to persistence, in which detector pixels that collect a large number of counts in one exposure continue to 'glow' with a count rate that decays gradually with time, producing afterimages in subsequent images. This occurs both due to astronomical sources (e.g., bright stars that leave afterimages in subsequent exposures) and to radiation events, especially after SAA passages when the entire array is heavily bombarded with energetic particles leaving a spatially mottled pattern that gradually fades throughout the subsequent orbit.

It may be possible to track and even correct such persistent afterimages. This functionality would require implementation of contemporaneous calibration reference files. Such reference files would not be part of the CDBS system, and would need to be included as part of the observation dataset. The generation of contemporaneous reference files will become more important for future IR observations using NICMOS, the WFC3/IR channel, and NGST science instruments.

For NICMOS post-SAA persistence, in Cycle 11 STScI will begin taking automatically scheduled 'post-SAA darks' in every SAA-impacted orbit. There is hope that software can be developed which will scale and subtract these 'darks' from subsequent images to reduce or remove the persistence signal, although this has not yet been generally demonstrated on-orbit. If this is successful, it could be implemented in a pipeline.

For "astronomical" persistence, bright sources could be identified in one exposure, and those pixels could be flagged (at least) or corrected (at best, if a suitable persistence model could be defined) in subsequent science images. The SIRTF data pipeline will attempt to do this. [I have contacted one of the SIRTF pipeline developers to get more information on this. - DAS]

It is quite likely that there are other examples involving other instruments, where time-dependent processing could improve data quality for many users.

Unique STScI Capabilities:

In general, time- or history-dependent processing would require some means to search for, link together, and multiply process exposures with a given instrument that were taken over some time frame, regardless of whether they are part of the same HST proposal or not. Only STScI is in a position to do this, using direct interfaces between the scheduling system, the data archive, and the data processing system.

Drawbacks:

Algorithms for removing the persistent images would need to be applied uniformly in a pipeline setting for all sources.

Assumptions:

For persistence found in archive data, a technique can be found to remove persistence found in the data processing stage without the advantage of scheduled contemporaneous darks.

Required Decisions:

To what degree does persistence from bright objects affect subsequent exposures from the same detector? What level of software sophistication will provide a general solution that would benefit HST data users?

Min and Max Goals:

Min. Goal - Perform removal of NICMOS SAA persistence in a pipeline.

Med. Goal - Extend NICMOS technique to scheduled observations known to contain bright sources, i.e., sources brighter than some instrument dependent threshold.

Max. Goal - Identify 'bright' sources in an object catalog, develop an algorithm from removing their persistence in subsequent exposures, and apply this calibration in OTFR when the affected data are retrieved from the archive.

Implementation Plan:

Contemporaneous Reference files - Fully implement the concept of contemporaneous reference files. This type of calibration reference file will not use the CDBS system since they are only applicable to a particular observation or set of observations taken with in short time period. NICMOS post-SAA darks are the first example of contemporaneous reference files for HST and will be operational in cycle 11.

NICMOS SAA persistence removal - In a pipeline setting, utilize post-SAA darks to generate a cosmic ray map that can be applied to an image in order to reduce noise from cosmic ray persistence induced by SAA charged particle hits on the detector. This depends on the pipeline removal of the DC pedestal common in NICMOS data.

Scheduling system - Observations scheduled on bright sources could be flagged in the scheduling system and automatic darks generated just after the bright object observation. These contemporaneous darks could then be applied to subsequent exposures using the same detector for a period of time that is a function of the magnitude of the bright source.

Archive - Extract photometry information from exposures into an object catalog. Determine for each HST detector, what source threshold magnitude will affect subsequent exposures. Determine a technique that can be used to 'calibrate' the affected data using only exposures contained in the archive.


Required Resources:

- For Min Goals
XXX FTE from the NICMOS team finds technique for implementing persistence calibration in the pipeline.

XXX FTE from SSG to implement application of CR map in CALNICA

- For Med Goals
XXX FTE from instrument teams to implement similar functional in other instrument calibration pipelines

XXX FTE to implement bright object identification in scheduling system and insert auto-darks into the observing schedule

0.5 FTE for DST to implement association of auto-darks with affected observations

- For Max Goals
XXX FTE for someone to determine an algorithm for finding bright object persistence in archived exposures using an object catalog as input

XXX FTE for someone to develop an algorithm for removing persistence from archived exposures

0.5 FTE for DST to implement association of persistence affected images with calibration files generated in above step

Time Scales:

- For Min Goals
cycle 11?

- For Med Goals
2 years?

- For Max Goals
After generation of object catalogs in the archive

==================================================================