OM data reduction with SAS: processing image data, step by step

The data package received by the investigator contains OM data which normally do not necessitate further processing for the purpose of calibration. However, a user may want to apply the most recent calibrations, or to change some default parameters to e.g. improve the source detection on his/her data. It may not be necessary to run the complete chains, but just some tasks. All this can be done interactively.

The SAS_ODF environment variable shall be set to the summary file produced by odfingest, or to a directory containing the data, and access to calibration files shall be set through cifbuild.

In addition to the fits files containing the OM science image data, the ODF needs to contain the OM house-keeping files:

and some spacecraft files of the following form:

Invoking omichain or omfchain will automatically start the processing of all OM data in the working directory. The duration of the process will depend on the number of exposures and windows and at the end we shall obtain the final processed files.

In the standard  automatic SSC pipeline processing, the temporary image files are re-used and thus overwritten. In the task by task processing, they can be  distinguished so that intermediate stage output can be looked at if desired. In the next examples we have used the same naming system than omichain. This allows us to maintain some intermediate files which can be helpful in understanding the whole process.

Currently the chains can accept parameters to limit the processing to a given filter or a given exposure. Some of the default parameters used by individual tasks can also be tuned. (See online SAS help pages for detailed instructions for each task parameters)
 
 

Example of image data processing

To process task by task a single exposure in image mode as
                            0644_0125321001_OMS00600IMI.FIT

the following files are necessary (they are normally part of the ODF):

and the process will be run in the following way. Note that output products of the different tasks have been given the names that omichain  would have applied.

It is assumed throughout this example that the ODF data are in: /path_to_your_data/
and we are running the tasks and writing their results in a subdirectory: /path_to_your_data/proc/
 
 

step 1  The OM tracking data are processed for later use, if applicable.

omprep  set=/path_to_your_data/0644_0125321001_OMS00600THX.FIT nphset=/path_to_your_data/0644_0125321001_OMX00000NPH.FIT pehset=/path_to_your_data/0644_0125321001_OMX00000PEH.FIT wdxset=/path_to_your_data/0644_0125321001_OMS00600WDX.FIT outset=/path_to_your_data/proc/I0125321001OMS006TRHIS0000.FIT modeset=3

In case there is no THX file, then set=DUMMYTHX.FIT. omprep will generate a dummy file needed for the rest of the chain, with zero drift in it.

step 2  A plot of the tracking data, useful to assess the S/C stability during the exposure, is generated. Note that tracking data may not be good enough, in which case no plot is produced.

omdrifthist set=/path_to_your_data/proc/I0125321001OMS006TRHIS0000.FITplotfile=/path_to_your_data/proc/P0125321001OMS006TSHPLT000.ps

step 3  The count rates of tracking stars are written into a file.

omthconv thxset=/path_to_your_data/proc/I0125321001OMS006TRHIS0000.FIT nphset=/path_to_your_data/0644_0125321001_OMX00000NPH.FIT outset=/path_to_your_data/proc/P0125321001OMS006TSTRTS000.FIT

step 4  Processing of the image data starts here. Some parameters are extracted for insertion into the FITS header.

omprep set=/path_to_your_data/0644_0125321001_OMS00600IMI.FIT nphset=/path_to_your_data/0644_0125321001_OMX00000NPH.FIT pehset=/path_to_your_data/0644_0125321001_OMX00000PEH.FIT wdxset=/path_to_your_data/0644_0125321001_OMS00600WDX.FIT outset=/path_to_your_data/proc/I0125321001OMS006IMAGE10000.FIT

step 5   Bad pixels in the image are flagged.

omcosflag  set=/path_to_your_data/proc/I0125321001OMS006IMAGE10000.FIT thxset=/path_to_your_data/proc/I0125321001OMS006TRHIS0000.FIT
 

Although no real flat field correction exists for OM, nor is it necessary, the processing requires such a file which can be generated using the task omflatgen as follows. (This task can also be run at the beginning of the processing)
 

step 6.0   A flat field is generated.

omflatgen outset=/path_to_your_data/proc/P06440125321001OM000FLAFLD0000.FIT

The output flatfield (primary extension) will be set to unity.
 

step 6  The flat field is applied to the data.

omflatfield  set=/path_to_your_data/proc/I0125321001OMS006IMAGE10000.FIT thxset=/path_to_your_data/proc/I0125321001OMS006TRHIS0000.FIT inorbitflatset=/path_to_your_data/proc/P06440125321001OM000FLAFLD0000.FIT tsflatset=/path_to_your_data/proc/I0125321001OMS006FLATF0000.FIT outset=/path_to_your_data/proc/I0125321001OMS006IMAGE20000.FIT

step 7   A very important correction is applied here: modulo-8 fixed pattern is removed.

ommodmap set=/path_to_your_data/proc/I0125321001OMS006IMAGE20000.FIT mod8product=yes mod8set=/path_to_your_data/proc/I0125321001OMS006MOD80000.FIT outset=/path_to_your_data/proc/P0125321001OMS006IMAGE_0000.FIT nsig=3 nbox=16

step 8   A searching algorithm is used to find the sources present in the image. Aperture photometry is performed on all sources to obtain corrected count rates.

omdetect nsigma=2 minsignificance=3 detectextended=1 set=/path_to_your_data/proc/P0125321001OMS006IMAGE_0000.FIT regionfile=/path_to_your_data/proc/I0125321001OMS006REGION0000.ASC levelimage=/path_to_your_data/proc/I0125321001OMS006LEVELIMAGE0000.FIT outset=/path_to_your_data/proc/P0125321001OMS006SWSRLI0000.FIT

Note that all functions from this step on can be accomplished with the interactive omsource task.

step 9  The count rates of all sources found with omdetect are converted to magnitudes in the OM instrumental system.

ommag set=/path_to_your_data/proc/P0125321001OMS006SWSRLI0000.FIT
 

step 10  Astrometry corrections are performed here to derive the coordinates of all detected sources. A sky image (North on top) is produced. This is just cosmetical, no photometric measurements should be made on this image  (SIMAGE file)

omatt set=/path_to_your_data/proc/P0125321001OMS006IMAGE_0000.FIT sourcelistset=/path_to_your_data/proc/P0125321001OMS006SWSRLI0000.FIT ppsoswset=/path_to_your_data/proc/P0125321001OMS006SIMAGE0000.FIT usecat=F rotateimage=T
 

Note that currently the usage of a catalogue for source cross correlation is disabled. The  detected sources can be overlaid on the OSW as follows:

step 10a  The success of the detection can be checked.

implot set=/path_to_your_data/proc/P0125321001OMS006SIMAGE0000.FIT withsrclisttab=yes  srclisttab=/path_to_your_data/proc/P0125321001OMS006SWSRLI0000.FIT device= '/XW'

A similar, perhaps easier result can be obtained by displaying the image with ds9 and loading the corresponding region produced by omdetect.

If multiple images are processed, the source lists can be combined as follows.
 

step 11  If we have several exposures, we can combine them. Color corrections will be applied to derive standard UBV magnitudes.

omsrclistcomb sourcelistsets="/path_to_your_data/proc/P0125321001OMS006SWSRLI0000.FIT /path_to_your_data/proc/any_other_SWSRLI_file"nsigma=3 outset=/path_to_your_data/proc/P0125321001OMCOMBOBSMLI0000.FIT

Note the list, surrounded by quotes, of SWSRLI files.

Finally, the sky images of the multiple exposures corresponding to the default image configuration, obtained with the same filter can be combined in a single sky image of the field of view. Note that this mosaiced image cannot be used for photometric purpose.
 

last step

ommosaic imagesets=''/path_to_your_data/proc/list of sky images'' mosaicedset=/path_to_your_data/proc/myfield_in_myfilter.fit