Документ взят из кэша поисковой машины. Адрес
оригинального документа
: http://www.eso.org/~qc/dfos/merge2Fits.html
Дата изменения: Fri May 11 13:09:47 2012 Дата индексирования: Tue Oct 2 01:47:34 2012 Кодировка: Поисковые слова: п п р р р р п п п п п п п п п п |
Common DFOS tools:
|
dfos = Data Flow Operations System, the common tool set for DFO |
![]() |
new: | See also : | ||||||||
v1.3: |
- the tool is distributed through the
utilities package (utilPack)
|
merge2Fits is a helper tool which prepares all input for creating product ancillary fits files. These files are effectively a wrapper around multiple log files, or plot (PNG, GIF, JPG) files, designed to be stored in NGAS. That mechanism offers three advantages:
The reason for ingestion of product files currently is mainly backup. There might be future use of the calibration products in the calSelector associations.
With the ancillary fits files, QC is able to ingest three types of product fits files with one standard mechanism (ingestProducts/cdbIngest):
what | where | NGAS internal name starts with |
calibration products | $DFO_CAL_DIR/$DATE | M. |
science products | $DFO_SCI_DIR/$DATE | S. [terminated as of 2011-10-01] |
ancillary files: logs | $DFO_LOG_DIR/$DATE | A. |
QC plots | $DFO_PLT_DIR/$DATE |
Normally this tool is called by moveProducts. You can also call it on the command line.
Type merge2Fits for on-line help.
Type
merge2Fits -a <ab> -d <date>
to create a pair of ancillary fits files, one with the logs for the specified AB, the other with the QC plots.
Ancillary fits files: names
The ancillary fits files have the following name convention, automatically enforced by merge2Fits:
<MCAL_CODE>_<LOGS/PLTS>_<AB_root without instrument part>.fits, e.g.
GI_LOGS_2008-09-12T23:20:35.043_tpl.fits and
GI_PLTS_2008-09-12T23:20:35.043_tpl.fits
Note that 'LOGS' and 'PLTS' are reserved tags and cannot be used for any other product name!
Ancillary fits files: header structure
Ancillary FITS files follow the FITS Foreign File Encapsulation Convention (see http://fits.gsfc.nasa.gov/registry/foreign.html).
They have a primary array with the primary header, and multiple extensions, one per input file. You can see the complete header by typing
dfits -x 0 GI_LOGS_2008-09-12T23:20:35.043_tpl.fits
The primary header has, among other keys, HIERACH.ESO.PRO.ANC = T, which is required for cdbIngest to recognize it as ancillary file, plus INSTRUME and MJD-OBS, plus PRO.CATG="ANCILLARY_FILE", plus some other PRO keys.
Each extension has EXTNAME = <original file name> (e.g. GIRAFFE.2008-09-12T23:20:35.043_tpl.ab for the AB in the above example); PRO.CATG, set to something meaningful (e.g. ASSOCIATION_BLOCK). [Note that fitsort will deliver you only the last value found. If you want to see all PRO.CATGs in a given fits file, use 'dfits -x 0 ...fits | grep "PRO CATG"' .]
Ancillary fits files: content
The content of ancillary fits files is controlled by merge2Fits. The following cases apply:
1. LOGS
PRO.CATG | extension | content | when? | source created by? |
ASSOCIATION_BLOCK | .ab | association block | always | createAB |
ASSOCIATION_LOG | .alog | association log | always | createAB |
PROCESSING_LOG | .rblog | pipeline/recipe log | if processed | pipeline |
SCORE_RESULT | .html | scoring result | if processed and scored | scoreQC |
PROCESSING_COMMENT | .cmt | extraction from NOTES files (about processed data) | if mode=CALIB | certifyProducts |
PRO.CATG | extension | content | when? | source created by? |
QC_PLOT (multiple possible) | .png | QC plot(s) in png format | if existing | QC procedures |
QC_PLOT (multiple possible) | .gif | QC plot (s) in gif format | if existing | QC procedures |
QC_PLOT (multiple possible) | .jpg | QC plot (s) in jpg format | if existing | QC procedures |
COVERSHEET | _cs.html | coversheet to multiple QC plots | if existing | scoreQC |
Ancillary files: ingestion and retrieval
Ingest an ancillary file with cdbIngest: cdbIngest <anc_file_name>. Operationally, use ingestProducts for this task (it automatically scans for calib, science, and ancillary products).
Retrieval of an archived ancillary file is done with ngasClient -c <anc_file_name>, just like retrieval of any other fits product file. Use fitsunpack afterwards to split it into its components.
Installation
The tool comes as part of the utilPack delivery.
none.
Operations
- In standard operations, this tool is wrapped into moveProducts.
[Turned off with v1.4.2:
- The tool automatically checks for previous versions
of the just created LOGS and PLTS files in NGAS. If found, it warns you and then overwrites
the older versions. Overwriting is implemented as combination of:
This is enforced since otherwise there might be inconsistencies between the archive backup and the local disk content. ]