Äîêóìåíò âçÿò èç êýøà ïîèñêîâîé ìàøèíû. Àäðåñ îðèãèíàëüíîãî äîêóìåíòà : http://www.eso.org/~qc/dfos/merge2Fits.html
Äàòà èçìåíåíèÿ: Fri May 11 13:09:47 2012
Äàòà èíäåêñèðîâàíèÿ: Tue Oct 2 01:47:34 2012
Êîäèðîâêà:
Ïîèñêîâûå ñëîâà: ï ï ð ð ð ð ð ð ð ð ð ð ð ð ð ð

merge2Fits

Common DFOS tools:
Documentation

dfos = Data Flow Operations System, the common tool set for DFO

make printable

new:

See also :

v1.3:
- support coversheet file _cs.html
v1.4:
- dropped: INFO messages about PROCESSING_LOG and SCORE_RESULT
- content changes: no PROCESSING_LOG for SCIENCE; no SCORE_RESULTS, no nightlog extracts
v1.4.2:
- dropped: consistency checks for ANC files (see bottom of page)

- the tool is distributed through the utilities package (utilPack)

databases	PANL2 for nightlog information; asm (thru qcm) ; dataproducts..calib for pre-existing ANC files
dfos tools	fitspack; qcm
output	twin ancillary fits files (LOGS and PLTS)
upload/download	none

merge2Fits

Description

merge2Fits is a helper tool which prepares all input for creating product ancillary fits files. These files are effectively a wrapper around multiple log files, or plot (PNG, GIF, JPG) files, designed to be stored in NGAS. That mechanism offers three advantages:

straightforward ingestion with existing tools: cdbIngest can handle any kind of product fits file
better ingestion performance: many small files ingest much slower than a single larger file
metadata description: with a valid fits header, all keys go into SAFIQ, without the need to additionally create and fill metadata tables

The reason for ingestion of product files currently is mainly backup. There might be future use of the calibration products in the calSelector associations.

With the ancillary fits files, QC is able to ingest three types of product fits files with one standard mechanism (ingestProducts/cdbIngest):

what	where	NGAS internal name starts with
calibration products	$DFO_CAL_DIR/$DATE	M.
science products	$DFO_SCI_DIR/$DATE	S. [terminated as of 2011-10-01]
ancillary files: logs	$DFO_LOG_DIR/$DATE	A.
QC plots	$DFO_PLT_DIR/$DATE	A.

How to use

Normally this tool is called by moveProducts. You can also call it on the command line.

Type merge2Fits for on-line help.

Type

merge2Fits -a <ab> -d <date>

to create a pair of ancillary fits files, one with the logs for the specified AB, the other with the QC plots.

Ancillary fits files: names

The ancillary fits files have the following name convention, automatically enforced by merge2Fits:

<MCAL_CODE>_<LOGS/PLTS>_<AB_root without instrument part>.fits, e.g.

GI_LOGS_2008-09-12T23:20:35.043_tpl.fits and
GI_PLTS_2008-09-12T23:20:35.043_tpl.fits

Note that 'LOGS' and 'PLTS' are reserved tags and cannot be used for any other product name!

Ancillary fits files: header structure

Ancillary FITS files follow the FITS Foreign File Encapsulation Convention (see http://fits.gsfc.nasa.gov/registry/foreign.html).

They have a primary array with the primary header, and multiple extensions, one per input file. You can see the complete header by typing

dfits -x 0 GI_LOGS_2008-09-12T23:20:35.043_tpl.fits

The primary header has, among other keys, HIERACH.ESO.PRO.ANC = T, which is required for cdbIngest to recognize it as ancillary file, plus INSTRUME and MJD-OBS, plus PRO.CATG="ANCILLARY_FILE", plus some other PRO keys.

Each extension has EXTNAME = <original file name> (e.g. GIRAFFE.2008-09-12T23:20:35.043_tpl.ab for the AB in the above example); PRO.CATG, set to something meaningful (e.g. ASSOCIATION_BLOCK). [Note that fitsort will deliver you only the last value found. If you want to see all PRO.CATGs in a given fits file, use 'dfits -x 0 ...fits | grep "PRO CATG"' .]

Ancillary fits files: content

The content of ancillary fits files is controlled by merge2Fits. The following cases apply:

1. LOGS

PRO.CATG	extension	content	when?	source created by?
ASSOCIATION_BLOCK	.ab	association block	always	createAB
ASSOCIATION_LOG	.alog	association log	always	createAB
PROCESSING_LOG	.rblog	pipeline/recipe log	if processed	pipeline
SCORE_RESULT	.html	scoring result	if processed and scored	scoreQC
PROCESSING_COMMENT	.cmt	extraction from NOTES files (about processed data)	if mode=CALIB	certifyProducts

2. PLTS: only if existing

PRO.CATG	extension	content	when?	source created by?
QC_PLOT (multiple possible)	.png	QC plot(s) in png format	if existing	QC procedures
QC_PLOT (multiple possible)	.gif	QC plot (s) in gif format	if existing	QC procedures
QC_PLOT (multiple possible)	.jpg	QC plot (s) in jpg format	if existing	QC procedures
COVERSHEET	_cs.html	coversheet to multiple QC plots	if existing	scoreQC

Ancillary files: ingestion and retrieval

Ingest an ancillary file with cdbIngest: cdbIngest <anc_file_name>. Operationally, use ingestProducts for this task (it automatically scans for calib, science, and ancillary products).

Retrieval of an archived ancillary file is done with ngasClient -c <anc_file_name>, just like retrieval of any other fits product file. Use fitsunpack afterwards to split it into its components.

Installation

The tool comes as part of the utilPack delivery.

Configuration file

none.

Operations

- In standard operations, this tool is wrapped into moveProducts.

[Turned off with v1.4.2:
- The tool automatically checks for previous versions of the just created LOGS and PLTS files in NGAS. If found, it warns you and then overwrites the older versions. Overwriting is implemented as combination of:

cdbDelete <previous version> -force
cdbIngest <new version>

This is enforced since otherwise there might be inconsistencies between the archive backup and the local disk content. ]

Common DFOS tools: Documentation

merge2Fits

Description

How to use

Configuration file

Common DFOS tools:
Documentation