make printable	new:	See also :
	v1.3: - support coversheet file _cs.html v1.4: - dropped: INFO messages about PROCESSING_LOG and SCORE_RESULT - content changes: no PROCESSING_LOG for SCIENCE; no SCORE_RESULTS, no nightlog extracts v1.4.2: - dropped: consistency checks for ANC files (see bottom of page)	- the tool is distributed through the utilities package (utilPack) databases PANL2 for nightlog information; asm (thru qcm) ; dataproducts..calib for pre-existing ANC files dfos tools fitspack; qcm output twin ancillary fits files (LOGS and PLTS) upload/download none

merge2Fits

Description

merge2Fits is a helper tool which prepares all input for creating product ancillary fits files. These files are effectively a wrapper around multiple log files, or plot (PNG, GIF, JPG) files, designed to be stored in NGAS. That mechanism offers three advantages:

• straightforward ingestion with existing tools: cdbIngest can handle any kind of product fits file
• better ingestion performance: many small files ingest much slower than a single larger file
• metadata description: with a valid fits header, all keys go into SAFIQ, without the need to additionally create and fill metadata tables

The reason for ingestion of product files currently is mainly backup. There might be future use of the calibration products in the calSelector associations.

With the ancillary fits files, QC is able to ingest three types of product fits files with one standard mechanism (ingestProducts/cdbIngest):

How to use

Normally this tool is called by moveProducts. You can also call it on the command line.

Type merge2Fits for on-line help.

Type

merge2Fits -a <ab> -d <date>

to create a pair of ancillary fits files, one with the logs for the specified AB, the other with the QC plots.

Ancillary fits files: names

The ancillary fits files have the following name convention, automatically enforced by merge2Fits:

<MCAL_CODE>_<LOGS/PLTS>_<AB_root without instrument part>.fits, e.g.

GI_LOGS_2008-09-12T23:20:35.043_tpl.fits and
GI_PLTS_2008-09-12T23:20:35.043_tpl.fits

Note that 'LOGS' and 'PLTS' are reserved tags and cannot be used for any other product name!

Ancillary fits files: header structure

Ancillary FITS files follow the FITS Foreign File Encapsulation Convention (see http://fits.gsfc.nasa.gov/registry/foreign.html).

They have a primary array with the primary header, and multiple extensions, one per input file. You can see the complete header by typing

dfits -x 0 GI_LOGS_2008-09-12T23:20:35.043_tpl.fits

The primary header has, among other keys, HIERACH.ESO.PRO.ANC = T, which is required for cdbIngest to recognize it as ancillary file, plus INSTRUME and MJD-OBS, plus PRO.CATG="ANCILLARY_FILE", plus some other PRO keys.

Each extension has EXTNAME = <original file name> (e.g. GIRAFFE.2008-09-12T23:20:35.043_tpl.ab for the AB in the above example); PRO.CATG, set to something meaningful (e.g. ASSOCIATION_BLOCK). [Note that fitsort will deliver you only the last value found. If you want to see all PRO.CATGs in a given fits file, use 'dfits -x 0 ...fits | grep "PRO CATG"' .]

Ancillary fits files: content

The content of ancillary fits files is controlled by merge2Fits. The following cases apply:

1. LOGS

2. PLTS: only if existing

Ancillary files: ingestion and retrieval

Ingest an ancillary file with cdbIngest: cdbIngest <anc_file_name>. Operationally, use ingestProducts for this task (it automatically scans for calib, science, and ancillary products).

Retrieval of an archived ancillary file is done with ngasClient -c <anc_file_name>, just like retrieval of any other fits product file. Use fitsunpack afterwards to split it into its components.

Installation

The tool comes as part of the utilPack delivery.

Configuration file

none.

Operations

- In standard operations, this tool is wrapped into moveProducts.

[Turned off with v1.4.2:
- The tool automatically checks for previous versions of the just created LOGS and PLTS files in NGAS. If found, it warns you and then overwrites the older versions. Overwriting is implemented as combination of:

• cdbDelete <previous version> -force
• cdbIngest <new version>

This is enforced since otherwise there might be inconsistencies between the archive backup and the local disk content. ]