Документ взят из кэша поисковой машины. Адрес оригинального документа : http://www.eso.org/~qc/dfos/productExplorer.html
Дата изменения: Mon Apr 20 15:14:29 2015
Дата индексирования: Sun Apr 10 01:08:29 2016
Кодировка:

Поисковые слова: md-11
productExplorer

Common DFOS tools:
Documentation

dfos = Data Flow Operations System, the common tool set for DFO
*make printable new: Note:
 

v1.4:
- display virtual products (for calSelector)
v1.5:
- new column 'pipefile'; mark multiple entries

[ used databases ] databases qc_metadata..qc_products; qc1_daily_comments; ngas..ngas_files
[ used dfos tools ] dfos tools ngasClient cpDelete histoMonitor
[ output used by ] output $DFO_MON_DIR/productExplorer.html and $DFO_MON_DIR/productExplorer1.html
[ output used by ] upload/download none

productExplorer

Description

This is a helper tool for exploring and managing the content of the qc_metada..qc_products database. This database stores all QC-ingested product fits files (calib, science: until 2011-10, ancillary: until 2014-05). It offers a user-friendly way to query ("explore") its content, and fine-tune the query. There are options to download the results or to delete entries.

It launches a direct isql query to qc_metada..qc_products.

It displays its result in a html table which can be sorted and filtered in the standard way, plus an executable download file, plus a deletion file for ingested products (to prepare for ingestion of an improved version).

How to use

Type productExplorer -h for on-line help, productExplorer -v for the version number,

productExplorer -d 2017-10-15

to explore calib for that date and the default $DFO_INSTRUMENT,

productExplorer -d 2017-10-15 -D 2007-10-30 -i CRIRES

for the specified time range and instrument,

productExplorer -d 2017-01-01 -p LINE_TABLE

for the specified pro_catg value only, and

productExplorer -d 2017-01-01 -p LINE_TABLE -s grism_256

to filter from the result set those files with setup value "grism_256". This filter option is done after the isql download, at the unix level, for performance reasons.

Result HTML file. The result file comes as an HTML file, similar in format to the (raw) data reports created by createReport. It uses color-coding to distinguish calib (green) from science (blue) files. It also supports ancillary (log) fits files (displayed yellow). In standard (date) mode, it displays daily comments just like the raw data reports.

Product report for 2008-03-13 (products ingested into NGAS)

  CAL SCI ANC ALL
number: 77
size (MB): 115.1 3.2 8.8 127.1
Date comment: Last night of old CCD Bruce; IFLATs taken; STD efficiency
[read query] | [read download file] | [read deletion file]
 
type* origfile setup** pipefile mjd_obs pro_catg ins_mode dp_id
VIRTUAL V.GIRAF.2008-03-14T02:29:04.977_005.fits   NULL 54539.163178 SCIENCE_RBN_SPECTRA MED V.GIRAF.2008-03-14T02:29:04.977_005.fits
SCIENCE GI_SRBS_287119_2008-03-14T02:29:04.977_IFU2_L543.1nm_o3.fits IFU2_L543.1nm_o3 r.GIRAF.2008-03-14T02:29:04.977_tpl_0001.fits 54539.103530 SCIENCE_RBNSPECTRA IFU S.GIRAFFE.2008-04-01T09:28:10.770
CALIB GI_PIME_080313A_Argus_L427.2nm_o4_LTT3218.fits Argus_L427.2nm_o4_LTT3218 r.GIRAF.2008-03-14T10:39:42.5009_tpl_0006.fits 54539.163178 STD_RCERRORS ARG M.GIRAFFE.2008-04-01T09:00:16.159

Filter and sort. The HTML table can be filtered (any string from the content can be used). You can filter e.g. to display only MBIA files, or files containing the string U_BESS etc. Sorting can be done by any column.

Virtual products. With v1.4, virtual products (defining the calSelector datasets) are displayed. You can toggle between displaying and hiding them, nearby the 'Filter' option.

Multiple entries. In particular static calibration files could have been entered multiple times with different origfile names. These files, as being identified by the combination of pipefile, pro_catg and mjd_obs, are marked in red font color.

Download. The files can be downloaded, by using the executable file $TMP_DIR/mcalibDownload, either as it stands or after further editing. It uses ngasClient for the download. All downloads go into $DFO_CAL_DIR/DOWNLOAD and can then be further distributed by the user.

Deletion. The tool supports deletion of products. It provides the results of the query in a text file, with the proper syntax prepared for deletion of these files. However, without further editing, deletion will be performed only in simulation mode. Only if the user edits the deletion file to the full syntax, e.g.

dpDelete GI_MBIA_...fits -force

these files will get deleted. Deletion means physical removal and deletion in all database tables. Deletion is not hiding: after deletion, you cannot reconstruct the file content from information stored anywhere in NGAS. You can, however, ingest a new version of this file from your local file system, and this is the main purpose of the tool.

The deletion mode should be used with care.

Installation

Use dfosInstall.

Configuration file

none.

Operational aspects