Phase 3 Release Validator
Local validation of the Phase 3 release is an essential preparatory step before uploading the data to ESO. In this process the internal consistency and the metadata content of the data products will be checked against the ESO/EDP data format standard. For this purpose ESO provides a dedicated Java toolтАУthe Phase 3 Release Validator.
Please make sure to obtain the latest versions of the Phase 3 validator (v2.3.0) and fitsverify (v4.16) for data release preparation.
Software Download
The current Phase 3 validator, version number 2.3.0, implements the ESO Science Data Product Standard, Issue 5, Date: 11/01/2013, including the amendments listed at http://www.eso.org/sci/observing/phase3/faq.html
The application version can be checked using the following command:
$ java -jar ./validator.jar -v
Starting up validation tool.
application.name = Phase 3 Release Validation Tool
application.command = java -jar validator.jar -r<reldir> -m{create|update} [other-options]
application.version = 2.0.0
Change History
Main changes in Phase 3 validator v2.3.0 with respect to the previous version v2.1.2:
- v2.3.0 is primarily a bug fix release and the verification of provenance information has been improved.
Previous Versions
Main changes in Phase 3 validator v2.1.2 with respect to the previous version v2.0.0:
- Phase 3 validator v2.1.2 significantly enhances the performance in case of large previous thereby reducing the feedback time.
Main changes in Phase 3 validator v2.0.0 with respect to the previous version v1.1.1:
- Phase 3 validator v2.0.0 supports recording the processing provenance of data products using one dedicated FITS binary table extension (cf. ESO SDP Standard, Issue 5, Sect. 2.4.2, for details);
- Performance improvements;
- Bugs fixed;
- Requires Java 7 (see below: System Requirements)
Main changes in Phase 3 validator v1.1.1 with respect to the previous version:
- Phase 3 validator v1.1.1 supports the submission of science catalogue data;
- Support the submission of the Phase 3 release description as PDF document;
- Improved checks on FITS keywords and their content to identify problems as early as possible in the validation process;
- Support for PRODCATG tokens in uppercase;
- User documentation updated (see below)
System Requirements
The validator has been developed and tested under Scientific Linux 5.5 (i386, 32-bit, based on Red Hat Enterprise 5). It requires Java version jre1.7.0_21 or higher. The validator is expected to run under Linux; other platforms are not supported at the moment.
FITSVERIFY
The FITS file format-verification tool fitsverify, which is maintained and distributed by NASA/HEASARC, must be installed on the system running the Phase 3 validator. Phase 3 data must pass fitsverify without any warning to be accepted. fitsverify can be obtained from http://heasarc.gsfc.nasa.gov/docs/software/ftools/fitsverify/
The Phase 3 infrastructure employs fitsverify version 4.16, which is currently the lastest version (October 2010). It is not recommended to run the Phase 3 validator in combination with any other tool or software version, because it was not tested and one cannot assure that everything would work correctly.
This binary is compiled for Linux (i386, 32-bit) and would not work on other operating systems like Mac OS X. You may download the source code from
http://heasarc.gsfc.nasa.gov/docs/software/ftools/fitsverify/
and compile it for your operating system. We cannot assure though that everything would work, as the tool was not tested for different operating systems.
Network connectivity
The validator must be able to establish a network connection to the ESO Phase 3 system at runtime.
Calling Sequence
Run the validator from the command line:
java -jar validator.jar -r <dir> -m create|update
Output
The validator displays a summary of the release content and produces one of the following status messages, either
OK - RELEASE CAN BE UPLOADED TO ESO
or
ERROR - PLEASE FIX THE ERRORS BEFORE UPLOADING THE RELEASE
The validator produces three files in the current working directory:
- validator.error - list of short error messages describing the errors found in validating the release. This file is empty if no errors are found.
- validator.toc - contains the status (release ok/error), a summary of the release in terms of number and type of validate files, number and type of errors found, number of parsed datasets and provenance information, and a dump of the parsed metadata for datasets, provenance, and product categories.
- validator.log - detailed list of messages generated during the validation.