45.1 Overview of Pipeline Processing
The data were received from the Space Telescope Data Capture Facility by the Post Observation Data Processing System (PODPS). There they were passed to Routine Science Data Processing (RSDP)-referred to as the pipeline-to be processed and calibrated. All of the steps performed by the pipeline were recorded in the trailer file for the dataset. Figure 45.1 shows an example of a trailer file and identifies comments made during the following pipeline steps:
- The data were partitioned (separated into individual files, e.g., the engineering and science data are separated).
- The data were edited to insert fill values in place of missing data.
- The data were evaluated to determine discrepancies between the subset of the planned and executed observational parameters.
- A list of calibration reference files to be used in the calibration of the data was created based on the executed observational parameters. This step did not generate comments in the example trailer file.
- The data were converted to a generic format and the header keywords populated.
- The data were calibrated using a standard WF/PC-1-specific calibration algorithm (STSDAS task calwfp).
Figure 45.1: Sample Trailer File
DPPBAN-***************************************************** TRAILER FILE *****************************************************
DPPSTA--------------------------------------------- DATA PARTITIONING STARTED ---------------------------------------------
DPPSTR-DATA PARTITIONING STARTED: 23-AUG-93 14:12:38 .
DPPOBS-Observation Identification: Program Number 1DR , Observation Set Number 08, Observation Number 06
DPPCAT-Msg Artifacts Catalog: DCF Msg 12 contained data for Info Set File with Root Name of W1DR0806T
DPPIST-Missing pkts processing is complete, processed 3200 science pkts, detected 0 as missing
DPVWM1-Weighted Quality Measure of 0 was calc for errors reported in Quality & Acctg Capsule entries for Info Set
DPPNPR- 3201 packets expected, 3201 packets received ( 1 SHPs, 0 UDLs, 0 Bad pkts) for Info Set W1DR0806T
DPPEND-DATA PARTITIONING ENDED: 23-AUG-93 14:23:31 .
DPPFIN--------------------------------------------- DATA PARTITIONING COMPLETED -------------------------------------------
CPE0ST-------------------------------------------- DATA EDITING STARTED -----------------------------------------------------
CPESTR-DATA EDITING STARTED 23-AUG-93 14:23:47 FOR W1DR0806T.
DPE4PC- 0 fill source packets were inserted by PODPS.
DPE4DC-In W1DR0806T, 0 source packets were found that contain fill data inserted by the ST DCF.
DPERSC-CP34SE: ...and 0 packets were found to have segments without Reed Solomon corrections
CPEEND-DATA EDITING ENDED 23-AUG-93 14:23:47 FOR W1DR0806T.
CPE0EN--------------------------------------------- DATA EDITING COMPLETED -----------------------------------------------------
CPG0VS--------------------------------------------- DATA EVALUATION STARTED -----------------------------------------------
DPVSTR-DATA EVALUATION STARTED 23-AUG-93 14:23:55 FOR W1DR0806T.
DPVEND-DATA EVALUATION ENDED 23-AUG-93 14:24:22 FOR W1DR0806T.
DPVWM2-Weighted Quality Measure of 0 was calc for discrepancies between predicted & actual Flags & Indicators for Info Set
DPVOKK-Info Set quality was OK as Weighted Quality Measure of 0 did not exceed Quality Threshold of 9
CPG0VE--------------------------------------------- DATA EVALUATION COMPLETED ---------------------------------------------
CGAI00-
CGAI01---------------------------------WIDE FIELD AND PLANETARY CAMERA GENERIC CONVERSION BEGINNING-----------------------------
CGAI00-
CGAT01-Routine CGWFPC, version 32.0.D , will perform generic conversion
CGAT02-GENERIC CONVERSION BEGINNING: processing information set W1DR0806T
CGAT03-CGWFPC START: 23-AUG-93, 14:24:22
CGAW06-No error values detected in flags and indicators
CGAI06-SHP generic conversion has completed successfully
CGAC02-Generated new WFPC reference file catalog : CCC:[SIS.WFPC]W1DR0806T.CTX
CGAI08-SCI generic conversion has completed successfully
CGAI22-DQM generic conversion has completed successfully
CGAT04-GENERIC CONVERSION TERMINATING: successful completion
CGAT09-CGWFPC END: 23-AUG-93, 14:29:08
CPG0EN--------------------------------------------- GENERIC CONVERSION COMPLETED -----------------------------------------------
######
###### This observation was reprocessed through the RSDP pipeline
###### because they were unreadable on the DMF disks. This allows
###### the data to be ingested into DADS for later/future retrievals
######
###### 26-APR-1995 Mk
######
CPE0ST-------------------------------------------- DATA EDITING STARTED -----------------------------------------------------
CPESTR-DATA EDITING STARTED 26-APR-95 18:05:56 FOR W1DR0806T.
DPE4PC- 0 fill source packets were inserted by PODPS.
DPE4DC-In W1DR0806T, 0 source packets were found that contain fill data inserted by the ST DCF.
DPERSC-CP34SE: ...and 0 packets were found to have segments without Reed Solomon corrections
CPEEND-DATA EDITING ENDED 26-APR-95 18:05:56 FOR W1DR0806T.
CPE0EN--------------------------------------------- DATA EDITING COMPLETED -----------------------------------------------------
CPG0VS--------------------------------------------- DATA EVALUATION STARTED -----------------------------------------------
DPVSTR-DATA EVALUATION STARTED 26-APR-95 18:16:57 FOR W1DR0806T.
DPVEND-DATA EVALUATION ENDED 26-APR-95 18:17:10 FOR W1DR0806T.
DPVWM2-Weighted Quality Measure of 0 was calc for discrepancies between predicted & actual Flags & Indicators for Info Set
DPVOKK-Info Set quality was OK as Weighted Quality Measure of 0 did not exceed Quality Threshold of 9
CPG0VE--------------------------------------------- DATA EVALUATION COMPLETED ---------------------------------------------
CGAI00-
CGAI01---------------------------------WIDE FIELD AND PLANETARY CAMERA GENERIC CONVERSION BEGINNING-----------------------------
CGAI00-
CGAT01-Routine CGWFPC, version SOGS35.3.A , will perform generic conversion
CGAT02-GENERIC CONVERSION BEGINNING: processing information set W1DR0806T
CGAT03-CGWFPC START: 26-APR-95, 18:17:10
CGAW06-No error values detected in flags and indicators
CGAI06-SHP generic conversion has completed successfully
CGAC02-Generated new WFPC reference file catalog : CCC:[SIS.WFPC]W1DR0806T.CTX
CGAI08-SCI generic conversion has completed successfully
CGAI22-DQM generic conversion has completed successfully
CGAT04-GENERIC CONVERSION TERMINATING: successful completion
CGAT09-CGWFPC END: 26-APR-95, 18:20:33
CPG0EN--------------------------------------------- GENERIC CONVERSION COMPLETED -----------------------------------------------
CCAI00-
CCAI01------------------------------------WIDE FIELD AND PLANETARY CAMERA CALIBRATION BEGINNING---------------------------------
CGAI00-
CCAI07-CALIBRATION BEGINNING: processing information set W1DR0806T
CCAI11-CCWFPC START: 26-APR-95, 18:45:08
CCAIRI- --- Message From: CALWFP While Processing:
CCAIRI- --- Message: WF/PC Calibration Starting: CALWFP Version 1.2.1
CCAIRI- --- Message From: CALWFP While Processing: W1DR0806T
CCAIRI- --- Message: Starting CALWFP: Input = wcal:w1dr0806t Output = wcal:w1dr0806t
CCAIRI- --- Message From: CALWFP While Processing: W1DR0806T
CCAIRI- --- Message: Start Process Element 1
CCAIRI- --- Message From: CALWFP While Processing: W1DR0806T
CCAIRI- --- Message: Start Process Element 2
CCAIRI- --- Message From: CALWFP While Processing: W1DR0806T
CCAIRI- --- Message: Start Process Element 3
CCAIRI- --- Message From: CALWFP While Processing: W1DR0806T
CCAIRI- --- Message: Start Process Element 4CCAIRI- --- Message From: CALWFP While Processing: W1DR0806T
CCAIRI- --- Message: WF/PC Calibration Ending for observation
CCAI10-CALIBRATION END: 26-APR-95, 18:54:35
The calibration software used by the pipeline is exactly the same as that provided within STSDAS (calwfp). The calibration files and tables used were taken from the Calibration Data Base (CDBS) at STScI and were the most up-to-date calibration files available at the time the data was processed. All CDBS files are available to users through the HST Data Archive.
Copyright © 1997, Association of Universities for Research in Astronomy. All rights reserved.