Документ взят из кэша поисковой машины. Адрес оригинального документа : http://www.adass.org/adass/proceedings/adass99/P3-06/
Дата изменения: Wed Oct 11 04:13:30 2000
Дата индексирования: Tue Oct 2 05:11:33 2012
Кодировка:
Поисковые слова: earth's atmosphere

Next: Data Quality Control at the Very Large Telescope
Up: Data Pipelines and Quality Control
Previous: The Chandra Monitoring System
Table of Contents - Subject Index - Author Index - PS reprint - Estes, A. N., DuPrie, K., Harbo, P., Pak, K., & Zografou, P. 2000, in ASP Conf. Ser., Vol. 216, Astronomical Data Analysis Software and Systems IX, eds. N. Manset, C. Veillet, D. Crabtree (San Francisco: ASP), 457

The Chandra Xray Center Data Archive Interfaces

A. N. Estes, K. DuPrie, P. Harbo, K. Pak, P. Zografou
Harvard-Smithsonian Center for Astrophysics Cambridge, MA 02138

Abstract:

The Chandra data archive at the Chandra Xray Center supports automatic pipeline processing via exchange of data and other information. The archive notifies automatic processing about events which change its contents and should trigger processing of an observation. It supports retrieval and ingestion of pipeline input and output data products and provides ancillary data in the form of dynamically generated parameter files. The data archive provides interfaces to access these data products and their associated metadata locally and by the Web.

This paper presents the notification mechanism which uses a specially developed server and a number of database triggers. The client interface for data products exchange is also presented, as is the mechanism to generate parameters for a pipeline run and store their updated values as returned by the pipeline for subsequent retrieval at higher levels of processing. In addition, the mechanism for the public access of the data files and metadata via the Web is described.

1. Introduction

The Chandra Xray Center (CXC) receives large amounts of data in the form of data files daily. The responsibility of the Chandra data archive is to ensure that these data files are stored in a file archive and to record pertinent processing information contained within these files in a database. It is essential that users such as the CXC automatic processing, CXC operators, Chandra observers, and the general public have access to this CXC data repository. The Chandra data archive has interfaces to the archive which allow users to view data locally using applications and globally through the Web.

2. Overview of Archive Interfaces

There are primarily two categories of interfaces to the archive: local and Web access. Automatic processing and CXC operators use local applications to communicate with the archive system while the observers and the general public use the Web applications. Figure 1 presents the general idea of how the interfaces interact with the archiving system.

Automatic processing (AP) uses the arc4gl client to make requests to the archive server for ingest and retrieval of data files. In addition, it utilizes the obspar program to extract ancillary data from database tables in the Sybase SQL server. Automatic processing has the capability to register with the AP notification server to be notified of events such as data arrivals or changes in an observation's status.

Observers and public users download Java applets from the Web server using a Web browser. Data browse and retrieve requests are made through applet buttons and are sent through the Gateway to the SQL server or to the archive server. Data is sent back to the applet for viewing or retrieval.

3. Automatic Processing

Figure 1: Overview of Chandra Data Archive Interfaces.

3.1. The Arc4gl and Archive Server Interface.

Automatic processing and CXC operators communicate data transfer requests through the arc4gl/archive server interface. This data archiving interface is two-fold, consisting of a client/server interaction (Zografou et al. 1997). The client, arc4gl, is a customized application which makes requests to the archive server for file archives and retrieves by issuing language commands. The archive server interprets and executes the arc4gl commands.

The arc4gl language used to make data requests is presented in the form ``keyword = value''. The server responds by executing an ingest, a browse, or a retrieve operation. Users must specify these operations along with other arc4gl keywords like filename, file type, level of processing, observation identifier (obsid), observation interval (obi), time range, etc. in order to correctly identify data products in the database and in the file storage area. If a file is ingested, pertinent archival information is extracted from the file and stored in the database by the archive server. This information is used for future browse and retrieve operations.

Included is an example of an arc4gl request for an observation that has an obsid, 1069. The user is requesting to browse the archive contents of level 1 event lists for the specified observation. The filename, file size, and the time it was archived is given back to the user.

arc4gl> operation=browse
arc4gl> dataset=flight
arc4gl> detector=acis
arc4gl> level=1
arc4gl> obsid=1069
arc4gl> filetype=evt1
arc4gl> go
Browsing...
                 Filename      FileSize(bytes) File_TimeStamp
acisf01069_000N001_evt1.fits   14276160        9/5/1999 22:50:09

1 record(s) browsed
Browse succeeded

arc4gl>

3.2. AP Notification Server

Automatic processing makes decisions when to start pipeline processing based on the timing of archival events, such as ingested telemetry files and updates made to certain observation information stored in database tables. AP can be notified of these archival events by using a client interface that registers with the AP notification server. When events occur, the notification server sends a message to the client describing the type of event that took place along with the time it occurred. This is one way that the AP can keep track of the changes within the database.

3.3. Obspar Customized Data Access

Planned observation data is accessed by automatic processing through the execution of obspar. The obspar program uses libraries to interface directly to the SQL server. It retrieves ancillary planned data from the observation catalog (OCAT) database for specified parameters e.g. obsid, obsid/obi, observation start and observation stop times. It then formats the extracted data into a parameter file. This parameter file is used in pipeline processing and is updated. The updated parameter file is then archived via the arc4gl client and updates ancillary databases for further obspar parameter extractions.

Included is an excerpt of an obspar-generated parameter file for planned data for obsid, 1069.

CLI PROMPT: obspar -o1069

title,s,h,"",,,"Proposal title"
observer,s,h,"DR. CAL",,,"Principal investigator"
object,s,h,"LMC X-1",,,"Source name"
obs_id,i,h,1069,,,"Observation id"
obi_num,i,h,0,,,"Obi number"
instrume,s,h,"ACIS",,,"Instrument"
detector,s,h,"ACIS-S",,,"Focus detector"
detnam,s,h,"ACIS-456789",,,"Detector"
ra_pnt,r,h,84.914583,,,"Pointing RA"
dec_pnt,r,h,-69.743611,,,"Pointing Dec"
tstart,r,h,51711568.184000,,,"Observation start time"
tstop,r,h,51713568.184000,,,"Observation end time"

4. Public Web Interface

The Chandra observers and public users can view and retrieve the processed CXC data through the Web via Java applets provided by the Chandra data archive. These applets communicate with data servers behind a firewall using Sybase jConnect as a Gateway. The requests in the applets are sent through the Gateway to the archive and SQL servers (Chary et al. 1997), and results are displayed in the applet.

Planned and scheduled observation data can be accessed by the OCAT browser applet and Chandra data files can be browsed and retrieved through the Search and Retrieve applet. Users set observation search criteria in the OCAT browser applet and submit the request. Observation records matching the criteria are returned for viewing. Data products for all observations or for a subset of these observations can then be selected and retrieved using the Search and Retrieve applet.

Acknowledgments

This project is supported by the Chandra Xray Center under NASA contract NAS8-39073.

References

Chary, S., DuPrie, K., Harbo, P., Pak, K., & Zografou P. 1997, Search and Retrieval of the AXAF Data Archive on the Web using Java, in ASP Conf. Ser., Vol. 145, Astronomical Data Analysis Software and Systems VII, ed. R. Albrecht, R. N. Hook, & H. A. Bushouse (San Francisco: ASP)

Zografou, P., Chary, S., DuPrie, K., Harbo, P., & Pak, K. 1997, AXAF Science Center Data Archive for AXAF Ground Calibration, in ASP Conf. Ser., Vol. 145, Astronomical Data Analysis Software and Systems VII, ed. R. Albrecht, R. N. Hook, & H. A. Bushouse (San Francisco: ASP)

© Copyright 2000 Astronomical Society of the Pacific, 390 Ashton Avenue, San Francisco, California 94112, USA Next: Data Quality Control at the Very Large Telescope
Up: Data Pipelines and Quality Control
Previous: The Chandra Monitoring System
Table of Contents - Subject Index - Author Index - PS reprint -