Next: ADASS Web Database XML Project
Up: Software Development Methodologies & Technologies
Previous: msg: A Message-Passing Library
Table of Contents -
Subject Index -
Author Index -
Search -
PS reprint -
PDF reprint
Yagi, M., Mizumoto, Y., Yoshida, M., Kosugi, G., Takata, T., Ogasawara, R., Ishihara, Y., Morita, Y., Nakamoto, H., & Watanabe, N. 2001, in ASP Conf. Ser., Vol. 238, Astronomical Data Analysis Software and Systems X, eds. F. R. Harnden, Jr., F. A. Primini, & H. E. Payne (San Francisco: ASP), 237
DASH--Distributed Analysis
System Hierarchy
Masafumi Yagi, Yoshihiko Mizumoto, Michitoshi Yoshida,
George Kosugi1, Tadafumi Takata2,
Ryusuke Ogasawara3
National Astronomical Observatory, Osawa, Mitaka, Tokyo 181-8588, Japan
Yasuhide Ishihara
Fujitsu Limited, Nakase, Mihama, Chiba 261-8588, Japan
Yasuhiro Morita, Hiroyuki Nakamoto, Noboru Watanabe
SEC Co. Ltd., 22-14 Sakuragaoka, Shibuya, Tokyo 150-0031, Japan
Abstract:
We developed the
Distributed Analysis Software Hierarchy (DASH), an object-oriented
data reduction and data analysis system
for efficient processing of data from the SUBARU telescope.
DASH consists of many objects (data management objects,
reduction engines, GUIs, etc.) distributed on CORBA.
We have also developed SASH, a stand-alone system
which has the same interface as DASH, but which does not use some of
the distributed services
such as DA/DB; visiting astronomers can
detach PROCube out of DASH and continue the analysis with SASH
at their home institute. SASH will be used as a quick reduction tool
at the summit.
DASH is an object-oriented data reduction and analysis system
for the 8.2m SUBARU telescope in Hawaii.
It is the Observatory's
system for managing,
reducing, and analyzing the huge amount of astronomical
data produced by various Subaru instruments. Design goals include
reproducibility and portability of the reduction,
seamless operability on heterogeneous computer systems,
open architecture, trial-and-error processing, pipeline processing,
and research-target-oriented analysis.
For these purposes, CORBA is adopted as a
distributed object environment (Takata et al. 1998).
With CORBA, we can easily connect the system backbone
coded in C++, and the GUIs coded in JAVA2 (Figure 1).
The details of the DASH system were reported in Yagi et al. (2000)
and Mizumoto et al. (2000).
Figure 1:
DASH application model (SUBARU restaurant)
|
There are three subsystems of the Subaru computer system:
the Subaru Observation Software System (SOSS), the Subaru Telescope
data ARchive System (STARS), and DASH.
The key idea connecting them is the
``Observation Dataset'' (Kosugi et al. 1998).
The Observation Dataset consists of Dataset rules and an Observation log.
Dataset rules define how the observation should be done.
For example, it defines how often bias frames should be taken
and which commands should be used.
Every observer makes his/her observation plan in SOSS
following Dataset rules, so that all calibration data
to reduce a target should be obtained during the observation.
The Observation log and the Dataset rules are
sent to STARS with the data.
After the observation, DASH reads the Observation Dataset
from STARS. In the dataset, a template reduction procedure
(skeleton PROCube) is also specified.
DASH fills the template with the data, and
the target is thus automatically reduced/analyzed
to output some data quality parameter, such as seeing size.
We can improve the Dataset rules by examining
the data quality and feed it back to SOSS.
Just prior to this conference, we released
SASH,
a stand-alone system extracted from a core part of DASH.
Because DASH is strongly connected with STARS
(Subaru data archiving system), and the SUBARU user/group authentication
system, DASH is available only at SUBARU Hilo (Hawaii) and Mitaka (Japan).
SASH is designed so that visiting astronomers can
continue the data analysis at home institutions
and feed the results back to the DASH systems.
For portability, the SASH platform is coded in JAVA2,
with some C++ backbones of DASH rewritten in JAVA2.
Users may add to the SASH platform using
reduction engines such as eclipse or IRAF.
There are few UNIX OSs on which JAVA2 1.2.2 is available.
We therefore selected Solaris Sparc and Linux
for the SASH environment at present.
Some engine wrapper scripts written in Bourne shells
must be rewritten because of the difference of behavior of
some binaries (eg. awk).
There are two DASH systems now, in Hawaii and Japan.
Each is connected to a local database and archive system, and
the DB/DA systems are not synchronized.
On the other hand, SASH has its 'database' as files.
So, DASHs and SASHs share 'reduction' by exporting/importing warehouses,
which contains engines, PROCubes, and data (Figure 2).
Users should explicitly detach and attach warehouses.
DASH and SASH use their own naming service for local objects.
In the next development phase, we plan is to connect the two DASHs at Hilo and
Mitaka via IIOP
(Figure 3). The naming service of each system will collaborate with
the other to resolve objects
so that users can continue the reduction at Hilo on the Mitaka system,
and vice versa.
If a requested object is not in its local system,
the object is transferred from another system through ORB.
This function depends on faster Internet speeds between Hilo and
Mikata than are currently available.
Our ultimate plan is to connect DASH and SASH via IIOP and to use
an interoperable naming service on all systems (Figure 4).
Importing/exporting would then no longer be needed.
If some data were needed, SASH would automatically
connect to the nearest DASH and to retrieve them.
Figure 2:
Phase 1: Current System
|
Figure 3:
Phase 2: Near Future
|
Figure 4:
Phase 3: Final System
|
References
Kosugi, G., et al. 1998, Proc. SPIE, vol. 3349, 421
Mizumoto, Y., et al. 2000, Proc. SPIE, vol. 4009, 429
Takata, T., et al. 1998, Proc. SPIE, vol. 3349, 247
Yagi, M., et al. 2000, in ASP Conf. Ser., Vol. 216, Astronomical Data
Analysis Software and Systems IX, ed. N. Manset, C. Veillet, &
D. Crabtree (San Francisco: ASP), 510
Footnotes
- ... Kosugi1
- Subaru Telescope, National Astronomical Observatory of Japan, Hilo, HI 96720, USA
- ... Takata2
- Subaru Telescope, National Astronomical Observatory of Japan, Hilo, HI 96720, USA
- ... Ogasawara3
- Subaru Telescope, National Astronomical Observatory of Japan, Hilo, HI 96720, USA
© Copyright 2001 Astronomical Society of the Pacific, 390 Ashton Avenue, San Francisco, California 94112, USA
Next: ADASS Web Database XML Project
Previous: msg: A Message-Passing Library
Table of Contents -
Subject Index -
Author Index -
Search -
PS reprint -
PDF reprint
adass-editors@head-cfa.harvard.edu