Документ взят из кэша поисковой машины. Адрес оригинального документа : http://www.atnf.csiro.au/research/workshops/2013/astroinformatics/talks/JMC_AstroInformatics.pdf Дата изменения: Fri Dec 13 04:35:01 2013 Дата индексирования: Sat Mar 1 10:31:06 2014 Кодировка: Поисковые слова: созвездие лебедя

CSIRO ASKAP Science Data Archive
CSIRO ASTRONOMY AND SPACE SCIENCE (CASS)

Jessica Chapman, AstroInformatics Meeting, 13 December 2013


CSIRO ASKAP Science Data Archive (CASDA)
Talk outline
A: CASDA overview

B: Requirements and use cases
C: Data access and data volumes

D: Communications with science users
E: High performance computing


Australian SKA Pathfinder (ASKAP) A fast survey instrument

· · · · ·

36 12 m dishes Max baseline = 6 km Phased array feeds ­ 188 elements Digital beamforming 30 deg2 FOV

· 700 ­ 1800 MHz · 300 MHz Bandwidth · 16,384 frequency channels


CASDA: Project roadmap

Early science: ~April 2015

A: CASDA overview and timeline


CASDA Stage 0 development team
IM&T:
Euan Sangster (Project leadership) Angus Vickery (Project leadership) Dan Miller (Project Manager) James Dempsey (Project engineer) Jared Pritchard (Business analyst) Dave Morrison (Infrastructure specialist) Simon Bear (Software developer) Adam de Laine (Testing) Bradford Greer (Software/architecture)

CASS:
Jessica Chapman (Project Leader) Ian Heywood (Project Scientist) Arkadi Kosmynin (Software developer) Matthew W hiting (Science data processing) Ben Humphreys (Science data processing)

Team uses an agile approach to project mgt and software development

A: CASDA overview and timeline


Stage 0 activities
· Requirements description · Use case model · Workflows model

· System Architecture description
· Decisions on VO, database and middleware solutions

· Getting started at Pawsey Centre
· DB and VO prototypes for test purposes · Preliminary operations and support model · Preliminary Design Review

A: CASDA overview and timeline


Murchison Radio Observatory

Pawsey Centre

Worldwide

Central processor: Processes the raw visibilities and outputs science data products. Science Data Archive Facility: Data storage and access to science data products

A: CASDA overview and timeline


ASKAP Central Processor

Data from same set of visibilities can be passed through three pipelines for Transient, Continuum and Spectral Line imaging.

In principle this allows the data for up to three different projects to be observed `commensally'.

A: CASDA overview and timeline


CASDA Data Products
Product Calibrated continuum visibility data (stored as `measurement sets') Continuum image cubes (small number of frequency channels) Data type CASA FITS

Spectral line image cubes (large number of frequency channels)
Postage stamp image cubes Continuum source detections Spectral line source detection catalogues Transient source detection catalogues

FITS
FITS Catalogue Catalogue Catalogue

Transient light curves ­ properties
Bright Source Catalogue (global sky model)

Catalogue
Catalogue

A: CASDA overview and timeline


ASKAP Data Processing levels

A: CASDA overview and timeline


CASDA use of Pawsey Centre infrastructure

Pulsars for Public & Pupils | Robert Hollow |


CASDA connections

~ 10

~ 350

1,000+ ~ 10
A: CASDA overview and timeline

~ 400


CASDA analysis and planning
(decisions in progress)

Middleware: NGAS Database backends: PostgreSQL

VO tools: Based on CDS VO libraries (looking at ADQL, UW S and SAVOT llibraries) with additional implementation CASDA application: W ill include components from CSIRO DAP

A: CASDA overview and timeline


CASDA Application capabilities


CASDA (Some) high-level requirements
Ref: CSIRO ASKAP Science Data Archive: Requirements and Use Cases

Essential Requirements (a subset as examples)
ASKAP data products are open access and made publically available as soon as possible. CASDA will provide access to images, image cubes and catalogues using VO protocols
Long term data storage will be provided at the Pawsey Centre. The CASDA design will not restrict the potential future requirement for one or more copies to be stored at other locations. CASDA will provide a repository for Survey Science Teams to upload predefined and VO-compatible science catalogues and will provide search tools for such catalogues (under negotiation).

B: Requirements and use cases


Survey Science Teams: Example use cases
· Run query to obtain a listing of the visibility files archived and

sky regions observed for project. · Set data validation flags following review of image quality reports · Simple cone searches · Complex catalogue queries · Download image cube `cut-outs' · Download selected image cubes for further analysis · Upload `final' science catalogues into archive and make available for general use.

B: Requirements and use cases


Survey Science Projects: File-based data sizes in CASDA
SSP Type Nfields Time per field Visibility data size per field (TB) 2.4 1.5
Not archived
Not archived Not archived

Image data size per field (TB) 0.003 1.0

EMU POSSUM

C C

1200 1200

12 8

WALLABY
DINGO FLASH

S
S S

1200
966 850

8
8 4

1.8
1.3 0.5

GASKAP
VAST

S
T

644
1200

12
8

Not archived

[0.5]
Probably not archived

<1

For full ASKAP, CASDA will archive on average about 15 TB per day


Science Data Access
Access to images and catalogues will be provided through Virtual Observatory services: Simple Image Access Protocol returns link to images/cubes identified for a given position. CASDA will provide tools to generate image `cut-outs'. Cone searches: Returns table results such as positions and fluxes for sources detected within an area around a given position. Table Access Protocol: Allows for complex querying of tables. For example ­ could return a list of detections for sources above a given flux density with negative spectral indices (slopes). We are setting up a VO demonstrator to trial and test VO implementations for ASKAP data. Need to build VO expertise in radio astronomy and engage community.

C: Data access and volumes


D: Communications with science users
· CASDA Science Reference Group (phase 0) ~ 15 members. Regular meetings since Oct 2013 · User Requirements draft document (v0.8) distributed 4 Nov 2013 for comments · User Requirements document (v1.0) will be released in early Jan 2014

· CASDA monthly newsletter distributed by email
· Articles in ASKAP newsletter (but more needed for CASS website)

D: Science communications


E: HPC data processing / high volume data
Issue Notes

Survey Science teams ­ will require access to temporary data storage and HPC ­ for post processing of ASKAP data products
Radio astronomy community is unfamiliar with applying for HPC and data storage on other national facilities Australian application processes for HPC and data storage on different facilities are not well integrated

Difficult / impossible to transfer `large' amounts of data over networks.
Information /education for Australian astronomy community is needed

E: High Performance Computing


iVEC Facilities
iVEC manages several high performance and data storage facilities including the Real Time Computer and Magnus supercomputers in the Pawsey Centre.

All iVEC facilities are allocated as follows:

Category

Allocation (compute + storage) 25% 25% 30%

Notes

Radio astronomy Geoscience Partners

Shared between ASKAP and MWA

CSIRO, UWA, Curtin Uni, Murdoch Uni and Edith Cowan Uni Managed through the NCMAS

National Researchers Director 's time

15% 5%

NCMAS: National Computational Merit Allocation Scheme
E: High Performance Computing


iVEC Magnus Supercomputer
Magnus ­ CRAY supercomputer. Currently has 69 TF compute power installed. Final configuration will be at least 1 PF. HPC allocations on Magnus include provision of scratch data storage.
Major application rounds for Magnus in Q4 each year for following calendar year: Science teams with Australian researchers may be able to apply: · to NCMAS [ncmas.nci.org.au] · to iVEC for the Major Partner share allocation [portal.ivec.org/ivecallocation/] The iVEC Director 's Share scheme (5% share) is always open.

E: High Performance Computing


Thank you
Jessica Chapman
CSIRO Astronomy and Space Science

t +61 2 9372 4196 e Jessica.Chapman@csiro.au w atnf.csiro.au
CSIRO ASTRONOMY AND SPACE SCIENCE