CASDA Users Guide

Version 1.1 March 2016

Welcome to the CSIRO ASKAP Science Data Archive (CASDA) Users Guide. This Guide is intended to help astronomers get started with finding and making use of data products from the Australian Square Kilometre Array Pathfinder (ASKAP).

The first ASKAP data products have been produced from science commissioning observations taken with the Boolardy Engineering Test Array (BETA). These have been selected to show the potential and unique wide-field fast survey capabilities of ASKAP and to provide some demonstration data sets to the astronomy community.

For information on the initial data release see the BETA Data Release Notes.

For a reference document that preceded the construction of CASDA see the pdf document:
CSIRO ASKAP Science Data Archive: Overview, Requirements and Use Cases (Chapman et al. 2014)

6 Publications and Acknowledgements

7 Links to external documentation

8 Document versions

1 CASDA Overview

1.1 About CASDA

The CSIRO ASKAP Science Data Archive (CASDA) provides the long term storage for ASKAP data products and the hardware and software facilities that enable astronomers to make use of these. Data products are stored at the Pawsey Supercomputing Centre in Perth, Western Australia.

ASKAP is a data driven facility where the data rates are extremely high. In full operations, the ASKAP data rates arriving at the Pawsey Centre will reach around 75 Petabytes (PB) per year. This is beyond the current ability to archive data and so raw data are not archived. Such high data rates require instead that ASKAP data processing is carried out using automated pipelines to produce 'data products' and associated metadata. These are stored and made available through the science archive. The archive can be thought of as the end stage of the full system.

ASKAP Early Science observations are due to begin in 2016. So that users can gain experience and provide feedback to the CASDA team, CASDA will provide access to demonstration data products obtained during science commissioning and pilot observing programs that have been carried out with a small number of ASKAP antennas.

1.2 CASDA data products

CASDA provides three types of data products:

Calibrated visibility files

Calibrated visibility files are stored for total intensity and polarisation continuum observations. During Early Science, calibrated visibility data files for spectral line observations will be archived on a best efforts basis.
Images and image cubes

Radio continuum and spectral line images and image cubes are stored in FITS format.
Catalogues

As part pf the data processing pipelines, detection algorithms are used to search images and image cubes cubes for source detections. Detection parameters such as positions and flux densities are captured in catalogues. The CASDA catalgoue registry includes source detection catalogues with parameters determined from continuum, polarisation and spectral line images or image cubes.

1.3 CASDA services on the CSIRO Data Access Portal

CASDA provides data services in two ways: using search interfaces and tools that are accessed through the CSIRO Data Access Portal (DAP), and using Virtual Observatory Services.

The CSIRO DAP provides access to many archival data products managed by CSIRO across the organisation. These include a wide range of science areas including astronomy. Many customised tools have been added to the DAP to support CASDA. See section 3 for instructions on how to use the CASDA DAP services.

1.4 CASDA Virtual Observatory services

The Virtual Observatory (VO) uses standard protocols to enable catalogue or image data obtained from one facility to be easily compared with similar data from other facilities. For example a user might wish to compare source detections from a radio survey with detections from an optical or infrared surveys, or he/she might wish to plot a set of radio source positions on top of an optical image.

For catalogues, we strongly recommend using TOPCAT (Tool for OPerations on Catalogues and Tables. TOPCAT is a freely available program that provides an interface to Virtual Observatory compatible data products. TOPCAT is like a 'browser' that provides access to VO supported data products with many tools for working with these.

TOPCAT is a richly tooled program with many features that support working with catalogues. In this guide we give some start-up instructions for using CASDA with TOPCAT. For full tutorial and reference documentation please see the TOPCAT website.

For images and image cubes, CASDA currently provides an initial implementation of the VO 'Simple Image Access protocol (v2)' with user access through a web browser. This allows users to progromatically access images using user scripts, and to make use of an image/image cube cut-out service.

1.5 User Authentication and OPAL registration

CASDA allows users to search for data and to see what is available in the archive, without any authentication. However, user authentication is required to download data files including calibrated visibilities, images and image cubes.

For science users CASDA supports two types of user authentication as follows:

OPAL CASDA uses the OPAL proposal applications user account system.
For general use, we recommend using OPAL accounts. To register with OPAL, go to the OPAL Home Page and click on the link to 'Register'. Enter your email address, name, affiliation and a password. The OPAL application will register you straight away and will then open a screen for you to login.
OPAL user accounts are self-managed. Please keep your account details up to date. To change user-registration details, or to request a new OPAL password, use the links to 'Update your details' and 'Change your password'. If you have forgotten your password you may request that a new one be sent via email.

CSIRO Nexus authentication is available for individuals who have CSIRO NEXUS accounts. This is needed for some adminstration-level tasks.

For general use we recommend that all CASDA users have an OPAL account.

1.6 Restricted CASDA tasks and project roles

In addition to general user authentication, a small number of CASDA tasks are restricted to archive administrators and or members of the Survey Science teams. These include, for example, setting data validation flags, and/or accessing project data prior to public release. Tasks with restricted access are described in section 5.

CASDA recognises special roles for members of Survey Science teams. These allow team members to find and access data products for their survey(s) prior to general release, and to carry out data validation tasks. The CASDA roles for Survey Science Team members are as follows:

Project administrators can assign roles to other team members, enter data validation information and access unreleased data products for their projects. (Note: At present the administration-level tasks are restricted to CASDA staff administrators. This will later be modified so that Survey Science team members can carry out project-level administration.)

'Validators' enter data validation information and access unreleased data products for their projects.

Other team members are given access to unreleased data products for their projects but do not have permission to enter data validation information.

1.7 Pawsey Supercomputing Centre user accounts

For users who have user accounts at the Pawsey Supercomputing Centre, CASDA provides fast downloads to the Pawsey Galaxy and/or Magnus supercomputers.

Pawsey accounts are restricted to individuals who are part of science teams that have been granted access to Pawsey facilities through competitive merit allocation processes. For further information please refer to the Pawsey website or contact the CASDA helpdesk.

1.8 Getting Help

CASDA provides help through documentation and a user support service in several ways:

This guide is intended to provide an overview of the CASDA data services and to help new users get started. To get going we recommend trying some of the items described in sections 2 and 3.
The CASDA application pages on the CSIRO Data Access Portal provide some 'tooltips'. These are shown as yellow question marks. Click on a question mark to read the tip.
Online documentation is available with the DAP . Click on the 'help' link at the top of the page.
For enquiries and staff support relating to all CSIRO radio astronomy data archives, including CASDA, please send an email to atnf-datasup@csiro.au. You will received an automated email from our helpdesk to acknowledge that your request has been logged. A CASS staff member will reply soon afterwards. We aim to send an initial reply to user queries within four business hours.
Suggestions for improvements to CASDA tools and to this Guide are always welcome. Please send these to the helpdesk.

2 Using CASDA with the CSIRO Data Access Portal

The information in this and the following section is presented as a set of 'how to instructions' using specific data sets as examples.

2.1 Login to the DAP using an OPAL or NEXUS account

Check that your browser options are set up to allow pop up windows.
For example, in Firefox - to allow pop up windows use the browser Options. Then select Content -> and check that the 'block pop up windows' tick box is unselected.
Go to the DAP Home Page
Select the registration scheme in the box headed 'Registered Users'

Enter your account details using either an OPAL or CSIRO Nexus account.

Notes

2.1 Login to the DAP using an OPAL or NEXUS account
Check that your browser options are set up to allow pop up windows. For example, in Firefox - to allow pop up windows use the browser Options. Then select Content -> and check that the 'block pop up windows' tick box is unselected. Go to the DAP Home Page Select the registration scheme in the box headed 'Registered Users' Enter your account details using either an OPAL or CSIRO Nexus account.
Notes

2.2 Find information about ASKAP Data Collections

Go to the DAP Home Page

Enter a keyword in the green search box. For example, enter the project code 'AS031' then click on 'Search'. The search will return links to two data 'collections'.

Click on the link to '...catalogues' and have a look at the information provided about the collection.
Note that at the top and bottom of the page there is a link to 'show all'. Click on this to reveal additional information about the collection including the list of individuals associated with the OPAL proposal for this project.

There are two tabs labelled 'Description' and 'Data'. Now click on the 'data' tab. This will bring up a page with a list of catalogues associated with AS031.

Navigate back to the DAP form and have a look at the other collection for '...images and visibilities'. You will see many visibility and image-related files listed for this data tab!

Notes
The CSIRO DAP holds data for thousands of collections. You might like to explore the DAP using different keyword searches. Some example keywords are 'pulsars', 'mosquitos' and 'climate change' or try guessing.
AS031 is the project code used for ASKAP BETA science commissioning observations. This is the only project code in use at present.
In general terms, a data collection is a group of similar data files. For CASDA, each ASKAP project has two collections. One collection holds the data catalogues and the other the data product files for images, image cubes and visibilities.
Lists of collections can be sorted using the sort options that are shown under the large blue search button.

2.2 Find information about ASKAP Data Collections
Go to the DAP Home Page Enter a keyword in the green search box. For example, enter the project code 'AS031' then click on 'Search'. The search will return links to two data 'collections'. Click on the link to '...catalogues' and have a look at the information provided about the collection. Note that at the top and bottom of the page there is a link to 'show all'. Click on this to reveal additional information about the collection including the list of individuals associated with the OPAL proposal for this project. There are two tabs labelled 'Description' and 'Data'. Now click on the 'data' tab. This will bring up a page with a list of catalogues associated with AS031. Navigate back to the DAP form and have a look at the other collection for '...images and visibilities'. You will see many visibility and image-related files listed for this data tab!
Notes The CSIRO DAP holds data for thousands of collections. You might like to explore the DAP using different keyword searches. Some example keywords are 'pulsars', 'mosquitos' and 'climate change' or try guessing. AS031 is the project code used for ASKAP BETA science commissioning observations. This is the only project code in use at present. In general terms, a data collection is a group of similar data files. For CASDA, each ASKAP project has two collections. One collection holds the data catalogues and the other the data product files for images, image cubes and visibilities. Lists of collections can be sorted using the sort options that are shown under the large blue search button.

2.3 Find a persistent link to a data collection

Use the above steps to bring up collections information for AS031

See the link near the top of the DAP page to 'copy this persistent link...'. This link provides an indefinite link for the collection.

Cut and paste this link into your browser. It will take you back to the same page.

Notes
CASDA collections remain 'open' for the duration of the project. The persistent link provides one way of finding the collection.
CASDA also provides Digital Object identifiers (DOIs) for data collections. A DOI provides a 'data citation' that allows a user to obtain a set of data corresponding to the date and time the DOI was issued (or 'minted'). For CASDA the DOIs will be updated at intervals of around six months from March 2016 onwards. DOIs will be included with the collections information.

2.3 Find a persistent link to a data collection
Use the above steps to bring up collections information for AS031 See the link near the top of the DAP page to 'copy this persistent link...'. This link provides an indefinite link for the collection. Cut and paste this link into your browser. It will take you back to the same page.
Notes CASDA collections remain 'open' for the duration of the project. The persistent link provides one way of finding the collection. CASDA also provides Digital Object identifiers (DOIs) for data collections. A DOI provides a 'data citation' that allows a user to obtain a set of data corresponding to the date and time the DOI was issued (or 'minted'). For CASDA the DOIs will be updated at intervals of around six months from March 2016 onwards. DOIs will be included with the collections information.

2.4 Search for data products using the DAP search form

Go to the DAP Home Page

Click on the large blue button that says 'Domain Search' and then select 'CASDA Observations Search' - this brings up the DAP search form.

Enter some search criteria. For example you could enter 'AS031' as the project code or enter 'BETA' as the project title.
At the bottom of the page check that the option for 'released' data is selected. Then click on 'search'.

You will see a page returned that has up to three tabs for 'Catalogues', 'Image cubes' and 'Visibilities'.

Have a look at each of these tabs to see what they contain.

Note the 'Refine Search' button that takes you back to the search page with your previous search criteria.

On the search form, notice the little question mark next to the frequency range option. This is a 'tooltip'. Click on this to see extra information.

Notes
The search form will return information for all collections and data products that match the criteria. Only tabs that have associated data files are shown.
Selecting a data tab and then clicking on the project number will open the collections information that is relevant to that tab.
'Unreleased' data are only available to team members. Data are released following a data validation process.

2.4 Search for data products using the DAP search form
Go to the DAP Home Page Click on the large blue button that says 'Domain Search' and then select 'CASDA Observations Search' - this brings up the DAP search form. Enter some search criteria. For example you could enter 'AS031' as the project code or enter 'BETA' as the project title. At the bottom of the page check that the option for 'released' data is selected. Then click on 'search'. You will see a page returned that has up to three tabs for 'Catalogues', 'Image cubes' and 'Visibilities'. Have a look at each of these tabs to see what they contain. Note the 'Refine Search' button that takes you back to the search page with your previous search criteria. On the search form, notice the little question mark next to the frequency range option. This is a 'tooltip'. Click on this to see extra information.
Notes The search form will return information for all collections and data products that match the criteria. Only tabs that have associated data files are shown. Selecting a data tab and then clicking on the project number will open the collections information that is relevant to that tab. 'Unreleased' data are only available to team members. Data are released following a data validation process.

2.5 Carry out a cone search using the CASDA Observation Search form

This example finds data products for a region observed in Tucana.

As above go to the CASDA search form

In the cone search box enter

right ascension = 22:58:30

declination = -55:56:00

search radius = 1 arcmin

This will return a page with a list of 21 images for the Tucana region that have central positions within this range.

Notes
Only one data tab is seen as CASDA has no corresponding visibility or data catalogues for this region.

2.5 Carry out a cone search using the CASDA Observation Search form
This example finds data products for a region observed in Tucana. As above go to the CASDA search form In the cone search box enter right ascension = 22:58:30 declination = -55:56:00 search radius = 1 arcmin This will return a page with a list of 21 images for the Tucana region that have central positions within this range.
Notes Only one data tab is seen as CASDA has no corresponding visibility or data catalogues for this region.

2.6 Download data products to external locations

In this example an image covering 150 square degrees of the Tucana region, created by Ian Heywood from data combined over four sub-bands and three epochs is downloaded to your local computer. This approach can be used to download any available data products including images, visibilities and catalogues.

Navigate to the CASDA Observation Search form as described in section 2.2

Select Scheduling Block ID = 1206 and then click on search.

You will see a single tab for image cubes - with a list of associated files.

To select the file corresponding to the demonstration image for this region, select the tick box next to the
filename = 'SB1206-1207-1229-1231_fullstack_reframed.fits'. (This is near the end of the list)

In the box below the list select the option for 'web' and click on the link to 'retrieve selected data'.

You will now see a new browser page or new tab on your browser with a link to the retrieved data and status information. The page also shows you how long you have left to retrieve the file(s)

The information on this page is refreshed automatically once a minute. When the status changes to green, click on the link to the data file and save it to a local location.

Notes
For this example, the image file size is 113 MB. This may take some time to download across the networks.
The Schedule Block ID (1206) corresponds to the first of four schedule blocks used for this data product. When more than one observation block is used to create a data product CASDA associates the data file with the first scheduling block.
An additional file is provided with checksum information. This could be used to check the download has fully worked.

2.6 Download data products to external locations
In this example an image covering 150 square degrees of the Tucana region, created by Ian Heywood from data combined over four sub-bands and three epochs is downloaded to your local computer. This approach can be used to download any available data products including images, visibilities and catalogues. Navigate to the CASDA Observation Search form as described in section 2.2 Select Scheduling Block ID = 1206 and then click on search. You will see a single tab for image cubes - with a list of associated files. To select the file corresponding to the demonstration image for this region, select the tick box next to the filename = 'SB1206-1207-1229-1231_fullstack_reframed.fits'. (This is near the end of the list) In the box below the list select the option for 'web' and click on the link to 'retrieve selected data'. You will now see a new browser page or new tab on your browser with a link to the retrieved data and status information. The page also shows you how long you have left to retrieve the file(s) The information on this page is refreshed automatically once a minute. When the status changes to green, click on the link to the data file and save it to a local location.
Notes For this example, the image file size is 113 MB. This may take some time to download across the networks. The Schedule Block ID (1206) corresponds to the first of four schedule blocks used for this data product. When more than one observation block is used to create a data product CASDA associates the data file with the first scheduling block. An additional file is provided with checksum information. This could be used to check the download has fully worked.

3 Using CASDA with Virtual Observatory Services for Catalogues

The notes given in this section provide an introduction to finding and working with CASDA VO tables, mostly together with TOPCAT. These are intended to be sufficient to get started but do not cover many of the available tools and features.

The notes given here correspond to TOPCAT version 4.3.

3.1 Install TOPCAT

Go to the webpage http://www.star.bris.ac.uk/~mbt/topcat/#install and download the full jar file.

You may also need to install a Java Runtime Environment (JRE) if this is not already installed on your computer. If so see links on this page.

Notes
TOPCAT and Java are freely available. To see which version of TOPCAT you are using open TOPCAT then click on Help > About TOPCAT.

3.1 Install TOPCAT
Go to the webpage http://www.star.bris.ac.uk/~mbt/topcat/#install and download the full jar file. You may also need to install a Java Runtime Environment (JRE) if this is not already installed on your computer. If so see links on this page.
Notes TOPCAT and Java are freely available. To see which version of TOPCAT you are using open TOPCAT then click on Help > About TOPCAT.

3.2 Find and download CSIRO astronomy catalogues using the VO Table Access Protocol (TAP)

In this example, you will find and download a list of VO catalogues supported by CSIRO and then select a catalogue, casda.continuum_island. This contains parameters for radio astronomy 'islands' detected using source find algorithms for an image of the Tucana region.

Open TOPCAT. You will see a window open up. In this guide this will be called the 'main window'. This window is like a 'command centre' so try not to close it!

Click on the link in the top menu bar to 'VO'

Click on the drop down option for 'Table Access Protocol Query'

In the Keywords box enter 'CSIRO' and then click on 'Find Services'. You will see two CSIRO TAP services. The first of these is for ASKAP catalogues

Select the 'CSIRO ASKAP TAP' service and then select 'Use Service'
You will now see a list of catalogues. Click on the catalogue called 'casda.continuum_island'.

Now enter a simple query to select everything in the table 'casda.continuum_island'.
In the text box near the bottom of the page type:

select *
from casda.continuum_island

You will now see a new TOPCAT window. This is used to store local versions of tables during your TOPCAT session.

To save a table to your laptop click select a table from the Table List. Then use: file > save Table(s)/Session and follow the prompts to choose an output table format and location etc.

Notes
TOPCAT write options include a range of table formats including VO-table, html, FITS, ascii and csv. To write out a VO table use the format option = 'votable-tabledata'.
To navigate to a selected location on your computer use the 'Filestore Browser'.

3.2 Find and download CSIRO astronomy catalogues using the VO Table Access Protocol (TAP)
In this example, you will find and download a list of VO catalogues supported by CSIRO and then select a catalogue, casda.continuum_island. This contains parameters for radio astronomy 'islands' detected using source find algorithms for an image of the Tucana region. Open TOPCAT. You will see a window open up. In this guide this will be called the 'main window'. This window is like a 'command centre' so try not to close it! Click on the link in the top menu bar to 'VO' Click on the drop down option for 'Table Access Protocol Query' In the Keywords box enter 'CSIRO' and then click on 'Find Services'. You will see two CSIRO TAP services. The first of these is for ASKAP catalogues Select the 'CSIRO ASKAP TAP' service and then select 'Use Service' You will now see a list of catalogues. Click on the catalogue called 'casda.continuum_island'. Now enter a simple query to select everything in the table 'casda.continuum_island'. In the text box near the bottom of the page type: select * from casda.continuum_island You will now see a new TOPCAT window. This is used to store local versions of tables during your TOPCAT session. To save a table to your laptop click select a table from the Table List. Then use: file > save Table(s)/Session and follow the prompts to choose an output table format and location etc.
Notes TOPCAT write options include a range of table formats including VO-table, html, FITS, ascii and csv. To write out a VO table use the format option = 'votable-tabledata'. To navigate to a selected location on your computer use the 'Filestore Browser'.

3.3 Run simple queries in Astronomical Data Query Language (ADQL)

Here are three simple examples using ADQL to query a catalogue.

Follow the steps above (section 3.2) and use TOPCAT to find the table casda.continuum_island

In the 'ADQL Text' box try out these simple queries.

Click 'Run Query' after each one is entered)

Example 1: Select the top 1000 rows

select TOP 1000 *
from casda.continuum_island

Example 2: Select first 400 rows and five columns

select top 400

id, catalogue_id, sbid, n_components

from casda.continuum_island

Example 3: Select all rows with schedule block id = 609

select *
from casda.continuum_island
where (sbid = 609)

Notes
The asterisk in examples 1 and 3 indicates that all columns are selected.
Line returns are optional in ADQL queries.

3.3 Run simple queries in Astronomical Data Query Language (ADQL)
Here are three simple examples using ADQL to query a catalogue. Follow the steps above (section 3.2) and use TOPCAT to find the table casda.continuum_island In the 'ADQL Text' box try out these simple queries. Click 'Run Query' after each one is entered) Example 1: Select the top 1000 rows select TOP 1000 * from casda.continuum_island Example 2: Select first 400 rows and five columns select top 400 id, catalogue_id, sbid, n_components from casda.continuum_island Example 3: Select all rows with schedule block id = 609 select * from casda.continuum_island where (sbid = 609)
Notes The asterisk in examples 1 and 3 indicates that all columns are selected. Line returns are optional in ADQL queries.

3.4 Try out some TOPCAT Table Icons

Follow the instructions in section 3.2 above and use TOPCAT to add one or more tables to your Table List.

Select a table from the Table List in the Main Window.

Click on each of the three TOPCAT table icons that have small squares (table cells):

The first of these opens a window with the table data.

The second opens a window with metadata that describe the table.

The third opens a window with metadata that describe the table columns. These include units and descriptions.

Have a look at each of these to get a feel for the data in the table and how this is described

Now try out the next two icons - find out what these do.

Notes
Using the mouse to hover over an icon will bring up a description of it.
Note that the icons are arranged in groups. This group of five icons provides diffferent displays of the table data and metadata.

3.4 Try out some TOPCAT Table Icons
Follow the instructions in section 3.2 above and use TOPCAT to add one or more tables to your Table List. Select a table from the Table List in the Main Window. Click on each of the three TOPCAT table icons that have small squares (table cells): The first of these opens a window with the table data. The second opens a window with metadata that describe the table. The third opens a window with metadata that describe the table columns. These include units and descriptions. Have a look at each of these to get a feel for the data in the table and how this is described Now try out the next two icons - find out what these do.
Notes Using the mouse to hover over an icon will bring up a description of it. Note that the icons are arranged in groups. This group of five icons provides diffferent displays of the table data and metadata.

3.5 Create sky plots showing positions of radio detections

This example shows how to do a sky plot for a set of source positions.

Follow the instructions in section 3.2 above and use TOPCAT to download the catalogue CASDA.continuum_island.

In the TOPCAT Main Window: the middle group of icons has five icons for different types of plots. Click on the sky plot icon (the middle one in this group). This will plot the positions for 1,578 locations identified as radio continuum 'islands'.

To get started with plots, here are a few suggestions to try out:
1. To change the plot centre - use the mouse with a left-click.
2. To zoom in and out - use the mouse with a right-click.
3. To write a plot to a file - select 'Export' (at top of window) and then select a file format.

Notes
TOPCAT provides powerful tools for plotting table data and allows a considerable degree of customization. Note the plot controls given below the plot. For full tutorial and reference documentation, see the TOPCAT user manual .

3.5 Create sky plots showing positions of radio detections
This example shows how to do a sky plot for a set of source positions. Follow the instructions in section 3.2 above and use TOPCAT to download the catalogue CASDA.continuum_island. In the TOPCAT Main Window: the middle group of icons has five icons for different types of plots. Click on the sky plot icon (the middle one in this group). This will plot the positions for 1,578 locations identified as radio continuum 'islands'. To get started with plots, here are a few suggestions to try out: 1. To change the plot centre - use the mouse with a left-click. 2. To zoom in and out - use the mouse with a right-click. 3. To write a plot to a file - select 'Export' (at top of window) and then select a file format.
Notes TOPCAT provides powerful tools for plotting table data and allows a considerable degree of customization. Note the plot controls given below the plot. For full tutorial and reference documentation, see the TOPCAT user manual .

3.6 Create a new column and add to existing table

In this example an extra column is added to a table, corresponding to the logarithm of the peak flux density.

Follow the instructions in section 3.2 above and use TOPCAT to download the catalogue CASDA.continuum_island.

In the TOPCAT Main Window, select the Menu option for 'Views' then select 'column information'

Click on the '+' sign. This opens up a form where you can enter details for a new column.

:

Name: flux_peak_log

Expression: log10(flux_peak)

Units: mJy/beam

Description: flux_peak_log

UCD: For now, leave this blank

Index:Default value (adds an extra column at end of table)

Click on OK

The new column is now added to the same table. To see this look again at the column information or open up the full table. Note that column neaders and table contents etc can be edited.

Notes
Universal Content Descriptors (UCDs) are part of a formal, controlled vocabulary for astronomical data, provided by the International Virtual Observatory Alliance. This is intended facilitates sharing information between different applications. For further information and a list of valid UCDs, see UCD1+ controlled vocabulary.

3.6 Create a new column and add to existing table
In this example an extra column is added to a table, corresponding to the logarithm of the peak flux density. Follow the instructions in section 3.2 above and use TOPCAT to download the catalogue CASDA.continuum_island. In the TOPCAT Main Window, select the Menu option for 'Views' then select 'column information' Click on the '+' sign. This opens up a form where you can enter details for a new column. : Name: flux_peak_log Expression: log10(flux_peak) Units: mJy/beam Description: flux_peak_log UCD: For now, leave this blank Index:Default value (adds an extra column at end of table) Click on OK The new column is now added to the same table. To see this look again at the column information or open up the full table. Note that column neaders and table contents etc can be edited.
Notes Universal Content Descriptors (UCDs) are part of a formal, controlled vocabulary for astronomical data, provided by the International Virtual Observatory Alliance. This is intended facilitates sharing information between different applications. For further information and a list of valid UCDs, see UCD1+ controlled vocabulary.

3.7 Carry out cone searches on a catalogue using the VO Cone Search Protocol

This example searches the CASDA table casda.continuum_island for sources within a given radius of a position and writes the results into another VO table.

Open TOPCAT

Select 'VO' then 'Cone Seach'

In the Keywords box enter 'CSIRO' and then click on 'Find Services'.

Select the 'CSIRO ASKAP Cone' Service

You now need to select the catalogue for the Cone Search. In the second box select the row with the description 'Continuum island data'.
Enter some cone search criteria. For the Tucana region try something like this:

RA = 340.5 degrees
dec = -60.5 degrees
radius = 4.5 degrees

Click on OK to run the query

In the main window, you will see a table added to the Table List. This table includes the subset of rows that correspond to positions within the selected search region.

Notes
CASDA currently restricts cone searches to a maximum radius of five degrees If this is a problem for you please let us know.

3.7 Carry out cone searches on a catalogue using the VO Cone Search Protocol
This example searches the CASDA table casda.continuum_island for sources within a given radius of a position and writes the results into another VO table. Open TOPCAT Select 'VO' then 'Cone Seach' In the Keywords box enter 'CSIRO' and then click on 'Find Services'. Select the 'CSIRO ASKAP Cone' Service You now need to select the catalogue for the Cone Search. In the second box select the row with the description 'Continuum island data'. Enter some cone search criteria. For the Tucana region try something like this: RA = 340.5 degrees dec = -60.5 degrees radius = 4.5 degrees Click on OK to run the query In the main window, you will see a table added to the Table List. This table includes the subset of rows that correspond to positions within the selected search region.
Notes CASDA currently restricts cone searches to a maximum radius of five degrees If this is a problem for you please let us know.

3.8 Search for image and catalogue data products programmatically using user scripts

The CASDA TAP service can be used together with user scripts. This can be helpful, for example, to facilitate access to large numbers of files.
In this example we provide a sample python script, tapquery.py. This connects to the CSIRO TAP Service and finds a catalogue, casda.continuum_component. A simple ADQL query is used to search the catalogue for radio components with peak flux densities below 600 mJy. The output is written in xml/VO-format to a file data.xml in the local folder.
To run the script:

Download the python script tapquery.py (text file) to your computer

If needed, install python, together with the python modules urllib2 and astropy. (You may need to ask computing staff for assistance with installing these.)

For unix/linux, run the script using > python tapquery.py

Notes
For this script the astropy module is used to write the output to a VO-format table. Here is a similar script that can be run without astropy tapquery_csv.py. This version writes the output to a csv file.
The sample script can be modified to connect to different catalogues and to carry out different ADQL commands. For additional documentation see the internal comments in the script.

3.8 Search for image and catalogue data products programmatically using user scripts
The CASDA TAP service can be used together with user scripts. This can be helpful, for example, to facilitate access to large numbers of files. In this example we provide a sample python script, tapquery.py. This connects to the CSIRO TAP Service and finds a catalogue, casda.continuum_component. A simple ADQL query is used to search the catalogue for radio components with peak flux densities below 600 mJy. The output is written in xml/VO-format to a file data.xml in the local folder. To run the script: Download the python script tapquery.py (text file) to your computer If needed, install python, together with the python modules urllib2 and astropy. (You may need to ask computing staff for assistance with installing these.) For unix/linux, run the script using > python tapquery.py
Notes For this script the astropy module is used to write the output to a VO-format table. Here is a similar script that can be run without astropy tapquery_csv.py. This version writes the output to a csv file. The sample script can be modified to connect to different catalogues and to carry out different ADQL commands. For additional documentation see the internal comments in the script.

3.9 Plot positions of radio detections on an optical image covering the sky region using TOPCAT (version 4.2) with Aladin (version 9)

If you don't already have this, download the Aladin desktop application. To do this:
Go to Aladin desktop and follow the instructions to download the Aladin.exe file

Open Aladin, enter a location (for this catalogue try 22:45:00 -60:00:00) and click on the link to the Digital Sky Survey (DSS).

If you are not familiar with using Aladin, take a moment to try out some of the Aladin options that allow you to navigate around the DSS and use zoom controls etc. For example, you can use the mouse wheel to zoom in and out. Hold the mouse down to pan around the sky.

Open TOPCAT

As described in section 3.2, find the CASDA table casda.continuum-island and add this to your local table list.
In the TOPCAT Main Window, select the table.

Select the Broadcast Table icon (3rd from left in Main Window). This will transmit the CASDA table data to Aladin. You may need to wait a moment for the information to be tranferred.

Now switch to Aladin to see the radio catalogue positions plotted on top of the DSS images.

Clicking on a catalogue position on the Aladin plot will list the catalogue parameters below the plot.

To see parameters for a region of sky hold the cursor down to select a region.

Notes
Aladin is an interactive sky atlas from the Centre de Donneées Astronomique de Strasbourg (CDS).
Note the other surveys provided by Aladin Desktop. The same approach can be used with any of these.

3.9 Plot positions of radio detections on an optical image covering the sky region using TOPCAT (version 4.2) with Aladin (version 9)
If you don't already have this, download the Aladin desktop application. To do this: Go to Aladin desktop and follow the instructions to download the Aladin.exe file Open Aladin, enter a location (for this catalogue try 22:45:00 -60:00:00) and click on the link to the Digital Sky Survey (DSS). If you are not familiar with using Aladin, take a moment to try out some of the Aladin options that allow you to navigate around the DSS and use zoom controls etc. For example, you can use the mouse wheel to zoom in and out. Hold the mouse down to pan around the sky. Open TOPCAT As described in section 3.2, find the CASDA table casda.continuum-island and add this to your local table list. In the TOPCAT Main Window, select the table. Select the Broadcast Table icon (3rd from left in Main Window). This will transmit the CASDA table data to Aladin. You may need to wait a moment for the information to be tranferred. Now switch to Aladin to see the radio catalogue positions plotted on top of the DSS images. Clicking on a catalogue position on the Aladin plot will list the catalogue parameters below the plot. To see parameters for a region of sky hold the cursor down to select a region.
Notes Aladin is an interactive sky atlas from the Centre de Donneées Astronomique de Strasbourg (CDS). Note the other surveys provided by Aladin Desktop. The same approach can be used with any of these.

3.10 Cross-match information from a CASDA catalogue with a catalogue obtained VizieR and generate a merged catalogue

Open TOPCAT

Select VO - TAP Query and find the table casda.continuum_island (see section 3.2).

In the query box enter the ADQL query:
SELECT TOP 1000 * FROM casda.continuum_island and run the query.

In the TOPCAT main window: Select VO > VizieR Catalogue Service

Enter the following criteria:

Go to the 'Row Selection' part of the screen.
Next to the 'Object Name' enter 'Tucana' then click on 'Resolve'

This will add in the RA and Dec for the Tucana region.
Set a radius = 10 degrees
Use the default Maximum Row Count: 50,000
Now go to the 'Catalogues Selection' and select the 'Surveys' tab
Select PGC – HYPERLEDA Catalog of galaxies and click OK. This will add a second tale to the Main Window.
In the TOPCAT Main Window: Select Joins > Pair Match

Enter criteria and set up the match. For example:

Max Error: 1 arcmin
Table 1: TAP_1_casda_continuum_island
RA Column: ra_deg_cont
Dec Column: dec_deg_cont
Table 2: VII_237_pgc
RA column: _RAJ2000
Dec Column: _DEJ2000

Click Go. This will create a third table using the matched results.

Select 'Plot Results' to see a sky plot with the two surveys and the matched results.

A matched table is also produced with the combined results. Use TOPCAT to save this if desired.

Notes
TOPCAT provides several options for joining tales and matching positions. For full details please see TOPCAT documentation.

3.10 Cross-match information from a CASDA catalogue with a catalogue obtained VizieR and generate a merged catalogue
Open TOPCAT Select VO - TAP Query and find the table casda.continuum_island (see section 3.2). In the query box enter the ADQL query: SELECT TOP 1000 * FROM casda.continuum_island and run the query. In the TOPCAT main window: Select VO > VizieR Catalogue Service Enter the following criteria: Go to the 'Row Selection' part of the screen. Next to the 'Object Name' enter 'Tucana' then click on 'Resolve' This will add in the RA and Dec for the Tucana region. Set a radius = 10 degrees Use the default Maximum Row Count: 50,000 Now go to the 'Catalogues Selection' and select the 'Surveys' tab Select PGC – HYPERLEDA Catalog of galaxies and click OK. This will add a second tale to the Main Window. In the TOPCAT Main Window: Select Joins > Pair Match Enter criteria and set up the match. For example: Max Error: 1 arcmin Table 1: TAP_1_casda_continuum_island RA Column: ra_deg_cont Dec Column: dec_deg_cont Table 2: VII_237_pgc RA column: _RAJ2000 Dec Column: _DEJ2000 Click Go. This will create a third table using the matched results. Select 'Plot Results' to see a sky plot with the two surveys and the matched results. A matched table is also produced with the combined results. Use TOPCAT to save this if desired.
Notes TOPCAT provides several options for joining tales and matching positions. For full details please see TOPCAT documentation.

4 Using CASDA Virtual Observatory Services for Images and Image Cubes

For images and image cues, the Virtual Oservatory provides two protocols. The SIAP (Simple Image Access Protocol) is used to 'discover' relevant files, whilst the SODA (Serverside Operations for Data Access) Protocol provides the data access.

These services are primarily intended for use with scripts written by science teams using python or other scripting languages. This approach allows a high degree of customisation together with the aility to handle many files at a time.

These protocols may be somewhat difficult to use whilst some knowledge of python is needed to develop scripts. We are happy to help users get started and would welcome any feedback to assist us improve the features described in this section. Please send any comments to ATNF Data Support.

4.1 Find images and image cubes and see access services.

This item describes how to navigate to a list of images and image cubes services. When one image or cube is selected, several data access services are shown. Sections 4.2 and 4.3 provide examples of accessing files and generating cut-outs using python scripts.

Open https://casda.csiro.au/casda_vo_tools/sia2/query in your web browser. You will see a table with a list of all available CASDA images and image cubes.

The following eaxmples include some search filters in the URL:

Cone search with radius = 2 degrees: https://casda.csiro.au/casda_vo_tools/sia2/query?POS=CIRCLE 340.4567 -64.4194 2

Polygon/Box search: https://casda.csiro.au/casda_vo_tools/sia2/query?POS=POLYGON 339 -63 342 -63 342 -65 339 -65

Band search: https://casda.csiro.au/casda_vo_tools/sia2/query?BAND=0.25 0.30

Combined search: https://casda.csiro.au/casda_vo_tools/sia2/query?BAND=0.25 0.30&POS=CIRCLE 340.4567 -64.4194 2

Click on a link to an image or image cube (given here as the access_url)

A new page is shown with two links. Select the second link and login using your OPAL account.

This will return a web page with a table showing five rows. These correspond to different actions that can be taken as follows:

Row What this does Notes

1 Direct file download Click on the link to download the file directly. This is a quick-access option that is available for filesizes below 50 GB and where the files are available in cache.

2 Scripted file access This allows highly customisable access full sized images or image cubes using user scripts. Files of any size can be downloaded (from either disk or tape). See section 4.2 for an example.

3 Scripted cut-out service This allows highly customisable access to 'cut-out' sections from images or image cubes using user scripts. Cut-outs can be obtained from 2-d images or 3-d image cubes. In both cases the first two axes must correspond to sky positions. See section 4.3 for an example.

4 Simple cut-out service Click on the link to bring up a simple form to use with a single cut-out.
Cut out images opr cubes have a square cross-section. For this form the term 'radius' corresponds to the distance from the image centre to the image edge along the x- or y- axis.

5 Access through the Data Access Portal This option will be discontinued.

Notes
The query string 'BAND = 0.25 0.30' refers to a wavelength range, with units given in metres.

4.2 Use the VO Simple Image Access Protocol for programmatic data access

Please note this item is documented here but has not yet been fully tested by the CASDA team.
In this example we provide a sample python script, siap.py, that can be used to download images and image cubes overlapping a particular sky region. Supported formats for right ascension are 22:58:04.88 22h58m4.88s and 344.52039 , and for declination -36:25:39.4 -36d25m39.4s and -36.42758
The script will query the CASDA SIAP service for images that overlap a 0.1 degree radius circle centred on the coordinates provided. All images and image cubes will then be downloaded in full.
To run this script:

Download the python script siap.py(text file) to your computer

Install python and astropy (if not already available)

Run the script using > python siap.py OPAL_username OPAL_password ra dec Destination_Directory

As an example, the following command will download HI images of the galaxy group IC1459. The output files are written into a folder 'output'. For a 0.1 radius, three images are produced.
>> python siap.py OPAL_username OPAL_password 344.52039 -36.42758 output

Notes
This script provides an example that can be customised for your own purposes. Extensive internal documentation is included.
To provide more detail, the script:

1. Creates the destination directory if provided.
2. Runs a SIA v2 query to retrieve a list of images and/or image cubes overlapping the sky location.
3. For each image/image cube it then:

Retrieves a datalink.

Retrieves the authorised id of the file

4. Creates an asynchronous task to access the images and/or image cubes
5. Adds the authorised id for each image and image cube.
6. Starts the task to access the images/image cubes.
7. Waits until the job has completed.
8. Downloads all images/image cubes.
To access files, the script uses OPAL authentication and this must be provided for all downloads. You will only be able to retrieve images which have been openly released or that you have access to as a member of a science team.
Please note that the OPAL_Password should be surrounded by single quotes on a mac or linux command line if there are any non-alphanumeric characters in the password.

4.2 Use the VO Simple Image Access Protocol for programmatic data access
Please note this item is documented here but has not yet been fully tested by the CASDA team. In this example we provide a sample python script, siap.py, that can be used to download images and image cubes overlapping a particular sky region. Supported formats for right ascension are 22:58:04.88 22h58m4.88s and 344.52039 , and for declination -36:25:39.4 -36d25m39.4s and -36.42758 The script will query the CASDA SIAP service for images that overlap a 0.1 degree radius circle centred on the coordinates provided. All images and image cubes will then be downloaded in full. To run this script: Download the python script siap.py(text file) to your computer Install python and astropy (if not already available) Run the script using > python siap.py OPAL_username OPAL_password ra dec Destination_Directory As an example, the following command will download HI images of the galaxy group IC1459. The output files are written into a folder 'output'. For a 0.1 radius, three images are produced. >> python siap.py OPAL_username OPAL_password 344.52039 -36.42758 output
Notes This script provides an example that can be customised for your own purposes. Extensive internal documentation is included. To provide more detail, the script: 1. Creates the destination directory if provided. 2. Runs a SIA v2 query to retrieve a list of images and/or image cubes overlapping the sky location. 3. For each image/image cube it then: Retrieves a datalink. Retrieves the authorised id of the file 4. Creates an asynchronous task to access the images and/or image cubes 5. Adds the authorised id for each image and image cube. 6. Starts the task to access the images/image cubes. 7. Waits until the job has completed. 8. Downloads all images/image cubes. To access files, the script uses OPAL authentication and this must be provided for all downloads. You will only be able to retrieve images which have been openly released or that you have access to as a member of a science team. Please note that the OPAL_Password should be surrounded by single quotes on a mac or linux command line if there are any non-alphanumeric characters in the password.

4.3 Generate cut outs from images and image cubes using a script

Please note this item is documented here but has not yet been fully tested by the CASDA team.
In this example we provide a sample python script, cutouts.py, that can be used to generate a set of cut outs obtained from the images and image cubes generated from observations associated with a scheduling block.
The script reads a VO-format catalogue, casda_continuum_component, that includes positions for radio continuum components. For each component with a peak flux density above 150 mJy it generates a small cut-out image from the associated image files where the detected position falss within the full-size image or cube.
The cut out images have sizes of 0.2 x 0.2 degrees in right ascension and declination and contain the same number of planes as is the full sized image or cubes. Thus a cut-out from a full spectral line image cube may have up to 16,200 channels along the third axis. A cut-out from a single plane image will also be a single plane image.
To run this script:

Download the python script cutouts.py (text file) to your computer
Install python and astropy (if not already available)
Run the script using > python cutouts.py OPAL_username OPAL_password scheduling_block_id Destination_Directory

As an example, the following command will generate cut outs from a radio continuum image of the Tucana region. The output files are written into a folder 'output'. For a flux-cut-off of 150 mJy, 89 cut-out images are produced.
>> python cutouts.py OPAL_username OPAL_password 609 output

Notes
This script provides an example that can be customised for your own purposes. Extensive internal documentation is included. Note that the script includes a switch to retrieve entire image cubes instead of cutouts.
To provide more detail, the script:

Creates the destination directory if provided.

Runs a TAP query to retrieve a list of images and/or image cubes associated with the scheduling block number.

For each image/image cube it then:

Retrieves a datalink.

Obtains a cut out URL and authenticated id for the image cube.

Create an asynchronous task to access the images and/or image cubes in the scheduling block.

Runs a second TAP query to retrieve the continuum catalogue components of interest.

Adds position criteria for each catalogue entry of interest.

Starts the task to generate a cut-out.

Waits until the job has completed.

Downloads all cutout images/image cubes.

To access files, the script uses OPAL authentication and this must be provided for all downloads. You will only be able to retrieve images which have been openly released or that you have access to as a member of a science team.
Please note that the OPAL_Password should be surrounded by single quotes on a mac or linux command line if there are any non-alphanumeric characters in the password.

4.3 Generate cut outs from images and image cubes using a script
Please note this item is documented here but has not yet been fully tested by the CASDA team. In this example we provide a sample python script, cutouts.py, that can be used to generate a set of cut outs obtained from the images and image cubes generated from observations associated with a scheduling block. The script reads a VO-format catalogue, casda_continuum_component, that includes positions for radio continuum components. For each component with a peak flux density above 150 mJy it generates a small cut-out image from the associated image files where the detected position falss within the full-size image or cube. The cut out images have sizes of 0.2 x 0.2 degrees in right ascension and declination and contain the same number of planes as is the full sized image or cubes. Thus a cut-out from a full spectral line image cube may have up to 16,200 channels along the third axis. A cut-out from a single plane image will also be a single plane image. To run this script: Download the python script cutouts.py (text file) to your computer Install python and astropy (if not already available) Run the script using > python cutouts.py OPAL_username OPAL_password scheduling_block_id Destination_Directory As an example, the following command will generate cut outs from a radio continuum image of the Tucana region. The output files are written into a folder 'output'. For a flux-cut-off of 150 mJy, 89 cut-out images are produced. >> python cutouts.py OPAL_username OPAL_password 609 output
Notes This script provides an example that can be customised for your own purposes. Extensive internal documentation is included. Note that the script includes a switch to retrieve entire image cubes instead of cutouts. To provide more detail, the script: Creates the destination directory if provided. Runs a TAP query to retrieve a list of images and/or image cubes associated with the scheduling block number. For each image/image cube it then: Retrieves a datalink. Obtains a cut out URL and authenticated id for the image cube. Create an asynchronous task to access the images and/or image cubes in the scheduling block. Runs a second TAP query to retrieve the continuum catalogue components of interest. Adds position criteria for each catalogue entry of interest. Starts the task to generate a cut-out. Waits until the job has completed. Downloads all cutout images/image cubes. To access files, the script uses OPAL authentication and this must be provided for all downloads. You will only be able to retrieve images which have been openly released or that you have access to as a member of a science team. Please note that the OPAL_Password should be surrounded by single quotes on a mac or linux command line if there are any non-alphanumeric characters in the password.

5 Restricted CASDA Tools

5.1 Download data to a location in the Pawsey Supercomputing Centre (Pawsey user account required)

In this example CASDA is used to find a set of data files to download. The user then logs into the Pawsey Supercomputing Centre using a Pawsey account. CASDA returns one or more links. These are copied into a .txt file and the files are retrieved to a location in the Pawsey Supercomputing Centre using unix commands.

The following steps are carried out from a Nexus or OPAL account:

Go to the DAP Home Page and login with your OPAL or Nexus account.

Click on the Domain Search and select ‘CASDA Observation Search’ to bring up the search form.

Enter your selection criteria and click on the search button to bring up a list of results.

Select the required products for downloading using the checkboxes on the left of the table.

At the bottom of the page, select ‘Access data with Pawsey HPC account’.

Open and read the link to 'this statement'.
Click the check box ‘To access data at Pawsey, I accept this statement.’

Click the ‘Retrieve selected data’.
You will now be presented with a ‘Retrieve Files’ page showing links to files. This page is updated automatically once a minute. The links are only accessible from Pawsey accounts.

So you do not lose the links provided it is advisable to bookmark this webpage or copy the page link to a file.

Wait until the status indicator goes green. Your data files are then ready for retrieval’.

The following steps use a Pawsey account:

Login to Pawsey using your Pawsey account details, for example with the command 'ssh username@galaxy.pawsey.org.au'.

Navigate to the desired path. For the Galaxy supercomputer the path will be similar to: > 'cd /scratch2/askap/username/foldername'.

To transfer a single file , the command 'wget link' can be used, where 'link' is the link provided for the file.

Alternatively to transfer many files:
Use a text editor (such as vi) to create and open a text file with a name such as 'linknames.txt'.

Cut and paste the links for your files into this text file using one row per link.

Save and close the file.

Use the command: 'xargs -i wget -c '{}' < links.txt' to copy all of the files across.

Notes
See section 1.6 for information on Pawsey accounts.
For Windows users: A handy hint to cut and paste text between files in different locations is to i) highlight the text to copy, ii) use cntrl-C to copy, iii) then right-click in the text file to insert.

5.1 Download data to a location in the Pawsey Supercomputing Centre (Pawsey user account required)
In this example CASDA is used to find a set of data files to download. The user then logs into the Pawsey Supercomputing Centre using a Pawsey account. CASDA returns one or more links. These are copied into a .txt file and the files are retrieved to a location in the Pawsey Supercomputing Centre using unix commands. The following steps are carried out from a Nexus or OPAL account: Go to the DAP Home Page and login with your OPAL or Nexus account. Click on the Domain Search and select ‘CASDA Observation Search’ to bring up the search form. Enter your selection criteria and click on the search button to bring up a list of results. Select the required products for downloading using the checkboxes on the left of the table. At the bottom of the page, select ‘Access data with Pawsey HPC account’. Open and read the link to 'this statement'. Click the check box ‘To access data at Pawsey, I accept this statement.’ Click the ‘Retrieve selected data’. You will now be presented with a ‘Retrieve Files’ page showing links to files. This page is updated automatically once a minute. The links are only accessible from Pawsey accounts. So you do not lose the links provided it is advisable to bookmark this webpage or copy the page link to a file. Wait until the status indicator goes green. Your data files are then ready for retrieval’. The following steps use a Pawsey account: Login to Pawsey using your Pawsey account details, for example with the command 'ssh username@galaxy.pawsey.org.au'. Navigate to the desired path. For the Galaxy supercomputer the path will be similar to: > 'cd /scratch2/askap/username/foldername'. To transfer a single file , the command 'wget link' can be used, where 'link' is the link provided for the file. Alternatively to transfer many files: Use a text editor (such as vi) to create and open a text file with a name such as 'linknames.txt'. Cut and paste the links for your files into this text file using one row per link. Save and close the file. Use the command: 'xargs -i wget -c '{}' < links.txt' to copy all of the files across.
Notes See section 1.6 for information on Pawsey accounts. For Windows users: A handy hint to cut and paste text between files in different locations is to i) highlight the text to copy, ii) use cntrl-C to copy, iii) then right-click in the text file to insert.

5.2 Set data validation flags and information (Survey Science Team members with validator permissions )

Data validation is carried out using the CSIRO DAP or Survey Science Projects by members of the Survey Science teams. This task can only be carried out by individuals with validation-level access to the data products.
The validation process involves setting a flag in the data base and adding additional information. Validation is applied at the level of individual data products. So for a set of images created for a project - a validation flag is set for each image. Following validation data products are 'released'. The steps below describe the process:

Go to the DAP Home Page and login with either an OPAL (recommended) or CSIRO NEXUS account

Click on Domain Search and select 'CASDA Observation Search' to bring up the search form.

Enter any selection criteria to restrict the search. For example Project code = 'AS031' will select data products for BETA science observations.

At the bottom of the page select "unreleased data" and then click on Search.

This will now bring up a list of any unreleased data products.
Click on the 'My Admin' tab that you will see near the top-right corner of the page.
Click on the title for the scheduling block of interest.
On the form select any of the indicators that you think are relevant.
Now click on the tab for 'Notes' and add any additional comments (if you wish).
Return to the validation tab, check the details, then click 'Submit'. This will change the validation status to allow the data product to be released.

Notes

For help with the validation process please contact the helpdesk: atnf-datasup@csiro.au.

5.2 Set data validation flags and information (Survey Science Team members with validator permissions )
Data validation is carried out using the CSIRO DAP or Survey Science Projects by members of the Survey Science teams. This task can only be carried out by individuals with validation-level access to the data products. The validation process involves setting a flag in the data base and adding additional information. Validation is applied at the level of individual data products. So for a set of images created for a project - a validation flag is set for each image. Following validation data products are 'released'. The steps below describe the process: Go to the DAP Home Page and login with either an OPAL (recommended) or CSIRO NEXUS account Click on Domain Search and select 'CASDA Observation Search' to bring up the search form. Enter any selection criteria to restrict the search. For example Project code = 'AS031' will select data products for BETA science observations. At the bottom of the page select "unreleased data" and then click on Search. This will now bring up a list of any unreleased data products. Click on the 'My Admin' tab that you will see near the top-right corner of the page. Click on the title for the scheduling block of interest. On the form select any of the indicators that you think are relevant. Now click on the tab for 'Notes' and add any additional comments (if you wish). Return to the validation tab, check the details, then click 'Submit'. This will change the validation status to allow the data product to be released.
Notes For help with the validation process please contact the helpdesk: atnf-datasup@csiro.au.

5.3 See the list of individuals assigned to a science team (CASDA Administrator Task)

Go to DAP Home Page and login with NEXUS account.

Click on the tab for 'My Admin' and then select the tab for 'Data Admin'.

Click on the link to 'Manage CASDA Team Members'.

In the section under 'Manage existing team members' select a project from the drop down list. This will bring up a list of project team members and their current project role.

Notes
In a later CASDA release, this task will later be made available to project-level adminstrators.

5.3 See the list of individuals assigned to a science team (CASDA Administrator Task)
Go to DAP Home Page and login with NEXUS account. Click on the tab for 'My Admin' and then select the tab for 'Data Admin'. Click on the link to 'Manage CASDA Team Members'. In the section under 'Manage existing team members' select a project from the drop down list. This will bring up a list of project team members and their current project role.
Notes In a later CASDA release, this task will later be made available to project-level adminstrators.

5.4 Manage roles for team members (CASDA Administrator Task)

Follow the steps in section 5.3 to bring up the DAP page for managing team members.

In the section for managing team members, select the project.

This will bring up a list of individuals that are recognised as team members in CASDA.

Update this list as needed to assign project roles and/or remove members.

Click on the 'Update team members' to complete.

Notes
In a later CASDA release, this task will also be made available to project-level administrators.

5.4 Manage roles for team members (CASDA Administrator Task)
Follow the steps in section 5.3 to bring up the DAP page for managing team members. In the section for managing team members, select the project. This will bring up a list of individuals that are recognised as team members in CASDA. Update this list as needed to assign project roles and/or remove members. Click on the 'Update team members' to complete.
Notes In a later CASDA release, this task will also be made available to project-level administrators.

5.5 Add an individual to a science team (CASDA Administrator Task)

Follow the steps in section 5.4 to bring up the DAP page for managing team members.

In the section for adding new members, start typing a surname for the person of interest. A list of matching names will appear.

Click on a name to select the required individual.

Select the appropriate project.

Select the required role.

Click 'Add Team Member' to complete.

Notes
In a later CASDA release, this task will later be made available to project-level adminstrators.
As a starting point, CASDA generates an list of team members from the PI and co-authors on the associated OPAL project proposals. This list is then managed through CASDA.

5.5 Add an individual to a science team (CASDA Administrator Task)
Follow the steps in section 5.4 to bring up the DAP page for managing team members. In the section for adding new members, start typing a surname for the person of interest. A list of matching names will appear. Click on a name to select the required individual. Select the appropriate project. Select the required role. Click 'Add Team Member' to complete.
Notes In a later CASDA release, this task will later be made available to project-level adminstrators. As a starting point, CASDA generates an list of team members from the PI and co-authors on the associated OPAL project proposals. This list is then managed through CASDA.

5.6 Publish a Science Team catalogue: General advice (for Survey Science Teams)

For some ASKAP surveys, science teams will generate catalogues with final science results. These are referred to here as 'Science Team catalogues'. As an example, a catalogue might contain a set of polarisation properties for a list of detected source. Another catalogue could contain properties dervied from HI spectra for nearby galaxies, together with optical identifications. Compiling such catalogues is the responsibility of Science Teams.
CASDA provides a service so that Science Team catalogues can be published as part of the ASKAP science archive. If your team is interested in publishing level 7 catalogues please note the following advice.

Catalogues must be provided in a VO-compliant .xml table format.

We strongly recommend that before submitting a catalogue for publication you consult with CASS staff for advice on generating VO-compliant tables. CASS staff can help you ensure your tables are VO compliant and can check whether your table is formatted correctly.

By publishing a catalogue a Science Team gives approval for the Catalogue to be made openly available through CASDA.

Once a catalogue is published it cannot be removed from the archive, or corrected for minor errors. However later versions can be provided, for example as data release 1, data release 2 etc.

Notes
CASDA catalogues are published together with digitial object identifiers (DOIs). These can be cited in journal publications.

5.6 Publish a Science Team catalogue: General advice (for Survey Science Teams)
For some ASKAP surveys, science teams will generate catalogues with final science results. These are referred to here as 'Science Team catalogues'. As an example, a catalogue might contain a set of polarisation properties for a list of detected source. Another catalogue could contain properties dervied from HI spectra for nearby galaxies, together with optical identifications. Compiling such catalogues is the responsibility of Science Teams. CASDA provides a service so that Science Team catalogues can be published as part of the ASKAP science archive. If your team is interested in publishing level 7 catalogues please note the following advice. Catalogues must be provided in a VO-compliant .xml table format. We strongly recommend that before submitting a catalogue for publication you consult with CASS staff for advice on generating VO-compliant tables. CASS staff can help you ensure your tables are VO compliant and can check whether your table is formatted correctly. By publishing a catalogue a Science Team gives approval for the Catalogue to be made openly available through CASDA. Once a catalogue is published it cannot be removed from the archive, or corrected for minor errors. However later versions can be provided, for example as data release 1, data release 2 etc.
Notes CASDA catalogues are published together with digitial object identifiers (DOIs). These can be cited in journal publications.

5.7 Submit a catalogue for CASDA publication (Nexus account required)

Go to the DAP Home Page and login using a Nexus account.

Click on the 'Deposit Data' button at the top of the page.

Read the deposit checklist and then click on the tick box next to this.

Under 'Select Deposit Method' select the option for 'ASKAP Science Team Catalogue Self Service Deposit'.

Click 'Start'.

Enter the ATNF Project Code and select 'Create Draft Collection'.

You will now see four different tabs for 'Description', 'Citation', 'Data' and 'Restrictions'.
Note that some information is pre-loaded using information provided on an associated OPAL proposal record. This can be edited if you wish.

Look carefully at the information on each tab and edit or add to these as needed. In particular:
For the description tab - you will need to add at least one 'Collaborating Organisation' and a 'Data Owner'.
Use the citations tab to modify the list of contributors.
Use the data tab to upload your VO catalogue.

At any time you can save your entries as a draft. When all entires are complete select 'Publish'

You will now be asked to select from a list of 'approvers'. Choose one of the options and then click on 'Submit'.

Your catalogue will now be parsed through some software to check for VO compliance and the approver will make sure it is an appropriate catalogue for CASDA.

You will be advised if there are any problems with the catalogue. Otherwise this will proceed to publication.

To see the status of collections that you have submitted, go to the DAP Home Page and select the 'Manage My Data' button at the top of the page. This will show your collections together with their current status.
Once published, your catalogue will be available using the CASDA web and VO search tools.

Notes
Improvements to this task are planned. In later releases the above steps should be easier to use. For now, you may need some assistance for catalogue deposits. Please see the help tips (yellow question marks) and/or contact ATNF data support.

5.7 Submit a catalogue for CASDA publication (Nexus account required)
Go to the DAP Home Page and login using a Nexus account. Click on the 'Deposit Data' button at the top of the page. Read the deposit checklist and then click on the tick box next to this. Under 'Select Deposit Method' select the option for 'ASKAP Science Team Catalogue Self Service Deposit'. Click 'Start'. Enter the ATNF Project Code and select 'Create Draft Collection'. You will now see four different tabs for 'Description', 'Citation', 'Data' and 'Restrictions'. Note that some information is pre-loaded using information provided on an associated OPAL proposal record. This can be edited if you wish. Look carefully at the information on each tab and edit or add to these as needed. In particular: For the description tab - you will need to add at least one 'Collaborating Organisation' and a 'Data Owner'. Use the citations tab to modify the list of contributors. Use the data tab to upload your VO catalogue. At any time you can save your entries as a draft. When all entires are complete select 'Publish' You will now be asked to select from a list of 'approvers'. Choose one of the options and then click on 'Submit'. Your catalogue will now be parsed through some software to check for VO compliance and the approver will make sure it is an appropriate catalogue for CASDA. You will be advised if there are any problems with the catalogue. Otherwise this will proceed to publication. To see the status of collections that you have submitted, go to the DAP Home Page and select the 'Manage My Data' button at the top of the page. This will show your collections together with their current status. Once published, your catalogue will be available using the CASDA web and VO search tools.
Notes Improvements to this task are planned. In later releases the above steps should be easier to use. For now, you may need some assistance for catalogue deposits. Please see the help tips (yellow question marks) and/or contact ATNF data support.

5.8 Approve a Science Team catalogue (Approvers only)

Login to the DAP using an appropriate account

Select 'My Admin' and then 'My Tasks'

For items with a task type = 'Collection Approval', click on the title to open and view the collection

Use the decision tab to enter your decision together with any comments.

Submit your decision

Notes

5.8 Approve a Science Team catalogue (Approvers only)
Login to the DAP using an appropriate account Select 'My Admin' and then 'My Tasks' For items with a task type = 'Collection Approval', click on the title to open and view the collection Use the decision tab to enter your decision together with any comments. Submit your decision
Notes

Author	Version	Date	TOPCAT Version	Notes
J Chapman	1.0	5 November 2015	4.2	Initial release
J Chapman	1.1	24 March 2016	4.3	Updates and new content in sections 1 to 5.

CASDA Users Guide

Contents