The AstroStat Slog

Generally, astronomical data archives are open to public. Also, astronomy has been the leading force of developing software and hardware to handle massive data, which nowadays receive spotlights from statistics. Although the astronomical data look easy to be accessed for some statistical challenges, compared to data sets of other disciplines, statistical applications on astronomical data are unlikely to be found. What is the cause of this long engagement period?

The data format…so called FITS.

Yet, ASCII format catalogues are available (for example, VizieR, http://vizier.u-strasbg.fr/viz-bin/VizieR) . Catalogues or data from VizieR are trimmed to enhance astrophysical interests. Many data sets contain only a few dozens of stars.

Therefore, for an interesting statistical research, one has to dig in raw data sets, which in astronomy are usually stored in FITS format. It seems like that not many statisticians are aware of this particular type of data format. Let’s defer
a discussion on FITS (need a help from astronomers). I only like to comment that there are ways for statisticians to access FITS format data.

1/ Bother astronomers. They will help you.

2/ R or SAS does not have FITS reader but there are free tools available. Here, I attach an email from 2006 Astrostatics Summer School at PSU, who kindly answer to my question on how to read files in FITS format.

—————————————from Patrick
The FTOOLS package I mentioned today is available at NASA-Goddard via
http://heasarc.gsfc.nasa.gov/ftools/ftools_menu.html
The tool is fdump. It’s manual page is available via fhelp fdump

A more modern and perhaps better package of FITS tools is contained in the data analysis package named CIAO, built for the Chandra mission (the NASA satellite Eric and I work with). It’s available via
http://asc.harvard.edu/ciao/download/
(You do NOT need the separate “calibration database” (CALDB) the you’ll see mentioned.) The tool you want in this package is called dmlist, with a syntax like this to dump the data to an ASCII file:
dmlist mydata.fits opt=data,clean outfile=mydata.txt

To see the column names for a table use:
dmlist mydata.fits cols

To see the keywords in the header of the FITS file use
dmlist mydata.fits header

FITS files can sometimes contain multiple data tables. To see the structure of your FITS file use this:
dmlist mydata.fits blocks

Anytime you specify a FITS filename you can optionally specify a block number, e.g.:
dmlist òÀÜmydata.fits[4]òÀÝ opt=data,clean outfile=mydata_part4.txt

You can even specify specific columns to dump, e.g.
dmlist òÀÜmydata.fits[4][cols x,y]òÀÝ opt=data,clean outfile=mydata_part4.txt

The manual page for dmlist is at http://asc.harvard.edu/ciao/ahelp/
dmlist.html or via
ahelp dmlist

If you ever need to look at images in FITS format the gold standard application is called “ds9″, included in the CIAO package.

—————————————————————

The 1st strategy looks easy and urges collaborations between astronomers and statisticians. On the other hand, astronomically knowledgeable statisticians may work on their own to verify their theories purely for statistical interests and eventually satisfy astrophysics. Overall, don’t be panic with this strange data format.

**Note that this email directs CIAO package on which CHANDRA X-ray data are processed.