Äîêóìåíò âçÿò èç êýøà ïîèñêîâîé ìàøèíû. Àäðåñ îðèãèíàëüíîãî äîêóìåíòà : http://xmm.vilspa.esa.es/sas/6.5.0/documentation/xmm_abc_guide.pdf Äàòà èçìåíåíèÿ: Tue Aug 16 18:05:23 2005 Äàòà èíäåêñèðîâàíèÿ: Sat Dec 22 02:44:37 2007 Êîäèðîâêà: Ïîèñêîâûå ñëîâà: ÷åþîùê ëáìåîäáòø

THE XMM-NEWTON ABC GUIDE AN INTRODUCTION TO XMM-NEWTON DATA ANALYSIS
NASA/GSFC XMM-Newton Guest Observer Facility Steve Snowden, Stefan Immler, Michael Arida, Brendan Perry, Martin Still, and Ilana Harrus

Version 2.01
23 July 2004

Copies of this guide are available in html, postscript and pdf formats.


Contents
1 Intro duction 1.1 ACKNOWLEDGMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Us 2.1 2.2 2.3 2.4 2.5 2.6 2.7 eful Information and References MAIN WEB SITES . . . . . . . . . . . . . . . . . XMM-NEWTON HELP DESKS . . . . . . . . . . MISSION PLANNING AND SPACECRAFT STAT PUBLIC DATA ARCHIVES . . . . . . . . . . . . CALIBRATION DATA . . . . . . . . . . . . . . . SOFTWARE . . . . . . . . . . . . . . . . . . . . . ANALYSIS, DOCUMENTATION AND HELPFUL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . T . . . . . . . . . . . . . . . P . . . . . . . . . . . . . . . . . . . . . . ..... ..... US . . . ..... ..... ..... HINTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . O . . . . . . . . . . . . . . . N . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1 2 2 2 2 3 3 3 3 5 5 5 6 6 6 8 8 8 8 9 9 9 10 10 10 10 11 12 12 13 13 14 16 19 19 21 21 24 26 27

3 Data 3.1 USEFUL DOCUMENTATION 3.2 THE DATA . . . . . . . . . . . 3.3 PI Data . . . . . . . . . . . . . 3.3.1 ODF Data . . . . . . . 3.3.2 Pipeline Product Data . 4 Setting Up and Running SAS 4.1 INSTALLATION . . . . . . . 4.2 CALIBRATION DATA . . . 4.3 SAS INVOCATION . . . . . 4.3.1 SAS Helpful Hints . . 4.4 SAS SYNTAX AND LOGIC 4.4.1 Command Line Syntax 4.4.2 Table Syntax . . . . . 4.4.3 Filtering Logic . . . . 4.5 GENERAL SAS TASKS FOR 4.5.1 cifbuild . . . . . . . . 4.5.2 odfingest . . . . . . .

.... .... .... .... .... .... .... .... DATA .... ....

.. .. .. .. .. .. .. .. SE .. ..

.. .. .. .. .. .. .. .. RE .. ..

..... ..... ..... ..... ..... ..... ..... ..... PARATI ..... .....

5 First Lo ok ­ An EPIC Data Primer 5.1 USING PIPELINE PROCESSED DATA PRODUCTS 5.1.1 A Quick Look at What You Have . . . . . . . . 5.2 EXAMINE AND FILTER THE DATA - PIPELINE P 5.2.1 Initialize SAS and Prepare the Data . . . . . . 5.2.2 Create and Display an Image . . . . . . . . . . 5.2.3 Create and Display a Light Curve . . . . . . . 5.2.4 Filter the Data and Create a New Event File . 5.3 EXTRACT AND FIT A SOURCE SPECTRUM . . . 5.3.1 Extract the Spectrum . . . . . . . . . . . . . . 5.3.2 Create RMFs and ARFs . . . . . . . . . . . . . 5.3.3 Prepare the Spectrum . . . . . . . . . . . . . . 5.3.4 Fit the Spectra . . . . . . . . . . . . . . . . . . i

...... ...... RODUCTS ...... ...... ...... ...... ...... ...... ...... ...... ......

... ... .. ... ... ... ... ... ... ... ... ...


ii 5 5 5 5 .4 .5 .6 .7 SOURCE DETECTION . . . . . . TIMING ANALYSIS . . . . . . . . ONCE MORE, THIS TIME WITH ODF DATA . . . . . . . . . . . . . 5.7.1 Rerunning the EPIC Chains A More-or-Less Complete Example .. .. FE .. .. .. . . E . . . . . L . . . .. .. ING .. .. .. .... .... AND .... .... .... . . . . . . . . . . . . . . . . . . . . . ce . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. .. FT .. .. .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . LS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ... ... .. ... ... ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 3 3 3 3 3 7 0 2 5 5 6

5.8

6 First Lo ok ­ RGS Data 6.1 A PRELIMINARY FIT . . . . . . . . . . . . . 6.1.1 Pipeline Products . . . . . . . . . . . . 6.1.2 Preparation for Running SAS Tasks . . 6.1.3 Creating Response Matrices . . . . . . . 6.1.4 Fitting a Spectral Model to the Data . . 6.2 FILTERING EVENTS . . . . . . . . . . . . . . 6.2.1 Creating and plotting a light curve . . . 6.2.2 Creating a GTI file . . . . . . . . . . . . 6.2.3 Running the RGS Pipeline . . . . . . . 6.2.4 Inspecting New Products . . . . . . . . 6.3 PIPELINE EXAMPLES . . . . . . . . . . . . . 6.3.1 A Nearby Bright Optical Source . . . . 6.3.2 A Nearby Bright X-ray Source . . . . . 6.3.3 User-defined Source Coordinates . . . . 6.4 PIPELINE ENTRY STAGES . . . . . . . . . . 6.5 COMBINING RGS1 AND RGS2 SPECTRA . 6.6 APPROACHES TO SPECTRAL FITTING . . 6.6.1 Spectral Rebinning . . . . . . . . . . . . 6.6.2 Maximum-Likelihood Statistics . . . . . 6.7 ANALYSIS OF EXTENDED SOURCES . . . 6.7.1 Region masks for extended sources . . . 6.7.2 Fitting spectral models to extended sour 6.7.3 Model limitations . . . . . . . . . . . . . 6.8 A MORE-OR-LESS COMPLETE EXAMPLE 7 First Lo ok ­ OM Data 7.1 PIPELINE PRODUCTS 7.1.1 Imaging Mode . 7.1.2 Fast Mode . . . . 7.1.3 Grism Mode . . 7.2 RE-PROSESSING OF O 7.2.1 Imaging Mode . 7.2.2 Fast Mode . . . . 7.2.3 Grism Analysis . .. .. .. .. DF .. .. .. ... ... ... ... FILE ... ... ... . . . . S . . . . . . . . .. .. .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

38 38 38 39 39 40 41 41 42 42 43 46 47 47 48 49 49 50 50 51 51 51 51 52 53 54 54 54 56 56 57 57 64 64


List of Tables
1 3.1 5.1 5.2 6.1 6.2 7.1 7.2 7.3 List of Acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pipeline Processing data files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv 7

EPIC Pipeline Processing data files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 EPIC ODF data files1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 RGS Pipeline Processing data files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 rgsproc output data files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 OM Pipeline Processing data files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Some of the important columns in the SWSRLI FITS file. . . . . . . . . . . . . . . . . . . . . . . 55 OM filter and file name correspondence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

iii


iv

Table 1: List of Acronyms ARF CAL CCD CCF C IF E P IC FITS GO GOF GSFC GUI HEASARC HTML OAL ODF OM PDF PP PPS PV RGS RMF SAS SOC SSC SV XMM Ancillary Region File Calibration Access Layer Charge Coupled Device Current Calibration File Calibration Index File European Photon Imaging Camera Flexible Image Transport System Guest Observer NASA/GSFC Guest Observer Facility Goddard Space Flight Center Graphical User Interface High Energy Astrophysics Science Archive Research Center Hyper Text Markup Language ODF Access Layer Observation Data File Optical Monitor Portable Data Format Pipeline Processing System Pipeline Processing Performance Validation Reflection Grating Spectrometer Redistribution Matrix File Science Analysis System Science Operations Center Survey Science Centre Science Validation X-ray Multi-Mirror Mission


Chapter 1

Introduction
The purpose of this ABC Guide to XMM-Newton data analysis is to provide a simple walk-through of basic data extraction and analysis tasks. Also included is a guide to references and help available to aid in the analysis of the data. We have tried to balance providing enough information to give the user a useful introduction to a variety of analysis tasks with not providing too much information, which would make a guide like this too ponderous to use. As such, there is no intention to replace the SAS Handbook, which should be considered the highest authority for the use of SAS. Therefore this document will not display the full versatility of the SAS tasks, and of SAS itself, but it will hopefully show a path through the forest. Chapter 2 provides lists of web-based references for the XMM-Newton pro ject, help desks, analysis guides, and science and calibration data. Chapter 3 provides a description of the data files provided for observation data sets. Chapter 4 discusses the installation and use of SAS. Chapters 5, 6, and 7 discuss the analysis of EPIC, RGS, and OM data respectively. This document will continue to evolve. Updated versions will be made available on our web site at: http://heasarc.gsfc.nasa.gov/docs/xmm/abc/

1.1

ACKNOWLEDGMENTS
the XMMcomments Guainazzi into EPIC

This guide would not have been possible without the help and comments from all people involved in Newton pro ject. In particular, we would like to thank Giuseppe Vacanti and Julian Osborne whose made this a more complete and accurate document. IMH wishes to thank all the OM calibration team and in particular Antonio Talavera, Matteo and Bing Chen for their help in the preparation of this and other documents related to the OM. SLS wishes to thank Dave Lumb, Richard Saxton, and Steve Sembay for their helpful insights data analysis.

1


Chapter 2

Useful Information and References
2.1 MAIN WEB SITES
http://xmm.vilspa.esa.es/ · NASA/GSFC GOF, source of US specific information and a mirror site for software and public data access: http://xmm.gsfc.nasa.gov/ · PPARC Information Center, source of UK specific information and mirror site for some software and data a cces s : http://www.xmm.ac.uk/ · Survey Science Centre http://xmmssc-www.star.le.ac.uk/

· XMM-Newton SOC, fount of all XMM-Newton pro ject information:

2.2

XMM-NEWTON HELP DESKS
http://xmm.vilspa.esa.es/external/xmm user support/helpdesk.shtml or via e-mail: xmmhelp@xmm.vilspa.esa.es The helpdesk also provides an archive of previously asked questions.

· The main pro ject helpdesk is located at Vilspa and can be accessed through the WWW:

· The NASA/GSFC GOF offers an e-mail helpdesk for both general support and for US-specific issues: xmmhelp@athena.gsfc.nasa.gov Some questions addressed to the NASA/GSFC GOF may be redirected to the Vilspa helpdesk.

2.3

MISSION PLANNING AND SPACECRAFT STATUS
http://xmm.vilspa.esa.es/external/xmm obs info/obs stat log.shtml The scheduling information from this data base has been extracted and incorporated into a Browse data base at GSFC: http://heasarc.gsfc.nasa.gov/db-perl/W3Browse/w3browse.pl 2

· Observation Log:


3 · Long-Term Timeline: http://xmm.vilspa.esa.es/external/xmm sched/advance plan.shtml

2.4

PUBLIC DATA ARCHIVES
http://xmm.vilspa.esa.es/external/xmm data acc/xsa/index.shtml

· SOC Public Data Archive via the XSA:

· GSFC Archive Mirror Site via Browse: http://heasarc.gsfc.nasa.gov/db-perl/W3Browse/w3browse.pl

2.5

CALIBRATION DATA

· XMM-Newton Calibration Page. Under this page can be found the Current Calibration File (CCF) archive, release notes for CCF updates, EPIC response and background files (top menu), and calibration information. http://xmm.vilspa.esa.es/external/xmm sw cal/calib/index.shtml · Caldb, NASA/GSFC GOF mirror site for canned response files: ftp://legacy.gsfc.nasa.gov/caldb/data/xmm/

2.6

SOFTWARE
http://xmm.vilspa.esa.es/external/xmm sw cal/sas frame.shtml

· XMM-Newton Standard Analysis System (SAS):

· HEASARC HEASoft Package: http://heasarc.gsfc.nasa.gov/docs/corp/software.html · CXC CIAO Package: http://asc.harvard.edu/ciao/

2.7

ANALYSIS, DOCUMENTATION AND HELPFUL HINTS
http://xmm.vilspa.esa.es/external/xmm sw cal/sas frame.shtml (click "Documentation", then click "its own documentation", then click "The SAS User's Guide"

· On-Line SAS Handbook:

· There is a "watchout" page for current SAS bugs at: http://xmm.vilspa.esa.es/sas/documentation/watchout/ · XMM-Newton Users Handbook: http://xmm.vilspa.esa.es/external/xmm user support/documentation/uhb frame.shtml · This Guide: http://heasarc.gsfc.nasa.gov/docs/xmm/abc/ · The MPE Analysis Guide:


4 http://wave.xray.mpe.mpg.de/xmm/cookbook/ · The Birmingham Analysis Guide (scripts etc. for EPIC extended source analysis): http://www.sr.bham.ac.uk/xmm2/


Chapter 3

Data
3.1 USEFUL DOCUMENTATION

There are a number of documents which the users of XMM-Newton data should be aware of. These documents include the SSC Products Specification, Data Files Handbook, Reading Data Products CD's (the most recent versions of these documents can be found in the SOC Document section under http://xmm.vilspa.esa.es/external/xmm user support/documentation/index.shtml), and the SAS Users Guide (http://xmm.vilspa.esa.es/external/xmm sw cal/sas frame.shtml). Additional information concerning XMM-Newton data files can be found in the Interface Control Document: Observation and Slew Data Files (XSCS to SSC) (SciSIM to SOCSIM) (XMM-SOC-ICD-0004-SSD). This is an impressive tome which goes into great detail about the file nomenclature and structure. This document can be found in the documents area of the SOC web pages: http://xmm.vilspa.esa.es/cgi-bin/docs/DOC list?Type=ICD. NOTE: For observation data sets going to US PIs, the GOF makes the data available on line after PGP encryption and after converting the file names to upper case. When the proprietary period for the observation expires the data are decrypted leaving the file names unchanged. A simple decryption script, minus the relevant keys of course, can be found at: ftp://legacy.gsfc.nasa.gov/xmm/software/decrypt.pl. NOTE: Laura Brenneman wrote a script and accompanying help file that gives explicit directions on how to most quickly pull over all the files in a data set from the archive, as well as decrypting, and uncompressing the files in preparation for data analysis. This package can be found at: ftp://legacy.gsfc.nasa.gov/xmm/software/prepare xmm data.tar.gz. and contains the following files: README, decrpyt.pl, and prepare xmm data.pl.

3.2

THE DATA

One of the first steps that should be taken when examining your data is to check to see what you actually have. XMM-Newton observations can be broken into several exposures which are each assigned separate observation numbers. These separate exposures can be radically different in length and can also have the different instruments in different modes. For example, in one case the full observation was 60 ks with EPIC and RGS active but there was one delivered exposure which was 3 ks and had only RGS active. (This can happen because the RGS can operate farther into regions of higher radiation than the EPIC detector. The additional observation time can be considered an additional exposure with only the RGS active.) Two files are useful for this examination. First, the primary HTML page is INDEX.HTM which is included in the Pipeline Products. This page lists basic information for the observation plus the operational modes, filters, and exposure start and stop times for the individual instruments. It also has links to various summary pages, including those for the instruments. (In the case above, the EPIC summary page simply stated that "EPIC exposures processed by PPS None.") Specifically, LOOK at the P*SUMMAR0000.HTM files in the pipeline products (easily available through the links). Second, to quickly access images from the various instruments examine the PPSGRA Pipeline Products page (§ 3.3.2).

NOTE:

5


6

3.3

PI Data

Proprietary XMM-Newton data is available for download via your XSA account. Email instructions from SOC at Vilspa are sent to the address on record with detailed directions on how to retrieve your data via the XSA. The data files can be considered to come in two groups in separate subdirectories when retrieved, the Observation Data Files (ODF) files and Pipeline Processing (PPS) files. The ODF data contain all of the observation-specific data necessary for reprocessing the observation. The PP data contain, among other things, calibrated photon event files and source lists. NOTE: For observation data sets going to US PIs, the GSFC GOF makes the data available in two directories containing the following groups of files. · ODF ­ The ODF (raw) data files · PIPEPROD ­ The pipeline processed data products

3.3.1

ODF Data

ODF data come with file names in the following format: · mmmm iiiiiijjkk aabcccddeee.FIT mmmm ­ orbit number iiiiiijjkk ­ observation number aa ­ detector (M1 ­ MOS1, M2 ­ MOS2, PN, OM, R1 ­ RGS1, R2 ­ RGS2, SC ­ space craft) b ­ S for scheduled data (U for unscheduled data, X for general purpose files) ccc ­ exposure number dd ­ CCD number or OM window number eee ­ type of data NOTE: For SAS processing, the file names should contain all upper case characters. However, at least with early CDs, the file names used lower case characters. The GSFC XMM-Newton GOF provides a script to rename the files.

3.3.2

Pipeline Product Data

PP data (listed in Table 3.1) contain some more immediately useful data products such as calibrated photon event lists, source lists, and images. While there are a large number of products which come in a single directory, they can be associated in up to 15 groupings. (The number of groups can vary depending on the number of operational instruments, e.g., if the OM is turned off there are no OM products.) Each group has an associated HTML file which organizes access to the files and provides a limited description of them. The names of the HTML files are of the following form: · PPiiiiiijjkkAAAAAA000 0.HTM iiiiiijjkk ­ observation number AAAAAA ­ group identifier (see Table 3.1)


7

Table 3.1: Pipeline Processing data files. Group ID Contents

CRSCOR1 EANCIL
1

EEVLIS1

ESKYIM1 ESRLIS1

Contains PDF files of POSS II finding charts, HTML files of cross correlations with the SIMBAD data base, FITS tables for the detected sources Contains the exposure maps in a variety of energy bands and the source-detection sensitivity maps for the EPIC instruments. The sensitivities are in units of counts s-1 corrected for vignetting and corresponding to a likelihood specified in the FITS header. The files are gzipped with a .FTZ extension. Contains calibrated photon event files for the EPIC detectors. If the files are sufficiently large they may be separated into two tar files. The files are gzipped fits files with a .FTZ extension. This group contains the event images in a variety of energy bands. The fits files are gzipped with a .FTZ extension, the full images also come as PNG images. Contains EPIC observation source lists. There is an HTML page of the merged source list and gzipped fits tables of source lists from the different instruments and source detection tasks. Contains OM sky images in gzipped FITS format. Contains OM observation source lists in gzipped FITS format. Contains OM star tracking time series in gzipped FITS format. Contains the Calibration Index File (CIF) used in the pipeline processing (*CALIND*), PPS information, and the attitude history time series (*ATTTSR*) in gzipped FITS or ASCII format. Contains the OM tracking history plots, PPS, EPIC, OM, RGS observation, and PPS run summaries. NOTE: CHECK THESE OUT ASCII file containing pipeline processing report Contains the RGS source and event lists in gzipped FITS format Contains the RGS exposure maps in gzipped FITS format Contains the RGS images (both energy dispersion and cross dispersion) in gzipped FITS and PNG formats Contains the RGS source and background spectra in gzipped FITS and PDF formats

OIMAGE2 OMSLIS2 OMSRTS2 PPSDAT

PPSGRA PPSMSG REVLIS3 REXPIM3 RIMAGE3 RSPECT3
1 2 3

Further information on the files can be found in Table 5.1. Further information on the files can be found in Table 7.1. Further information on the files can be found in Table 6.1.


Chapter 4

Setting Up and Running SAS
The Science Analysis Software (SAS, http://xmm.vilspa.esa.es/external/xmm sw cal/sas frame.shtml), developed by the Survey Science Centre (SSC) and Science Operations Centre (SOC), is a suite of about 125 programs and scripts that perform data reduction, extraction, and some analysis of XMM-Newton data. The Pipeline Processing System (PPS), comprised of a superset of the SAS suite and Perl scripts, is run at Leicester University (http://xmmssc-www.star.le.ac.uk/) to create the basic data products provided to the Guest Observer from the satellite ancillary and science data. SAS is not designed for higher level scientific analysis such as spectral fitting and temporal analysis, but does provide for the creation of detector response files and barycentric corrected event timing information. SAS includes extensive EPIC and OM source-detection software. The SAS product files conform to OGIP FITS standards so any high-level analysis package used in high-energy astrophysics should theoretically be capable of processing XMM-Newton data. For example, the HEASoft package, http://heasarc.gsfc.nasa.gov/docs/corp/software.html, of the High Energy Astrophysics Science Archive Research Center (HEASARC, http://heasarc.gsfc.nasa.gov/) at NASA/GSFC and the CIAO package (http://asc.harvard.edu/ciao/) of the Chandra X-ray Observatory Center (http://chandra.harvard.edu/) can both be used with XMM-Newton data files.

4.1

INSTALLATION

The primary guide for the installation of SAS can be found through the SOC at http://xmm.vilspa.esa.es/external/xmm sw cal/sas frame.shtml (note that the final "/" is often required for SOC pages). Because of the complexity of the SAS installation, it is strongly recommended that users download and install the binary executables rather than compiling SAS from source code (which also necessitates the purchase of commercial software). It should also be noted that "optional" components, while not needed for running SAS tasks from the command-line, are critical to running SAS from the GUI. These optional components are listed at the SOC page http://xmm.vilspa.esa.es/sas/installation/requirements.shtml.

4.2

CALIBRATION DATA

XMM-Newton data reduction and analysis requires extensive calibration data which must be available under a Current Calibration File (CCF) directory. Information on the CCF and instructions for downloading/mirroring the files can be found under the SOC XMM-Newton Calibration page (http://xmm.vilspa.esa.es/ccf/). The calibration page also has links to the CCF release notes. In addition, background event files and canned spectral response files can be found under http://xmm.vilspa.esa.es/external/xmm sw cal/calib/epic files.shtml.

4.3

SAS INVOCATION

There are a few parameters which need to be set for the proper operation of SAS. Many are taken care of by the initialization script, but it doesn't hurt to repeat them. The commands should, of course, be modified to be appropriate for your specific setup. setenv SAS_DIR /path/to/xmmsas_yyyymmdd_hhmmsource 8


9 source $SAS_DIR/sas-setup.csh $SAS_DIR/sas-setup.sh setenv SAS_ODF /path/to/odf_data S I A S e n l e ts the SAS directory path itializes SAS ternate SAS initialization ts the directory path to the ODF data, it is probably a good idea to have this be the full path. ts the directory path to the CCF data ts the Calibration Index File (CIF) path and file name (note that the CIF file is normally part of an event list, so SAS_CCF can also be pointed at the list, this should probably be the full path as well ts the verbosity, 1 => little, 10 => lot ts the warning level, 1 => little, 10 => lot vokes the SAS GUI, SAS tasks can also be run on the command line

setenv SAS_CCFPATH /path/to/CCF setenv SAS_CCF $SAS_ODF/ccf.cif

Se Se

setenv SAS_VERBOSITY 3 setenv SAS_SUPPRESS_WARNING 3 sas &

Se Se In

NOTE: To verify the SAS-specific settings, use the task sasversion (alternatively, the command env | grep SAS can be used). SAS need not be run in the directory where the data are stored (for example, it will be possible to run off of the data CDs when the file names are changed to be upper case). To do so only requires that the setenv SAS CCF $SAS ODF/ccf.cif be reset to the directory string for the working directory, see § 4.5.1. This can also be done using the SAS Preferences GUI (found under the "File" menu). From the command line invocation of tasks the input and output directories, when relevant, can be set as parameters (e.g., see command line input for odfingest, § 4.5.2). SAS tasks can be run equally well from the command line and from the SAS GUI. In this document we will demonstrate the use of most tasks from the command line. In many cases parameters where the default values are acceptable are not included in the command list, which can be done in practice as well. If the GUI interface is being used then simply set the parameters there. The MPE Analysis Guide, http://wave.xray.mpe.mpg.de/xmm/data analysis demonstrates many of the common tasks using GUIs.

4.3.1

SAS Helpful Hints

Command lines can often be quite long with a variety of parameters. To avoid considerable typing when creating command scripts a feature of the GUI interface can be of assistance. When invoking a task through the GUI a copy of the full command appears in the dialog box, from where it can then be cut and pasted. There are several useful features of the command-line interface that users should be aware of. 1) If the dialog parameter is included in the command line, the task GUI will pop up with all parameters in the command line preset. This allows the use of the GUI interfaces at the task level without having to go through the main SAS GUI. 2) If the manpage parameter is included in the command line, the task documentation will pop up in a Netscape window. 3) In addition, the command sashelp doc=sas task will pop up a Netscape window with the documentation for the task sas task as well. NOTE: The command documentation (i.e., the pages brought up by sashelp doc=sas task or sas task manpage) has an Errors section. Common warning messages produced by the tasks and their meanings are listed here. This feature is very useful.

4.4
4.4.1

SAS SYNTAX AND LOGIC
Command Line Syntax

There is some flexibility in command line syntax in SAS. The following are all valid task calls on the command line that result in identical operations: rgsproc withsrc=F rgsproc withsrc=no rgsproc withsrc='no' rgsproc withsrc="no"


10 rgsproc --withsrc=no rgsproc --withsrc='no' rgsproc --withsrc="no" However, rgsproc -withsrc=F rgsproc -withsrc=no rgsproc -withsrc='no' rgsproc -withsrc="no" are not correct syntax. One format is not "more correct" than another, and the choice of which to use is left to user preference. In this ABC guide we adopt the simplest format, and use no dashes and only single quotation marks only when required, e.g., rgsproc withsrc=no orders='1 2 3' where, in this case, the quotes provide the task with a list.

4.4.2

Table Syntax

When a task requires the use of a table within a file there are also several valid syntaxes, e.g., xmmselect table=filtered.fits:EVENTS xmmselect table="filtered.fits:EVENTS" xmmselect table=filtered.fits%EVENTS do an identical operation in opening the EVENTS table inside the file filtered.fits.

4.4.3

Filtering Logic

Filtering event files requires some command of the SAS logical language which consists of familiar arithmetic and Boolean operators and functions. These, and their syntax, are described within the on-line documentation supplied with the software. Pull up the help document using: sashelp doc=selectlib

4.5

GENERAL SAS TASKS FOR DATA SET PREPARATION

WARNING: Before running the following tasks make sure that the ODF file names are all upper case. NOTE: Run these tasks.

4.5.1

cifbuild

Many SAS tasks require calibration information from the Calibration Access Layer (CAL). Relevant files are accessed from the set of Current Calibration File (CCF) data using a CCF Index File (CIF). A CIF is included in the pipeline products but if the CCF has been updated it can be recreated by the user. In practice, it is perhaps easiest to determine whether the CCF has been updated by recreating the CIF using the SAS task cifbuild (default name ccf.cif) and then using the SAS task cifdiff to compare the new CIF with the old. If the CAL has changed the user may want to reprocess the data using the new CIF (e.g., see § 5.7.1). To help determine whether it is reasonable to reprocess the data, the CCF release notes (http://xmm.vilspa.esa.es/user/calib top.html) should be examined. CCF files can be downloaded directly from the SOC web site (see § 4.2) WARNING: The CIF file contains a list of files to be used in the calibration/processing of your data. The task cifbuild looks at the CCF directory and builds the CIF file accordingly. If the data are processed with two different CIF files (e.g., because they were generated at different times, with different files under the CCF directory) you can end up with different results (although most often not significantly different). Note that the pipeline product *CALIND* is the CIF file used for the pipeline processing. To run cifbuild and cifdiff on the command line use: · cifbuild withccfpath=no analysisdate=now category=XMMCCF fullpath=yes > withccfpath ­ flag to look for the CCF constituents in a specific directory (the parameter SAS CCFPATH should be set, see § 4.3)


11 > analysisdate ­ date when analysis was performed. > category ­ XMMCCF (SCISIMCCF if data were constructed by the SciSim simulation package). > fullpath ­ include the full path to each constituent within the CIF. · cifdiff calindex1set=ccf.cif calindex2set=CALIND.FIT > calindex1set ­ name of the first file to be compared, in this case the output from the current run of cifbuild > calindex2set ­ name of the second file to be compared, in this case the (renamed) PP file

4.5.2

odfingest

The task odfingest extends the Observation Data File (ODF) summary file with data extracted from the instrument housekeeping data files and the calibration database. It is required for reprocessing the ODF data with the pipeline tasks as well as for many other tasks. To run odfingest on the command line use: · odfingest odfdir=$SAS ODF outdir=$SAS ODF > odfdir ­ ODF directory > outdir ­ directory to deposit summary file (the ODF directory in this case although the user might want to deposit the summary file in the working directory)


Chapter 5

First Look ­ An EPIC Data Primer
So, you've received an XMM-Newton EPIC data set. What are you going to do with it? After checking what the observation consists of (see § 3.2), you can start with the Pipeline Processed data. As noted in Chapter 4, a variety of analysis packages can be used for the following steps. However, as the SAS was designed for the basic reduction and analysis of XMM-Newton data (extraction of spatial, spectral, and temporal data), it will be used here for demonstration purposes (although see § 5.6 for a short tutorial on the use of Xselect for data extraction). SAS will be required at any rate for the production of detector response files (RMFs and ARFs) and other observatory-specific requirements. (Although for the simple case of on-axis point sources the canned response files provided by the SOC can be used.) NOTE: For PN observations with very bright sources, out-of-time events can provide a serious contamination of the image. Out-of-time events occur because the read-out period for the CCDs can be up to 6.3% of the frame time. Since events that occur during the read-out period can't be distinguished from others events, they are included in the event files but have invalid locations. For observations with bright sources, this can cause bright stripes in the image along the CCD read-out direction. For a more detailed description of this issue, check: http://wave.xray.mpe.mpg.de/xmm/cookbook/EPIC PN/ootevents.html

5.1

USING PIPELINE PROCESSED DATA PRODUCTS

The Pipeline Processing (PP) produces quite a number of useful products which allow a first look at the data, but can overwhelm the user by their sheer numbers. The first place to look is the INDEX.HTM page which organizes the presentation of the data and provides links to other PP pages. The INDEX.HTM page also lists general observation information (target, date, time, etc.) and instrument modes. The INDEX.HTM page provides links to various observation summary pages, which have names with the following nomenclature: · PPiiiiiijjkkAAX000SUMMAR0000.HTM, where iiiiii ­ proposal number jj ­ target ID - target number in proposal kk ­ exposure ID - exposure number for target NOTE: The ten-digit combination of iiiiiijjkk is the observation number and is used repetitively throughout the file nomenclature AA ­ ID (EP ­ EPIC Summary, OM ­ Optical Monitor Summary, RG ­ RGS Summary OB ­ Observation Summary, OB with SUMMAR - > PPSSUM ­ Pipeline Processing Summary, CA with SUMMAR - > XCORRE ­ Source Correlation Summary) Each grouping of the pipeline products (Tables 3.1 and 5.1) there is an HTML (.HTM extension) file which lists the associated files and gives a few-word description of those files. It is useful to set up your web browser to automatically display a number of file types, e.g., PDF files. The HTML file names are of the following format: · PPiiiiiijjkkAAAAAA000 0.HTM, where iiiiii ­ proposal number 12


13 jj ­ target ID - target number in proposal kk ­ exposure ID - exposure number for target AAAAAA ­ Group ID (Table 5.1) The data file names are of the form (see Table 41 in the XMM Data Files Handbook, ftp://xmm.vilspa.esa.es/pub/odf/data/sv/docs/datafileshb 2 0.pdf.gz, or *.ps.gz): · PiiiiiijjkkaablllCCCCCCnmmm.zzz, where iiiiiijjkk ­ observation number aa ­ detector, M1 ­ MOS1, M2 ­ MOS2, PN ­ PN, CA ­ for files from the CRSCOR group b ­ S for scheduled observation, U for unscheduled, X for files from the CRSCOR group (and any product that is not due to a single exposure) lll ­ exposure number CCCCCC ­ file identification (Table 5.1) n ­ exposure map band number, unimportant otherwise for EPIC data mmm ­ source number in hexidecimal zzz ­ file type (e.g., PDF, PNG, FTZ, HTM) ASC ­ ASCII file, use Netscape, other web browser, or the "more" command ASZ ­ gzipped ASCII file FTZ ­ gzipped FITS format, use ds9, Ximage, Xselect, fv HTM ­ HTML file, use Netscape or other web browser PDF ­ Portable Data Format, use Acrobat Reader PNG ­ Portable Networks Graphics file, use Netscape or other web browser TAR ­ TAR file

5.1.1

A Quick Look at What You Have

The ESKYIM files contain EPIC sky images in different energy bands whose ranges are listed in Table 5.1. While the zipped FITS files may need to be unzipped before display in ds9 (depending on the version of ds9), they can be displayed when zipped using fv (fv is FITS file viewer available in the HEASoft package). In addition, the image of the total band pass for all three EPIC detectors is also provided in PNG format which can be displayed with Netscape. The PP source list is provided in both zipped FITS format (readable by fv) and as an HTML file.

5.2

EXAMINE AND FILTER THE DATA - PIPELINE PRODUCTS

The EPIC event lists in the EEVLIS group of the Pipeline Processing will have names of the form: · PiiiiiijjkkaaSlllcIEVLI0000.FTZ, where i a l c iiiiijjkk ­ observation number a ­ detector (M1 ­ MOS1, M2 ­ MOS2, PN ­ PN) ll ­ exposure number within the observation ­ detector (M ­ MOS1 or MOS2, P ­ PN, T ­ Timing Mode)

These are OGIP standard calibrated photon event FITS files in gzipped format. Some tasks and software will require that these files be gunzipped, which usually means renaming as well, e.g.: mv PiiiiiijjkkaaSlllcIEVLI0000.FTZ PiiiiiijjkkaaSlllcIEVLI0000.FIT.gz gunzip PiiiiiijjkkaaSlllcIEVLI0000.FIT.gz The following sections describe the use of SAS tasks using the both the command-line and GUI interfaces, except in cases where one of the methods is particularly easy. The SAS xmmselect GUI provides a very simple method for producing and displaying images, spectra, and light curves, and is the recommended method for extracting data unless large numbers of sources are being analyzed.


14 Table 5.1: EPIC Pipeline Processing data files.

Group ID

File ID

Contents

File Type

View With

CRSCOR

F R S D F I I I I I I E E E E E E E
2

C O N N N M M M M M M X X X X X X X

H S N N N A A A A A A P P P P P P S

A I N N N G G G G G G M M M M M M N

R M N N N E E E E E E A A A A A A M

T G N N N

1 1 1

Finding chart ROSAT image of region Source cross-correlation Results Catalog descriptions FOV cross-correlation Result Sk Sk Sk Sk Sk Sk y y y y y y p p p p p p p ima ima ima ima ima ima o o o o o o o s s s s s s s g g g g g g e e e e e e 0 0 0 2 4 7 .2 .2 .5 .0 .5 .5 1 0 2 4 7 1 2.0 keV .5 keV .0 keV .5 keV .5 keV 2.0 keV 12.0 keV 0.5 keV 2.0 keV 4.5 keV 7.5 keV 12.0 keV map

PDF PDF Zipped FITS PDF Zipped FITS Z Z Z Z Z Z Z Z Z Z Z Z Z ipp ipp ipp ipp ipp ipp ipp ipp ipp ipp ipp ipp ipp e e e e e e d d d d d d d d d d d d d FI FI FI FI FI FI T T T T T T S S S S S S

Acrobat Reader Acrobat Reader fv Acrobat Reader fv ds ds ds ds ds ds ds ds ds ds ds ds ds 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, X X X X X X X X X X X X X im im im im im im im im im im im im im age, age, age, age, age, age, age, age, age, age, age, age, age, fv fv fv fv fv fv fv, Netscape fv fv fv fv fv fv

ESKYIM

8 1 2 3 4 5 P P P P P P P 8 1 2 3 4 5

EANCIL

E E E E E E E

x x x x x x x

ur ur ur ur ur ur ur

e e e e e e e

map 0.2 map 0.2 map 0.5 map 2.0 map 4.5 map 7.5 sensitivity

e e e e e e e

FITS, PNG FITS FITS FITS FITS FITS FITS

EEVLIS

MIEVLI PIEVLI TIEVLI E E E O B B M B L M S S S S R M L L L L I I I I

MOS imaging mode event list PN imaging mode event list PN, MOS timing mode event list Box-local detect source list Box-map detect source list Max-like detect source list Summary source list

Zipped FITS Zipped FITS Zipped FITS Z Z Z Z ipp ipp ipp ipp ed ed ed ed FITS FITS FITS FITS, HTML

xmmselect, fv, Xselect xmmselect, fv, Xselect xmmselect, fv, Xselect fv fv fv fv, Netscape

ESRLIS

1 2

NNNNN ­ Alphanumeric ID Files for only those modes which were active will be included

5.2.1

Initialize SAS and Prepare the Data

For the following demonstration, the PP data are assumed to be in the /PIPE directory, the ODF data (with upper case file names, and uncompressed) are in the directory /ODF, the analysis is taking place in the /PROC directory, and the CCF data are in the directory /CCF. The data used are from the Lockman Hole SV1 observation. 1) gunzip the PP event list to be examined (not really necessary), and for practical purposes shorten the file name as well, e.g.: mv P0123700101M1S001MIEVLI0000.FTZ mos1.fits.gz gunzip mos1.fits.gz 2 a ) In se se se preparation set a few SAS parameters (directory pointers): tenv SAS ODF /ODF tenv SAS CCFPATH /CCF tenv SAS CCF /PROC/ccf.cif


15 To verify the SAS-specific settings, use the command env | grep SAS, and remember that for SAS ODF and SAS CCF it is best to use the full path 2b) If it doesn't already exist, create a CIF file using the SAS task cifbuild (§ 4.5.1). ­ cifbuild fullpath=yes 2c) If it hasn't already been done (don't do it twice), prepare the ODF data by using the SAS task odfingest (necessary for many SAS tasks) (see § 4.5.2). ­ odfingest odfdir=$SAS ODF outdir=$SAS ODF 3) Invoke the SAS GUI (Figure 5.1). ­ sas &

Figure 5.1: The SAS GUI. To locate and invoke a task one need only start typing the task name, and when it is high-lighted hit a carriage return. Otherwise, double-click on the task name.

4) Invoke the xmmselect GUI (Figure 5.2) from the SAS GUI. To invoke a task one need only start typing the task name, and when it is high-lighted hit a carriage return. ­ When xmmselect is invoked a dialog box will first appear requesting a file name. One can either use the browser button or just type the file name in the entry area, "mos1.fits" in this case. To use the browser, first click on the file folder icon button on the right which will bring up a second GUI for the file selection. Double click on the desired event file in the right-hand column (you may have to open the appropriate directory first), click on the "EVENTS" extension in the right-hand column (which selects the extension), and then click "Ok". The directory GUI will then disappear and then click "Run" on the selection GUI.


16 ­ When the file name has been submitted the xmmselect GUI (Figure 5.2) GUI will appear, along with a dialog box offering to display the selection expression. The selection expression will include the filtering done to this point on the event file, which for the pipeline processing includes for the most part CCD and GTI selections.

Figure 5.2: The xmmselect GUI. The top dialog area is for the selection expression. The central part of the GUI provides a list of the parameters available in the table (note the scroll bar on the right hand side). Twodimensional data are selected using the square boxes on the left hand side (in this case X,Y, sky coordinates, have been selected) while one-dimensional data are selected using the round boxes (Time in this example).

5.2.2

Create and Display an Image

5a) Create an image in sky coordinates by using the xmmselect GUI.


17 Figure 5.3: The evselect GUI. Additional parameters for the selected process can be accessed through the tabs at the top of the GUI.

­ To create an image of the data in sky coordinates check the square boxes to the left of the "X" and "Y" entries. ­ Click on the "Image" button near the bottom of the page. This brings up the evselect GUI (Figure 5.3). ­ The default settings are reasonable for a basic image so click on the "Run" button at the lower left corner of the evselect GUI. Different binnings and other selections can be invoked by accessing the "Image" tab at the top of the GUI. ­ The resultant image is written to the file image.ds, and the image is automatically displayed using ds9, and is shown in Figure 5.4. 5b) Using the command line interface, create an image in sky coordinates by using the task evselect. The same image produced in 5a) can be created using the following command.


18 ­ evselect table=/PIPE/mos1.fits:EVENTS withimageset=yes imageset=image.fits xcolumn=X ycolumn=Y imagebinning=imageSize ximagesize=600 yimagesize=600 > > > > > > ta wi im xc yc im of > xi > xi ble ­ input event table. thimageset ­ make an image. ageset ­ name of output image. olumn ­ event column for X axis. olumn ­ event column for Y axis. agebinning ­ form of binning, force entire image into a given size or bin by a specified number pixels. magesize ­ output image pixels in X. magesize ­ output image pixels in Y.

Display the output file image.fits using, e.g., ds9 image.fits &.

Figure 5.4: ds9 window showing the unfiltered image of the MOS1 data from the Lockman Hole SV1 observation, displayed on a square root scale with an upper cut value of 40 using the SLS color look-up table.


19

5.2.3

Create and Display a Light Curve
­ To create a light curve check the round box to the left of the "Time" entry. ­ Click on the "OGIP Rate Curve" button near the bottom of the page. This brings up the evselect GUI (Figure 5.3). ­ The default setting is for a one-second bin which is a bit fine, so access the "Lightcurve" tab and change the "timebinsize" to, e.g., 100 (100 s). Click on the "Run" button at the lower left corner of the evselect GUI. ­ The resultant light curve is written to the file rates.ds, and is displayed automatically using Grace (Figure 5.5).

6a) Create a light curve of the observation by using the xmmselect GUI (Figure 5.2).

6b) Using the command line interface, create a light curve of the observation using the task evselect then display with dsplot. ­ evselect table=/PIPE/mos1.fits:EVENTS withrateset=yes rateset=rate.fits maketimecolumn=yes timecolumn=TIME timebinsize=100 makeratecolumn=yes > > > > > > > table ­ withrat rateset maketim timecol timebin makerat created input event table. eset ­ make an light curve. ­ name of output light curve file. ecolumn ­ control to create a time column umn ­ time column label size ­ time binning (seconds) ecolumn ­ control to create a count rate column, otherwise a count column will be

­ dsplot table=rate.fits x=TIME y=RATE.ERROR withoffsetx=yes & > table ­ input event table > x ­ column for plotting on X axis > y ­ column for plotting on Y axis, the nomenclature RATE.ERROR plots the count rate column (RATE) with the count-rate error column (ERROR) as uncertainties > withoffsetx ­ creates an offset to the X axis (-73194570.96472888 s in Figure 5.5)

5.2.4

Filter the Data and Create a New Event File

7) Next apply some filtering to the data. The "expressions" for the MOS and PN, (PATTERN <= 12)&&(PI in [200:12000])&&#XMMEA EM and (PATTERN <= 4)&&(PI in [200:15000])&&#XMMEA EP, will select good events with PATTERN in the 0 to 12 range (single, double, triple, and quadruple pixel events) and pulse height in the range of 200 to 12000 eV for the MOS and good events with PATTERN in the 0 to 4 range (single and double pixel events) and pulse height in the range of 200 to 15000 eV for the PN. This should clean up the image significantly with most of the rest of the obvious contamination due to low pulse-height events. Setting the lower PI channel limit somewhat higher (e.g., to 300 eV) will eliminate much of the rest. The selection on the PATTERN value is similar the GRADE selection for ASCA data, and is related to the number and pattern of the CCD pixels triggered for a given event. The PATTERN assignments are: single pixel events ­ PATTERN == 0, double pixel events ­ PATTERN in [1:4], triple and quadruple events ­ PATTERN in [5:12]. The #XMMEA EM (#XMMEA EP for the PN) filter provides a canned screening set of FLAG values for the event. (The FLAG value provides a bit encoding of various event conditions, e.g., near hot pixels or outside of the field of view. Setting FLAG == 0 in the selection expression provides the most conservative screening criteria. The definitions of the FLAG values can be found in the FITS headers of the EVENTS extensions of the event files. FITS headers can easily be examined using fv.) An output file will be created for further processing. 7a) Filter the data using the xmmselect GUI.


20 Figure 5.5: Grace window showing the unfiltered light curve of the MOS1 data from the Lockman Hole SV1 observation. Also shown is the time selection interval.

­ Since MOS data are being used, in the selection expression area at the top of the xmmselect GUI enter: (PATTERN <= 12)&&(PI in [200:12000])&&#XMMEA EM. ­ Click on the "Filtered Table" box at the lower left of the xmmselect GUI. ­ Change the evselect filteredset parameter, the output file name, to something useful, e.g., mos1-filt.fits. Click "Run". 7b) Filter the data using evselect on the command line. ­e e f u v x i p s p l d e r t a l e e t e s r e c s e e t i d x table=mos1.fits:EVENTS withfilteredset=yes on='(PATTERN <= 12)&&(PI in [200:12000])&&#XMMEA EM' set=mos1-filt.fits filtertype=expression keepfilteroutput=yes posure=yes filterexposure=yes ­ rt ss il re il ee re input event table. ype ­ method of filtering ion ­ filtering expression. teredset ­ create a filtered set. dset ­ output file name. teroutput ­ save the filtered output xposure ­ for use with temporal filtering xposure ­ for use with temporal filtering

> > > > > > > >

t f e w f k u f

a i x i i e p i

b l p t l e d l

l t r h t p a t

e e e f e f t e


21 8) If necessary (and for the Lockman Hole SV1 data it most definitely is), add a temporal filtering clause to the evselect selection "expression". This is most often required because of soft proton flaring which can be painfully obvious with count rates of 50­100 counts a second, or more. Note that how much flaring needs to be excluded depends on the science goals of the analysis, a whopping bright point source will clearly be less affected than a faint extended ob ject. A temporal filter can be easily created from the Grace light curve plot window. ­ Create a light-curve plot through the xmmselect GUI ­ In the Grace window, pull down the "Edit" menu, select "Regions", and select "Define" ­ For this case, select "Left of Line" for the "Region type" ­ Click the "Define" button and then click at two points to create a vertical line at the upper end of the desired range on the Grace plot. (It is possible to define up to five regions at one time by changing the "Define region" counter.) ­ Back on the xmmselect GUI, click on the "1D region" button. This will transfer the selection criteria to the "Selection expression" location. The syntax for the time selection is (TIME <= 73227600). A more complicated expression which would remove a small flare within an otherwise good interval (e.g., the soft proton flares observed in the light curve plot of Figure 5.5) could be: (TIME <= 73227600)&&!(TIME IN [73221920:73223800]). The syntax &&(TIME < 73227600) includes only events with times less than 73227600. Use &&!(TIME in [73221920:73223800]) to exclude events in the time interval 73221920 to 73223800, the "!" symbol stands for the logical "not". The full "expression" would then be: (PATTERN <= 12)&&(PI in [200:12000])&&#XMMEA EM &&(TIME <= 73227600) &&!(TIME in [73221920:73223800]) Again, give the new file a useful name (mos1-filt-time.fits) and make sure that the updateexposure and filterexposure boxes are checked on the evselect GUI. Time filtering can also be done directly using the light curve by the creation of a secondary GTI file using the routine tabgtigen task. ­ tabgtigen table=rate.fits:RATE expression='RATE<5' gtiset=gtisel.fits timecolumn=TIME > table ­ input count rate table and extension (§ 5.2.3). > expression ­ filtering expression, in this case include those intervals where the count rate is < 5 counts s-1 in the individual 100 s intervals. > gtiset ­ output file name for selected GTI intervals. > timecolumn ­ time column. The output GTI table can then be used in the filtering expression in evselect with the syntax &>I(gtisel.fits,TIME). The full "expression" would then be: (PATTERN <= 12)&&(PI in [200:15000])&&#XMMEA EM&>I(gtisel.fits,TIME). Figures 5.6 and 5.7 show the image and light curve generated from the filtered data.

5.3
5.3.1

EXTRACT AND FIT A SOURCE SPECTRUM
Extract the Spectrum

While all of the data extraction can be done on the original file keeping the final selection "expression", it can save significant time and memory to operate on the filtered event file. For instance, in the case of the Lockman Hole data, the original MOS1 event file is 48.4 Mb while the filtered (spatial, temporal, and spectral) list is only 4.0 Mb. To change the event file, pull down the "file" menu on the xmmselect GUI and select "New Table". This will bring up the file selection browser and just follow the instructions in Item 4 of Section 5.2.1. The extraction of region-specific data (e.g., source spectra and light curves) is simplified by using the GUI again because of the treatment of selection regions. 1) With xmmselect running on the filtered file, create an image by selecting the small boxes to the left of the X and Y columns, clicking on the "Image" button, and then clicking on the "Run" button on the


22 Figure 5.6: Filtered image of the MOS1 data from the Lockman Hole SV1 observation. Displayed with a square root scale and an upper cut value of 20.

pop-up evselect GUI (for these purposes the default parameters are fine). To select a file name for the image rather than using the default image.ds, select the "Image" page on the evselect GUI and change the imageset entry. 2) On the ds9 window, create a region for a source of interest. Click once on the ds9 image and a region circle will appear. Click on the region circle and the region will be activated, allowing the region to be moved and its size to be changed. Having created, placed, and sized the region appropriate for the source, click the "2D region" button on the xmmselect GUI. This transfers the region information into the "Selection expression" text area, e.g., ((X,Y) IN circle(26144,22838,600)) for the bright source at the lower center of the Lockman Hole observation. The circle parameters are the X and Y positions and the radius of the circle in units of 0. 05, so the above region description is for a circle of 30 radius. Note: For serious spectral analysis the phrase &&(FLAG == 0) should be added to the selection expression. This provides the most stringent screening of the data and will exclude events such as those next to the edges of the CCDs and next to bad pixels which may have incorrect energies. 3) To extract the spectrum, first click the circular button next to the PI column on the xmmselect GUI.


23 Figure 5.7: Filtered light curve of the MOS1 data from the Lockman Hole SV1 observation.

Figure 5.8: Spectrum of a source from the Lockman Hole SV1 observation.

Next click the "OGIP Spectrum" button. Select the "Spectrum" page of the evselect GUI to set the


24 file name and binning parameters for the spectrum. For example, set spectrumset to source.ds. The spectralbinsize must be set to 15 for the MOS or 5 for the PN. withspecranges must be checked, specchannelmin set to 0, and specchannelmax set to 11999 for the MOS or 20479 for the PN. Figure 5.8 shows the spectrum. 4) To extract a background spectrum from an annulus surrounding the source, first clear the "Selection expression". Next repeat step 2) except create two circles defining the inner and outer edges of the background annulus. Use the "Properties" menu under the ds9 "Region" menu to set the inner circle to "exclude". Then click the "2D region" button on the xmmselect GUI to transfer the region description of both circles to the "Selection expression". This may need to be edited. For example, for an annulus it should be as follows: ((X,Y) IN circle(26144,22838,1500))&&!((X,Y) IN circle(26144,22838,900)). This will include data within a circle of radius 75 but not within a concentric circle of 45 (the values are in units of 0. 05). Finally, repeat step 3) except set the filteredset parameter to a different file name, e.g., back.ds. 5) To extract the source light curve, put the source "Selection expression" (the region descriptor used in Step 3) in place and click the circular button next to the "TIME" column on the xmmselect GUI. (Note: if you forgot to record it, the region selection criteria can be found in the FITS header of the spectrum extension of the spectrum file, e.g., source.ds.) Next click the "OGIP Rate curve" button. Select the "Lightcurve" tab of the evselect GUI to set the file name and binning parameters for the light curve. For example, set filteredset to source.rate and timebinsize to 1000 for a reasonable binning for the source examined in the spectral analysis section. (NOTE: set timebinsize=1 and deselect makeratecolumn to create the light curve for the temporal analysis example in § 5.5. The first forces the time interval to be 1 s and the second creates a count rather than a count rate column.) 6) Depending on how bright the source is and what modes the EPIC detectors are in, event pile up can possibly be a problem. Pile up occurs when a source is so bright that there is the non-negligible possibility that X-rays will strike two neighboring pixels or the same pixel in the CCD more than once in a read-out cycle. In such cases the energies of the two events are in effect added together to form one event. If this happens sufficiently often it will skew the spectrum to higher energies. To check whether pile up may be a problem, use the SAS task epatplot. To run epatplot create source and background event files by extracting data from the original event file using the time and region selection expressions combined with the FLAG == 0 filtering (all PATTERN values are required). On the xmmselect GUI click the "Filtered Table" button and check the updateexposure on the evselect "General" page and provide a filteredset name, e.g., mos1-source.fits and mos1-back.fits, for the resultant files. Invoke epatplot from the SAS GUI, enter the source event file name (e.g., mos1-source.fits) for the set parameter on Tab 0 and set withbackgroundset to yes and provide the background event file name (e.g., mos1-back.fits) for the backgroundset parameter on Tab 1, and click on "Run". If the plot shows the model distributions for single and double events diverging significantly from the measured distributions then pileup must be considered. Figure 5.9 shows an example of a bright source (from a different observation) which is not strongly affected by pileup. The source used in this example is too faint to provide reasonable statistics for epatplot and is far from being affected by pile up.

5.3.2

Create RMFs and ARFs

The following assumes that an appropriate source spectrum, named source.ds, has been extracted as in § 5.3.1. 7a) Create the photon redistribution matrix, the RMF, using the task rmfgen GUI. ­ From the SAS GUI, invoke the rmfgen GUI (Figure 5.10) ­ Set the spectrumset keyword to the spectrum file name, e.g., source.ds ­ Set the rmfset keyword to the RMF file name, e.g., rmf.ds ­ Click "Run" (if your xmmselect GUI is still running, a dialog box will occur asking whether rmfgen can be run, it can as there is no conflict) 7b Create the photon RMF from the command line.


25 Figure 5.9: A MOS1 epatplot plot for a moderately bright source which does not show evidence for pileup. The central source from the observation of G21.5-09 (0122700101) is used.

­ rmfgen rmfset=response.ds spectrumset=source.ds > rmfset ­ output RMF file name > spectrumset ­ input spectrum file name 8a) Create the ancillary region file, the ARF, using the task arfgen GUI. ­ From the SAS GUI, invoke the arfgen GUI (Figure 5.11) ­ On the "main" tab set the spectrumset keyword to the spectrum file name, e.g., source.ds


26 Figure 5.10: The rmfgen GUI.

Figure 5.11: The arfgen GUI.

­ On the "main" tab set the arfset keyword to the ARF file name, e.g., arf.ds ­ On the "effects" tab set the badpixlocation keyword to the event file name from which the spectrum was extracted, e.g., mos1-filt-time.fits ­ On the "calibration" tab check the "withrmfset" box and set the rmfset keyword to the RMF file name, e.g., rmf.ds ­ Click "Run" (if your xmmselect GUI is still running, a dialog box will occur asking whether rmfgen can be run, it can as there is no conflict) 8b) Create the ARF from the command line. ­ arfgen arfset=arf.ds spectrumset=source.ds withrmfset=yes rmfset=rmf.ds badpixlocation=mos1-filt-time.fits > > > > > > arfse spect withr rmfse withb badpi event t­ rum mfs t­ adp xlo file output ARF file name set ­ input spectrum file name et ­ flag to use the RMF RMF file created by rmfgen ixcorr ­ flag to include the bad pixel correction cation ­ point to the file containing the bad pixel information, which should be the from which the spectrum was extracted

5.3.3

Prepare the Spectrum

Assuming that source and background spectra have been extracted as in § 5.3 and the RMF and ARF created as in § 5.3.2, spectral fitting will be demonstrated using HEASoft software.


27 9) Nearly all spectra will need to be binned for statistical purposes. The FTOOL grppha provides an excellent mechanism to do just that. The following commands not only group the source spectrum for Xspec but also associate the appropriate background and response files for the source. > grppha P P G G G G G l l R R R R R e e P P P P P a a P P P P P s s H H H H H e e A A A A A enter PHA filename[] source.ds ! inp enter output filename[] source-grp.ds ! out [] chkey BACKFILE back.ds ! inc [] chkey RESPFILE rmf.ds ! inc [] chkey ANCRFILE arf.ds ! inc [] group min 25 ! gro [] exit u p l l l u t u u u u p spectrum file name t grouped spectrum de the background spectrum de the RMF de the ARF the data by 25 counts/bin

5.3.4

Fit the Spectra
> xspec XSPEC> data source-grp.ds XSPEC> ignore 0.0-0.2,6.6-** X 1 2 3 4 5 w r X X X X S : : : : : a e S S S S P w p p p p b n P P P P E a o o o o s o E E E E C b w w w w : r C C C C > s e e e e n m > > > > ! ! ! model wabs(pow+pow) ! :nH> 0.01 ! rlaw:PhoIndex> 2.0 ! rlaw:norm> ! rlaw:PhoIndex> 1.0 ! rlaw:norm> ! H> 0.01 ! ! fit ! setplot device /xw ! setplot energy ! plot ldata ratio ! ! exit ! really want to exit? (y) y i i s s s s d s d s r f s p p ut data ore unusable energy ranges, in keV a range appropriate for the data spectral model to two absorbed power laws model absorption column density to 1.e20 the first model power law index to -2.0 ault model normalization the second model power law index to -1.0 ault model normalization model absorption column density to 1.e20 ormalize the model spectrum the model to the data the plot device t energy along the X axis t two panels with the log of the data and the data/model ratio values along the Y axes exit Xspec n g e e e e e e e e e i e l l p n t t t t f t f t n t t o o

10) Next use Xspec to fit the spectrum.

XSPEC> Do you

Figure 5.12 shows the fit to the spectrum.

5.4

SOURCE DETECTION

The edetect chain does nearly all the work involved with EPIC source detection. In the example below source detection is done on images in two bands for all three detectors. The example uses the filtered event files produced as in § 5.2.4 with the assumption that they are located in the current directory. 1) Create an attitude file using atthkgen, this is required for the creation of the exposure maps. Note that the file *ATTTSR* is the attitude file created by the pipeline processing and can also be used. Both the atthkgen GUI and command line are easy to use. ­ atthkgen atthkset=attitude.fits timestep=1 > atthkset ­ output file name > timestep ­ time step in seconds for attitude file 2) Create images can be used as evselect is run of the three de in sky coordinates over the PI channel ranges of interest using the task evselect (the GUI well). It will use the filtered event list mos1-filt.fits produced above. In this example six times to create the images in two bands (300 - 2000 eV, and 2000 - 10000 eV) for each tectors.


28 Figure 5.12: Fitted spectrum of the Lockman Hole source.

­ evselect table=mos1-filt-time.fits withimageset=yes imageset=mos1-s.fits imagebinning=binSize xcolumn=X ximagebinsize=50 ycolumn=Y yimagebinsize=50 filtertype=expression expression='(FLAG == 0)&&(PI in [300:2000])' > table ­ event list > withimageset ­ flag to create an image > imageset ­ fits image name to be created, image1.fits for band 1 > imagebinning ­ how to bin the image > xcolumn ­ table column to use for the X axis > ximagebinsize ­ binning in X axis (original pixels are 0.05 ) > ycolumn ­ table column to use for the Y axis > yimagebinsize ­ binning in Y axis (original pixels are 0.05 ) > filtertype ­ type of filtering > expression ­ filtering expression, select events in the PI channel range 300-2000 eV ­ evselect table=mos1-filt-time.fits withimageset=yes imageset=mos1-h.fits imagebinning=binSize xcolumn=X ximagebinsize=50 ycolumn=Y yimagebinsize=50 filtertype=expression expression='(FLAG == 0)&&(PI in [2000:10000])' ­ evselect table=mos2-filt-time.fits withimageset=yes imageset=mos2-s.fits imagebinning=binSize xcolumn=X ximagebinsize=50 ycolumn=Y yimagebinsize=50 filtertype=expression expression='(FLAG == 0)&&(PI in [300:2000])' ­ evselect table=mos2-filt-time.fits withimageset=yes imageset=mos2-h.fits imagebinning=binSize xcolumn=X ximagebinsize=50 ycolumn=Y yimagebinsize=50 filtertype=expression expression='(FLAG == 0)&&(PI in [2000:10000])' ­ evselect table=pn-filt-time.fits withimageset=yes imageset=pn-s.fits imagebinning=binSize xcolumn=X ximagebinsize=50 ycolumn=Y yimagebinsize=50 filtertype=expression expression='(FLAG == 0)&&(PI in [300:2000])' ­ evselect table=pn-filt-time.fits withimageset=yes imageset=pn-h.fits imagebinning=binSize xcolumn=X ximagebinsize=50 ycolumn=Y yimagebinsize=50 filtertype=expression expression='(FLAG == 0)&&(PI in [2000:10000])'


29 3) Create a merged count image for later display purposes. ­ emosaic imagesets='mos1-s.fits mos1-h.fits mos2-s.fits mos2-h.fits pn-h.fits pn-s.fits' mosaicedset=mosaic.fits > imagesets ­ list of count images > mosaicedset ­ output file name 4) Run edetect chain. ­e p a p e e e d n t i c b s e t m f o p tect chain imagesets='mos1-s.fits mos1-h.fits mos2-s.fits mos2-h.fits pn-s.fits h.fits' eventsets='mos1-filt-time.fits mos2-filt-time.fits pn-filt-time.fits' itudeset=attitude.fits pimin='300 2000 300 2000 300 2000' ax='2000 10000 2000 10000 2000 10000' likemin=10 witheexpmap=yes ='0.878 0.220 0.878 0.220 3.652 0.632' eboxl list=eboxlist l.fits xm list=eboxlist m.fits eml list=emllist.fits esp withootset=yes ooteventset=pn-oot-filt-time.fits

> > > > > > > > > > >

imagesets ­ list of count images attitudeset ­ attitude file name pimin ­ list of minimum PI channels for the bands pimax ­ list of maximum PI channels for the bands likemin ­ maximum likelihood threshold witheexpmap ­ create and use exposure maps ecf ­ energy conversion factors for the bands eboxl list ­ output file name for the local sliding box source detection list eboxm list ­ output file name for the sliding box source detection in background map mode list eml list ­ output file name for maximum likelihood source detection list esp withootset ­ Flag to use an out-of-time processed PN event file, useful in cases where bright point sources have left streaks in the PN data > esp ooteventset ­ The out-of-time processed PN event file

The ecfs are in units of 1011 counts cm2 erg-1 . Those used here are derived from PIMMS using the flux in the 0.1­10.0 keV band, a source power-law index of 1.9, an absorption of 0.5 â 1020 cm-2 , and the thin filters. 5) Display the results of eboxdetect using the task srcdisplay. ­ srcdisplay boxlistset=eboxlist m.fits imageset=mosaic.fits regionfile=regionfile.txt sourceradius=0.01 withregionfile=yes > > > > > > b i i r s w o m n e o i x a c g u t l g l i r h i e u o c r s s d n e e t e e f r g s t s i a i et ­ ou le di on ­ eboxdetect source list image file name over which the source circles are to be plotted rces ­ flag to include the source positions on the display ­ file name of output file containing source regions us ­ radius of circle plotted to locate sources file ­ flag to create a region file

6) Display the results of emldetect using the task implot, in this case as a GIF file (pgplot.gif ). ­ implot set=mosaic.fits device=/GIF srclisttab=emllist.fits > set ­ input image for the plot > device ­ type of output (/GIF, /PS, /XW) > srclisttab ­ source list file name Figure 5.13 shows the output of implot for the maximum likelihood source detection (emldetect).


30 Figure 5.13: EPIC count image with the detected sources from the maximum likelihood task created by implot.

5.5

TIMING ANALYSIS

This section will demonstrate some basic timing analysis of EPIC image-mode data using the Xronos analysis package. (Note: for PN timing and burst mode data, the task epchain must be run with datamode=TIMING|BURST.) These examples assume that an appropriate light curve, named source.lc, has been created as in § 5.3 with timebinsize set to 1 and makeratecolumn set to "no". For this exercise the central source from the observation of G21.5-09 (0122700101) is used. For the aficionado, the task barycen can be used for the barycentric correction of the source event arrival times. 1) Use the Xronos command lcurve to produce a binned lightcurve. The following command will also produce a screen plot using QDP ("quit" or "exit" will exit the QDP session). ­ lcurve nser=1 cfile1=source.lc window=- dtnb=500 nbint=450 outfile=lightcurve.fits plot=yes plotdev=/xw > > > > > > > > nser ­ number of time series cfile1 ­ filename first series window ­ name of window file (if a subset of the time series is required) dtnb ­ bin size (time) nbint ­ number of bins per interval outfile ­ output file name (FITS format light curve) plot ­ plot flag plotdev ­ device for plotting, output shown in Figure 5.14

2) Use the Xronos command powspec calculate power spectrum density. The following command will also produce a screen plot using QDP ("quit" or "exit" will exit the QDP session). ­ powspec cfile1=source.lc window=- dtnb=100.0 nbint=300 nintfm=INDEF rebin=5 plot=yes plotdev=/xw outfile=power.fits > cfile1 ­ filename first series


31 Figure 5.14: Light curve for the source analyzed in § 5.3.

Figure 5.15: Power spectrum density for the source analyzed in § 5.3.

> > > > > >

window ­ name of window file (if a subset of the time series is required) dtnb ­ bin size (time) nbint ­ number of bins per interval nintfm ­ number of intervals in each power spectrum rebin ­ rebin factor for power spectrum (0 for no rebinning) plot ­ plot flag


32 > plotdev ­ device for plotting, output shown in Figure 5.15 > outfile ­ output file name (FITS format power spectrum) 3) Use the Xronos command efsearch to search for periodicities in the time series. The following command will also produce a screen plot using QDP ("quit" or "exit" will exit the QDP session). ­ efsearch cfile1=source.lc window=- sepoch=INDEF dper=20 nphase=10 nbint=INDEF nper=100 dres=INDEF plot=yes plotdev=/xw outfile=efsearch.fits > > > > > > > > > > > cfile1 ­ filename first series window ­ name of window file (if a subset of the time series is required) sepoch ­ value for epoch used for phase zero when folding the time series dper ­ value for the period used in the folding nphase ­ number of phases per period nbint ­ number of bins per interval nper ­ number of sampled periods during search dres ­ sampling resolution of search plot ­ plot flag plotdev ­ device for plotting outfile ­ output file name (FITS format)

4) Use the Xronos command autocor to calculate the auto correlation for an input time series. The following command will also produce a screen plot using QDP ("quit" or "exit" will exit the QDP session). ­ autocor cfile1=source.lc window=- dtnb=24.0 nbint=2048 nintfm=INDEF rebin=0 plot=yes plotdev=/xw outfile=auto.fits > > > > > > > > > cfile1 ­ filename first series window ­ name of window file (if a subset of the time series is required) dtnb ­ bin size (time) nbint ­ number of bins per interval nintfm ­ number of intervals to be summed in each autocorrelation function rebin ­ rebin factor for autocorrelation function (0 for no rebinning) plot ­ plot flag plotdev ­ device for plotting outfile ­ output file name (FITS format autocorrelation spectrum)

5) Use the Xronos command lcstats to calculate statistical quantities for an input time series. The following command will write the output to an ASCII file. (Leave off the > fname to write the results to the screen.) ­ lcstats cfile1=source.lc window=- dtnb=6.0 nbint=8192 > fname > > > > > cfile1 ­ filename first series window ­ name of window file dtnb ­ integration time (binning) nbint ­ number of bins fname ­ output file name

5.6

ONCE MORE, THIS TIME WITH FEELING AND FTOOLS

Most of the data extraction described in the previous sections can be done equally well in Ftools, and will be illustrated here using fselect and Xselect. Note that the HEASoft package is incorporated into SAS and so if SAS is operational, fselect and Xselect will be available. Keith Arnaud is responsible for the XMM-Newton-specific tools mentioned below, which he describes at: http://lheawww.gsfc.nasa.gov/users/kaa/xselect/xmm.html


33 1) Filter the event file using the xmmclean perl script provided by Arnaud at the HTML page above. ­ fselect mos1.fits mos1-filt.fits "FLAG ==0 && TIME <= 73227600) &&!(TIME in [73221920:73223800]) && PATTERN <= 12 && PI <= 12000 && PI >= 200" > FLAG < 65536 is the equivalent of the xmmselect expression #XMMEA EM, for PN data also use "FLAG < 65536" 2) Invoke an Xselect session. ­ xselect > Enter a session name or default with a carriage return 3) Read in the event list. ­ read events mos1-filt.fits > Enter the directory containing the event file > Enter yes to reset the mission 4) Create and plot an image (this will spawn a ds9 window). ­ extract image ­ plot image 5) Create and plot a light curve (this will spawn a Pgplot window). ­ Invoke the command extract curve ­ Invoke the command plot curve 6a) Filter on time using the cursor and light curve plot. ­ filter time cursor, then follow the instructions ­ Enter "quit" at the PLT prompt ­ Right click at the start and end points of the time intervals to keep ­ When done entering intervals enter "x" on the keyboard 6b) Filter on time using a threshold intensity. ­ filter intensity, then give the filter range, e.g., "0.01-3.0" 7a) Create the extraction region for the source. ­ Extract and plot a new image with the temporal filter ­ Create a region on the ds9 window > In the ds9 window pull down the Region menu and set 1) the File Format to DS9/Funtools, 2) the File Coordinate System to Equatorial J2000, and 3) the Region Coordinate System to Degrees ­ Adjust the region to be appropriate for the source of interest ­ Under the region menu select the Save Regions option ­ Save the region as a file (e.g., ds9-source.reg) 7b) Create an annulus extraction region for the background. ­ If necessary, resize the existing region to be appropriate for the inner annulus radius ­ Pull down the Region menu and select Exclude under Properties ­ Create a second region on the ds9 window ­ Adjust the region to be appropriate for the outer boundary of the annulus


34 ­ Pull down the Region menu and select Include under Properties ­ Make sure that the outer annulus is "in front" by selecting the Move to Front option under the Region menu. ­ Under the region menu select the Save Regions option ­ Save the region (e.g., ds9-back.reg) 8) Filter the data using the source region. ­ filter region ds9-source.reg 9) Extract, plot, save the spectrum from the source region and create RMF and ARF files. ­ extract spectrum ­ plot spectrum ­ save spectrum resp=yes The resp=yes runs the perl script xsl xmm epic makeresp which is available from Arnaud's web page above > Enter a file name for the spectrum, e.g., mos1-source.pi > Bin the data (i.e., enter yes at the query) 10) Filter the data using the background region. ­ First remove the source filter expression: clear region all ­ filter region ds9-back.reg 11) Extract, plot, and save the spectrum from the background region. ­ extract spectrum ­ plot spectrum ­ save spectrum > Enter a file name for the spectrum, e.g., mos1-back.pi > Bin the data (i.e., enter yes at the query) 12) Extract, plot, and save the light curve from the region. ­ First remove the source filter expression: clear region all ­ filter region ds9-source.reg ­ extract curve binsize=1000 phalcut t=300 phahcut t=10000 > use binsize=1 to create a light curve for timing analysis > use phalcut t to set the lower energy bound for the light curve > use phahcut t to set the upper energy bound for the light curve ­ plot curve ­ save curve > Enter a file name for the light curve From this point follow the procedures in § 5.3.3 and § 5.3.4 for spectral analysis and § 5.5 for temporal analysis.


35

5.7

ODF DATA

The ODF names for the EPIC data will look something like: · mmmm iiiiiijjkk aabeeeccfff.zzz mmmm ­ revolution orbit number iiiiiijjkk ­ observation number aa ­ detector ID (M1 - MOS1, M2 - MOS2, PN - PN). b ­ flag for scheduled (S), unscheduled (U) observations, or (X) for general use files. eee ­ exposure number within the observation cc ­ CCD identifier. fff ­ data identifier (see Table 5.2) zzz ­ Format (FITS - FIT, ASCII - ASC)

Table 5.2: EPIC ODF data files1 . Data ID Contents

I R C T B A C H H H P P D P P
1

M I T I U U C B C T E T L A M

E E E E E X X H H H H H I H H

Event list for individual CCDs, imaging mode Event list for individual CCDs, reduced imaging mode Event list for individual CCDs, compressed timing mode Event list for individual CCDs, timing mode Event list for individual CCDs, burst mode Auxiliary file Counting cycle report (auxiliary file) HBR buffer size, non-periodic housekeeping HBR configuration, non-periodic housekeeping HBR threshold values, non-periodic housekeeping Periodic housekeeping Bright pixel table, non-periodic housekeeping Discarded lines data Additional periodic housekeeping Main periodic housekeeping

From the document GEN-ICD-0004-2-8.

5.7.1

Rerunning the EPIC Chains

When the CCF is updated it may be necessary to rerun the basic pipeline processing (see § 4.5.1), and luckily the process is reasonably simple. This next set of tasks will reproduce the calibrated photon event files found in the pipeline products. (Note: for reference, an executable log file of the entire pipeline processing can be found in the pipeline product *SCRLOG*.) Since data in the public archive are typically at least a year old (although with the coming reprocessing this will change, for a while) older versions of both the CCF and SAS were used, it is therefore useful to rerun the pipeline processing to reproduce the event files. 1) If necessary, rename all files in the ODF directory to upper case. This can be done using the script provided by the NASA/GSFC XMM-Newton GOF. 2) Initialize SAS (see § 4). 3) Create a CIF file using the SAS task cifbuild (§ 4.5.1). If a CIF file has previously been produced, it is only necessary to rerun cifbuild if the CCF has changed.


36 4) Run the SAS task odfingest (§ 4.5.2). It is only necessary to run it once on any data set (and will cause problems if it is run a second time). If for some reason odfingest must be rerun, first delete the earlier *.SAS (the file produced by odfingest). 5) Run the enter: emchain emchain data set e.g.: emchain where n SAS task emchain. From the command line of a window where SAS has been initialized, simply

processes the data from both MOS instruments producing calibrated photon event files. If the has more than one exposure, a specific exposure can be accessed using the exposure parameter, exposure=n is the exposure number.

6) Run the SAS task epchain, which processes the data from PN instrument producing a calibrated photon event file. From the command line of a window where SAS has been initialized, simply enter: epchain To create an out-of-time event file, use the command: epchain withoutoftime=yes Adding the parameter keepintermediate=none causes epchain to discard a number of intermediate files. Once the chains have completed with new event files the same analysis techniques described in the previous sections can used.

5.8

A More-or-Less Complete Example

The Lockman Hole ODF data have been used for a reasonably complete example of the EPIC data reduction. The data can be found via the XSA at: http://xmm.vilspa.esa.es/xsa/ or via Browse at: http://heasarc.gsfc.nasa.gov/db-perl/W3Browse/w3browse.pl while the script (run.com) and output data files (except for the unfiltered event lists which are huge) can be found at: ftp://legacy.gsfc.nasa.gov/xmm/data/examples/epic/ The script assumes that SAS V6.0 has been set up to run. The commands to set the CCF and ODF directories as well as the ccf.cif file are included but will need to be changed for the specific setup. The data were processed using SAS V6.0. The entire process took two hours on a relatively new linux RH7.3 machine (1.67 GHz processor, 2 GB RAM). The result is about a gigabyte of new files. The script uses the SAS command-line interface, however in its creation the GUI interface to xmmselect was used to find the time filtering and source extraction parameters. The script goes through the following steps. 1) Sets a few SAS parameters. 2) Runs cifbuild and odfingest to prepare for SAS analysis. 3) Runs emchain to produce calibrated photon event files for the MOS1 and MOS2 detectors 4) Creates images and light curves of the MOS data 5) Filters the MOS event files to exclude bad events and times of background flares 6) Creates images and light curves of the filtered MOS data 7a) Runs epchain to produce a calibrated photon event file for the PN detector 7b) Runs epchain a second time to produce a calibrated photon event file for the PN detector of out-of-time events 8) Repeats items 4 - 6 for the PN data 9) Does source detection for two bands for each of the three detectors


37 10) Extracts source and background spectra for a brighter field source 11) Creates an RMFs and ARFs for the source 12) Groups the spectra using grppha 13) Included in the script, but commented out, are the commands to fit the source spectra using Xspec 14) Creates a light curve for the source 15) Included in the script, but commented out, are the commands to analyze the source light curve using Xronos


Chapter 6

First Look ­ RGS Data
Before beginning this chapter please consult the "watchout" page at the VILSPA SOC: · http://xmm.vilspa.esa.es/sas/documentation/watchout This web site discusses current and past SAS bugs and analysis issues, e.g., regarding missing libraries when using rgsproc with SAS V6.

6.1
6.1.1

A PRELIMINARY FIT
Pipeline Products

You will find a variety of RGS-specific files in XMM-Newton data sets. Generally there are two of each because there are two RGS instruments. Table 6.1 lists typical file names, their purpose, the file format, and a list of tools that will enable the user to inspect their data. As usual, there are some HTML products to help you inspect the data with file names of the form (note that we will use the generic form of the name in the following examples): · PPiiiiiijjkkAAAAAA000 0.HTM, where iiiiii ­ proposal number jj ­ observation ID - target number in proposal kk ­ observation ID - observation number for target AAAAAA ­ Group ID (Table 6.1) NOTE: The ten-digit combination of iiiiiijjkk is the observation number and is used repetitively throughout the file nomenclature The INDEX.HTML file will help you navigate. The data file names are of the form: · PiiiiiijjkkaablllCCCCCCnmmm.zzz, where iiiiii ­ proposal number jj ­ observation ID - target number in proposal kk ­ observation ID - observation number for target aa ­ detector, R1 ­ RGS1, R2 ­ RGS2 b ­ S for scheduled observation, U for unscheduled lll ­ exposure number CCCCCC ­ file identification (Table 6.1) n ­ spectral order number, unimportant otherwise mmm ­ source number zzz ­ file type (e.g., PDF, PNG, FTZ, HTM) 38


39 Table 6.1: RGS Pipeline Processing data files. Group ID File ID Contents File Type View With

REVLIS

SRCLI EVENLI EXPMAP S S B B S R R G G R S S S S S P P P P P E E E E E C C C C C 1 2 1 2

RGS Source Lists RGS Event lists RGS Exposure Maps 1st Order Source Spectra 2nd Order Source Spectra 1st Order Back. Spectra 2nd Order Back. Spectra Spectra Plots Images, disp. vs. X-disp Images, disp. vs. PI

Zipped FITS Zipped FITS Zipped FITS Zipped Zipped Zipped Zipped PDF fo FITS FITS FITS FITS rmat

fv xmmselect, fv ds9, Ximage, fv Xspec, fv Xspec, fv Xspec, fv Xspec, fv Acrobat reader ds9, Ximage, fv, Netscape ds9, Ximage, fv, Netscape

REXPIM RSPECT

RIMAGE

ORDIMG IMAGE

Zipped FITS, PNG Zipped FITS, PNG

F P H P

T N T D

Z G M F

­ ­ ­ ­

gzipped FITS format, use ds9, Ximage, Xselect, fv use Netscape or other web browser use Netscape or other web browser Portable Data Format, use Acrobat Reader

6.1.2

Preparation for Running SAS Tasks

1) Ensure that you have created a Calibration Index File, using cifbuild (§ 4.5.1). 2) Ensure that you have created a summary file of your ODF constituents (just once) and deposited it in your ODF directory using the SAS task odfingest (§ 4.5.2).

6.1.3

Creating Response Matrices

Response matrices and ancillary response files are not provided as part of the pipeline product package, so a user must create their own before analyzing data. The SAS package rgsrmfgen generates an RMF and ARF and combines them within a single RSP file. The following command demonstrates this using the pipeline products above: · rgsrmfgen file=RGS1 ORDER1.RSP evlist=PiiiiiijjkkaablllEVENLInmmm.FTZ withspectrum=yes spectrumset=PiiiiiijjkkaablllSRSPEC1mmm.FTZ emin=0.3 emax=2.8 ebins=4000 > file ­ the name of the output response matrix. > evlist ­ the event list from which the spectrum was extracted. > withspectrum ­ Use the spectrum file product to calculate the RMF. > spectrumset ­ name of the spectrum file from the pipeline products. Source, order, background, and response channel binning will be taken from this file. > emin ­ the lower energy limit of the RSP file. > emax ­ the upper energy limit of the RSP file. > ebins ­ The number of bins calculated between emin and emax. The task documentation suggests this number be > 3000.


40 The response files take many factors into account, such as pointing, pile-up, telemetry saturation, hot pixels, instrument temperatures, etc. Therefore it is imperative to create new response files after any filtering of the data and the same response should never be used for fitting two different observations or pointings. Note that if the pipeline products were created with SAS v5.0 this procedure will fail because of a more recent code alteration. To construct a response matrix, the pipeline should be re-run first using the latest software (SAS V6) and calibration.

6.1.4

Fitting a Spectral Model to the Data

Now use XSPEC to fit an appropriate model to your spectrum: x s p ec XSPEC>data PiiiiiijjkkaablllSRSPEC1mmm.FTZ XSPEC>back PiiiiiijjkkaablllBGSPEC1mmm.FTZ XSPEC>resp RGS1 ORDER1.RSP XSPEC>ignore bad XSPEC>model wabs*mekal wabs:nH>1 mekal:kT>1 mekal:nH> mekal:Abundanc>0.4 mekal:Redshift> mekal:Switch>0 mekal:norm>1 XSPEC>renorm XSPEC>fit XSPEC>setplot device /xs XSPEC>setplot wave XSPEC>setplot command window all XSPEC>setplot command log x off XSPEC>setplot command wind 1 XSPEC>setplot command r y 1e-5 1.6 XSPEC>setplot command wind 2 XSPEC>setplot command r y -9.99 9.99 XSPEC>plot data residuals XSPEC>exit Do you really want to exit? The plot is provided in Figure 6.1. Please note: · PiiiiiijjkkaablllSRSPECnmmm.FTZ is a "net" spectrum. This is the source+background events minus the background events which were extracted from a different detector region. A number of xspec functions will yield erroneous results using net spectra ­ see Sec. 6.6 · PiiiiiijjkkaaSlllBGSPECnmmm.FTZ is a background spectrum. Consequently, when analyzing the net file with XSPEC or CIAO, DO NOT employ this file as a background for your data. It has already been subtracted. (y)y


41 Figure 6.1: 1st order RGS1 spectrum of AB Dor. The fit is an absorbed single-temperature mekal model. The gap between 10­15° is due to the absence of CCD7. A

6.2

FILTERING EVENTS

Solar flares result in periods of high background. Observers may find an appreciable increase in signal-to-noise if they remove flare events from their data. The general SAS task evselect does not correct the RGS exposure maps during filtering which is vital in order to fit data accurately. Consequently the RGS-specific task rgsfilter must be run in order to perform any filtering of the data. As with the ma jority of RGS tasks, rgsfilter can be called from the meta-task rgsproc which provides a convenient interface between the user and the entire RGS pipeline. This section provides an example of how to produce a time-filtered spectrum.

6.2.1

Creating and plotting a light curve

Create a FITS light curve with 100 second binning from the pipeline product event file using the SAS task evselect (alternatively use the xmmselect GUI). Being closer to the optical axis, CCD9 is most susceptible to proton events and generally records the least source events, therefore we will extract events over this CCD only. Also, to avoid confusing solar flares for source variability, a region filter that that removes the source from the final event list should be used. The region filters are kept in the source file product PiiiiiijjkkaablllSRCLI nmmm.FTZ ·e m e B v a x E s k p T e e r A lect table=PiiiiiijjkkaablllEVENLInmmm.FTZ withrateset=yes rateset=RGS1 RATE.FIT ratecolumn=yes maketimecolumn=yes timebinsize=100 ession='(CCDNR==9)&&(REGION(PiiiiiijjkkaablllSRCLI nmmm.FTZ:RGS1 BACKGROUND, CORR,XDSP CORR))' t w r m m t e a i a a a i x b t t k k m p l h e e e e r e r s r t b e ­ at et at im in ss event list from the pipeline products. eset ­ create a light curve. ­ name of the resulting FITS file. ecolumn ­ create a rate column. ecolumn ­ create a time column. size ­ bin the time column to this size (in units of seconds). ion ­ filter expression.

> > > > > > >

Plot the light curve using the SAS tool dsplot (see Figure 6.2): · dsplot table=RGS1 RATE.FIT x=TIME y=RATE


42 Figure 6.2: Background event rate from the RGS1 CCD9 chip. The flares are solar events. The time units are elapsed mission time.

R1_RATE.FIT
RATE 1

0.8

RATE [count/s]

0.6

0.4

0.2

0 9.639e+07

9.64e+07

9.641e+07

9.642e+07 TIME [s]

9.643e+07

9.644e+07

9.645e+07

6.2.2

Creating a GTI file

The CCD9 quiescent count rate within the background region mask is 0.05 counts per second. In this example there are two intervals of significant background flaring. Determine which intervals should be rejected and write these time intervals to an ASCII file, gti.asc, as follows: 9.6405e7 9.6413e7 - 9.6422e7 9.6425e7 - The first two columns provide the start and stop times (in seconds since the start of the mission) of intervals to be filtered. The third column can be a '+' (good time interval) or '-' (bad time interval). In this case two intervals of high background activity are excluded. Execute gtibuild to convert the above into a FITS format GTI file: · gtibuild file=gti.asc table=GTI.FIT > file ­ ASCII file of time intervals > table ­ GTI FITS file Alternatively, a GTI table can be created using a Boolean expression to record times of acceptably low count rate with the task tabgtigen: · tabgtigen table=R1 RATE.FIT gtiset=GTI.FIT expression='(RATE<0.2)' > table ­ Input data file. > gtiset ­ Output GTI table. > expression ­ Boolean expression.

6.2.3

Running the RGS Pipeline

One can now re-run the complete RGS pipeline using the SAS meta-task rgsproc.


43 Table 6.2: rgsproc output data files.

Data Type

Extension

File Type

Contents

A a h S m E E S B m f

T t k R e V X R G a l

T t g C r E P S S t u

T g t L g N M P P r x

S t i I e L A E E i e

R i

d I P C C x d

FI FI FI FI FI FI FI FI FI FI FI

T T T T T T T T T T T

FI FI FI FI FI FI FI FI FI FI FI

T T T T T T T T T T T

S S S S S S S S S S S

table table table table table table image table table table table

attitude information for the complete observation. good time intervals from the attitude history. good time intervals from the housekeeping files. list of sources and extraction masks. event list merged from all CCDs. merged and filtered event list. exposure map. source spectrum. background spectrum. response matrix. fluxed spectrum. For quick and dirty inspection only.

· rgsproc orders='1 2' auxgtitables=GTI.FIT bkgcorrect=no withmlambdacolumn=yes > orders ­ the spectral orders to extract. > auxgtitables ­ a list of GTI files > bkgcorrect ­ Subtract background from source spectra? > withmlambdacolumn ­ include a wavelength column in the event file product (we will use this to generate a "dirty" spectrum plot later). Note: If an error message that a library is missing occurs, follow these steps: · go to the XMM-Newton SAS v6.0 ftp site ftp://xmm.vilspa.esa.es/pub/sas/6.0.0/ · go to the directory which contains the SAS V6 version for your platform · download the package: platform-htrframes.tar.gz · go to your $SAS DIR · gunzip platform-htrframes.tar.gz · tar -xvf platform-htrframes.tar bkgcorrect=no will yield a source spectrum with background events included. The background level will be automatically subtracted if bkgcorrect=yes. Unless the spectra are of high signal-to-noise, it is recommended that scientific analysis only be carried out on those spectra where bkgcorrect=no. However, note that the fluxed spectrum (which is only suitable for initial data inspection) is best examined at after declaring bkgcorrect=yes. New files are written to the working directory. Table 6.2 lists these, and all are uncompressed FITS files. The filenames are of the same form given in Section 6.1.1: Even if no solar flares occurred during the observation, it is recommended that the pipeline is re-run in order to take advantage of the most up-to-date calibration and ensure that region filters more appropriate for the source are created.

6.2.4

Inspecting New Products

To take a first look at these new products try the following recipes. 1) To examine images of dispersion versus PI and cross-dispersion directions:


44 Figure 6.3: Images over the dispersion­cross-dispersion plane (top) and the dispersion­pulse height plane (bottom). The lower and upper bananas are 1st and 2nd order events respectively. The blue lines define the source extraction regions, one spatial and the other over PI. Horizontal blue lines delineate the internal calibration sources. The regular chevron background pattern in the right hand CCDs (1 and 2) are a manifestation of electronic cross-talk. These events have low PI values and are filtered out by the PI masks.

· set srclst = 'PiiiiiijjkkaablllSRCLI nmmm.FIT' · evselect table=PiiiiiijjkkaablllEVENLInmmm.FIT:EVENTS withimageset=yes imageset=spatialimage.fit xcolumn=BETA CORR ycolumn=XDSP CORR > table ­ input events table. > withimageset ­ create an image. > imageset ­ output image file. > xcolumn ­ column in events file to extract


45 > ycolumn ­ column in events file to extract · evselect table=PiiiiiijjkkaablllEVENLInmmm.FIT:EVENTS withimageset=yes imageset=orderimage.fit xcolumn=BETA CORR ycolumn=PI withyranges=yes yimagemin=0 yimagemax=3000 expression='region($srclst:RGS1 SRC1 SPATIAL,BETA CORR,XDSP CORR)' > withyranges ­ set the y range of the image. > yimagemin ­ minimum y limit. > yimagemax ­ maximum y limit. > expression ­ filter expression. This example is filtering events found inside the spatial mask for the source. · rgsimplot withspatialset=yes withendispset=yes spatialset=spatialimage.fit endispset=orderimage.fit withspatialregionsets=yes withendispregionsets=yes srclistset=$srclst srcidlist=1 orderlist='1 2' colourmap=LOG colour=3 device=/XS > withspatialset ­ include spatial image. > withendispset ­ include PI image. > spatialset ­ name of spatial image. > endispset ­ name of PI image. > withspatialregionsets ­ include spatial mask in plot. > withendispregionsets ­ include PI mask in plot. > srclistset ­ name of source list. > srcidlist ­ source number of the target within the source list. Source 1 will correspond to the target coordinates provided in the original proposal. Source 2 will be the camera boresight. > orderlist ­ order of masks to plot > colourmap ­ colour scale. > colour ­ colour scheme for plot. > device ­ plotting device (upper case is mandatory; e.g., /XS, /XSERVE, /PS, /CPS) The output from rgsimplot is provided in Figure 6.3. 2) To plot a light curve from all events: · evselect table=PiiiiiijjkkaablllEVENLInmmm.FIT:EVENTS withrateset=yes rateset=PiiiiiijjkkaablllRATES nmmm.FIT makeratecolumn=yes maketimecolumn=yes timebinsize=100 > table ­ input event list. > withrateset ­ create a light curve. > rateset ­ name of output light curve. > maketimecolumn ­ include an absolute time column. > makeratecolumn ­ create a rate column. > timebinsize ­ bin size (in seconds). · dsplot table=PiiiiiijjkkaablllRATES nmmm.FIT x=TIME y=RATE The resulting curve is provided in Figure 6.4. Note that unlike Figure 6.2 these events have been extracted across the whole detector and that our Good Time constraint has been adhered to. 3) To plot a spectrum with an approximate wavelength scale, use the mlambda table column rather than a response matrix. One important caveat here is that all orders are superimposed on this table:


46 Figure 6.4: Total event rate from RGS1 after Good Time filtering.

P0134520301R1S001RATES_0000.FIT
RATE 6

5

4

RATE [count/s]

3

2

1

0 9.639e+07

9.64e+07

9.641e+07

9.642e+07 TIME [s]

9.643e+07

9.644e+07

9.645e+07

·e h w e r

v i i x e

s s t p g

e t h r i

l o h e o

e g i s n

c r s s (

t a t i $

table=PiiiiiijjkkaablllEVENLInmmm.FIT:EVENTS withhistogramset=yes mset=PiiiiiijjkkaablllQKSPECnmmm.FIT histogramcolumn=M LAMBDA oranges=yes histogrammin=5 histogrammax=40 histogrambinsize=0.0116667 on='region($srclst:RGS1 SRC1 SPATIAL,BETA CORR,XDSP CORR)&& srclst:RGS1 SRC1 ORDER 1,BETA CORR,PI)'

> table ­ input event table. > withhistogramset ­ make a histogram table. > histogramset ­ name of output histogram table. > histogramcolumn ­ event column to histogram. > withhistoranges ­ include only certain ranges. > histogrammin ­ lower limit. > histogrammax ­ upper limit. > histogrambinsize ­ size of histogram bins. > expression ­ filter expression. This one takes only events from inside both the spatial and 1st order PI masks defined within the source list. · dsplot table=PiiiiiijjkkaablllHISTOGnmmm.FIT x=M LAMBDA y=COUNTS The resulting histogram is provided in Figure 6.5.

6.3

PIPELINE EXAMPLES

Several examples of the flexibility of the RGS pipeline are provided below, and these address some of the potential pitfalls for RGS users.


47 Figure 6.5: RGS1 spectrum binned on the approximate wavelength scale provided in the M LAMBDA column. the gap between 10 and 15° is the missing chip CCD7. CCD4 is similarly missing in the RGS2 camera. Both A failed after space operations began.

P0134520301R1S001QKSPEC0000.FIT
HISTO 600

500

400

COUNTS [count]

300

200

100

0

0

10

20 M_LAMBDA [Angstrom]

30

40

6.3.1

A Nearby Bright Optical Source

With certain pointing angles, zeroth-order optical light may be reflected off the telescope optics and cast onto the RGS CCD detectors. If this falls on an extraction region, the current energy calibration will require a wavelength-dependent zero-offset. Stray light can be detected on RGS DIAGNOSTIC images taken before, during and after the observation. This test, and the offset correction, are not performed on the data before delivery. To check for stray light and apply the appropriate offsets use: · rgsproc orders='1 2' bkgcorrect=no calcoffsets=yes withoffsethistogram=no > orders ­ dispersion orders to extract > calcoffsets ­ calculate pha offsets from diagnostic images > withoffsethistogram ­ produce a histogram of uncalibrated excess for the user

6.3.2

A Nearby Bright X-ray Source

In the example above, it is assumed that the field around the source contains sky only. Provided a bright background source is well-separated from the target in the cross-dispersion direction, a mask can be created that excludes it from the background region. Here the source has been identified in the EPIC images and its coordinates have been taken from the EPIC source list which is included among the pipeline products. The bright neighboring ob ject is found to be the third source listed in the sources file. The first source is the target: · rgsproc orders='1 2' bkgcorrect=no withepicset=yes epicset=PiiiiiijjkkaablllEMSRLInmmm.FTZ exclsrcsexpr='INDEX==1&&INDEX==3' > orders ­ dispersion orders to extract. > withepicset ­ calculate extraction regions for the sources contained in an EPIC source list. > epicset ­ name of the EPIC source list.


48 > exclsrcsexpr ­ expression to identify which source(s) should be excluded from the background extraction region. Since this operation will alter only the size of the regions in the sources file, it saves time to not re-make the event table or re-calculate the exposure map. The pipeline can be entered at five different points. In this case one only need start from the spectral extraction stage: · rgsproc orders='1 2' entrystage=spectra finalstage=fluxing bkgcorrect=no withepicset=yes epicset=PiiiiiijjkkaablllEMSRLInmmm.FTZ exclsrcsexpr='INDEX==1&&INDEX==3' > orders ­ dispersion orders to extract. > entrystage ­ entry stage to the pipeline (see Sec. 6.4). > finalstage ­ exit stage for the pipeline (see Sec. 6.4). > withepicset ­ calculate extraction regions for the sources contained in an EPIC source list. > epicset ­ name of the EPIC source list. > exclsrcsexpr ­ expression to identify which source(s) should be excluded from background extraction region. Note that this last example will only work if one has retained the event file from a previous re-running of the pipeline.

6.3.3

User-defined Source Coordinates

If the true coordinates of an ob ject are not included in the EPIC source list or the science proposal, the user can define the coordinates of a new source: · rgsproc orders='1 2' bkgcorrect=no withsrc=yes srcra=81.823317 srcdec=-6.532072 > orders ­ dispersion orders to extract. > withsrc ­ with a user-defined source. > srcra ­ decimal RA of source. > srcdec ­ decimal Dec of source. These coordinates are written to the RGS source list PiiiiiijjkkaablllEMSRLInmmm.FIT with a source ID which, in this example, will be '3'. Creating the source file is one of the first tasks of the pipeline. If these new coordinates correspond to the prime source then the entire pipeline must be run again in order to calculate the correct aspect drift corrections in the dispersion direction. However, if these new coordinates refer to a background source that should be ignored during background extraction, then the ma jority of pipeline processing (drift correction, filtering etc) will remain identical to the previous examples. To save processing time one can create a new source list by hand and then enter the pipeline at a later stage. · rgssources filemode=create srclist=PiiiiiijjkkaablllEMSRLInmmm.FIT atthkset='P0iiiiiijjkkaablllATTTSRnmmm.FIT' writeobskwds=yes writeexpkwds=yes instexpid='R1S001' addusersource=yes label='BACK SOURCE' ra=81.823317 dec=-6.532072 > filemode ­ create or modify an existing source list. > srclist ­ name of resulting source list. > atthkset ­ attitude history file. > writeobskwds ­ write observation keywords to the source list. > writeexpkwds ­ write exposure keywords to the source list. > instexpid ­ instrument/exposure ID. > addusersource ­ add a source to the list. > label ­ label for the new source. > changeprime ­ change the prime source from the proposal coordinates.


49 > userasprime ­ change the prime source to the user added coordinates. > ra ­ RA of user's source. > dec ­ Dec of user's source. · rgsproc orders='1 2' entrystage=spectra finalstage=fluxing bkgcorrect=no exclsrcsexpr='INDEX==1&&INDEX==3' > orders ­ dispersion orders to extract. > entrystage ­ entry stage to the pipeline (see Sec. 6.4). > finalstage ­ exit stage for the pipeline (see Sec. 6.4). > exclsrcsexpr ­ expression to identify which source(s) should be excluded from the background extraction region.

6.4

PIPELINE ENTRY STAGES

There are five stages at which the user can enter or leave the pipeline: · events ­ Creates attitude time series, attitude-drift and housekeeping GTI tables, pulse height offsets, the source list, and unfiltered, combined event lists. · angles ­ Corrects event coordinates for aspect drift and establishes the dispersion and cross-dispersion coordinates. · filter ­ produces filtered event lists, creates exposure maps. · spectra ­ Constructs extraction regions and source and background spectra. · fluxing ­ creates "fluxed" spectra for quick data inspection and response matrices. Provided the filtered event list is retained, users can apply their own filtering by entering the pipeline at the filter stage. Changes in the extraction region sizes can be handled by entering at the spectra stage. If the coordinates of the source differ from those in the original proposal, the pipeline must be run from events. Extraction of spectra with different binning can be achieved at the spectra stage. Recalculation of the response matrices can be done in the final fluxing stage.

6.5

COMBINING RGS1 AND RGS2 SPECTRA

While it is tempting to merge the RGS1 and RGS2 data, or data from different pointings, to provide a single spectrum with a signal-to-noise improvement over either individual spectrum, this is strongly discouraged since it results in data degradation. The pointings of the two instruments are not identical, resulting in different dispersion angles and wavelength scales. Separate response files are always required for each unit. While it is possible to merge spectra and response files, great care must be taken to account for different exposure times, background subtractions, error propagation, and so on. However the resulting response will always have inferior resolution to the originals. It is therefore simpler and more accurate to keep data from the two RGS units separate and use both sets to fit one model in tandem: xspec XSPEC>data 1:1 PiiiiiijjkkaablllSRSPEC1mmm.FIT 1:2 PiiiiiijjkkaablllSRSPEC2mmm.FIT XSPEC>ignore bad XSPEC>model phabs*mekal etc...


50

6.6

APPROACHES TO SPECTRAL FITTING

For data sets of high signal-to-noise and low background, where counting statistics are within the Gaussian regime, the data products above are suitable for analysis using the default fitting scheme in XSPEC, 2 minimization. However for low count rates, in the Poisson regime, 2 -minimization is no longer suitable. With low count rates in individual channels, the error per channel can dominate over the count rate. Since channels are weighted by the inverse-square of the errors during 2 model fitting, channels with the lowest count rates are given overly-large weights in the Poisson regime. Spectral continua are consequently often fit incorrectly ­ the model lying underneath the true continuum level. This will be a common problem with most RGS sources. Even if count rates are large, much of the flux from these sources can be contained within emission lines, rather than continuum. Consequently even obtaining correct equivalent widths for such sources is non-trivial. There are two approaches to fitting low signal-to-noise RGS data, and the correct approach would normally be to use an optimization of the two.

6.6.1
By the cha the

Spectral Rebinning

grouping channels in appropriately large numbers, the combined signal-to-noise of groups will jump into Gaussian regime. The FTOOL grppha can group channels using an algorithm which bins up consecutive nnels until a count rate threshold is reached. This method conserves the resolution in emission lines above threshold while improving statistics in the continuum. · grppha > Please enter PHA filename[] PiiiiiijjkkaablllSRSPEC1mmm.FIT > Please enter output filename[] !PiiiiiijjkkaablllSRSPEC1mmm.FIT > GRPPHA[] group min 30 > GRPPHA[] exit

The disadvantage of using grppha is that, although channel errors are propagated through the binning process correctly, the errors column in the original spectrum product is not strictly accurate. The problem arises because there is no good way to treat the errors within channels containing no counts. To allow statistical fitting, these channels are arbitrarily given an error value of unity, which is subsequently propagated through the binning. Consequently the errors are over-estimated in the resulting spectra. An alternative approach is to bin the data during spectral extraction. The easiest way to do this is call the RGS pipeline after the pipeline is complete. The following rebins the pipeline spectrum by a factor 3: · rgsproc orders='1 2' rebin=3 rmfbins=4000 entrystage=spectra finalstage=fluxing bkgcorrect=no > orders ­ dispersion orders to extract. > rebin ­ wavelength rebinning factor. > rmfbins ­ number of bins in the response file (> 3000 is recommended by the SAS documentation). > entrystage ­ entry stage to the pipeline (see Sec. 6.4). > finalstage ­ exit stage for the pipeline (see Sec. 6.4). One disadvantage of this approach is that one can only choose integer binning of the original channel size. To change the sampling of the events the pipeline must be run from angles or earlier: · rgsproc orders='1 2' nbetabins=1133 rmfbins=4000 entrystage=angles finalstage=fluxing bkgcorrect=no > nbetabins ­ number of bins in the dispersion direction. The default is 3400. The disadvantage of using rgsproc, as opposed to grppha, is that the binning is linear across the dispersion direction. Velocity resolution is lost in the lines; e.g., the accuracy of redshift determinations will be degraded, transition edges will be smoothed and neighboring lines will become blended.


51

6.6.2

Maximum-Likelihood Statistics

The second alternative is to replace the 2 -minimization scheme with the Cash maximum-likelihood scheme when fitting data. This method is much better suited to data with low count rates and is a suitable option only if one is running XSPEC v11.1.0 or later. The reason for this is that RGS spectrum files have prompted a slight modification to the OGIP standard. Because the RGS spatial extraction mask has a spatial-width which is a varying function of wavelength, it has become necessary to characterize the BACKSCL and AREASCL parameters as vectors (i.e., one number for each wavelength channel), rather than scalar keywords as they are for data from the EPIC cameras and past missions. These quantities map the size of the source extraction region to the size of the background extraction region and are essential for accurate fits. Only XSPEC v11.1.0, or later versions, are capable of reading these vectors, so ensure that one has an up-to-date installation at your site. One caveat of using the cstat option is that the scheme requires a "total" and "background" spectrum to be loaded into XSPEC. This is in order to calculate parameter errors correctly. Consequently, be sure not to use the "net" spectra that were created as part of product packages by SAS v5.2 or earlier. To change schemes in XSPEC before fitting the data, type: · XSPEC>statistic cstat

6.7
6.7.1

ANALYSIS OF EXTENDED SOURCES
Region masks for extended sources

The optics of the RGS allow spectroscopy of reasonably extended sources, up to a few arc minutes. The width of the spatial extraction mask is defined by the fraction of total events one wishes to extract. With the default pipeline parameter values, 90% of events are extracted, assuming a point-like source. Altering and optimizing the mask width for a spatially-extended source may take some trial and error, and, depending on the temperature distribution of the source, may depend on which lines one is currently interested in. While AB Dor is not an extended source, the following example increases the width of the extraction mask and ensures that the size of the background mask is reduced so that the two do not overlap: · rgsproc orders='1 2' entrystage=spectra finalstage=fluxing bkgcorrect=no xpsfincl=99 xpsfexcl=99 pdistincl=95 > orders ­ dispersion orders to extract. > xpsfincl ­ Include this fraction of point-source events inside the spatial source extraction mask. > xpsfexcl ­ Exclude this fraction of point-source events from the spatial background extraction mask. > pdistincl ­ Include this fraction of point-source events inside the pulse height extraction mask. Observing extended sources effectively broadens the psf of the spectrum in the dispersion direction. Consequently it is prudent to also increase the width of the PI masks using the pdistincl parameter in order to prevent event losses.

6.7.2

Fitting spectral models to extended sources

RGS response matrices are consistent for point sources only. Since extended source spectra are broadened, the simplest way to deal with this problem during spectral fitting is to reproduce the broadening function, and convolve it across the spectral model. XSPEC v11.2 contains the convolution model rgsxsrc. It requires two external files to perform the operation. 1. An OGIP FITS image of the source. The better the resolution of the image, the more accurate the convolution. For example, if a Chandra image of the source is available, this will provide a more accurate result than an EPIC image. 2. An ASCII file called, e.g. xsource.mod, containing three lines of input. It defines three environment variables and should look like this example:


52 RGS XSOURCE IMAGE ./MOS1.fit RGS XSOURCE BORESIGHT 23:25:19.8 -12:07:25 247.302646 RGS XSOURCE EXTRACTION 2.5 > RGS XSOURCE IMAGE ­ path to the source image. > RGS XSOURCE BORESIGHT ­ RA, Dec of the center of the source and PA of the telescope. > RGS XSOURCE EXTRACTION ­ The extent (in arcmin), centered on the source, over which you want to construct the convolution function. You want this "aperture" to be larger than the source itself. To set these environment variables within XSPEC execute the command: · xset rgs xsource file xsource.mod Here is an example (Note that the spectral order is always negative, e.g. -1, -2...): x s p ec XSPEC>data P0108460201R1S004SRSPEC1003.FIT XSPEC>ignore bad XSPEC>xset rgs xsource file xsource.mod XSPEC>model rgsxsrc*wabs*mekal rgsxsrc:order>-1 wabs:nH>1 mekal:kT>2 mekal:nH>1 mekal:Abundanc>1 mekal:Redshift> mekal:Switch>0 mekal:norm>1 XSPEC>renorm XSPEC>fit XSPEC>setplot device /xs XSPEC>setplot wave XSPEC>setplot command window all XSPEC>setplot command log x off XSPEC>plot data residuals XSPEC>exit Do you really want to exit? (y)y

Fig. 6.6 compares a point source model with an extended source counterpart.

6.7.3

Model limitations

Users should be aware that this method assumes an isothermal source (or uniform emissivity from line to line in the case of a non-thermal spectrum) where the spatial distributions of all the lines are identical. In reality, however, the thermal structure of the source is likely to be more complicated. The broad-band convolution function may bear little resemblance to the correct function for particular line transitions. One way around this problem would be to have a temperature map of the source to define line emissivity across the source and convolve the model spectrum accordingly. The RGS instrument team at the Columbia Astrophysics Laboratory are developing a Monte Carlo code to perform an operation with this effect. While it is unlikely the code will be publicly available in the near future, the team welcomes investigators who would be interested in collaboration. Contact John Peterson .


53 Figure 6.6: The top figure is a thin, thermal plasma at 2 keV from a point source. The lower figure is the same spectral model, but convolved by the MOS1 0.3­2.0 keV spatial profile of a low-redshift cluster.

6.8

A MORE-OR-LESS COMPLETE EXAMPLE

The AB Doradus PV ODF data (ObsID: 0134520301 from orbit 0205) have been used for a reasonably complete example of RGS data reduction. The script can be found at: · ftp://heasarc.gsfc.nasa.gov/xmm/data/examples/rgs/RGS ABC.SC The lines of the script for setting up and running SAS are specific to installation at GSFC and so will need to be modified as appropriate. The script uses the SAS command-line interface and goes through the following steps: 1) Copies the raw and pipelined data from the XMM archive. 2) Initializes SAS. 3) Creates a Current Calibration file. 4) Builds an ODF summary file. 5) Constructs a GTI file based on background activity. 6) Runs the RGS pipeline. 7) Makes a few useful data inspection products. 8) Fits a model to one of the resulting spectra.


Chapter 7

First Look ­ OM Data
The OM is somewhat different from the other instruments on-board XMM-Newton, and not only because it is not an X-ray instrument. Since the OM pipeline products can be used directly for most science analysis tasks, a re-processing of the data is not needed in most cases. So in principle one can ignore the files in the ODF directory and go directly to §7.1, which describes the files in the PPS (or PIPEPROD) directory. Users interested in re-processing of the OM data can go directly to §7.2 which explains the pipeline processing. For the analysis of OM data obtained in FAST or GRISM mode, however, a re-processing of data is needed, which is explained in more detail in §7.2.2 and §7.2.3.

7.1

PIPELINE PRODUCTS

You will find a variety of OM-specific files in your data directories. The pipeline products differ slightly with different versions of the SAS software. We give a brief description of the files produced by SAS V6, and discuss the important differences with older pipeline products. For a complete description of all files check the pipeline products documentation, which can be found at: http://xmmssc-www.star.le.ac.uk/pubdocs/SSC-LUX-SP-0004.ps.gz

7.1.1

Imaging Mode

The PPS directory for the OM products contains files with the following nomenclature: · PjjjjjjkkkkOMlmmmNNNoooo.zzz jjjjjj ­ Proposal number kkkk ­ Observation ID l ­ S (scheduled), U (unscheduled), or X (general) mmm ­ A number either of the form of 005/6 or 401/2 NNN ­ File ID (see Table 7.1) oooo ­ Either 0000 (high res) or 1000 (low res) zzz ­ File type (FTZ, PNG, PDF, HTM,..) The pipeline produces a summed sky image for each of the filters in low resolution. The results are in files with the nomenclature: · PjjjjjjkkkkOMX000RSIMAGbb000.QQQ jjjjjj ­ Proposal number kkkk ­ Observation ID b ­ Filter keyword: B, V, U, L (UVW1) and S (UVW2) zzz ­ File type (e.g., PNG, FTZ) 54


55 Table 7.1: OM Pipeline Processing data files. Group ID File ID Contents File Type View With

OIMAGE OMSLIS OMSRTS

SIMAGE SWSRLI TSTRTS

OM Sky Image OM Source Lists Tracing Star Time Series

Gzipped FITS Zipped FITS Zipped FITS

ds9, Ximage, fv fv fv

For example, P0123456789OMX000RSIMAGB000.FTZ is the low-resolution final image in the B filter of the observation 0123456789 in sky coordinates (indicated by the S before the IMAG). The letter L is used for the UVW1 filter and S for UVW2. The keyword XPROC0 in the FITS header lists the files which have been added to create the final image P0123456789OMX000RSIMAGB000.FTZ. The keyword looks like this: X C C C C C P O O O O O R N N N N N O T T T T T C I I I I I 0 N N N N N U U U U U = E E E E E ' ' ' ' ' ' ommosaic imagesets=''"product/P0123456 "product/P0123456789OMS409SIMAGE1000. 89OMS410SIMAGE1000.FIT" "product/P0123 FIT" "product/P0123456789OMS412SIMAGE1 product/P0123456789OMX000RSIMAGS000.FI ommosaic-1.2.1) [xmmsas_20011206_171378 FI 45 00 T' no 9OMS008SIMAGE1000.FIT"&' T" "product/P01234567&' 6789OMS411SIMAGE1000.&' 0.FIT"'' mosaicedset=''&' ' sampling=''point'' # (&' -aka]'

The identification/coupling of the files (product/P0123456789OMS410SIMAGE1000.FIT) are identical to the ones described at the beginning of the previous section. Table 7.2: Some of the important columns in the SWSRLI FITS file. Column name Contents

SRCNUM RA DEC POSERR RATE RATE ERR SIGNIFICANCE MAG MAGERR

Source number RA of the detected source Dec of the detected source Positional uncertainty extracted count rate error estimate on the count rate Significance of the detection (in ) Brightness of the source in magnitude uncertainty on the magnitude

Creating images with the OM pro ducts If the observation data products do not contain any mosaic files for all the exposures but about 10 files per filter, it means that they were processed with an older version of the pipeline. In this case we recommend a re-processing of the OM data with SAS V6 using omichain. The omichain task automatically produces one single final file per filter. If there are multiple OM exposures of the same field, ommosaic can be used to create one single image covering the full field of view. One must specify which files are to be added (the program does not do this automatically) so one must know which files were produced for which filters, and at which


56 Figure 7.1: Merged OM image of the Lockman Hole SV1 observation obtained with the V filter. The image is displayed in logarithmic scale with an upper cut value of 20,000.
DEC 57:40

57:35

57:30

57:25

57:20

10h54m00s

10h53m30s

10h53m00s 10h52m30s RA

10h52m00s

10h51m30s

resolution. The task ommosaic be also be used to combine images observed with different filters. Note that the final image is not corrected for coincidence losses or for deadtime. Thr (OBS-ID data and Hole SV1 oughout the OM section of this ABC Guide, public data from the Lockman Hole SV1 observation 0123700101) have been used to illustrate the SAS tasks. We suggest that the user download these to retrace the following procedures. Figure 7.1 shows the merged V-band image from the Lockman observation using the ommosaic task.

You can also use a program written at the NASA/GSFC XMM-Newton GOF. The task is meant to be used on files in the PPS directory (which contains the outputs of the OM pipeline). It produces a final event and exposure images in sky coordinates for each of the filters used in the observation. Low and high-resolution images are treated separately. The task requires that FTOOLS and Perl are installed on your machine and the script must be run from a writable directory which contains the OM files. The tar file with the script is available at the GOF site: ftp://legacy.gsfc.nasa.gov/xmm/software/om tools/om prod all.tar.gz. The program is fairly easy to use, and to modify. If any problems arise with the task please contact the GOF.

7.1.2

Fast Mode

Most OM data are obtained in Imaging mode. If the default included the Fast mode, there will be an additional event list file corresponding to the Fast window (*FAE.FIT). We suggest a re-processing of data obtained in FAST mode, which is explained in detail in §7.2.2.

7.1.3

Grism Mode

OM Grism data require a re-processing, which is explained in detail in §7.2.3.


57

7.2

RE-PROSESSING OF ODF FILES

The OM can operate in IMAGING, FAST, and GRISM mode. Each of these modes has dedicated chain commands, omichain, omfchain, and omgchain. If you run these chains, it is helpful to inspect the sas log file to get a detailed list of the performed tasks. In general, the ODF file names for the data will look like this: · mmmm iiiiiijjjj OMbeeeccfff.zzz mmmm ­ Revolution orbit number iiiiii ­ Proposal identifier number jjjj ­ Observation ID (target and exposure) b ­ Flag for scheduled (S) or unscheduled (U) exposures eee ­ Exposure number within the observation cc ­ CCD or OM window Identifier fff ­ Data identifier (imaging, timing, reduced imaging...) zzz ­ Format (FITS - FIT, ASCII - ASC) The IMAGING, FAST, and GRISM chains (omichain, omfchain, and omgchain) are described below. We have also written an equivalent to the omichain which allows one to vary the input parameters of each task, or to run the pipeline on only a subset of the data.

7.2.1

Imaging Mode

The Stray-light Problem All OM images are affected by the so-called "stray-light" problem (see Fig. 7.1). This problem does not affect source detection and magnitude determination but contributes to a higher background (and an ugly appearance of the images). The stray-light problem is less noticeable at UV wavelengths. A (proprietary) program to produce clean images exists but the results are strictly for display purposes only since the routine does not conserve flux. Because the stray-light problem is mainly aesthetic, there are no plans to develop publicly available routines to deal with it. ODF Pro ducts In IMAGING mode, OM files in the ODF directory looks like: 0 0 0 0 0 0 0 . 0 0 0 0 0 0 0 . 7 7 7 7 7 7 7 . 0 0 0 0 0 0 0 _ _ _ _ _ _ _ 0 0 0 0 0 0 0 1 1 1 1 1 1 1 2 2 2 2 2 2 2 3 3 3 3 3 3 3 7 7 7 7 7 7 7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 0 0 1 1 1 1 1 1 1 _O _O _O _O _O _O _O M M M M M M M S0 S0 S0 S0 S0 S0 S0 04 04 04 04 04 05 05 00 00 00 00 01 00 00 I R T W I I R MI FX HX DX MI MI FX .F .F .F .F .F .F .F I I I I I I I T T T T T T T 0 0 0 0 0 0 0 0 0 0 0 0 0 0 7 7 7 7 7 7 7 0 0 0 0 0 0 0 _ _ _ _ _ _ _ 0 0 0 0 0 0 0 1 1 1 1 1 1 1 2 2 2 2 2 2 2 3 3 3 3 3 3 3 7 7 7 7 7 7 7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 0 0 1 1 1 1 1 1 1 _ _ _ _ _ _ _ OM OM OM OM OM OM OM S4 S4 S4 S4 S4 S4 S4 2 2 2 2 2 2 2 20 20 20 20 30 30 30 0R 0T 0W 1I 0I 0R 0T F H D M M F H X. X. X. I. I. X. X. FI FI FI FI FI FI FI T T T T T T T

For each exposure there are: an image file (IMI), a tracking history file (THX), and a window data auxiliary file (WDX). There is one non-periodic (NPH) and periodic (PEH) housekeeping file per observation. In order to run a task, you will also need three files that are not specific to the OM. The first one (0070 0123700101 SCX00000SUM.SAS) is an ASCII file containing a summary of the observation, which is created by the odfingest task (§ 4.5.2): / / / / / X / / / / MM/Mydata/ODF: more 0070_0123700101_SCX00000SUM.SAS ---------------------------------XMM-Newton Science Analysis System ----------------------------------


58 / / / / / / / P / / / / / / / / A / ODF Summary File By: odfingest(odfingest-3.4) [xmmsas_20020109_1903-no-aka] on 2002-01-25T20:42:44.000

Directory where the ODF constituents were found. This may have to be edited to match the local file system structure. TH /XMM/Mydata/ODF

Please note that the keyword PATH can b e edited to match your current location of the data. The second general file (0070 0123700101 SCX00000TCS.FIT) is the spacecraft time correlation file, while the third (0070 0123700101 SCX00000ATS.FIT) contains the spacecraft attitude file. Re-processing of Imaging data can be done automatically by using omichain. The task omichain runs on filters specified by the user. If no arguments are given, the chain runs on all the files present in the $SAS ODF directory. If the omichain tasks are re-run one by one, there may be small differences between the files obtained in this manner and the pipeline products in the PPS directory. The main reasons for the differences are improvements made to the SAS software, the type of products produced by the pipeline (for example, only the most recent products have a final image for each filter), and some changes in the calibration products. The following explains the step-by-step processing of OM files. At the end of this section, a script is provided which goes automatically through all of the steps described below. The script is essentially an annotated version of omichain and shows what the processing does. We suggest that the user goes through all of the steps at least once manually before using the script. Preparation for Data Pro cessing If one wants to group the ODF files by filter values, one must extract the FILTER keyword from their headers. This can be done by using the FTOOLS task fkeyprint, e.g.: fkeyprint odfile name FILTER The FILTER keyword in the initial ODF file is a number between 0 and 2100. The correspondence between number and filter value is given in Table 7.3. Table 7.3: OM filter and file name correspondence. File ID Filter

1 1 1 1 2 0 0 0 0 0 1 2

2 4 6 8 0 0 2 4 6 8 0 1

0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0

blocked V Magnifier U (no bar) B White (datum) Grism 2 (Optical) UVW1 UVM2 UVW2 Grism 1 (UV) Ba r


59 We have written a script which goes through the complete list of files and gives back the filter used for each exposure. The script is available at: ftp://legacy.gsfc.nasa.gov/xmm/software/om tools/file examine.shell Running this script provides a list of files and their associated filters. The details of the association are less complicated than it may appear at first. In the standard configuration (the so-called Rudi-5 mode) one gets exposures in groups of 5, in high- and low-resolution mode, for a total of 10 files per filter. The high-resolution mode covers the same small central window in all five exposures while the low-resolution mode covers different parts of the detector in each of the 5 exposures. The sum of the low-resolution exposures covers the entire FOV. In general, the number following OMS will either be of the form 00400, 00401, 00500... or 40100, 40101, 40200,.. The last two digits indicate the resolution. 00 is high-resolution and 01 is low-resolution. In this example, the high-resolution window will be called 0070 0123700101 OMS00400IMI.FIT.gz while the lowresolution window will be 0070 0123700101 OMS00401IMI.FIT.gz. The low-resolution images for each of the five frames are taken consecutively to obtain the full FOV. For each low-resolution frame there is a high-resolution frame of the inner part of the detector. Here is an example of what running the script, file examine.shel l, produces: / 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 X 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 M 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 M 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 / _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ M 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 y 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 d 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 a 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 t 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 a 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 / 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 O 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 D 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 F 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 : _O _O _O _O _O _O _O _O _O _O _O _O _O _O _O _O ./file_examine.shell MS00400IMI.FIT FILTER MS00401IMI.FIT FILTER MS00500IMI.FIT FILTER MS00501IMI.FIT FILTER MS00600IMI.FIT FILTER MS00601IMI.FIT FILTER MS41500IMI.FIT FILTER MS41501IMI.FIT FILTER MS41600IMI.FIT FILTER MS41601IMI.FIT FILTER MS41700IMI.FIT FILTER MS41701IMI.FIT FILTER MS41800IMI.FIT FILTER MS41801IMI.FIT FILTER MS41900IMI.FIT FILTER MS41901IMI.FIT FILTER

V V U U WHITE WHITE V V V V V V V V U U

As noted above, the last three digits are paired so that in general one gets a low- and a high-resolution image. In this example the following images for the V filter are produced: 00401/00400 (inner part of the lowresolution image plus high-resolution frame of the inner part) 41501/41500 (left-hand frame of the low-resolution image plus high-resolution frame of the inner part), 41601/41600 (bottom frame of the low-resolution image plus high-resolution frame of the inner part), 41701/41700 (right-hand frame of the low-resolution image plus highresolution frame of the inner part), and 41801/41800 (top frame of the low-resolution image plus high-resolution frame of the inner part). This means that usually five high-resolution images are produced which can be co-added to achieve deeper exposures. Please be aware that one should NOT add low-resolution and high resolution images, even if they cover the same part of the FOV (e.g., one can not add 0070 0123700101 OMS00401IMI.FIT and 0070 0123700101 OMS00400IMI.FIT). Once one has decided which data to process (for example one exposure of one filter taken with a certain resolution), one should make sure that 1) A Calibration Index File has been created using cifbuild (§ 4.5.1). 2) A summary file of the ODF constituents has been created using odfingest (§ 4.5.2). 3) One set of exposures has been chosen on which to run omprep. As an example, we use the first high-resolution exposure for the Lockman Hole SV1 data and have copied them into the /XMM/Mydata directory. The files associated with the exposure are:


60 / 0 0 0 0 X 0 0 0 0 M 7 7 7 7 M 0 0 0 0 / _ _ _ _ M 0 0 0 0 y 1 1 1 1 d 2 2 2 2 a 3 3 3 3 t 7 7 7 7 a 0 0 0 0 : 0 0 0 0 1 1 1 1 ls 01_O 01_O 01_O 01_O M M M M S0 S0 S0 X0 04 04 04 00 00 00 00 00 I T W N MI HX DX PH .F .F .F .F I I I I T T T T 0 0 0 0 0 0 0 0 7 7 7 7 0 0 0 0 _ _ _ _ 0 0 0 0 1 1 1 1 2 2 2 2 3 3 3 3 7 7 7 7 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 _ _ _ _ OM SC SC SC X0 X0 X0 X0 0 0 0 0 00 00 00 00 0P 0A 0S 0S E T U U H. S. M. M. FI FI AS SA T T* C* S

The file 0070 0123700101 SCX00000SUM.SAS has been edited to point to that directory, SAS ODF is also pointing to this directory, and SAS CCF points to the file ccf.cif generated by cifbuild (§ 4.5.1). Examine the Guide Star Record · Both the ODF and the THX files should be processed by the omprep task before running any other SAS task (except cifbuild and odfingest). ­o p n w o m m e p d u o p h h x t d r s s s s e e e e e e s p t t t t e set='Mydata/0070 0123700101 OMS00400THX.FIT' ='Mydata/0070 0123700101 OMX00000PEH.FIT' ='Mydata/0070 0123700101 OMX00000NPH.FIT' ='Mydata/0070 0123700101 OMS00400WDX.FIT' ='Mydata/0070 0123700101 OMS00400THX OUT OMPREP.FIT' t=0 ­ se se se se es Tracking history file t ­ Periodic Housekeeping file t ­ Non-Periodic Housekeeping file t ­ Window Data Auxiliary file t ­ Output file et ­ Are these slew data (0=no)?

> > > > > >

s p n w o m

e e p d u o

t h h x t d

· Now the output THX file is ready to be used by the rest of the SAS tasks. One can examine the OM tracking history using the task omdrifhist. The output is a postscript file containing plots and statistics on the tracking history. ­ omdrifthist set='Mydata/0070 0123700101 OMS00400THX OUT OMPREP.FIT' plotfile='Mydata/0070 0123700101 OMS00400THX drift.ps' trackradius=0.5 hardcopy=yes pages='1 2' > > > > > s p t h p e l r a a t o a r g ­ THX file output of the omprep task tfile ­ Output name ckradius ­ Radius of pointing accuracy. dcopy ­ Yes/no ? e ­ Pages to plot (maximum pages produced is 2)

· Now One can inspect the output PS file. The other check on the tracking is to look at the count rates of the guide stars. To do this, use the task omthconv to produce a FITS file containing up to 10 columns with the guide stars' count rates. ­ omthconv thxset='Mydata/0070 0123700101 OMS00400THX OUT OMPREP.FIT' nphset='Mydata/0070 0123700101 OMX00000NPH.FIT' outset='Mydata/THX trackingStar.FIT' > thxset ­ Corrected THX file (output from omprep) > nphset ­ Non Periodic Housekeeping data > outset ­ Output file Examine the Images (IMI) files Bad Pixels IMI ODF files containing the OM images should also be corrected before running any other SAS task. The arguments of the omprep task are identical to the ones given for the THX file except for the input file which is now the IMI file.


61 ·o p n w o m m e p d u o p h h x t d r s s s s e e e e e e s p t t t t e set='Mydata/0070 0123700101 OMS00400IMI.FIT' ='Mydata/0070 0123700101 OMX00000PEH.FIT' ='Mydata/0070 0123700101 OMX00000NPH.FIT' ='Mydata/0070 0123700101 OMS00400WDX.FIT' ='Mydata/0070 0123700101 OMS00400IMI OUT OMPREP.FIT' t=0

> for parameter definitions see above, except for the input file which is now the IMI file. Once omprep has been run, the omcosflag task looks at the (corrected) OM tracking history and applies it to the map of bad pixels defined in the CCF. The resulting new bad pixel map is then used by the source detection algorithms. Bad pixels are set to 1, good pixels are set to 0. · omcosflag samplefactor=1 timefactor=1 set='Mydata/0070 0123700101 OMS00400IMI OUT OMPREP.FIT' thxset='Mydata/0070 0123700101 OMS00400THX OUT OMPREP.FIT' > samplefactor ­ Spatial oversampling factor (default 1) > timefactor ­ Temporal sampling factor (default 1) > set ­ Corrected IMI file > thxset ­ Corrected THX file Note: The output file is a modified IMI file. One can not run this task twice as it fails if it detects an existing QUALITY extension. To avoid problems, keep a copy of the original file (output from omprep) before running omcosflag. Note: The timefactor allows sub-sampling of the spacecraft jitter for tracking shifting of the bad pixel map. Although this has not yet been studied in detail, it appears that the tracking is generally so good that sub-sampling does not seem necessary. This parameter should be set to one (the default). Flat Field Generation OM flat field generation is implemented in the omichain command, but there is no flat field generation in the OM pipeline. Instead, users can run the task omflatgen to produce a unit flatfield. · omflatgen outset= 'Mydata/OUT FLATGEN.FIT' > outset ­ Name of the output file Once the OUT FLATGEN.FIT file has been created, one should run the omflatfield task which creates a tracking-shifted flatfield and applies it to an OM Science Window (OSW) Image. The omflatfield task creates two output files: one is the actual image and the other (specified by the output parameter ppsflatset) contains the tracking-shifted version of the omflatgen file. ·o t i t o m h n s u f x o f t l s r l s a e b a e t t i t t f = t s = ield samplefactor ='1' set='Mydata/0070 0123700101 OMS00400IMI OUT OMPREP.FIT' 'Mydata/0070 0123700101 OMS00400THX OUT OMPREP.FIT' flatset='Mydata/OUT FLATGEN.FIT' et='Mydata/0070 0123700101 OMS00400PPSFLATSET.FIT' 'Mydata/0070 0123700101 OMS00400IMI OUT FLATFIELD.FIT'

> samplefactor Sampling factor (to be set to 1) > set ­ Corrected IMI file (output of the omcosflag task) > thxset ­ Corrected THX file (output of the omprep task) > inorbitflatset ­ Unit file (Output of the omflatgen task) > tsflatset ­ Output name for the tracking history flatfield > outset ­ Output name for the flat field image Note: Not (too) surprisingly omflatfield produces the following warning: ** omflatfield: warning (Uniform flatfield- no correction to image will be applied)


62 Correct for Fixed-Pattern Noise The task ommodmap corrects a given OM Science Window (OSW) image for "modulo-8" spatial fixed-pattern noise that results from the OM centroiding algorithm performed by the on-board electronics (see documentation at "$SAS PATH/doc/ommodmap/ommodmap.html" for more details). Note that the ommodmap task does not lose counts, it simply redistributes them. ·o m o n m o u s m d t i o 8 s g d p e = m r t 3 ap set='Mydata/0070 0123700101 OMS00400IMI OUT FLATFIELD.FIT' oduct=yes mod8set='Mydata/0070 0123700101 OMS00400PPSMODE8SET OUT.FIT' ='Mydata/0070 0123700101 OMS00400OUT OMMODMAP.FIT' nbox=16

> set ­ Input file (output of omflatfield) > mod8product ­ Produce a Pipeline Processing System (PPS) file? > mod8set ­ Name of the output modulo-8 tile > outset ­ Name of the corrected image > nsig ­ Significance level for sigma clipping > nbox ­ Size of the sliding box in units of 8 pixels Perform Source Detection The task omdetect employs a simple two-stage process to locate sources are corrected for a 0.5 pixel position error in both FAST and IMAGE stage in source detection is to determine the background. The second of pixels above the sigma significance cut-off are identified and grouped lot of parameters (see below) but only set and outset are mandatory. ·o o n b l s s b p m o r m u s a e i m o i o u e d t i c v g o x x d t g e s g k e n o i e 8 p i t e m g l i t m l s u o e t a r i f h a c e t n c = = o m i s g o t r f t ' 6 u a m i e n = e i in an OSW image. OM source positions Mode exposures in SAS V6. The first stage is an island search, in which sets into individual ob jects. The task has a

set='Mydata/0070 0123700101 OMS00400OUT OMMODMAP.FIT' Mydata/0070 0123700101 OMS00400IMI OUT OMDETECT.FIT' contrast=0.001 nd='Mydata/0070 0123700101 OMS00400BACKGROUND.FIT' ge='Mydata/0070 0123700101 OMS00400LEVELIMAGE.FIT' age='Mydata/0070 0123700101 OMS00400SIGNIFIMAGE.FIT' ze=64 boxscale=3 maxscale=1 ='Mydata/0070 0123700101 OMS00400BOXIMAGE.FIT' nect=1 flatset='Mydata/OUT FLATGEN.FIT' 'Mydata/0070 0123700101 OMS00400PPSMODE8SET OUT.FIT' gionfile=yes le='Mydata/0070 0123700101 OMS00400oswList.reg'

> set ­ Input file (output of omflatfield) > outset ­ Name of the output source list file > nsigma ­ Number of above background for a detection > contrast ­ Blended source OK if source flux larger than contrast X total flux > background ­ Name of output background image > levelimage ­ Name of output image before deblending > signifimage ­ Name of output significance ( ) image > smoothsize ­ Size of smoothing box for background determination > boxscale ­ Minimum sliding box size for source detection > maxscale ­ Maximum binning to search > boximage ­ Name of output sliding box image > pixelconnect ­ Not used (keep to 1) > flatset ­ Name of input flat field image


63 > mod8set ­ Name of the modulo-8 noise map (mod8set output parameter from ommodmap) > outputregionfile ­ Do you want to produce an saoimage region file? > regionfile ­ Name of saoimage region file Note: omdetect does a variable job with the stray-light features and it may sometimes be fooled by them. One way to separate them from real detections is to look at the FWHM max and min parameter in the source list. Spurious source detections associated with stray-light features will have large value associated with these parameters. Convert Source Counts to Magnitudes The task ommag converts the list of given source counts to magnitudes in the appropriate instrumental band passes. The accuracy is estimated to be a few tenths of a magnitude. SAS V6 takes a new PSF for the UVW1, UVM2, and UVW2 filters into account, leading to an improved OM photometry on the order of tenths of a magnitude. · ommag set='Mydata/0070 0123700101 OMS00400IMI OUT OMDETECT.FIT' wdxset='Mydata/0070 0123700101 OMS00400WDX.FIT' > set ­ Input list (output of omdetect) > wdxset ­ Window Data Auxiliary file Note: There is a "recipe" to convert the UV count rates to flux. The recipe was provided by Alice Breeveld (MSSL) and can be accessed at: http://xmm.vilspa.esa.es/sas/documentation/watchout/uvflux.shtml

Convert Source OM Positions to Sky Co ordinates The task omatt converts an OM OSW source list from pixels to sky coordinates. These sky coordinates are then used to produce a sky coordinate image. ·o s p d m o p e a u s v t r o i t c s c set='Mydata/0070 0123700101 OMS00400OUT OMMODMAP.FIT' elistset='Mydata/0070 0123700101 OMS00400IMI OUT OMDETECT.FIT' wset='Mydata/0070 0123700101 OMS00400FINAL IMAGE.FIT' e='/NULL' usecat=no tolerance=3 catfile=''

> set ­ Input file (output of the ommodmap task) > sourcelistset ­ Source list (output of omdetect task) > ppsoswset ­ Output name for the corrected sky image > device ­ Output device > usecat ­ Do you want to use the USNO-SA 1 catalog? > tolerance ­ Tolerance for catalog search in arc seconds > catfile ­ Name of the USNO star catalog (default: 'usnocat.fit') Note: Due to the large size of the catalog, it is not distributed. Users, however, can provide their own catalog if they wish. The format is that used for the USNO cross-correlation FITS products. In general, the usecat keyword should be set to no. Note: The pointing stability about the spacecraft boresight position is better than 1 (look at the tracking plots derived at the beginning). There is still a scatter of about 4 between the planned and actual pointing position. There is a script which does all this step by step and allows one to run the pipeline only on the desired file. The script is available at: ftp://legacy.gsfc.nasa.gov/xmm/software/om tools/omproc gof. Please contact the GOF if you have any problems with it.


64

7.2.2

Fast Mode

SAS has a working fast mode pipeline. If the data have not been processed by the latest version of SAS, the task omfchain should be run. The chain works similarly to the imaging chain explained above, and consists of a Perl script which calls all the necessary tasks sequentially. It produces images of the detected sources, extracts events related to the sources and the background, and extracts the corresponding light curves. A more detailed description of the chain can be found in the SAS on-line help available at http://xmm.vilspa.esa.es/sas/current/doc/index.html. You can also access the general description of the task at: ftp://legacy.gsfc.nasa.gov/xmm/doc/fastmode.ps.gz. A summary of the task is shown in Figure 7.2.

Figure 7.2: OM fast chain­diagram of the different tasks run.

7.2.3

Grism Analysis

SAS V6 allows, for the first time, the analysis of data obtained with the OM grisms. A new metatask, omgchain, can be used to extract and automatically calibrate spectra produced by the OM grisms. OM grism data are taken in Image Mode. Hence omgchain uses already existing tasks, such as omprep and ommodmap, to handle housekeeping information and to perform some corrections (the "modulo-8" noise reduction for example). Also, omdetect is designed to find the spectra, zero and first orders, producing a source list. Other new tasks are grism specific. omgprep is used to correct for geometric distortion of the detector and to rotate the image so as to have the dispersion direction aligned with the image Y axis. omgprep performs the spectral extraction and the wavelength and flux calibration. Finally, the extracted spectra are plotted using omgplot. The sequence of tasks used by omgchain is illustrated in Fig. 7.3. An output spectrum produced by omgchain is given in Fig. 7.4. Each of these tasks can be run individually. SAS V6 also includes a new interactive task, omgsource, which allows the user to select with the cursor the spectrum to be extracted.


65 Figure 7.3: Diagram of the different tasks used by omgchain. The first four tasks are preparatory, the other three tasks execute the source detection and spectral identification procedure, and produce the output files.

The task omgchain has many parameters, but none of them are mandatory. Below is a description of the calling sequence and the individual parameters. ·o n b e m s k x g i g t c g o r h m f a a a f c i = s t n 3 e i inpdirectory=MyData/ODF outdirectory=MyData comment='' combine=yes spectrumhalfwidth=-8 bkgoffsetleft=0 bkgwidthleft=-8 tright=0 bkgwidthright=-8 spectrumsmoothlength=0 mod8correction=1 onmode=0 plotbinsize=1 plotflux=2 scalebkgplot=no

> inpdirectory ­ Input file directory > outdirectory ­ Output file directory > comment ­ User's comments for output > nsigma ­ Number of above the background required for a detection (this parameter is passed to omdetect > combine ­ Condition for combining the Engineering-2 subwindows > spectrumhalfwidth ­ Half-width of the spectrum extraction region (in pixels, if negative, and in FWHMs otherwise) > bkgoffsetleft ­ Offset of the left background extraction region from the edge of the spectrum extraction area; in pixels, if negative, or in FWHMs otherwise. > bkgwidthleft ­ Width of the left background extraction region; in pixels, if negative, or in FWHMs otherwise > bkgoffsetright ­ Offset for the right background extraction region; in pixels, if negative, and in FWHMs otherwise > bkgwidthright ­ Width of the right background extraction region; in pixels, if negative, or in FWHMs otherwise.


66 > spectrumsmoothlength ­ Length of the smoothing window for smoothing the extracted spectra, if necessary. Values 0 or 1 of this parameter imply no smoothing > mod8correction ­ Condition for removing the modulo-8 noise: 0: correction not applied; 1: correction applied using the modulo-8 map extracted from the input image; 2: correction applied using the modulo-8 map extracted from the OM CCF flat field; 3: correction applied multiplying the input image by the OM CCF flat field > extractionmode ­ Switch between different extraction modes. The value 0 corresponds to the normal extraction (summation of counts in the cross-dispersion direction); 1 corresponds to the Gaussian fit > plotbinsize ­ Size of spectrum wavelength bins for the output plot (in °) A > plotflux ­ Flag for plotting the spectrum only (value 0), the background only (value 1), or both of them (value 2) > scalebkgplot ­ Condition for scaling the background plot differently from the spectrum plot Note: if a source is not detected by omdetect, or does not fall within the grism window, omgchain will run without warning, but will not produce output files.

Figure 7.4: OM optical grism spectrum obtained from a 4.7 ks observation of Mrk 478.