|
Документ взят из кэша поисковой машины. Адрес
оригинального документа
: http://hea-www.harvard.edu/RD/funtools/programs.html
Дата изменения: Unknown Дата индексирования: Tue Oct 2 00:28:41 2012 Кодировка: Поисковые слова: п п п п п п п п п п п п п п |
funcalc [-n] [-a argstr] [-e expr] [-f file] [-l link] [-p prog] [-u] <iname> [oname [columns]] funcen [-i] [-n iter] [-t tol] [-v lev] <iname> <region> funcnts [switches] <source_file> [source_region] [bkgd_file] [bkgd_region|bkgd_cnts] funcone [-n] [-x|-X|-j|-J] [[-l|-L] list] [-r ra_col] [-d dec_col] <iname> <oname> <ra[hdr]> <dec[hdr]> <radius[dr'"]> [columns] fundisp [-f format] [-l] [-n] [-T] <iname> [columns|bitpix=n] funhead [-a] [-l] [-s] [-t] [-L] <iname> [oname ename] funhist [-n|-w|-T] <iname> [column] [[lo_edge:hi_edge:]bins] funimage [-a] [-l] [-p x|y] <iname> <oname> [bitpix=n] funindex <iname> <key> [oname] funjoin [switches] <ifile1> <ifile2> ... <ifilen> <ofile> funmerge <iname1> <iname2> ... <oname> funsky [switches] <iname1> [<lname2> <col1> <col2>] funtable [-a] [-i|-z] [-m] [-s cols] <iname> <oname> [columns] funtbl [-c cols] [-h] [-n table] [-p prog] [-s sep] [-T] <iname>
funcalc [-n] [-a argstr] [-e expr] [-f file] [-l link] [-p prog] <iname> [oname [columns]]
-a argstr # user arguments to pass to the compiled program -e expr # funcalc expression -f file # file containing funcalc expression -l libs # libs to add to link command -n # output generated code instead of compiling and executing -p prog # generate named program, no execution -u # die if any variable is undeclared (don't auto-declare)
funcalc is a calculator program that allows arbitrary expressions to be constructed, compiled, and executed on columns in a Funtools table (FITS binary table or raw event file). It works by integrating user-supplied expression(s) into a template C program, then compiling and executing the program. funcalc expressions are C statements, although some important simplifications (such as automatic declaration of variables) are supported.
funcalc expressions can be specified in three ways: on the command line using the -e [expression] switch, in a file using the -f [file] switch, or from stdin (if neither -e nor -f is specified). Of course a file containing funcalc expressions can be read from stdin.
Each invocation of funcalc requires an input Funtools table file to be specified as the first command line argument. The output Funtools table file is the second optional argument. It is needed only if an output FITS file is being created (i.e., in cases where the funcalc expression only prints values, no output file is needed). If input and output file are both specified, a third optional argument can specify the list of columns to activate (using FunColumnActivate()). Note that funcalc determines whether or not to generate code for writing an output file based on the presence or absence of an output file argument.
A funcalc expression executes on each row of a table and consists of one or more C statements that operate on the columns of that row (possibly using temporary variables). Within an expression, reference is made to a column of the current row using the C struct syntax cur->[colname], e.g. cur->x, cur->pha, etc. Local scalar variables can be defined using C declarations at very the beginning of the expression, or else they can be defined automatically by funcalc (to be of type double). Thus, for example, a swap of columns x and y in a table can be performed using either of the following equivalent funcalc expressions:
double temp; temp = cur->x; cur->x = cur->y; cur->y = temp;or:
temp = cur->x; cur->x = cur->y; cur->y = temp;When this expression is executed using a command such as:
funcalc -f swap.expr itest.ev otest.evthe resulting file will have values of the x and y columns swapped.
By default, the data type of the variable for a column is the same as the data type of the column as stored in the file. This can be changed by appending ":[dtype]" to the first reference to that column. In the example above, to force x and y to be output as doubles, specify the type 'D' explicitly:
temp = cur->x:D; cur->x = cur->y:D; cur->y = temp;Data type specifiers follow standard FITS table syntax for defining columns using TFORM:
Of course, it is important to handle the data type of the columns correctly. One of the most frequent cause of error in funcalc programming is the implicit use of the wrong data type for a column in expression. For example, the calculation:
dx = (cur->x - cur->y)/(cur->x + cur->y);usually needs to be performed using floating point arithmetic. In cases where the x and y columns are integers, this can be done by reading the columns as doubles using an explicit type specification:
dx = (cur->x:D - cur->y:D)/(cur->x + cur->y);Alternatively, it can be done using C type-casting in the expression:
dx = ((double)cur->x - (double)cur->y)/((double)cur->x + (double)cur->y);
In addition to accessing columns in the current row, reference also can be made to the previous row using prev->[colname], and to the next row using next->[colname]. Note that if prev->[colname] is specified in the funcalc expression, the very first row is not processed. If next->[colname] is specified in the funcalc expression, the very last row is not processed. In this way, prev and next are guaranteed always to point to valid rows. For example, to print out the values of the current x column and the previous y column, use the C fprintf function in a funcalc expression:
fprintf(stdout, "%d %d\n", cur->x, prev->y);
New columns can be specified using the same cur->[colname] syntax by appending the column type (and optional tlmin/tlmax/binsiz specifiers), separated by colons. For example, cur->avg:D will define a new column of type double. Type specifiers are the same those used above to specify new data types for existing columns.
For example, to create and output a new column that is the average value of the x and y columns, a new "avg" column can be defined:
cur->avg:D = (cur->x + cur->y)/2.0Note that the final ';' is not required for single-line expressions.
As with FITS TFORM data type specification, the column data type specifier can be preceded by a numeric count to define an array, e.g., "10I" means a vector of 10 short ints, "2E" means two single precision floats, etc. A new column only needs to be defined once in a funcalc expression, after which it can be used without re-specifying the type. This includes reference to elements of a column array:
cur->avg[0]:2D = (cur->x + cur->y)/2.0; cur->avg[1] = (cur->x - cur->y)/2.0;
The 'X' (bits) data type is treated as a char array of dimension (numeric_count/8), i.e., 16X is processed as a 2-byte char array. Each 8-bit array element is accessed separately:
cur->stat[0]:16X = 1; cur->stat[1] = 2;Here, a 16-bit column is created with the MSB is set to 1 and the LSB set to 2.
By default, all processed rows are written to the specified output file. If you want to skip writing certain rows, simply execute the C "continue" statement at the end of the funcalc expression, since the writing of the row is performed immediately after the expression is executed. For example, to skip writing rows whose average is the same as the current x value:
cur->avg[0]:2D = (cur->x + cur->y)/2.0;
cur->avg[1] = (cur->x - cur->y)/2.0;
if( cur->avg[0] == cur->x )
continue;
If no output file argument is specified on the funcalc command line, no output file is opened and no rows are written. This is useful in expressions that simply print output results instead of generating a new file:
fpv = (cur->av3:D-cur->av1:D)/(cur->av1+cur->av2:D+cur->av3); fbv = cur->av2/(cur->av1+cur->av2+cur->av3); fpu = ((double)cur->au3-cur->au1)/((double)cur->au1+cur->au2+cur->au3); fbu = cur->au2/(double)(cur->au1+cur->au2+cur->au3); fprintf(stdout, "%f\t%f\t%f\t%f\n", fpv, fbv, fpu, fbu);In the above example, we use both explicit type specification (for "av" columns) and type casting (for "au" columns) to ensure that all operations are performed in double precision.
When an output file is specified, the selected input table is processed and output rows are copied to the output file. Note that the output file can be specified as "stdout" in order to write the output rows to the standard output. If the output file argument is passed, an optional third argument also can be passed to specify which columns to process.
In a FITS binary table, it sometimes is desirable to copy all of the other FITS extensions to the output file as well. This can be done by appending a '+' sign to the name of the extension in the input file name. See funtable for a related example.
funcalc works by integrating the user-specified expression into a template C program called tabcalc.c. The completed program then is compiled and executed. Variable declarations that begin the funcalc expression are placed in the local declaration section of the template main program. All other lines are placed in the template main program's inner processing loop. Other details of program generation are handled automatically. For example, column specifiers are analyzed to build a C struct for processing rows, which is passed to FunColumnSelect() and used in FunTableRowGet(). If an unknown variable is used in the expression, resulting in a compilation error, the program build is retried after defining the unknown variable to be of type double.
Normally, funcalc expression code is added to funcalc row processing loop. It is possible to add code to other parts of the program by placing this code inside special directives of the form:
[directive name]
... code goes here ...
end
The directives are:
global
double v1, v2;
double init(void);
double finish(double v);
end
before
v1 = init();
end
... process rows, with calculations using v1 ...
after
v2 = finish(v1);
if( v2 < 0.0 ){
fprintf(stderr, "processing failed %g -> %g\n", v1, v2);
exit(1);
}
end
Routines such as init() and finish() above are passed to the generated
program for linking using the -l [link directives ...]
switch. The string specified by this switch will be added to the link
line used to build the program (before the funtools library). For
example, assuming that init() and finish() are in the library
libmysubs.a in the /opt/special/lib directory, use:
funcalc -l "-L/opt/special/lib -lmysubs" ...
User arguments can be passed to a compiled funcalc program using a string argument to the "-a" switch. The string should contain all of the user arguments. For example, to pass the integers 1 and 2, use:
funcalc -a "1 2" ...The arguments are stored in an internal array and are accessed as strings via the ARGV(n) macro. For example, consider the following expression:
local
int pmin, pmax;
end
before
pmin=atoi(ARGV(0));
pmax=atoi(ARGV(1));
end
if( (cur->pha >= pmin) && (cur->pha <= pmax) )
fprintf(stderr, "%d %d %d\n", cur->x, cur->y, cur->pha);
This expression will print out x, y, and pha values for all rows in which
the pha value is between the two user-input values:
funcalc -a '1 12' -f foo snr.ev'[cir 512 512 .1]' 512 512 6 512 512 8 512 512 5 512 512 5 512 512 8 funcalc -a '5 6' -f foo snr.ev'[cir 512 512 .1]' 512 512 6 512 512 5 512 512 5
Note that it is the user's responsibility to ensure that the correct number of arguments are passed. The ARGV(n) macro returns a NULL if a requested argument is outside the limits of the actual number of args, usually resulting in a SEGV if processed blindly. To check the argument count, use the ARGC macro:
local
long int seed=1;
double limit=0.8;
end
before
if( ARGC >= 1 ) seed = atol(ARGV(0));
if( ARGC >= 2 ) limit = atof(ARGV(1));
srand48(seed);
end
if ( drand48() > limit ) continue;
The macro WRITE_ROW expands to the FunTableRowPut() call that writes the current row. It can be used to write the row more than once. In addition, the macro NROW expands to the row number currently being processed. Use of these two macros is shown in the following example:
if( cur->pha:I == cur->pi:I ) continue; a = cur->pha; cur->pha = cur->pi; cur->pi = a; cur->AVG:E = (cur->pha+cur->pi)/2.0; cur->NR:I = NROW; if( NROW < 10 ) WRITE_ROW;
If the -p [prog] switch is specified, the expression is not executed. Rather, the generated executable is saved with the specified program name for later use.
If the -n switch is specified, the expression is not executed. Rather, the generated code is written to stdout. This is especially useful if you want to generate a skeleton file and add your own code, or if you need to check compilation errors. Note that the comment at the start of the output gives the compiler command needed to build the program on that platform. (The command can change from platform to platform because of the use of different libraries, compiler switches, etc.)
As mentioned previously, funcalc will declare a scalar variable automatically (as a double) if that variable has been used but not declared. This facility is implemented using a sed script named funcalc.sed, which processes the compiler output to sense an undeclared variable error. This script has been seeded with the appropriate error information for gcc, and for cc on Solaris, DecAlpha, and SGI platforms. If you find that automatic declaration of scalars is not working on your platform, check this sed script; it might be necessary to add to or edit some of the error messages it senses.
In order to keep the lexical analysis of funcalc expressions (reasonably) simple, we chose to accept some limitations on how accurately C comments, spaces, and new-lines are placed in the generated program. In particular, comments associated with local variables declared at the beginning of an expression (i.e., not in a local...end block) will usually end up in the inner loop, not with the local declarations:
/* this comment will end up in the wrong place (i.e, inner loop) */ double a; /* also in wrong place */ /* this will be in the the right place (inner loop) */ if( cur->x:D == cur->y:D ) continue; /* also in right place */ a = cur->x; cur->x = cur->y; cur->y = a; cur->avg:E = (cur->x+cur->y)/2.0;Similarly, spaces and new-lines sometimes are omitted or added in a seemingly arbitrary manner. Of course, none of these stylistic blemishes affect the correctness of the generated code.
Because funcalc must analyze the user expression using the data file(s) passed on the command line, the input file(s) must be opened and read twice: once during program generation and once during execution. As a result, it is not possible to use stdin for the input file: funcalc cannot be used as a filter. We will consider removing this restriction at a later time.
Along with C comments, funcalc expressions can have one-line internal comments that are not passed on to the generated C program. These internal comment start with the # character and continue up to the new-line:
double a; # this is not passed to the generated C file # nor is this a = cur->x; cur->x = cur->y; cur->y = a; /* this comment is passed to the C file */ cur->avg:E = (cur->x+cur->y)/2.0;
As previously mentioned, input columns normally are identified by their being used within the inner event loop. There are rare cases where you might want to read a column and process it outside the main loop. For example, qsort might use a column in its sort comparison routine that is not processed inside the inner loop (and therefore not implicitly specified as a column to be read). To ensure that such a column is read by the event loop, use the explicit keyword. The arguments to this keyword specify columns that should be read into the input record structure even though they are not mentioned in the inner loop. For example:
explicit pi phawill ensure that the pi and pha columns are read for each row, even if they are not processed in the inner event loop. The explicit statement can be placed anywhere.
Finally, note that funcalc currently works on expressions involving FITS binary tables and raw event files. We will consider adding support for image expressions at a later point, if there is demand for such support from the community.
funcen [-i] [-n iter] [-t tol] [-v lev] <iname> <region>
-i # use image filtering (default: event filtering) -n iter # max number of iterations (default: 0) -t tol # pixel tolerance distance (default: 1.0) -v [0,1,2,3] # output verbosity level (default: 0)
funcen iteratively calculates the centroid position within one or more regions of a Funtools table (FITS binary table or raw event file). Starting with an input table, an initial region specification, and an iteration count, the program calculates the average x and y position within the region and then uses this new position as the region center for the next iteration. Iteration terminates when the maximum number of iterations is reached or when the input tolerance distance is met for that region. A count of events in the final region is then output, along with the pixel position value (and, where available, WCS position).
The first argument to the program specifies the Funtools table file to process. Since the file must be read repeatedly, a value of "stdin" is not permitted when the number of iterations is non-zero. Use Funtools Bracket Notation to specify FITS extensions and filters.
The second required argument is the initial region descriptor. Multiple regions are permitted. However, compound regions (accelerators, variable argument regions and regions connected via boolean algebra) are not permitted. Points and polygons also are illegal. These restrictions might be lifted in a future version, if warranted.
The -n (iteration number) switch specifies the maximum number of iterations to perform. The default is 0, which means that the program will simply count and display the number of events in the initial region(s). Note that when iterations is 0, the data can be input via stdin.
The -t (tolerance) switch specifies a floating point tolerance value. If the distance between the current centroid position value and the last position values is less than this value, iteration terminates. The default value is 1 pixel.
The -v (verbosity) switch specifies the verbosity level of the output. The default is 0, which results in a single line of output for each input region consisting of the following values:
counts x y [ra dec coordsys]The last 3 WCS values are output if WCS information is available in the data file header. Thus, for example:
[sh] funcen -n 0 snr.ev "cir 505 508 5" 915 505.00 508.00 345.284038 58.870920 j2000 [sh] funcen -n 3 snr.ev "cir 505 508 5" 1120 504.43 509.65 345.286480 58.874587 j2000The first example simply counts the number of events in the initial region. The second example iterates the centroid calculation three times to determine a final "best" position.
Higher levels of verbosity obviously imply more verbose output. At level 1, the output essentially contains the same information as level 0, but with keyword formatting: [sh] funcen -v 1 -n 3 snr.ev "cir 505 508 5" event_file: snr.ev initial_region: cir 505 508 5 tolerance: 1.0000 iterations: 1 events: 1120 x,y(physical): 504.43 509.65 ra,dec(j2000): 345.286480 58.874587 final_region1: cir 504.43 509.65 5 Level 2 outputs results from intermediate calculations as well.
Ordinarily, region filtering is performed using analytic (event) filtering, i.e. that same style of filtering as is performed by fundisp and funtable. Use the -i switch to specify image filtering, i.e. the same style filtering as is performed by funcnts. Thus, you can perform a quick calculation of counts in regions, using either the analytic or image filtering method, by specifying the -n 0 and optional -i switches. These two method often give different results because of how boundary events are processed:
[sh] funcen snr.ev "cir 505 508 5" 915 505.00 508.00 345.284038 58.870920 j2000 [sh] funcen -i snr.ev "cir 505 508 5" 798 505.00 508.00 345.284038 58.870920 j2000See Region Boundaries for more information about how boundaries are calculated using these two methods.
funcnts [switches] <source_file> [source_region] [bkgd_file] [bkgd_region|bkgd_value]
-e "source_exposure[;bkgd_exposure]"
# source (bkgd) FITS exposure image using matching files
-w "source_exposure[;bkgd_exposure]"
# source (bkgd) FITS exposure image using WCS transform
-t "source_timecorr[;bkgd_timecorr]"
# source (bkgd) time correction value or header parameter name
-g # output using nice g format
-G # output using %.14g format (maximum precision)
-i "[column;]int1;int2..." # column-based intervals
-m # match individual source and bkgd regions
-p # output in pixels, even if wcs is present
-r # output inner/outer radii (and angles) for annuli (and pandas)
-s # output summed values
-v "scol[;bcol]" # src and bkgd value columns for tables
-T # output in starbase/rdb format
-z # output regions with zero area
funcnts counts photons in the specified source regions and reports the results for each region. Regions are specified using the Spatial Region Filtering mechanism. Photons are also counted in the specified bkgd regions applied to the same data file or a different data file. (Alternatively, a constant background value in counts/pixel**2 can be specified.) The bkgd regions are either paired one-to-one with source regions or pooled and normalized by area, and then subtracted from the source counts in each region. Displayed results include the bkgd-subtracted counts in each region, as well as the error on the counts, the area in each region, and the surface brightness (cnts/area**2) calculated for each region.
The first argument to the program specifies the FITS input image, array, or raw event file to process. If "stdin" is specified, data are read from the standard input. Use Funtools Bracket Notation to specify FITS extensions, image sections, and filters.
The optional second argument is the source region descriptor. If no region is specified, the entire field is used.
The background arguments can take one of two forms, depending on whether a separate background file is specified. If the source file is to be used for background as well, the third argument can be either the background region, or a constant value denoting background cnts/pixel. Alternatively, the third argument can be a background data file, in which case the fourth argument is the background region. If no third argument is specified, a constant value of 0 is used (i.e., no background).
In summary, the following command arguments are valid:
[sh] funcnts sfile # counts in source file [sh] funcnts sfile sregion # counts in source region [sh] funcnts sfile sregion bregion # bkgd reg. is from source file [sh] funcnts sfile sregion bvalue # bkgd reg. is constant [sh] funcnts sfile sregion bfile bregion # bkgd reg. is from separate file
NB: unlike other Funtools programs, source and background regions are specified as separate arguments on the command line, rather than being placed inside brackets as part of the source and background filenames. This is because regions in funcnts are not simply used as data filters, but also are used to calculate areas, exposure, etc. If you put the source region inside the brackets (i.e. use it simply as a filter) rather than specifying it as argument two, the program still will only count photons that pass the region filter. However, the area calculation will be performed on the whole field, since field() is the default source region. This rarely is the desired behavior. On the other hand, with FITS binary tables, it often is useful to put a column filter in the filename brackets, so that only events matching the column filter are counted inside the region.
For example, to extract the counts within a radius of 22 pixels from the center of the FITS binary table snr.ev and subtract the background determined from the same image within an annulus of radii 50-100 pixels:
[sh] funcnts snr.ev "circle(502,512,22)" "annulus(502,512,50,100)"
# source
# data file: snr.ev
# degrees/pix: 0.00222222
# background
# data file: snr.ev
# column units
# area: arcsec**2
# surf_bri: cnts/arcsec**2
# surf_err: cnts/arcsec**2
# background-subtracted results
reg net_counts error background berror area surf_bri surf_err
---- ------------ --------- ------------ --------- --------- --------- ---------
1 3826.403 66.465 555.597 5.972 96831.98 0.040 0.001
# the following source and background components were used:
source region(s)
----------------
circle(502,512,22)
reg counts pixels
---- ------------ ---------
1 4382.000 1513
background region(s)
--------------------
annulus(502,512,50,100)
reg counts pixels
---- ------------ ---------
all 8656.000 23572
The area units for the output columns labeled "area", "surf_bri"
(surface brightness) and "surf_err" will be given either in
arc-seconds (if appropriate WCS information is in the data file
header(s)) or in pixels. If the data file has WCS info, but you do not
want arc-second units, use the -p switch to force output in
pixels. Also, regions having zero area are not normally included in
the primary (background-subtracted) table, but are included in the
secondary source and bkgd tables. If you want these regions to be
included in the primary table, use the -z switch.
Note that a simple sed command will extract the background-subtracted results for further analysis:
[sh] cat funcnts.sed 1,/---- .*/d /^$/,$d [sh] sed -f funcnts.sed funcnts.out 1 3826.403 66.465 555.597 5.972 96831.98 0.040 0.001
If separate source and background files are specified, funcnts will attempt to normalize the the background area so that the background pixel size is the same as the source pixel size. This normalization can only take place if the appropriate WCS information is contained in both files (e.g. degrees/pixel values in CDELT). If either file does not contain the requisite size information, the normalization is not performed. In this case, it is the user's responsibility to ensure that the pixel sizes are the same for the two files.
Normally, if more than one background region is specified, funcnts will combine them all into a single region and use this background region to produce the background-subtracted results for each source region. The -m (match multiple backgrounds) switch tells funcnts to make a one to one correspondence between background and source regions, instead of using a single combined background region. For example, the default case is to combine 2 background regions into a single region and then apply that region to each of the source regions:
[sh] funcnts snr.ev "annulus(502,512,0,22,n=2)" "annulus(502,512,50,100,n=2)"
# source
# data file: snr.ev
# degrees/pix: 0.00222222
# background
# data file: snr.ev
# column units
# area: arcsec**2
# surf_bri: cnts/arcsec**2
# surf_err: cnts/arcsec**2
# background-subtracted results
reg net_counts error background berror area surf_bri surf_err
---- ------------ --------- ------------ --------- --------- --------- ---------
1 3101.029 56.922 136.971 1.472 23872.00 0.130 0.002
2 725.375 34.121 418.625 4.500 72959.99 0.010 0.000
# the following source and background components were used:
source region(s)
----------------
annulus(502,512,0,22,n=2)
reg counts pixels
---- ------------ ---------
1 3238.000 373
2 1144.000 1140
background region(s)
--------------------
annulus(502,512,50,100,n=2)
reg counts pixels
---- ------------ ---------
all 8656.000 23572
Note that the basic region filter rule "each photon is counted once
and no photon is counted more than once" still applies when using The
-m to match background regions. That is, if two background
regions overlap, the overlapping pixels will be counted in only one of
them. In a worst-case scenario, if two background regions are the same
region, the first will get all the counts and area and the second
will get none.
Using the -m switch causes funcnts to use each of the two background regions independently with each of the two source regions:
[sh] funcnts -m snr.ev "annulus(502,512,0,22,n=2)" "ann(502,512,50,100,n=2)"
# source
# data file: snr.ev
# degrees/pix: 0.00222222
# background
# data file: snr.ev
# column units
# area: arcsec**2
# surf_bri: cnts/arcsec**2
# surf_err: cnts/arcsec**2
# background-subtracted results
reg net_counts error background berror area surf_bri surf_err
---- ------------ --------- ------------ --------- --------- --------- ---------
1 3087.015 56.954 150.985 2.395 23872.00 0.129 0.002
2 755.959 34.295 388.041 5.672 72959.99 0.010 0.000
# the following source and background components were used:
source region(s)
----------------
annulus(502,512,0,22,n=2)
reg counts pixels
---- ------------ ---------
1 3238.000 373
2 1144.000 1140
background region(s)
--------------------
ann(502,512,50,100,n=2)
reg counts pixels
---- ------------ ---------
1 3975.000 9820
2 4681.000 13752
Note that most floating point quantities are displayed using "f" format. You can change this to "g" format using the -g switch. This can be useful when the counts in each pixel is very small or very large. If you want maximum precision and don't care about the columns lining up nicely, use -G, which outputs all floating values as %.14g.
When counting photons using the annulus and panda (pie and annuli) shapes, it often is useful to have access to the radii (and panda angles) for each separate region. The -r switch will add radii and angle columns to the output table:
[sh] funcnts -r snr.ev "annulus(502,512,0,22,n=2)" "ann(502,512,50,100,n=2)"
# source
# data file: snr.ev
# degrees/pix: 0.00222222
# background
# data file: snr.ev
# column units
# area: arcsec**2
# surf_bri: cnts/arcsec**2
# surf_err: cnts/arcsec**2
# radii: arcsecs
# angles: degrees
# background-subtracted results
reg net_counts error background berror area surf_bri surf_err radius1 radius2 angle1 angle2
---- ------------ --------- ------------ --------- --------- --------- --------- --------- --------- --------- ---------
1 3101.029 56.922 136.971 1.472 23872.00 0.130 0.002 0.00 88.00 NA NA
2 725.375 34.121 418.625 4.500 72959.99 0.010 0.000 88.00 176.00 NA NA
# the following source and background components were used:
source region(s)
----------------
annulus(502,512,0,22,n=2)
reg counts pixels
---- ------------ ---------
1 3238.000 373
2 1144.000 1140
background region(s)
--------------------
ann(502,512,50,100,n=2)
reg counts pixels
---- ------------ ---------
all 8656.000 23572
Radii are given in units of pixels or arc-seconds (depending on the presence of WCS info), while the angle values (when present) are in degrees. These columns can be used to plot radial profiles. For example, the script funcnts.plot in the funtools distribution) will plot a radial profile using gnuplot (version 3.7 or above). A simplified version of this script is shown below:
#!/bin/sh
if [ x"$1" = xgnuplot ]; then
if [ x`which gnuplot 2>/dev/null` = x ]; then
echo "ERROR: gnuplot not available"
exit 1
fi
awk '
BEGIN{HEADER=1; DATA=0; FILES=""; XLABEL="unknown"; YLABEL="unknown"}
HEADER==1{
if( $1 == "#" && $2 == "data" && $3 == "file:" ){
if( FILES != "" ) FILES = FILES ","
FILES = FILES $4
}
else if( $1 == "#" && $2 == "radii:" ){
XLABEL = $3
}
else if( $1 == "#" && $2 == "surf_bri:" ){
YLABEL = $3
}
else if( $1 == "----" ){
printf "set nokey; set title \"funcnts(%s)\"\n", FILES
printf "set xlabel \" radius(%s)\"\n", XLABEL
printf "set ylabel \"surf_bri(%s)\"\n", YLABEL
print "plot \"-\" using 3:4:6:7:8 with boxerrorbars"
HEADER = 0
DATA = 1
next
}
}
DATA==1{
if( NF == 12 ){
print $9, $10, ($9+$10)/2, $7, $8, $7-$8, $7+$8, $10-$9
}
else{
exit
}
}
' | gnuplot -persist - 1>/dev/null 2>&1
elif [ x"$1" = xds9 ]; then
awk '
BEGIN{HEADER=1; DATA=0; XLABEL="unknown"; YLABEL="unknown"}
HEADER==1{
if( $1 == "#" && $2 == "data" && $3 == "file:" ){
if( FILES != "" ) FILES = FILES ","
FILES = FILES $4
}
else if( $1 == "#" && $2 == "radii:" ){
XLABEL = $3
}
else if( $1 == "#" && $2 == "surf_bri:" ){
YLABEL = $3
}
else if( $1 == "----" ){
printf "funcnts(%s) radius(%s) surf_bri(%s) 3\n", FILES, XLABEL, YLABEL
HEADER = 0
DATA = 1
next
}
}
DATA==1{
if( NF == 12 ){
print $9, $7, $8
}
else{
exit
}
}
'
else
echo "funcnts -r ... | funcnts.plot [ds9|gnuplot]"
exit 1
fi
Thus, to run funcnts and plot the results using gnuplot (version 3.7
or above), use:
funcnts -r snr.ev "annulus(502,512,0,50,n=5)" ... | funcnts.plot gnuplot
The -s (sum) switch causes funcnts to produce an additional table of summed (integrated) background subtracted values, along with the default table of individual values:
[sh] funcnts -s snr.ev "annulus(502,512,0,50,n=5)" "annulus(502,512,50,100)"
# source
# data file: snr.ev
# degrees/pix: 0.00222222
# background
# data file: snr.ev
# column units
# area: arcsec**2
# surf_bri: cnts/arcsec**2
# surf_err: cnts/arcsec**2
# summed background-subtracted results
upto net_counts error background berror area surf_bri surf_err
---- ------------ --------- ------------ --------- --------- --------- ---------
1 2880.999 54.722 112.001 1.204 19520.00 0.148 0.003
2 3776.817 65.254 457.183 4.914 79679.98 0.047 0.001
3 4025.492 71.972 1031.508 11.087 179775.96 0.022 0.000
4 4185.149 80.109 1840.851 19.786 320831.94 0.013 0.000
5 4415.540 90.790 2873.460 30.885 500799.90 0.009 0.000
# background-subtracted results
reg counts error background berror area surf_bri surf_err
---- ------------ --------- ------------ --------- --------- --------- ---------
1 2880.999 54.722 112.001 1.204 19520.00 0.148 0.003
2 895.818 35.423 345.182 3.710 60159.99 0.015 0.001
3 248.675 29.345 574.325 6.173 100095.98 0.002 0.000
4 159.657 32.321 809.343 8.699 141055.97 0.001 0.000
5 230.390 37.231 1032.610 11.099 179967.96 0.001 0.000
# the following source and background components were used:
source region(s)
----------------
annulus(502,512,0,50,n=5)
reg counts pixels sumcnts sumpix
---- ------------ --------- ------------ ---------
1 2993.000 305 2993.000 305
2 1241.000 940 4234.000 1245
3 823.000 1564 5057.000 2809
4 969.000 2204 6026.000 5013
5 1263.000 2812 7289.000 7825
background region(s)
--------------------
annulus(502,512,50,100)
reg counts pixels
---- ------------ ---------
all 8656.000 23572
The -t and -e switches can be used to apply timing and exposure corrections, respectively, to the data. Please note that these corrections are meant to be used qualitatively, since application of more accurate correction factors is a complex and mission-dependent effort. The algorithm for applying these simple corrections is as follows:
C = Raw Counts in Source Region Ac= Area of Source Region Tc= Exposure time for Source Data Ec= Average exposure in Source Region, from exposure map B= Raw Counts in Background Region Ab= Area of Background Region Tb= (Exposure) time for Background Data Eb= Average exposure in Background Region, from exposure mapThen, Net Counts in Source region is
Net= C - B * (Ac*Tc*Ec)/(Ab*Tb*Eb)with the standard propagation of errors for the Error on Net. The net rate would then be
Net Rate = Net/(Ac*Tc*Ec)The average exposure in each region is calculated by summing up the pixel values in the exposure map for the given region and then dividing by the number of pixels in that region. Exposure maps often are generated at a block factor > 1 (e.g., block 4 means that each exposure pixel contains 4x4 pixels at full resolution) and funcnts will deal with the blocking automatically. Using the -e switch, you can supply both source and background exposure files (separated by ";"), if you have separate source and background data files. If you do not supply a background exposure file to go with a separate background data file, funcnts assumes that exposure already has been applied to the background data file. In addition, it assumes that the error on the pixels in the background data file is zero.
NB: The -e switch assumes that the exposure map overlays the image file exactly, except for the block factor. Each pixel in the image is scaled by the block factor to access the corresponding pixel in the exposure map. If your exposure map does not line up exactly with the image, do not use the -e exposure correction. In this case, it still is possible to perform exposure correction if both the image and the exposure map have valid WCS information: use the -w switch so that the transformation from image pixel to exposure pixel uses the WCS information. That is, each pixel in the image region will be transformed first from image coordinates to sky coordinates, then from sky coordinates to exposure coordinates. Please note that using -w can increase the time required to process the exposure correction considerably.
A time correction can be applied to both source and background data using the -t switch. The value for the correction can either be a numeric constant or the name of a header parameter in the source (or background) file:
[sh] funcnts -t 23.4 ... # number for source [sh] funcnts -t "LIVETIME;23.4" ... # param for source, numeric for bkgdWhen a time correction is specified, it is applied to the net counts as well (see algorithm above), so that the units of surface brightness become cnts/area**2/sec.
The -i (interval) switch is used to run funcnts on multiple column-based intervals with only a single pass through the data. It is equivalent to running funcnts several times with a different column filter added to the source and background data each time. For each interval, the full funcnts output is generated, with a linefeed character (^L) inserted between each run. In addition, the output for each interval will contain the interval specification in its header. Intervals are very useful for generating X-ray hardness ratios efficiently. Of course, they are only supported when the input data are contained in a table.
Two formats are supported for interval specification. The most general format is semi-colon-delimited list of filters to be used as intervals:
funcnts -i "pha=1:5;pha=6:10;pha=11:15" snr.ev "circle(502,512,22)" ...Conceptually, this will be equivalent to running funcnts three times:
funcnts snr.ev'[pha=1:5]' "circle(502,512,22)" funcnts snr.ev'[pha=6:10]' "circle(502,512,22)" funcnts snr.ev'[pha=11:15]' "circle(502,512,22)"However, using the -i switch will require only one pass through the data.
Note that complex filters can be used to specify intervals:
funcnts -i "pha=1:5&&pi=4;pha=6:10&&pi=5;pha=11:15&&pi=6" snr.ev ...The program simply runs the data through each filter in turn and generates three funcnts outputs, separated by the line-feed character.
In fact, although the intent is to support intervals for hardness ratios, the specified filters do not have to be intervals at all. Nor does one "interval" filter have to be related to another. For example:
funcnts -i "pha=1:5;pi=6:10;energy=11:15" snr.ev "circle(502,512,22)" ...is equivalent to running funcnts three times with unrelated filter specifications.
A second interval format is supported for the simple case in which a single column is used to specify multiple homogeneous intervals for that column. In this format, a column name is specified first, followed by intervals:
funcnts -i "pha;1:5;6:10;11:15" snr.ev "circle(502,512,22)" ...This is equivalent to the first example, but requires less typing. The funcnts program will simply prepend "pha=" before each of the specified intervals. (Note that this format does not contain the "=" character in the column argument.)
Ordinarily, when funcnts is run on a FITS binary table (or a raw event table), one integral count is accumulated for each row (event) contained within a given region. The -v "scol[;bcol]" (value column) switch will accumulate counts using the value from the specified column for the given event. If only a single column is specified, it is used for both the source and background regions. Two separate columns, separated by a semi-colon, can be specified for source and background. The special token '$none' can be used to specify that a value column is to be used for one but not the other. For example, 'pha;$none' will use the pha column for the source but use integral counts for the background, while '$none;pha' will do the converse. If the value column is of type logical, then the value used will be 1 for T and 0 for F. Value columns are used, for example, to integrate probabilities instead of integral counts.
If the -T (rdb table) switch is used, the output will conform to starbase/rdb data base format: tabs will be inserted between columns rather than spaces and line-feed will be inserted between tables.
Finally, note that funcnts is an image program, even though it can be run directly on FITS binary tables. This means that image filtering is applied to the rows in order to ensure that the same results are obtained regardless of whether a table or the equivalent binned image is used. Because of this, however, the number of counts found using funcnts can differ from the number of events found using row-filter programs such as fundisp or funtable For more information about these difference, see the discussion of Region Boundaries.
funcone <switches> <iname> <oname> <ra[hdr]> <dec[hdr]> <radius[dr'"]> [columns]
-d deccol:[hdr] # Dec column name, units (def: DEC:d) -j # join columns from list file -J # join columns from list file, output all rows -l listfile # read centers and radii from a list -L listfile # read centers and radii from a list, output list rows -n # don't use cone limits as a filter -r racol:[hdr] # RA column name, units (def: RA:h) -x # append RA_CEN, DEC_CEN, RAD_CEN, CONE_KEY cols -X # append RA_CEN, DEC_CEN, RAD_CEN, CONE_KEY cols, output all rows
Funcone performs a cone search on the RA and Dec columns of a FITS binary table. The distance from the center RA, Dec position to the RA, Dec in each row in the table is calculated. Rows whose distance is less than the specified radius are output.
The first argument to the program specifies the FITS file, raw event file, or raw array file. If "stdin" is specified, data are read from the standard input. Use Funtools Bracket Notation to specify FITS extensions, and filters. The second argument is the output FITS file. If "stdout" is specified, the FITS binary table is written to the standard output.
The third and fourth required arguments are the RA and Dec center position. By default, RA is specified in hours while Dec is specified in degrees. You can change the units of either of these by appending the character "d" (degrees), "h" (hours) or "r" (radians). Sexagesimal notation is supported, with colons or spaces separating hms and dms. (When using spaces, please ensure that the entire string is quoted.)
The fifth required argument is the radius of the cone search. By default, the radius value is given in degrees. The units can be changed by appending the character "d" (degrees), "r" (radians), "'" (arc minutes) or '"' (arc seconds).
By default, all columns of the input file are copied to the output file. Selected columns can be output using an optional sixth argument in the form:
"column1 column1 ... columnN"A seventh argument allows you to output selected columns from the list file when -j switch is used. Note that the RA and Dec columns used in the cone calculation must not be de-selected.
Also by default, the RA and Dec column names are named "RA" and "Dec", and are given in units of hours and degrees respectively. You can change both the name and the units using the -r [RA] and/or -d [Dec] switches. Once again, one of "h", "d", or "r" is appended to the column name to specify units but in this case, there must be a colon ":" between the name and the unit specification.
If the -l [listfile] switch is used, then one or more of the center RA, center Dec, and radius can be taken from a list file (which can be a FITS table or an ASCII column text file). In this case, the third (center RA), fourth (center Dec), and fifth (radius) command line arguments can either be a column name in the list file (if that parameter varies) or else a numeric value (if that parameter is static). When a column name is specified for the RA, Dec, or radius, you can append a colon followed by "h", "d", or "r" to specify units (also ' and " for radius). The cone search algorithm is run once for each row in the list, taking RA, Dec, and radius values from the specified columns or from static numeric values specified on the command line.
When using a list, all valid rows from each iteration are written to a single output file. Use the -x switch to help delineate which line of the list file was used to produce the given output row(s). This switch causes the values for the center RA, Dec, radius, and row number to be appended to the output file, in columns called RA_CEN, DEC_CEN, RAD_CEN and CONE_KEY, respectively. Alternatively, the -j (join) switch will append all columns from the list row to the output row (essentially a join of the list row and input row), along with the CONE_KEY row number. These two switches are mutually exclusive.
The -X and -J switches write out the same data as their lower case counterparts for each row satisfying a cone search. In addition, these switches also write out rows from the event file that do not satisfy any cone search. In such cases, that CONE_KEY column will be given a value of -1 and the center and list position information will be set to zero for the given row. Thus, all rows of the input event file are guaranteed to be output, with rows satisfying at least one cone search having additional search information.
The -L switch acts similarly to the -l switch in that it takes centers from a list file. However, it also implicitly sets the -j switch, so that output rows are the join of the input event row and the center position row. In addition, this switch also writes out all center position rows for which no event satisfies the cone search criteria of that row. The CONE_KEY column will be given a value of -2 for center rows that were not close to any data row and the event columns will be zeroed out for such rows. In this way, all centers rows are guaranteed to be output at least once.
If any of "all row" switches (-X, -J, or -L) are specified, then a new column named JSTAT is added to the output table. The positive values in this column indicate the center position row number (starting from 1) in the list file that this data row successful matched in a cone search. A value of -1 means that the data row did not match any center position. A value of -2 means that the center position was not matched by any data row.
Given a center position and radius, the cone search algorithm calculates limit parameters for a box enclosing the specified cone, and only tests rows whose positions values lie within those limits. For small files, the overhead associated with this cone limit filtering can cause the program to run more slowly than if all events were tested. You can turn off cone limit filtering using the -n switch to see if this speeds up the processing (especially useful when processing a large list of positions).
For example, the default cone search uses columns "RA" and "Dec" in hours and degrees (respectively) and RA position in hours, Dec and radius in degrees:
funone in.fits out.fits 23.45 34.56 0.01To specify the RA position in degrees:
funcone in.fits out.fits 23.45d 34.56 0.01To get RA and Dec from a list but use a static value for radius (and also write identifying info for each row in the list):
funcone -x -l list.txt in.fits out.fits MYRA MYDec 0.01User specified columns in degrees, RA position in hours (sexagesimal notation), Dec position in degrees (sexagesimal notation) and radius in arc minutes:
funcone -r myRa:d -d myDec in.fits out.fits 12:30:15.5 30:12 15'
fundisp [-f format] [-l] [-n] [-T] <iname> [columns|bitpix=n]
-f # format string for display -l # display image as a list containing the columns X, Y, VAL -n # don't output header -F [c] # use specified character as column separator (def: space) -T # output in rdb/starbase format (tab separators)
fundisp displays the data in the specified FITS Extension and/or Image Section of a FITS file, or in a Section of a non-FITS array or raw event file.
The first argument to the program specifies the FITS input image, array, or raw event file to display. If "stdin" is specified, data are read from the standard input. Use Funtools Bracket Notation to specify FITS extensions, image sections, and filters.
If the data being displayed are columns (either in a FITS binary table or a raw event file), the individual rows are listed. Filters can be added using bracket notation. Thus:
[sh] fundisp "test.ev[time-(int)time>.15]"
X Y PHA PI TIME DX DY
------- ------- ------- --------- ---------------- ---------- ----------
10 8 10 8 17.1600 8.50 10.50
9 9 9 9 17.1600 9.50 9.50
10 9 10 9 18.1600 9.50 10.50
10 9 10 9 18.1700 9.50 10.50
8 10 8 10 17.1600 10.50 8.50
9 10 9 10 18.1600 10.50 9.50
9 10 9 10 18.1700 10.50 9.50
10 10 10 10 19.1600 10.50 10.50
10 10 10 10 19.1700 10.50 10.50
10 10 10 10 19.1800 10.50 10.50
[NB: The FITS binary table test file test.ev, as well as the FITS
image test.fits, are contained in the funtools funtest directory.]
When a table is being displayed using fundisp, a second optional argument can be used to specify the columns to display. For example:
[sh] fundisp "test.ev[time-(int)time>=.99]" "x y time"
X Y TIME
-------- -------- ---------------------
5 -6 40.99000000
4 -5 59.99000000
-1 0 154.99000000
-2 1 168.99000000
-3 2 183.99000000
-4 3 199.99000000
-5 4 216.99000000
-6 5 234.99000000
-7 6 253.99000000
The special column $REGION can be specified to display the region id of each row:
[sh $] fundisp "test.ev[time-(int)time>=.99&&annulus(0 0 0 10 n=3)]" 'x y time $REGION'
X Y TIME REGION
-------- -------- --------------------- ----------
5 -6 40.99000000 3
4 -5 59.99000000 2
-1 0 154.99000000 1
-2 1 168.99000000 1
-3 2 183.99000000 2
-4 3 199.99000000 2
-5 4 216.99000000 2
-6 5 234.99000000 3
-7 6 253.99000000 3
Here only rows with the proper fractional time and whose position also is within one of the three annuli are displayed.
Columns can be excluded from display using a minus sign before the column:
[sh $] fundisp "test.ev[time-(int)time>=.99]" "-time"
X Y PHA PI DX DY
-------- -------- -------- ---------- ----------- -----------
5 -6 5 -6 5.50 -6.50
4 -5 4 -5 4.50 -5.50
-1 0 -1 0 -1.50 0.50
-2 1 -2 1 -2.50 1.50
-3 2 -3 2 -3.50 2.50
-4 3 -4 3 -4.50 3.50
-5 4 -5 4 -5.50 4.50
-6 5 -6 5 -6.50 5.50
-7 6 -7 6 -7.50 6.50
All columns except the time column are displayed.
The special column $N can be specified to display the ordinal value of each row. Thus, continuing the previous example:
fundisp "test.ev[time-(int)time>=.99]" '-time $n'
X Y PHA PI DX DY N
------- -------- -------- ---------- ----------- ----------- ----------
5 -6 5 -6 5.50 -6.50 337
4 -5 4 -5 4.50 -5.50 356
-1 0 -1 0 -1.50 0.50 451
-2 1 -2 1 -2.50 1.50 465
-3 2 -3 2 -3.50 2.50 480
-4 3 -4 3 -4.50 3.50 496
-5 4 -5 4 -5.50 4.50 513
-6 5 -6 5 -6.50 5.50 531
-7 6 -7 6 -7.50 6.50 550
Note that the column specification is enclosed in single quotes to protect
'$n' from begin expanded by the shell.
In general, the rules for activating and de-activating columns are:
If the data being displayed are image data (either in a FITS primary image, a FITS image extension, or an array file), an mxn pixel display is produced, where m and n are the dimensions of the image. By default, pixel values are displayed using the same data type as in the file. However, for integer data where the BSCALE and BZERO header parameters are present, the data is displayed as floats. In either case, the display data type can be overridden using an optional second argument of the form:
bitpix=nwhere n is 8,16,32,-32,-64, for unsigned char, short, int, float and double, respectively.
Of course, running fundisp on anything but the smallest image usually results in a display whose size makes it unreadable. Therefore, one can uses bracket notation (see below) to apply section and/or blocking to the image before generating a display. For example:
[sh] fundisp "test.fits[2:6,2:7]" bitpix=-32
2 3 4 5 6
---------- ---------- ---------- ---------- ----------
2: 3.00 4.00 5.00 6.00 7.00
3: 4.00 5.00 6.00 7.00 8.00
4: 5.00 6.00 7.00 8.00 9.00
5: 6.00 7.00 8.00 9.00 10.00
6: 7.00 8.00 9.00 10.00 11.00
7: 8.00 9.00 10.00 11.00 12.00
Note that is is possible to display a FITS binary table as an image simply by passing the table through funimage first:
[sh] ./funimage test.ev stdout | fundisp "stdin[2:6,2:7]" bitpix=8
2 3 4 5 6
------- ------- ------- ------- -------
2: 3 4 5 6 7
3: 4 5 6 7 8
4: 5 6 7 8 9
5: 6 7 8 9 10
6: 7 8 9 10 11
7: 8 9 10 11 12
If the -l (list) switch is used, then an image is displayed as a
list containing the columns: X, Y, VAL. For example:
fundisp -l "test1.fits[2:6,2:7]" bitpix=-32
X Y VAL
---------- ---------- -----------
2 2 6.00
3 2 1.00
4 2 1.00
5 2 1.00
6 2 1.00
2 3 1.00
3 3 5.00
4 3 1.00
5 3 1.00
6 3 1.00
2 4 1.00
3 4 1.00
4 4 4.00
5 4 1.00
6 4 1.00
2 5 1.00
3 5 1.00
4 5 1.00
5 5 3.00
6 5 1.00
2 6 1.00
3 6 1.00
4 6 1.00
5 6 1.00
6 6 2.00
2 7 1.00
3 7 1.00
4 7 1.00
5 7 1.00
6 7 1.00
If the -n (nohead) switch is used, then no header is output for tables. This is useful, for example, when fundisp output is being directed into gnuplot.
The fundisp program uses a default set of display formats:
datatype TFORM format -------- ----- -------- double D "%21.8f" float E "%11.2f" int J "%10d" short I "%8d" byte B "%6d" string A "%12.12s" bits X "%8x" logical L "%1x"Thus, the default display of 1 double and 2 shorts gives:
[sh] fundisp snr.ev "time x y"
TIME X Y
--------------------- -------- --------
79494546.56818075 546 201
79488769.94469175 548 201
...
You can change the display format for individual columns or for all
columns of a given data types by means of the -f switch. The format
string that accompanies -f is a space-delimited list of keyword=format
values. The keyword values can either be column names (in which case
the associated format pertains only to that column) or FITS table
TFORM specifiers (in which case the format pertains to all columns
having that data type). For example, you can change the double and
short formats for all columns like this:
[sh] fundisp -f "D=%22.11f I=%3d" snr.ev "time x y"
TIME X Y
---------------------- --- ---
79494546.56818075478 546 201
79488769.94469174743 548 201
...
Alternatively, you can change the format of the time and x columns like this:
[sh] fundisp -f "time=%22.11f x=%3d" snr.ev "time x y"
TIME X Y
---------------------- --- --------
79494546.56818075478 546 201
79488769.94469174743 548 201
...
Note that there is a potential conflict if a column has the same name
as one of the TFORM specifiers. In the examples above, the the "X"
column in the table has the same name as the X (bit) datatype. To
resolve this conflict, the format string is processed such that
TFORM datatype specifiers are checked for first, using a
case-sensitive comparison. If the specified format value is not an
upper case TFORM value, then a case-insensitive check is made on the
column name. This means that, in the examples above, "X=%3d" will refer
to the X (bit) datatype, while "x=%3d" will refer to the X column:
[sh] fundisp -f "X=%3d" snr.ev "x y"
X Y
-------- --------
546 201
548 201
...
[sh] fundisp -f "x=%3d" snr.ev "x y"
X Y
--- --------
546 201
548 201
...
As a rule, therefore, it is best always to specify the column name in
lower case and TFORM data types in upper case.
The -f [format] will change the format for a single execution of fundisp. You also can use the FUN_FORMAT envronment variable to change the format for all invocations of fundisp. The format of this environment variable's value is identical to that used with the -f switch. This global value can be overridden in individual cases by use of the -f [format] switch.
Caveats: Please also note that it is the user's responsibility to match the format specifier to the column data type correctly. Also note that, in order to maintain visual alignment between names and columns, the column name will be truncated (on the left) if the format width is less than the length of the name. However, truncation is not performed if the output is in RDB format (using the -T switch).
[An older-style format string is supported but deprecated. It consists of space-delimited C format statements for all data types, specified in the following order:
double float int short byte string bit.This order of the list is based on the assumption that people generally will want to change the float formats.
If "-" is entered instead of a format statement for a given data type, the default format is used. Also, the format string can be terminated without specifying all formats, and defaults will be used for the rest of the list. Note that you must supply a minimum field width, i.e., "%6d" and "%-6d" are legal, "%d" is not legal. By using -f [format], you can change the double and short formats like this:
[sh] fundisp -f "22.11f - - 3d" snr.ev "time x y"
TIME X Y
---------------------- --- ---
79494546.56818075478 546 201
79488769.94469174743 548 201
...
NB: This format is deprecated and will be removed in a future release.]
The -F[c] switch can be used to specify a (single-character) column separator (where the default is a space). Note that column formatting will almost certainly also add spaces to pad individual columns to the required width. These can be removed with a program such as sed, at the cost of generating unaligned columns. For example:
fundisp -F',' snr.ev'[cir 512 512 .1]'
X, Y, PHA, PI, TIME, DX, DY
--------,--------,--------,--------,---------------------,--------,--------
512, 512, 6, 7, 79493997.45854475, 578, 574
512, 512, 8, 9, 79494575.58943175, 579, 573
512, 512, 5, 6, 79493631.03866175, 578, 575
512, 512, 5, 5, 79493290.86521725, 578, 575
512, 512, 8, 9, 79493432.00990875, 579, 573
fundisp -F',' snr.ev'[cir 512 512 .1]' | sed 's/ *, */,/g'
X,Y,PHA,PI,TIME,DX,DY
--------,--------,--------,--------,---------------------,--------,--------
512,512,6,7,79493997.45854475,578,574
512,512,8,9,79494575.58943175,579,573
512,512,5,6,79493631.03866175,578,575
512,512,5,5,79493290.86521725,578,575
512,512,8,9,79493432.00990875,579,573
fundisp -f "x=%3d y=%3d pi=%1d pha=%1d time=%20.11f dx=%3d dy=%3d" -F',' snr.ev'[cir 512 512 .1]' | sed 's/ *, */,/g'
X,Y,A,I,TIME,DX,DY
---,---,-,-,--------------------,---,---
512,512,6,7,79493997.45854474604,578,574
512,512,8,9,79494575.58943174779,579,573
512,512,5,6,79493631.03866174817,578,575
512,512,5,5,79493290.86521725357,578,575
512,512,8,9,79493432.00990875065,579,573
If the -T (rdb table) switch is used, the output will conform to starbase/rdb data base format: tabs will be inserted between columns rather than spaces. This format is not available when displaying image pixels (except in conjunction with the -l switch).
Finally, note that fundisp can be used to create column filters from the auxiliary tables in a FITS file. For example, the following shell code will generate a good-time interval (GTI) filter for X-ray data files that contain a standard GTI extension:
#!/bin/sh
sed '1,/---- .*/d
/^$/,$d' | awk 'tot>0{printf "||"};{printf "time="$1":"$2; tot++}'
If this script is placed in a file called "mkgti", it can be used in a
command such as:
fundisp foo.fits"[GTI]" | mkgti > gti.filterThe resulting filter file can then be used in various funtools programs:
funcnts foo.fits"[@gti.filter]" ...to process only the events in the good-time intervals.
funhead [-a] [-s] [-t] [-L] <iname> [oname ename]
-a # display all extension headers -s # display 79 chars instead of 80 before the new-line -t # prepend data type char to each line of output -L # output in rdb/starbase list format
funhead displays the FITS header parameters in the specified FITS Extension.
The first argument to the program specifies the Funtools input file to display. If "stdin" is specified, data are read from the standard input. Funtools Bracket Notation is used to specify particular FITS extension to process. Normally, the full 80 characters of each header card is output, followed by a new-line.
If the -a switch is specified, the header from each FITS extensions in the file is displayed. Note, however, that the -a switch does not work with FITS files input via stdin. We hope to remove this restriction in a future release.
If the -s switch is specified, only 79 characters are output before the new-line. This helps the display on 80 character terminals.
If the -t switch is specified, the data type of the parameter is output as a one character prefix, followed by 77 characters of the param. The parameter data types are defined as: FUN_PAR_UNKNOWN ('u'), FUN_PAR_COMMENT ('c'), FUN_PAR_LOGICAL ('l'), FUN_PAR_INTEGER ('i'), FUN_PAR_STRING ('s'), FUN_PAR_REAL ('r'), FUN_PAR_COMPLEX ('x').
If the -L (rdb table) switch is used, the output will conform to starbase/rdb data base list format.
For example to display the EVENTS extension (binary table):
[sh] funhead "foo.fits[EVENTS]"
XTENSION= 'BINTABLE' / FITS 3D BINARY TABLE
BITPIX = 8 / Binary data
NAXIS = 2 / Table is a matrix
NAXIS1 = 20 / Width of table in bytes
NAXIS2 = 30760 / Number of entries in table
PCOUNT = 0 / Random parameter count
GCOUNT = 1 / Group count
TFIELDS = 7 / Number of fields in each row
EXTNAME = 'EVENTS ' / Table name
EXTVER = 1 / Version number of table
TFORM1 = '1I ' / Data type for field
TTYPE1 = 'X ' / Label for field
TUNIT1 = ' ' / Physical units for field
TFORM2 = '1I ' / Data type for field
etc. ...
END
To display the third header:
[sh] funhead "foo.fits[3]"
XTENSION= 'BINTABLE' / FITS 3D BINARY TABLE
BITPIX = 8 / Binary data
NAXIS = 2 / Table is a matrix
NAXIS1 = 32 / Width of table in bytes
NAXIS2 = 40 / Number of entries in table
PCOUNT = 0 / Random parameter count
GCOUNT = 1 / Group count
TFIELDS = 7 / Number of fields in each row
EXTNAME = 'TGR ' / Table name
EXTVER = 1 / Version number of table
TFORM1 = '1D ' / Data type for field
etc. ...
END
To display the primary header (i.e., extension 0):
sh> funhead "coma.fits[0]" SIMPLE = T /STANDARD FITS FORMAT BITPIX = 16 /2-BYTE TWOS-COMPL INTEGER NAXIS = 2 /NUMBER OF AXES NAXIS1 = 800 / NAXIS2 = 800 / DATATYPE= 'INTEGER*2' /SHORT INTEGER END
The funhead program also can edit (i.e. add, delete, or modify) or display individual headers parameters. Edit mode is signalled by the presence of two additional command-line arguments: output file and edit command file, in that order. Edit mode acts as a filter: the output file will contain the entire input FITS file, including other extensions. The edit command file can be "stdin", in which case edit command are read from the standard input.
The edit command file contains parameter comments (having '#' in the first column) and delete and assignment(modify or add) operations. A delete operation is specified by preceding the parameter name with a minus sign "-". A display operation (very useful in interactive sessions, i.e., where the edit commands are taken from stdin) is specified by preceding the parameter name with a question mark "?". In either case, a parameter value need not be specified. An assignment operation is specified in the same two ways that a parameter is specified in a text header (but without the comment character that precedes header params), i.e.:
For example, the following interactive session checks for the existence of parameters, adds new parameters, modifies them, and modifies and deletes existing parameters:
sh$ ./funhead snr.ev foo.fits - # look for FOO1 ? FOO1 WARNING: FOO1 not found # add new foo1 FOO1 = 100 # add foo2 FOO2 = 200 # reset foo1 to a different value FOO1 -1 # delete foo2 -FOO2 # change existing value EXTVER 2 ? XS-SORT XS-SORT = 'EOF ' / type of event sort # delete existing value -XS-SORT # exit ^D
See Column-based Text Files for more information about header parameter format.
funhist [-n|-w|-T] <iname> [column] [[lo:hi:]bins]
-n # normalize bin value by the width of each bin -w # specify bin width instead of number of bins in arg3 -T # output in rdb/starbase format (tab separators)
funhist creates a one-dimensional histogram from the specified columns of a FITS Extension binary table of a FITS file (or from a non-FITS raw event file), or from a FITS image or array, and writes that histogram as an ASCII table. Alternatively, the program can perform a 1D projection of one of the image axes.
The first argument to the program is required, and specifies the Funtools file: FITS table or image, raw event file, or array. If "stdin" is specified, data are read from the standard input. Use Funtools Bracket Notation to specify FITS extensions, and filters.
For a table, the second argument also is required. It specifies the column to use in generating the histogram. If the data file is of type image (or array), the column is optional: if "x" (or "X"), "y" (or "Y") is specified, then a projection is performed over the x (dim1) or y (dim2) axes, respectively. (That is, this projection will give the same results as a histogram performed on a table containing the equivalent x,y event rows.) If no column name is specified or "xy" (or "XY") is specified for the image, then a histogram is performed on the values contained in the image pixels.
The argument that follows is optional and specifies the number of bins to use in creating the histogram and, if desired, the range of bin values. For image and table histograms, the range should specify the min and max data values. For image histograms on the x and y axes, the range should specify the min and max image bin values. If this argument is omitted, the number of output bins for a table is calculated either from the TLMIN/TLMAX headers values (if these exist in the table FITS header for the specified column) or by going through the data to calculate the min and max value. For an image, the number of output bins is calculated either from the DATAMIN/DATAMAX header values, or by going through the data to calculate min and max value. (Note that this latter calculation might fail if the image cannot be fit in memory.) If the data are floating point (table or image) and the number of bins is not specified, an arbitrary default of 128 is used.
For binary table processing, the -w (bin width) switch can be used to specify the width of each bin rather than the number of bins. Thus:
funhist test.ev pha 1:100:5means that 5 bins of width 20 are used in the histogram, while:
funhist -w test.ev pha 1:100:5means that 20 bins of width 5 are used in the histogram.
The data are divvied up into the specified number of bins and the resulting 1D histogram (or projection) is output in ASCII table format. For a table, the output displays the low_edge (inclusive) and hi_edge (exclusive) values for the data. For example, a 15-row table containing a "pha" column whose values range from -7.5 to 7.5 can be processed thus:
[sh] funhist test.ev pha
# data file: /home/eric/data/test.ev
# column: pha
# min,max,bins: -7.5 7.5 15
bin value lo_edge hi_edge
------ --------- --------------------- ---------------------
1 22 -7.50000000 -6.50000000
2 21 -6.50000000 -5.50000000
3 20 -5.50000000 -4.50000000
4 19 -4.50000000 -3.50000000
5 18 -3.50000000 -2.50000000
6 17 -2.50000000 -1.50000000
7 16 -1.50000000 -0.50000000
8 30 -0.50000000 0.50000000
9 16 0.50000000 1.50000000
10 17 1.50000000 2.50000000
11 18 2.50000000 3.50000000
12 19 3.50000000 4.50000000
13 20 4.50000000 5.50000000
14 21 5.50000000 6.50000000
15 22 6.50000000 7.50000000
[sh] funhist test.ev pha 1:6
# data file: /home/eric/data/test.ev
# column: pha
# min,max,bins: 0.5 6.5 6
bin value lo_edge hi_edge
------ --------- --------------------- ---------------------
1 16 0.50000000 1.50000000
2 17 1.50000000 2.50000000
3 18 2.50000000 3.50000000
4 19 3.50000000 4.50000000
5 20 4.50000000 5.50000000
6 21 5.50000000 6.50000000
[sh] funhist test.ev pha 1:6:3
# data file: /home/eric/data/test.ev
# column: pha
# min,max,bins: 0.5 6.5 3
bin value lo_edge hi_edge
------ --------- --------------------- ---------------------
1 33 0.50000000 2.50000000
2 37 2.50000000 4.50000000
3 41 4.50000000 6.50000000
For a table histogram, the -n(normalize) switch can be used to normalize the bin value by the width of the bin (i.e., hi_edge-lo_edge):
[sh] funhist -n test.ev pha 1:6:3
# data file: test.ev
# column: pha
# min,max,bins: 0.5 6.5 3
# width normalization (val/(hi_edge-lo_edge)) is applied
bin value lo_edge hi_edge
------ --------------------- --------------------- ---------------------
1 16.50000000 0.50000000 2.50000000
2 6.16666667 2.50000000 4.50000000
3 4.10000000 4.50000000 6.50000000
This could used, for example, to produce a light curve with values
having units of counts/second instead of counts.
For an image histogram, the output displays the low and high image values (both inclusive) used to generate the histogram. For example, in the following example, 184 pixels had a value of 1, 31 had a value of 2, while only 2 had a value of 3,4,5,6, or 7:
[sh] funhist test.fits
# data file: /home/eric/data/test.fits
# min,max,bins: 1 7 7
bin value lo_val hi_val
------ --------------------- --------------------- ---------------------
1 184.00000000 1.00000000 1.00000000
2 31.00000000 2.00000000 2.00000000
3 2.00000000 3.00000000 3.00000000
4 2.00000000 4.00000000 4.00000000
5 2.00000000 5.00000000 5.00000000
6 2.00000000 6.00000000 6.00000000
7 2.00000000 7.00000000 7.00000000
For the axis projection of an image, the output displays the low and high image bins (both inclusive) used to generate the projection. For example, in the following example, 21 counts had their X bin value of 2, etc.:
[sh] funhist test.fits x 2:7
# data file: /home/eric/data/test.fits
# column: X
# min,max,bins: 2 7 6
bin value lo_bin hi_bin
------ --------------------- --------------------- ---------------------
1 21.00000000 2.00000000 2.00000000
2 20.00000000 3.00000000 3.00000000
3 19.00000000 4.00000000 4.00000000
4 18.00000000 5.00000000 5.00000000
5 17.00000000 6.00000000 6.00000000
6 16.00000000 7.00000000 7.00000000
[sh] funhist test.fits x 2:7:2
# data file: /home/eric/data/test.fits
# column: X
# min,max,bins: 2 7 2
bin value lo_bin hi_bin
------ --------------------- --------------------- ---------------------
1 60.00000000 2.00000000 4.00000000
2 51.00000000 5.00000000 7.00000000
You can use gnuplot or other plotting programs to graph the results, using a script such as:
#!/bin/sh
sed -e '1,/---- .*/d
/^$/,$d' | \
awk '\
BEGIN{print "set nokey; set title \"funhist\"; set xlabel \"bin\"; set ylabel \"counts\"; plot \"-\" with boxes"} \
{print $3, $2, $4-$3}' | \
gnuplot -persist - 1>/dev/null 2>&1
Similar plot commands are supplied in the script funhist.plot:
funhist test.ev pha ... | funhist.plot gnuplot
funimage [-a] <iname> <oname> [bitpix=n] funimage [-l] <iname> <oname> <xcol:xdims> <ycol:ydims> <vcol> [bitpix=n] funimage [-p x|y] <iname> <oname> [bitpix=n]
-a # append to existing output file as an image extension -l # input is a list file containing xcol, ycol, value -p [x|y] # project along x or y axis to create a 1D image
funimage creates a primary FITS image from the specified FITS Extension and/or Image Section of a FITS file, or from an Image Section of a non-FITS array, or from a raw event file.
The first argument to the program specifies the FITS input image, array, or raw event file to process. If "stdin" is specified, data are read from the standard input. Use Funtools Bracket Notation to specify FITS extensions, image sections, and filters. The second argument is the output FITS file. If "stdout" is specified, the FITS image is written to the standard output. By default, the output pixel values are of the same data type as those of the input file (or type "int" when binning a table), but this can be overridden using an optional third argument of the form:
bitpix=nwhere n is 8,16,32,-32,-64, for unsigned char, short, int, float and double, respectively.
If the input data are of type image, the appropriate section is extracted and blocked (based on how the Image Section is specified), and the result is written to the FITS primary image. When an integer image containing the BSCALE and BZERO keywords is converted to float, the pixel values are scaled and the scaling keywords are deleted from the output header. When converting integer scaled data to integer (possibly of a different size), the pixels are not scaled and the scaling keywords are retained.
If the input data is a binary table or raw event file, these are binned into an image, from which a section is extracted and blocked, and written to a primary FITS image. In this case, it is necessary to specify the two columns that will be used in the 2D binning. This can be done on the command line using the bincols=(x,y) keyword:
funcnts "foo.ev[EVENTS,bincols=(detx,dety)]"The full form of the bincols= specifier is:
bincols=([xname[:tlmin[:tlmax:[binsiz]]]],[yname[:tlmin[:tlmax[:binsiz]]]])where the tlmin, tlmax, and binsiz specifiers determine the image binning dimensions:
dim = (tlmax - tlmin)/binsiz (floating point data) dim = (tlmax - tlmin)/binsiz + 1 (integer data)Using this syntax, it is possible to bin any two columns of a binary table at any bin size. Note that the tlmin, tlmax, and binsiz specifiers can be omitted if TLMIN, TLMAX, and TDBIN header parameters (respectively) are present in the FITS binary table header for the column in question. Note also that if only one parameter is specified, it is assumed to be tlmax, and tlmin defaults to 1. If two parameters are specified, they are assumed to be tlmin and tlmax. See Binning FITS Binary Tables and Non-FITS Event Files for more information about binning parameters.
By default, a new 2D FITS image file is created and the image is written to the primary HDU. If the -a (append) switch is specified, the image is appended to an existing FITS file as an IMAGE extension. (If the output file does not exist, the switch is effectively ignored and the image is written to the primary HDU.) This can be useful in a shell programming environment when processing multiple FITS images that you want to combine into a single final FITS file.
funimage also can take input from a table containing columns of x, y, and value (e.g., the output from fundisp -l which displays each image x and y and the number of counts at that position.) When the -l (list) switch is used, the input file is taken to be a FITS or ASCII table containing (at least) three columns that specify the x and y image coordinates and the value of that image pixel. In this case, funimage requires four extra arguments: xcolumn:xdims, ycolumn:ydims, vcolumn and bitpix=n. The x and y col:dim information takes the form:
name:dim # values range from 1 to dim name:min:max # values range from min to max name:min:max:binsiz # dimensions scaled by binsizeIn particular, the min value should be used whenever the minimum coordinate value is something other than one. For example:
funimage -l foo.lst foo.fits xcol:0:512 ycol:0:512 value bitpix=-32
The list feature also can be used to read unnamed columns from standard input: simply replace the column name with a null string. Note that the dimension information is still required:
funimage -l stdin foo.fits "":0:512 "":0:512 "" bitpix=-32 240 250 1 255 256 2 ... ^D
The list feature provides a simple way to generate a blank image. If you pass a Column-based Text File to funimage in which the text header contains the required image information, then funimage will correctly make a blank image. For example, consider the following text file (called foo.txt):
x:I:1:10 y:I:1:10 ------ ------ 0 0This text file defines two columns, x and y, each of data type 32-bit int and image dimension 10. The command:
funimage foo.txt foo.fits bitpix=8will create an empty FITS image called foo.fits containing a 10x10 image of unsigned char:
fundisp foo.fits
1 2 3 4 5 6 7 8 9 10
------ ------ ------ ------ ------ ------ ------ ------ ------ ------
10: 0 0 0 0 0 0 0 0 0 0
9: 0 0 0 0 0 0 0 0 0 0
8: 0 0 0 0 0 0 0 0 0 0
7: 0 0 0 0 0 0 0 0 0 0
6: 0 0 0 0 0 0 0 0 0 0
5: 0 0 0 0 0 0 0 0 0 0
4: 0 0 0 0 0 0 0 0 0 0
3: 0 0 0 0 0 0 0 0 0 0
2: 0 0 0 0 0 0 0 0 0 0
1: 1 0 0 0 0 0 0 0 0 0
Note that the text file must contain at least
one row of data. However, in the present example, event position 0,0 is
outside the limits of the image and will be ignored. (You can, of course,
use real x,y values to seed the image with data.)
Furthermore, you can use the TEXT filter specification to obviate the need for an input text file altogether. The following command will create the same 10x10 char image without an actual input file:
funimage stdin'[TEXT(x:I:10,y:I:10)]' foo.fits bitpix=8 < /dev/null or funimage /dev/null'[TEXT(x:I:10,y:I:10)]' foo.fits bitpix=8
You also can use either of these methods to generate a region mask simply by appending a region inside the filter brackets and specfying mask=all along with the bitpix. For example, the following command will generate a 10x10 char mask using 3 regions:
funimage stdin'[TEXT(x:I:10,y:I:10),cir(5,5,4),point(10,1),-cir(5,5,2)]' \ foo.fits bitpix=8,mask=all < /dev/nullThe resulting mask looks like this:
fundisp foo.fits
1 2 3 4 5 6 7 8 9 10
------ ------ ------ ------ ------ ------ ------ ------ ------ ------
10: 0 0 0 0 0 0 0 0 0 0
9: 0 0 0 0 0 0 0 0 0 0
8: 0 0 1 1 1 1 1 0 0 0
7: 0 1 1 1 1 1 1 1 0 0
6: 0 1 1 0 0 0 1 1 0 0
5: 0 1 1 0 0 0 1 1 0 0
4: 0 1 1 0 0 0 1 1 0 0
3: 0 1 1 1 1 1 1 1 0 0
2: 0 0 1 1 1 1 1 0 0 0
1: 0 0 0 0 0 0 0 0 0 2
You can use funimage to create 1D image projections along the x or y axis using the -p [x|y] switch. This capability works for both images and tables. For example consider a FITS table named ev.fits containing the following rows:
X Y
-------- --------
1 1
1 2
1 3
1 4
1 5
2 2
2 3
2 4
2 5
3 3
3 4
3 5
4 4
4 5
5 5
A corresponding 5x5 image, called dim2.fits, would therefore contain:
1 2 3 4 5
---------- ---------- ---------- ---------- ----------
5: 1 1 1 1 1
4: 1 1 1 1 0
3: 1 1 1 0 0
2: 1 1 0 0 0
1: 1 0 0 0 0
A projection along the y axis can be performed on either the table or
the image:
funimage -p y ev.fits stdout | fundisp stdin
1 2 3 4 5
---------- ---------- ---------- ---------- ----------
1: 1 2 3 4 5
funimage -p y dim2.fits stdout | fundisp stdin
1 2 3 4 5
---------- ---------- ---------- ---------- ----------
1: 1 2 3 4 5
Furthermore, you can create a 1D image projection along any column of a table by using the bincols=[column] filter specification and specifying a single column. For example, the following command projects the same 1D image along the y axis of a table as use of the -p y switch:
funimage ev.fits'[bincols=y]' stdout | fundisp stdin
1 2 3 4 5
---------- ---------- ---------- ---------- ----------
1: 1 2 3 4 5
Examples:
Create a FITS image from a FITS binary table:
[sh] funimage test.ev test.fits
Display the FITS image generated from a blocked section of FITS binary table:
[sh] funimage "test.ev[2:8,3:7,2]" stdout | fundisp stdin
1 2 3
--------- --------- ---------
1: 20 28 36
2: 28 36 44
funindex <switches> <iname> [oname]
NB: these options are not compatible with Funtools processing. Please use the defaults instead. -c # compress output using gzip" -a # ASCII output, ignore -c (default: FITS table)" -f # FITS table output (default: FITS table)" -l # long output, i.e. with key value(s) (default: long)" -s # short output, i.e. no key value(s) (default: long)"
The funindex script creates an index for the specified column (key) by running funtable -s (sort) and then saving the column value and the record number for each sorted row. This index will be used automatically by funtools filtering of that column, provided the index file's modification date is later than that of the data file.
The first required argument is the name of the FITS binary table to index. Please note that text files cannot be indexed at this time. The second required argument is the column (key) name to index. While multiple keys can be specified in principle, the funtools index processing assume a single key and will not recognize files containing multiple keys.
By default, the output index file name is [root]_[key].idx, where [root] is the root of the input file. Funtools looks for this specific file name when deciding whether to use an index for faster filtering. Therefore, the optional third argument (output file name) should not be used for funtools processing.
For example, to create an index on column Y for a given FITS file, use:
funindex foo.fits YThis will generate an index named foo_y.idx, which will be used by funtools for filters involving the Y column.
funjoin [switches] <ifile1> <ifile2> ... <ifilen> <ofile>
-a cols # columns to activate in all files -a1 cols ... an cols # columns to activate in each file -b 'c1:bvl,c2:bv2' # blank values for common columns in all files -bn 'c1:bv1,c2:bv2' # blank values for columns in specific files -j col # column to join in all files -j1 col ... jn col # column to join in each file -m min # min matches to output a row -M max # max matches to output a row -s # add 'jfiles' status column -S col # add col as status column -t tol # tolerance for joining numeric cols [2 files only]funjoin joins rows from two or more (up to 32) FITS Binary Table files, based on the values of specified join columns in each file. NB: the join columns must have an index file associated with it. These files are generated using the funindex program.
The first argument to the program specifies the first input FITS table or raw event file. If "stdin" is specified, data are read from the standard input. Subsequent arguments specify additional event files and tables to join. The last argument is the output FITS file.
NB: Do not use Funtools Bracket Notation to specify FITS extensions and row filters when running funjoin or you will get wrong results. Rows are accessed and joined using the index files directly, and this bypasses all filtering.
The join columns are specified using the -j col switch (which specifies a column name to use for all files) or with -j1 col1, -j2 col2, ... -jn coln switches (which specify a column name to use for each file). A join column must be specified for each file. If both -j col and -jn coln are specified for a given file, then the latter is used. Join columns must either be of type string or type numeric; it is illegal to mix numeric and string columns in a given join. For example, to join three files using the same key column for each file, use:
funjoin -j key in1.fits in2.fits in3.fits out.fitsA different key can be specified for the third file in this way:
funjoin -j key -j3 otherkey in1.fits in2.fits in3.fits out.fits
The -a "cols" switch (and -a1 "col1", -a2 "cols2" counterparts) can be used to specify columns to activate (i.e. write to the output file) for each input file. By default, all columns are output.
If two or more columns from separate files have the same name, the second (and subsequent) columns are renamed to have an underscore and a numeric value appended.
The -m min and -M max switches specify the minimum and maximum number of joins required to write out a row. The default minimum is 0 joins (i.e. all rows are written out) and the default maximum is 63 (the maximum number of possible joins with a limit of 32 input files). For example, to write out only those rows in which exactly two files have columns that match (i.e. one join):
funjoin -j key -m 1 -M 1 in1.fits in2.fits in3.fits ... out.fits
A given row can have the requisite number of joins without all of the files being joined (e.g. three files are being joined but only two have a given join key value). In this case, all of the columns of the non-joined file are written out, by default, using blanks (zeros or NULLs). The -b c1:bv1,c2:bv2 and -b1 'c1:bv1,c2:bv2' -b2 'c1:bv1,c2:bv2' ... switches can be used to set the blank value for columns common to all files and/or columns in a specified file, respectively. Each blank value string contains a comma-separated list of column:blank_val specifiers. For floating point values (single or double), a case-insensitive string value of "nan" means that the IEEE NaN (not-a-number) should be used. Thus, for example:
funjoin -b "AKEY:???" -b1 "A:-1" -b3 "G:NaN,E:-1,F:-100" ...means that a non-joined AKEY column in any file will contain the string "???", the non-joined A column of file 1 will contain a value of -1, the non-joined G column of file 3 will contain IEEE NaNs, while the non-joined E and F columns of the same file will contain values -1 and -100, respectively. Of course, where common and specific blank values are specified for the same column, the specific blank value is used.
To distinguish which files are non-blank components of a given row, the -s (status) switch can be used to add a bitmask column named "JFILES" to the output file. In this column, a bit is set for each non-blank file composing the given row, with bit 0 corresponds to the first file, bit 1 to the second file, and so on. The file names themselves are stored in the FITS header as parameters named JFILE1, JFILE2, etc. The -S col switch allows you to change the name of the status column from the default "JFILES".
A join between rows is the Cartesian product of all rows in one file having a given join column value with all rows in a second file having the same value for its join column and so on. Thus, if file1 has 2 rows with join column value 100, file2 has 3 rows with the same value, and file3 has 4 rows, then the join results in 2*3*4=24 rows being output.
The join algorithm directly processes the index file associated with the join column of each file. The smallest value of all the current columns is selected as a base, and this value is used to join equal-valued columns in the other files. In this way, the index files are traversed exactly once.
The -t tol switch specifies a tolerance value for numeric columns. At present, a tolerance value can join only two files at a time. (A completely different algorithm is required to join more than two files using a tolerance, somethng we might consider implementing in the future.)
The following example shows many of the features of funjoin. The input files t1.fits, t2.fits, and t3.fits contain the following columns:
[sh] fundisp t1.fits
AKEY KEY A B
----------- ------ ------ ------
aaa 0 0 1
bbb 1 3 4
ccc 2 6 7
ddd 3 9 10
eee 4 12 13
fff 5 15 16
ggg 6 18 19
hhh 7 21 22
fundisp t2.fits
AKEY KEY C D
----------- ------ ------ ------
iii 8 24 25
ggg 6 18 19
eee 4 12 13
ccc 2 6 7
aaa 0 0 1
fundisp t3.fits
AKEY KEY E F G
------------ ------ -------- -------- -----------
ggg 6 18 19 100.10
jjj 9 27 28 200.20
aaa 0 0 1 300.30
ddd 3 9 10 400.40
Given these input files, the following funjoin command:
funjoin -s -a1 "-B" -a2 "-D" -a3 "-E" -b \ "AKEY:???" -b1 "AKEY:XXX,A:255" -b3 "G:NaN,E:-1,F:-100" \ -j key t1.fits t2.fits t3.fits foo.fitswill join the files on the KEY column, outputting all columns except B (in t1.fits), D (in t2.fits) and E (in t3.fits), and setting blank values for AKEY (globally, but overridden for t1.fits) and A (in file 1) and G, E, and F (in file 3). A JFILES column will be output to flag which files were used in each row:
AKEY KEY A AKEY_2 KEY_2 C AKEY_3 KEY_3 F G JFILES
------------ ------ ------ ------------ ------ ------ ------------ ------ -------- ----------- --------
aaa 0 0 aaa 0 0 aaa 0 1 300.30 7
bbb 1 3 ??? 0 0 ??? 0 -100 nan 1
ccc 2 6 ccc 2 6 ??? 0 -100 nan 3
ddd 3 9 ??? 0 0 ddd 3 10 400.40 5
eee 4 12 eee 4 12 ??? 0 -100 nan 3
fff 5 15 ??? 0 0 ??? 0 -100 nan 1
ggg 6 18 ggg 6 18 ggg 6 19 100.10 7
hhh 7 21 ??? 0 0 ??? 0 -100 nan 1
XXX 0 255 iii 8 24 ??? 0 -100 nan 2
XXX 0 255 ??? 0 0 jjj 9 28 200.20 4
funmerge [-w|-x] -f [colname] <iname1> <iname2> ... <oname>
-f # output a column specifying file from which this event came -w # adjust position values using WCS info -x # adjust position values using WCS info and save old values
funmerge merges FITS data from one or more FITS Binary Table files or raw event files.
The first argument to the program specifies the first input FITS table or raw event file. If "stdin" is specified, data are read from the standard input. Use Funtools Bracket Notation to specify FITS extensions and row filters. Subsequent arguments specify additional event files and tables to merge. (NB: Stdin cannot not be used for any of these additional input file arguments.) The last argument is the output FITS file. The columns in each input table must be identical.
If an input file begins with the '@' character, it is processed as an include file, i.e., as a text file containing event file names (as well as blank lines and/or comment lines starting with the '#' sign). If standard input is specified as an include file ('@stdin'), then file names are read from the standard input until EOF (^D). Event files and include files can be mixed on a command line.
Rows from each table are written sequentially to the output file. If the switch -f [colname] is specified on the command line, an additional column is added to each row containing the number of the file from which that row was taken (starting from one). In this case, the corresponding file names are stored in the header parameters having the prefix FUNFIL, i.e., FUNFIL01, FUNFIL02, etc.
Using the -w switch (or -x switch as described below), funmerge also can adjust the position column values using the WCS information in each file. (By position columns, we mean the columns that the table is binned on, i.e., those columns defined by the bincols= switch, or (X,Y) by default.) To perform WCS alignment, the WCS of the first file is taken as the base WCS. Each position in subsequent files is adjusted by first converting it to the sky coordinate in its own WCS coordinate system, then by converting this sky position to the sky position of the base WCS, and finally converting back to a pixel position in the base system. Note that in order to perform WCS alignment, the appropriate WCS and TLMIN/TLMAX keywords must already exist in each FITS file.
When performing WCS alignment, you can save the original positions in the output file by using the -x (for "xtra") switch instead of the -w switch (i.e., using this switch also implies using -w) The old positions are saved in columns having the same name as the original positional columns, with the added prefix "OLD_".
Examples:
Merge two tables, and preserve the originating file number for each row in the column called "FILE" (along with the corresponding file name in the header):
[sh] funmerge -f "FILE" test.ev test2.ev merge.ev
Merge two tables with WCS alignment, saving the old position values in 2 additional columns:
[sh] funmerge -x test.ev test2.ev merge.ev
This program only works on raw event files and binary tables. We have not yet implemented image and array merging.
funsky iname[ext] # RA,Dec (deg) or image pix from stdin funsky iname[ext] [lname] # RA, Dec (deg) or image pix from list funsky iname[ext] [col1] [col2] # named cols:units from stdin funsky iname[ext] [lname] [col1] [col2] # named cols:units from list
-d # always use integer tlmin conversion (as ds9 does) -r # convert x,y to RA,Dec (default: convert RA,Dec to x,y) -o # include offset from the nominal target position (in arcsec) -v # display input values also (default: display output only) -T # output display in rdb format (w/header,tab delimiters)
Funsky converts input sky coordinates (RA, Dec) to image coordinates (or vice versa) using the WCS information contained in the specified FITS file. Several calling sequences are supported in order to make it easy to specify coordinate positions in different ways.
The first required argument is always the input FITS file (or extension) containing the WCS information in an extension header. Note that the data from this file is not used. By default, the program converts input RA and Dec values to X and Y using this WCS information. If the WCS is associated with a FITS image, then the X,Y values are image values. If the WCS is associated with a binary table, then the X, Y values are physical values. To convert X,Y to RA and Dec, use the -r (reverse) switch.
If no other command arguments are supplied, then the input positions are read from the standard input. Each line is assumed to contain a single coordinate position consisting of an RA in hours (or X in pixels) followed by a Dec in degrees (or Y in pixels). The usual delimiters are supported (spaces, commas, tabs). For example:
# read from stdin, default column names and units
[sh] funsky snr.ev
22.982695 58.606523 # input RA (hrs), Dec(deg)
510.00 510.00
22.982127 58.607634 # input
512.00 510.50
22.981700 58.614301 # input
513.50 513.50
^D # end of input
If a second argument is supplied, this argument is assumed to be a file containing RA (X) and Dec (Y) positions. The file can either be an ASCII table or a FITS binary table. The order of columns is unimportant, if the table has a column header. In this case, the names of the columns must be one of "RA", "DEC", or "X", "Y" for sky to image and image to sky conversions, respectively. If the table has no header, then once again, RA (X) is assumed to first, followed by DEC (Y). For example:
# read from file, default column names and units
[sh] cat hd.in
RA DEC
--------- ---------
22.982695 58.606523
22.982127 58.607634
22.981700 58.614301
[sh] funsky snr.ev hd.in
510.00 510.00
512.00 510.50
513.50 513.50
If three arguments are supplied, then the input positions again are read from the standard input. Each line is assumed to contain a single coordinate position consisting of an RA (or X in pixels) followed by a Dec (or Y in pixels), with the usual delimiters supported. However, the second and third arguments now specify the column names and/or sky units using a colon-delimited syntax:
[colname]:[h|d|r]If the colname is omitted, the names default to "RA", "DEC", "X", "Y", "COL1", or "COL2" as above. If the units are omitted, the default is hours for RA and degrees for Dec. When the -r switch is used (convert from image to sky) the units are applied to the output instead of the input. The following examples will serve to illustrate the options:
# read from stdin, specifying column names (def. units: hours, degrees)
[sh] cat hd.in
MYRA MYDEC
--------- ---------
22.982695 58.606523
22.982127 58.607634
22.981700 58.614301
[sh] funsky snr.ev MYRA MYDEC < hd.in
510.00 510.00
512.00 510.50
513.50 513.50
# read from stdin, specifying column names and units
[sh] cat dd.in
MYRA MYDEC
--------- ---------
344.740432 58.606523
344.731900 58.607634
344.725500 58.614301
[sh] funsky snr.ev MYRA:d MYDEC:d < dd.in
510.00 510.00
512.00 510.50
513.50 513.50
# read stdin, convert image to sky, specifying output sky units
[sh] cat im.in
510.00 510.00
512.00 510.50
513.50 513.50
[sh] cat im.in | funsky -r snr.ev :d :d
344.740432 58.606523
344.731900 58.607634
344.725500 58.614301
Finally, four command arguments specify both and input file and column names and/or units:
[sh] cat dd.in
MYRA MYDEC
--------- ---------
344.740432 58.606523
344.731900 58.607634
344.725500 58.614301
[sh] funsky snr.ev dd.in MYRA:d MYDEC:d
510.00 510.00
512.00 510.50
513.50 513.50
# read file, convert image to sky, specifying output sky units
[sh] cat im.in
510.00 510.00
512.00 510.50
513.50 513.50
[sh] funsky -r snr.ev im.in :d :d
344.740432 58.606523
344.731900 58.607634
344.725500 58.614301
By default, the output of funsky consists only of the converted coordinate position(s), one per output line. This makes parsing in shell scripts easy. Use the -v (verbose) switch to specify that the input coordinates should be pre-pended to each line. For example:
[sh] cat dd.in
MYRA MYDEC
--------- ---------
344.740432 58.606523
344.731900 58.607634
344.725500 58.614301
[sh] funsky snr.ev dd.in MYRA:d MYDEC:d
510.00 510.00
512.00 510.50
513.50 513.50
[sh] funsky -v snr.ev dd.in MYRA:d MYDEC:d
344.740432 58.606523 510.00 510.00
344.731900 58.607634 512.00 510.50
344.725500 58.614301 513.50 513.50
In addition, a full starbase table can be output using the -T (table) switch. This switch can be used with or without the -v switch. If the -T and -v are both specified, then a descriptive header parameters are output before the table (mainly to remind you of the sky units):
# output table in non-verbose mode
[sh] funsky -T snr.ev dd.in MYRA:d MYDEC:d
X Y
------------ ------------
510.00 510.00
512.00 510.50
513.50 513.50
# output table in verbose mode
[sh] funsky -T -v snr.ev dd.in MYRA:d MYDEC:d
# IFILE = /Users/eric/data/snr.ev
# ICOL1 = MYRA
# ICOL2 = MYDEC
# IUNITS1 = d
# IUNITS2 = d
# OCOL1 = X
# OCOL2 = Y
MYRA MYDEC X Y
------------ ------------ ------------ ------------
344.740432 58.606523 510.00 510.00
344.731900 58.607634 512.00 510.50
344.725500 58.614301 513.50 513.50
Finally, the -d (ds9) switch mimicks ds9's use of integer TLMIN and TLMAX values for all coordinate transformations. FITS conventions seem to call for use of floating point TLMIN and TLMAX when the data are floats. This convention is followed by funsky but results in a small discrepancy with ds9's converted values for floating point data. We will remedy this conflict in the future, maybe.
funtable [-a] [-i|-z] [-m] [-s cols] <iname> <oname> [columns]
-a # append to existing output file as a table extension -i # for image data, only generate X and Y columns -m # for tables, write a separate file for each region -s "col1 ..." # columns on which to sort -z # for image data, output zero-valued pixels
funtable selects rows from the specified FITS Extension (binary table only) of a FITS file, or from a non-FITS raw event file, and writes those rows to a FITS binary table file. It also will create a FITS binary table from an image or a raw array file.
The first argument to the program specifies the FITS file, raw event file, or raw array file. If "stdin" is specified, data are read from the standard input. Use Funtools Bracket Notation to specify FITS extensions, and filters. The second argument is the output FITS file. If "stdout" is specified, the FITS binary table is written to the standard output. By default, all columns of the input file are copied to the output file. Selected columns can be output using an optional third argument in the form:
"column1 column1 ... columnN"
The funtable program generally is used to select rows from a FITS binary table using Table Filters and/or Spatial Region Filters. For example, you can copy only selected rows (and output only selected columns) by executing in a command such as:
[sh] funtable "test.ev[pha==1&&pi==10]" stdout "x y pi pha" | fundisp stdin
X Y PHA PI
------- ------- ------- ---------
1 10 1 10
1 10 1 10
1 10 1 10
1 10 1 10
1 10 1 10
1 10 1 10
1 10 1 10
1 10 1 10
1 10 1 10
1 10 1 10
The special column $REGION can be specified to write the region id of each row:
[sh $] funtable "test.ev[time-(int)time>=.99&&annulus(0 0 0 10 n=3)]" stdout 'x y time $REGION' | fundisp stdin
X Y TIME REGION
-------- -------- --------------------- ----------
5 -6 40.99000000 3
4 -5 59.99000000 2
-1 0 154.99000000 1
-2 1 168.99000000 1
-3 2 183.99000000 2
-4 3 199.99000000 2
-5 4 216.99000000 2
-6 5 234.99000000 3
-7 6 253.99000000 3
Here only rows with the proper fractional time and whose position also is within one of the three annuli are written.
Columns can be excluded from display using a minus sign before the column:
[sh $] funtable "test.ev[time-(int)time>=.99]" stdout "-time" | fundisp stdin
X Y PHA PI DX DY
-------- -------- -------- ---------- ----------- -----------
5 -6 5 -6 5.50 -6.50
4 -5 4 -5 4.50 -5.50
-1 0 -1 0 -1.50 0.50
-2 1 -2 1 -2.50 1.50
-3 2 -3 2 -3.50 2.50
-4 3 -4 3 -4.50 3.50
-5 4 -5 4 -5.50 4.50
-6 5 -6 5 -6.50 5.50
-7 6 -7 6 -7.50 6.50
All columns except the time column are written.
In general, the rules for activating and de-activating columns are:
Ordinarily, only the selected table is copied to the output file. In a FITS binary table, it sometimes is desirable to copy all of the other FITS extensions to the output file as well. This can be done by appending a '+' sign to the name of the extension in the input file name. For example, the first command below copies only the EVENT table, while the second command copies other extensions as well:
[sh] funtable "/proj/rd/data/snr.ev[EVENTS]" events.ev [sh] funtable "/proj/rd/data/snr.ev[EVENTS+]" eventsandmore.ev
If the input file is an image or a raw array file, then funtable will generate a FITS binary table from the pixel values in the image. Note that it is not possible to specify the columns to output (using command-line argument 3). Instead, there are two ways to create such a binary table from an image. By default, a 3-column table is generated, where the columns are "X", "Y", and "VALUE". For each pixel in the image, a single row (event) is generated with the "X" and "Y" columns assigned the dim1 and dim2 values of the image pixel, respectively and the "VALUE" column assigned the value of the pixel. With sort of table, running funhist on the "VALUE" column will give the same results as running funhist on the original image.
If the -i ("individual" rows) switch is specified, then only the "X" and "Y" columns are generated. In this case, each positive pixel value in the image generates n rows (events), where n is equal to the integerized value of that pixel (plus 0.5, for floating point data). In effect, -i approximately recreates the rows of a table that would have been binned into the input image. (Of course, this is only approximately correct, since the resulting x,y positions are integerized.)
If the -s [col1 col2 ... coln] ("sort") switch is specified, the output rows of a binary table will be sorted using the specified columns as sort keys. The sort keys must be scalar columns and also must be part of the output file (i.e. you cannot sort on a column but not include it in the output). This facility uses the _sort program (included with funtools), which must be accessible via your path.
For binary tables, the -m ("multiple files") switch will generate a separate file for each region in the filter specification i.e. each file contains only the rows from that region. Rows which pass the filter but are not in any region also are put in a separate file.
The separate output file names generated by the -m switch are produced automatically from the root output file to contain the region id of the associated region. (Note that region ids start at 1, so that the file name associated with id 0 contains rows that pass the filter but are not in any given region.) Output file names are generated as follows:
funtable -m input.fits'[cir(512,512,1);cir(520,520,1)...]' 'foo.goo_$n.fits'will generate files named foo.goo_0.fits (for rows not in any region but still passing the filter), foo.goo_1.fits (rows in region id #1, the first region), foo.goo_2.fits (rows in region id #2), etc. Note that single quotes in the output root are required to protect the '$' from the shell.
funtable -m input.fits'[cir(512,512,1);cir(520,520,1)...]' foo.evt.fitswill generate files named foo0.evt.fits (for rows not in any region but still passing the filter), foo1.evt.fits (rows in region id #1), foo2.evt.fits (rows in region id #2), etc.
funtable -m input.fits'[cir(512,512,1);cir(520,520,1)...]' 'foo_evt'will generate files named foo_evt0 (for rows not in any region but still passing the filter), foo_evt1 (rows in region id #1), foo_evt2 (rows in region id #2), etc.
By default, a new FITS file is created and the binary table is written to the first extension. If the -a (append) switch is specified, the table is appended to an existing FITS file as a BINTABLE extension. Note that the output FITS file must already exist.
If the -z ("zero" pixel values) switch is specified and -i is not specified, then pixels having a zero value will be output with their "VALUE" column set to zero. Obviously, this switch does not make sense when individual events are output.
funtable [-c cols] [-h] [-n table] [-p prog] [-s sep] <iname>
[NB: This program has been deprecated in favor of the ASCII text processing support in funtools. You can now perform fundisp on funtools ASCII output files (specifying the table using bracket notation) to extract tables and columns.] The funtbl script extracts a specified table (without the header and comments) from a funtools ASCII output file and writes the result to the standard output. The first non-switch argument is the ASCII input file name (i.e. the saved output from funcnts, fundisp, funhist, etc.). If no filename is specified, stdin is read. The -n switch specifies which table (starting from 1) to extract. The default is to extract the first table. The -c switch is a space-delimited list of column numbers to output, e.g. -c "1 3 5" will extract the first three odd-numbered columns. The default is to extract all columns. The -s switch specifies the separator string to put between columns. The default is a single space. The -h switch specifies that column names should be added in a header line before the data is output. Without the switch, no header is prepended. The -p program switch allows you to specify an awk-like program to run instead of the default (which is host-specific and is determined at build time). The -T switch will output the data in rdb format (i.e., with a 2-row header of column names and dashes, and with data columns separated by tabs). The -help switch will print out a message describing program usage.
For example, consider the output from the following funcnts command:
[sh] funcnts -sr snr.ev "ann 512 512 0 9 n=3"
# source
# data file: /proj/rd/data/snr.ev
# arcsec/pixel: 8
# background
# constant value: 0.000000
# column units
# area: arcsec**2
# surf_bri: cnts/arcsec**2
# surf_err: cnts/arcsec**2
# summed background-subtracted results
upto net_counts error background berror area surf_bri surf_err
---- ------------ --------- ------------ --------- --------- --------- ---------
1 147.000 12.124 0.000 0.000 1600.00 0.092 0.008
2 625.000 25.000 0.000 0.000 6976.00 0.090 0.004
3 1442.000 37.974 0.000 0.000 15936.00 0.090 0.002
# background-subtracted results
reg net_counts error background berror area surf_bri surf_err
---- ------------ --------- ------------ --------- --------- --------- ---------
1 147.000 12.124 0.000 0.000 1600.00 0.092 0.008
2 478.000 21.863 0.000 0.000 5376.00 0.089 0.004
3 817.000 28.583 0.000 0.000 8960.00 0.091 0.003
# the following source and background components were used:
source_region(s)
----------------
ann 512 512 0 9 n=3
reg counts pixels sumcnts sumpix
---- ------------ --------- ------------ ---------
1 147.000 25 147.000 25
2 478.000 84 625.000 109
3 817.000 140 1442.000 249
There are four tables in this output. To extract the last one, you can execute:
[sh] funcnts -s snr.ev "ann 512 512 0 9 n=3" | funtbl -n 4 1 147.000 25 147.000 25 2 478.000 84 625.000 109 3 817.000 140 1442.000 249Note that the output has been re-formatted so that only a single space separates each column, with no extraneous header or comment information.
To extract only columns 1,2, and 4 from the last example (but with a header prepended and tabs between columns), you can execute:
[sh] funcnts -s snr.ev "ann 512 512 0 9 n=3" | funtbl -c "1 2 4" -h -n 4 -s "\t" #reg counts sumcnts 1 147.000 147.000 2 478.000 625.000 3 817.000 1442.000
Of course, if the output has previously been saved in a file named foo.out, the same result can be obtained by executing:
[sh] funtbl -c "1 2 4" -h -n 4 -s "\t" foo.out #reg counts sumcnts 1 147.000 147.000 2 478.000 625.000 3 817.000 1442.000