SIGPROC

PSRFITS2FIL

• psrfits2fil converts AO psrfits files from the pdev spectrometer to sigproc filterbank program. This standalone routine will probably one day be incorporated in the sigproc filterbank program.
• psrfits data taken with the mock pdev spectrometer is scan based. During a single scan one or more files are written to disk until the end of the scan. Sequence numbers will increment by 1 for  each file in a scan. The sequence number for the first file of a scan is a multiple of 100. 
• Conversion of psrfits->sigproc.fil format is scan based. The routine will read all of the psrfits files from a scan and output a single sigprog.fil file.
• Calling psrfits2file:
• The options/filenames have been switched to normal unix format. The options come first preceded by -   and then followed by the input filename:
  

Output data:

• by default output is written to a file in the current directory. The output filename is the same name as the input file after replacing the trailing .fits with .fil:
• If current directory is /share/psrdat
• inpfile=/share/pdata1/pdev/x101.20090113.b0s0g0.00500.fits
• outfile will be: /share/psrdat/x101.20090113.b0s0g0.00500.fil
• The -filterout option will cause the data to be written to stdout.
  
• All of the files of the input scan is written to a single output file.
• By default the sigproc header is written to the start of the .fil file.
• -header will write the sigproc header to the file ./head in ascii. The output data will not include the header (it will just have data).
• By default the datatype of the output data matches that of the input data (normally 2 byte shorts). the --floats will force the output data to be floats.

Examples:

• Output to a single file with same format as input:
• psrfits2fil  /share/pdata1/pdev/x101.20090113.b0s0g0.00500.fits
• Output to a single file as floats. Leave the header in the output file:
• psrfits2fil  -floats  /share/pdata1/pdev/x101.20090113.b0s0g0.00500.fits
• Send data output thru a filter. Write the header to disc:
• psrfits2fil -filterout -floats -headerfile\   /share/pdata1/pdev/x101.20090113.b0s0g0.00500.fits | rest of filter

Disc locations:

• Program\
• executable: /share/megs/phil/svn/pdev/pdev/psrfits/sigproc/psrfits2fil.
  
• It is currently compiled for 32 bits but also runs on galfas1.
• Source: /share/megs/phil/svn/pdev/pdev/psrfits/sigproc/psrfits2fil.c
• Compiling:
• Look at makefile . Need to set INCFITS=, LFITSD=..
  
• need to make sure that the cfitsio lib you link in is the same architecture (32/64) as the executable you are creating.
  
• Data files:
  
• see /share/phil/sigproc. Look at the two Readme for what data was taken.
• The psrfits files for the test data  are located in /share/pdata1/pdev/
• Look at /share/megs/phil/x101/090113/makefil.sc for a script that converted the 090113 files.
  

Notes:

• If the input files (.fits) are > 2gb then I've seen the the cfitsio open routine abort. The problem occurs if you run nfs version 2 on linux. The open tries to position to the end of the file  and it looks like the nfs call can't handle a int > 2gb (signed 32 bits). nfs version 3 doesn't seem to have the problem. The problem also doesn't occur if you read the file locally (no nfs read).
• cfitsio can complain of different versions. The cfitsiolib is a shared object. The cfitsio code wants to make sure that the library version on your system matches the version used when  psrfits2fil was compiled (CFITSIO_VERSION 3.03 as of 27jan09).
  

Changes from duncs psrfits2fil used at gbt:

• got source from dunc for gbt version psrfits -> .fil
• updates to work with pdev spectrometer psrfits format
• differences in psrfits format
• added 2 bintables after the primary header and before the subint table
• extname: AOGEN  .. generic ao if/lo setup info
• extname: PDEV   .. pdev spectrometer config info
• These bintables have keyword values but no rows..
• extname: SUBINT  the psrfits data table. I added a STAT column (after the data column). This holds the stat info for each spectra: seqNum, numAccums, blanking info.
• program differences:
• psrfits.h
• added struct: SUBINT_COL. It maps column names to column number of subint bintable
•  few other variable additions
• read_psrfits.c:
• psrfits_open
• parameterized number of digits in filename seq nun.
•  grab info out of PDEV bintable needed for blanking.
• keep track of bytes in statblock each row, fix fold mode to use nbits to compute databytes in row.
• on first call map colnames to column numbers (needed for read).
• only malloc the data buffers on the first open.. No sense doing it every row since the size can't change...
• psrfits_read_subint
• define args for cfitsio that are longlong to be longlong.
• change readcol routines to use the colNums we got in open
• changed the read of the subint header info to only occur on the first read. Sigproc isn't using any of this info that changes...
• switched to read byte,short,int,float,doubles. original
           version only read bytes..
• psrfits2fil.c
• options now come before filenames (normal unix convention).
• flip is now an option -flip
•  fcentMhz is now an option: -fcent  cfrFrqMhz
• added options: filterout,floats,headerfile,odir,help (see list above).
• by default the output filename is the same as the input filename
• change filename parsing to just grab the seqnumber (parse from the end backwards).. don't bother with all the other fields in the name.
• if -header write ascii file. Make it look like the output of the header command.
•  Got rid of -o.
• process 1 row at a time.
• key off of the datatype to determine what the input/output data types are (and -floatOut).
• use a macro to work with all the different datatypes.
• include blanking in the row processing. If blanked then scale so that the mean of all spectra are the same. If ints then scale and then round.
• use header scanlen to only output good data
• free the alloced data once at the end

 home_~phil