STScI_Python Version 2.12 Release Notes August 2011 This release includes new versions of PyRAF, PyFITS, STWCS, pytools, Multidrizzle, and related tasks. This release also includes instrument-specific packages to support ACS, NICMOS, and COS. The convolve, image, and ndimage packages have been included with this release to insure compatibility with the rest of our code. Platform Support ================ Normally, platform support issues do not apply to Python tasks, as most Python code will run on all platforms on which Python has been installed. This distribution was tested to correctly support installation on Linux and Mac OS X 10.6 (Snow Leopard), while an installation for Windows will be provided at a later date. As of the June 2010 release, PyRAF no longer requires the installation of IRAF. Note that the only IRAF distribution for Windows runs under Cygwin, but no testing of PyRAF has been conducted under Cygwin. No Solaris Support ------------------ This release has NOT been tested on the Solaris platform. We no longer build Solaris binaries for STSDAS, although the source code can always be downloaded and compiled locally as needed. Documentation ============= Documentation for these tasks has been consolidated into a single location complete with an index viewable as a local web page. The documentation includes any available user guides and API documentation generated using 'EPYDOC' for the modules or packages. This index can be accessed using: --> import stscidocs --> stscidocs.viewdocs() --> stscidocs.help() This will automatically bring up the default web browser application to display the index page. All the available documentation for software packaged with this distribution can be accessed through this interface, as this documentation resides locally upon installation. Python Environment ================== This release has been tested using Python Versions 2.5.4 and 2.7. This release also requires at least numpy 1.3. All packages within this release have been revised to support easy_install of the code, while still allowing the code to be installed using the disutils command 'python setup.py'. Namespace Changes ================== The organization of the stsci_python package has been significantly changed to move many packages into the 'stsci' namespace and remove any package name confusion (such as 'pytools' which is used as a name by other Python software). The namespace changes require dependent packages to update to point to the new names of the packages based on this translation table: old name ---> new name ---------- ------------- convolve stsci.convolve image stsci.image imagemanip stsci.imagemanip imagestats stsci.imagestats numdisplay stsci.numdisplay pytools stsci.tools stimage stsci.stimage ndimage stsci.ndimage The 'ndimage', 'image', and 'convolve' packages included in our release are forked from SciPy in order to support our software needs without requiring all of SciPy to be installed separately. PyFITS Version 3.0.0 ==================== This release now includes and relies on PyFITS 3.0.0, a version which contains changes to interfaces that are incompatible with previous versions of PyFITS. These changes were implemented to allow PyFITS to be translated to work with Python 3.0 while refactoring the code to make it easier to maintain. The PyFITS Release Notes (and file 'CHANGES.txt' included in the release) contain detailed descriptions of all the changes made to PyFITS 3.0.0. The code in this release has been modified to be compatible with both PyFITS 3.0.0 and earlier versions of PyFITS, however, this compatibility will be removed at some point in the future. This release addresses an extensive list of bug fixes and improvements, including: - Calls to deprecated functions will display a Deprecation warning. However, in Python 2.7 and up Deprecation warnings are ignored by default, so run Python with the `-Wd` option to see if you're using any deprecated functions. If we get close to actually removing any functions, we might make the Deprecation warnings display by default. - Added basic Python 3 support - Added support for multi-dimensional columns in tables as specified by the TDIMn keywords (#47) - Fixed a major memory leak that occurred when creating new tables with the `new_table()` function (where strings are padded with zero-bytes) vs ASCII tables (where strings are padded with spaces) - Fixed a bug in which the case of Random Access Group parameters names was not preserved when writing - Added support for binary table fields with zero width - Added support for wider integer types in ASCII tables; although this is non- standard, some GEIS images require it - Fixed a bug that caused the index_of() method of HDULists to crash when the HDUList object is created from scratch - Fixed the behavior of string padding in binary tables (where strings should be padded with nulls instead of spaces) - Fixed a rare issue that caused excessive memory usage when computing checksums using a non-standard block size - Add support for forced uint data in image sections - Fixed tuple and list-based indexing of FITS_rec objects - Fixed an issue where BZERO and BSCALE keywords were appended to headers in the wrong location - `FITS_record` objects (table rows) have full slicing support, including stepping, etc. - Fixed a bug where updating multiple files simultaneously (such as when running parallel processes) could lead to a race condition with mktemp() - Fixed a bug where compressed image headers were not in the order expected by the funpack utility stsci.tools (formerly: pytools) ============ This package provides utilities which support multiple packages due to the more general nature. Updates include: - New modules were added to work with GEIS (IRAF) images: specifically, the 'swapgeis' module to byteswap a GEIS image and the 'convertgeis' to replace the 'stwfits' task in IRAF to write out a waivered-FITS image from a GEIS image. - Removed the use of 'numerixenv' so that code will now only support numpy. - Updated the 'buildRootname()' function in 'tools.fileutil' to retain and properly handle any paths provided for the input filename - There were many improvements to the TEAL parameter interface: - TEAL now handles .cfgspc changes/upgrades with a notice stating which parameters are "problems" in the changed version (#728, r11322, PyRAF ticket #127). - Changes were made to the TEAL code to prepare for the port to Python 3 (#584), although Python 2.x will be supported for quite some time. TEAL does now run under Python 3 (via 2to3), but this has not yet been heavily tested (expected next release). - TEAL is now smarter about showing task help in HTML where possible, and it also has hooks to jump to parameter-specific help topics (#614, r12000, r12863) - Some GUI items were fixed: font colors, Save-As dialog, a FileDialog which changes directory when it shouldn't (#727, r12344, r12462, r12995). - Parameter mismatch errors are now more easy to read/understand. - TEAL supports two new trigger keywords: "set_yes_if" and "set_no_if". Set the env. variable TEAL_DEBUG to have TEAL print out trigger logic (#725). calcos ============ Version 2.14.4 was previously released as a patch in April. This release includes CALCOS Version 2.15.4 with updates from 2.14.4 that include: - A 'walk' correction was added to CALCOS to correct the pixel coordinates of events in an FUV TIME-TAG exposure for shifts which are dependent on the pulse height amplitude (PHA). - Use exposure appropriate for each FUV data segment in the x1d correction. - A bug in background smoothing during spectral extraction has been fixed. This mostly affects values near the endpoints of the spectrum. In this release, there is an unexplained problem on Windows; CALCOS cannot be imported. costools ============ A new package containing COS-specific analysis tasks has been added to this release. Currently, the only task included in this new package is the 'timefilter' task. This program reads a COS corrtag file and sets flags in the DQ column of the EVENTS extension depending on the filter that was specified. imagestats (now: stsci.imagestats) ============ The imagestats package was updated to take into account any user-specified limits when clipping has been requested so that it behaves more like IRAF's imagestats task. numdisplay (now: stsci.numdisplay) ============ Numdisplay version 1.6.1 corrects a problem with compatibility with numpy versions > 2.0. This version has been tested to work under Python 2.7 using all the latest versions of numpy. pydrizzle ============ Only minor changes were made to allow this version to work with PyFITS 3.0.0. PyRAF v1.11 ============ This release supports Python 2.5, 2.6, and 2.7. Since the 1.10 bundled release, the following enhancements have been made: - PyRAF is being prepared for the port to Python 3. Many code changes were made, though there should be no behavior changes noticeable to the user (#137, #154). - PyRAF was modified to maintain compatibility with IRAF 2.15.*, both 32 and 64-bit (mostly due to graphics WCS changes). Also, better error messaging was implemented within PyRAF's core in order to debug related problems, often seen during startup. See PyRAF FAQ #1.5 (#156 and r1324). - PyRAF is now more robust in the face of outdated or changing CL scripts, e.g. login.cl. The ErrorTracker class, used internally, is now keeping better track of offensive CL script code for reporting to the user. - The 'unlearn' command is now available for TEAL-enabled tasks (r1412,3). TEAL also now automatically switches focus like EPAR (#159). - PyRAF on Windows now has a desktop icon for launching the command-line interface (#130). and the following bugs have been fixed: - EPAR for TEAL-enabled tasks (.cfg file) now more elegantly handles changes to that task's .cfgspc definition: no traceback, simpler error message or, in some cases, even a dialog (#127). Validation errors are also handled more gracefully now (#158). - It was noticed that text-based help in the terminal would not run without graphics capabilities, even though it should not require them (#147). - On OSX, Python itself will no longer crash in a remote session if aqutil fails to load (#149). - A bug in the Page menu (graphics windows) was fixed for the matplotlib graphics kernel (#150). Also fixed in this graphics kernel was redraws not occurring with 't' during implot (r1387). - CL script parsing is now catching cases where a "#" is found in the right hand side of a set/reset statement (#155). - Integer division was not behaving correctly in parsed CL code. See PyRAF FAQ #6.6 for a full discussion of the expected behavior (#157). - PyRAF was throwing a traceback during a warning about certain CL system/shell commands (#162). - PyRAF on Windows no longer requires the use of the HOME environment variable (#130). Python 3 Note: It is anticipated that the next full release of PyRAF (late 2011/early 2012) will have basic capabilities working under Python 3. Python 2.x however will be supported for quite a while. Many users will be cautious about converting their scripts and processing. pysynphot ============ The following changes have been made since the 0.8.2 release: Bugfixes --------- - Calculation of the pivot wavelength is now correctly implemented as a method (.pivot) on Bandpass objects. - The GaussianSource, Powerlaw, FlatSpectrum, and BlackBody classes have been fixed so that they are consistent after unit changes using the convert() method. - Fixed multiple bugs in the InterpolatedSpectralElement class so that it correctly handles tables with different column designs and provides correctly interpolated results for all table input types. (Previously, interpolation was implemented correctly only for ramp filters.) - Handling of MergedWavesets was corrected to strictly eliminate duplicate entries, which previously sometimes occurred due to numerical rounding during merge operations. Performance ----------- - Calculating binned flux for Observation objects has been moved to an optional C extension to improve speed. If the C extension is not available at run time, the calculations are performed in Python. - Objects created when opening and reading tables are cached to improve speed in situations where a table previously would have been opened and closed many times within a program. Other changes ------------- - Custom pysynphot exceptions have been implemented for some conditions that previously raised ValueErrors. Additional custom exceptions will be implemented in the next release. - The CompTable and GraphTable classes have been moved to a new pysynphot module called tables. - The etc module has been deprecated. The etc.parse_spec function has been relocated to spparser.parse_spec. stwcs ============ The package has been updated with these changes: - added logging to STWCS - added support for a WCSCORR table which summarizes all updates made to the WCS information of each SCI extension - The HSTWCS method 'wcs2header' was modified to correctly append the SIP and IDC distortion model keywords to the header - The logic for using the 'readModel()' method of HSTWCS was also improved to not try and read the IDCModel when no 'idctab' had been specified in the header. - eliminated the 1 pixel offset introduced when creating an output WCS with an input WCS which has no distortion model. - makes recognition of non-HST data (slightly) more general so that it will work on data created with IRAF's artdata package - added 'all_sky2pix()' method to HSTWCS objects which implements an iterative solution using the full distortion model to convert sky positions into pixel positions - Added a tool to create a virtual mosaic based on the WCS of the input observations, with the option to provide an output tangent plane for the virtual mosaic - Added a method to update the scale of a WCS object: updatePscale()