Документ взят из кэша поисковой машины. Адрес
оригинального документа
: http://www.eso.org/~qc/tqs/pyqc/python_2.6.2.html
Дата изменения: Fri Dec 18 13:22:23 2009 Дата индексирования: Thu Apr 8 11:01:03 2010 Кодировка: Поисковые слова: п п п п п п п п п п п п п п п п п п п п п п п п п п п п п п п п п п п п п п |
Common Trending and QC tools:
|
tqs = Trending and Quality Control System |
![]() |
new: | see also: | ||||||||
A Radical Update of Python on the DFO machines. Python 2.6.2 + most up-to-date modules (December 2009) vimos_stdimg.tar scripts for QC on VIMOS imaging standard star fields (July 2009) qc_imadisp.py (May 2009, V1.2 with symbol overplotting July 2009) release of V1.1 for qclib and plot modules (March 2009) HAWK-I example scripts (March 2009) repository of example and tutorial scripts (December 2009) qc_detlin.py (February 2009) |
- description of library QClib
|
The current installation of python and its associated modules on our DFO machines is extremely old (i.e. python 2.4.2 was released in Sept. 2005). Since that time python has undergone a number of significant changes. More importantly, the modules used by the DFO installation are extremely out-dated. This means that many new and useful features are unavailable, and that those that do run are far from optimal in terms of processing time. This was the motivation for installing python version 2.6.2 (April 2009) and the newest versions of the modules.
However, in order to support python routines historical to DFO (i.e. qc1Ingest, cdbIngest, cdbQuery, dfscheck, etc.) requires that we maintain the old modules and python intrinsic to the DFO installation (python 2.4 with numarry). However, since we also want to make use of the large number of improvements in speed and features (numpy, scipy, matplotlib, etc.) that have been created since python 2.4, we need to have these two versions coexist. Therefore, the new python will be installed in a separate directory, and cleverly addressed within our new routines.
The new python executable lives in: /qcdp/bin/python
The new modules are located at: /qcdp/lib/python2.6/site-packages
1. First modify the PYTHONPATH in your .qcrc file to contain the following:
export PATH=/opsw/util/python/bin:$PATH
export PYTHONPATH=${PYTHONPATH}:/opsw/packages:/opsw/util/python2.4.2.1/lib/python2.4/site-packages
export PYTHONPATH=${PYTHONPATH}:/qcdp/lib/python2.6/site-packagesThis works because the first python path points to the old installation. If you are running a
python script that requires the new installation, it will first look in the old path, not find it,
and then look at the new site packages. This allows the old python routines to work along side the
new ones.
2. Each of your python scripts that require the new installation should begin with:
#!/qcdp/bin/python
Python scripts using the old DFO installation will begin with:
#!/usr/bin/env python or,
#!/scisoft/bin/python
3. Here is a summary of potential changes that may exist in your python routines:
Point of Change |
Old DFO Python Installation (2.4) |
New Python Installation (2.6.2) |
---|---|---|
top line of your python script |
#!/scisoft/bin/python or #!/usr/bin/env python | #!/qcdp/bin/python |
modules | import numarray | import numpy (numarray is no longer supported) |
array functions 1 (for example:) | x = zeros(N, Float32) x = zeros(N, Int) |
x = zeros(N, float) x = zeros(N, int) |
array functions 2 (for example:) | min_x = min(x) |
min_x = x.min() |
array functions 3 (for example:) | semilogy(x, where(hist>0, hist, 1), linestyle='steps', linewidth=1, color='blue')
|
semilogy(x, hist, linestyle='steps', linewidth=1, color='blue')
In the new system, negative values are automatically filtered |
array methods (for example:) | x.getflat() |
x.ravel() x.std() x.size |
matplotlib/pylab color definitions: character strings instead of floats | plot(..., color=0.00) | plot(..., color='0.00') |
scipy optimize package must be imported | import scipy | import scipy import scipy.optimize |
python 2.6.2
scipy
numpy
matplotlib
asciidata
dateutil
matplotlib
pyfits
pylab
Using the new DFO python 2.6.2 installation give you the most up-to-date version of python and, more importantly, its modules. As a result, there are now a very large number of extremely useful additions and debugging that have vastly improved the standard modules. This is particularly true forr matplotlib, numpy, scipy, and pylab.
Finally, when running your scripts using the new installation, you will find a huge increase in performance. This is mainly due to inprovements within numpy (versus the older numarray), but are also due to improvents in matplotlib (in particular, a speed enhancement in creating hardcopies of the QC plots) and scipy. On average, my QC python routines now run a remarkable six times faster!
Here is the proof. Below are the summaries of from the AB Monitor for 2009-11-08. The first figure shows the execution times for the old python installation, while the second figure shows the execution times based on scripts running the python 2.6.2 installation with the new modules. Summing up the total QC processing times, the latter is 6.1 times faster.
2009-11-08 Processed with the old DFO python installation:
Total times: 62.5 + 160.8 minutes
2009-11-08 Processed with the new DFO python installation:
Total times: 77.0 + 26.2 minutes
Send
comments to <mneeser@eso.org>
Last update: December 18, 2009 |