ST-ECF History

(Excerpts from articles by R. Fosbury and R. Albrecht, published in STScI and ST-ECF Newsletters)

What is the ECF? Why was it established, and what is its function?

It all started with the Memorandum of Understanding (MoU), signed by NASA and ESA in October 1977, governing their collaboration on the Hubble Space Telescope project. This stipulated that ESA provide one of the science instruments, the Faint Object Camera (FOC), the solar arrays plus their associated electronics, and fifteen ESA staff members on assignment to the Institute. In exchange, ESA astronomers would get a minimum of 15% of the observing time on Hubble and a complete copy of the science data archive.

The concept of a European Coordinating Facility for the Space Telescope developed subsequently and was stimulated in part by the FOC instrument science team, whose task it was to advise ESA on the science requirements for the instrument and on its design. It became clear that, for astronomers with the relatively primitive software tools available in the1980s, the process of observing with and reducing the data from the FOC and the other Hubble instruments would be a formidable task. Planning and specifying the observations would require access to an extensive library of technical information and, furthermore, few European astronomical institutes were equipped to do the extensive digital image processing that was going to be necessary. It was also realised at an early stage that, for a digital archive to be established and utilized properly, a lot of new ground would need to be broken.

In response to these concerns, ESA issued the Call for Proposals for the Space Telescope – European Coordinating Facility in mid-1980. The idea was that the successful bidder would host the organization and provide half the financial support in the form of a fraction of the staff as well as an institutional infrastructure within which the ECF could operate. Five organizations responded, and the European Southern Observatory (ESO) was selected as the host institute. The formal agreement to establish the ST-ECF as a separate unit at the ESO HQ in Garching, near Munich, Germany was signed in February 1983 by the respective Directors General of ESA (E. Quistgaard) and ESO (L. Woltjer). Strong points of the ESO proposal were the large, although not 100%, overlap of the ESA and ESO scientific communities and the fact that ESO was already operating a major multi-national observatory in Chile using an operating concept similar in many respects to that foreseen for Hubble. With the benefit of (a long) hindsight, it is rewarding to see that the synergy between operational models has been exceedingly beneficial to both the ground and space communities of observational astronomers. The fact that ESO operates its Very Large Telescope (VLT) on Paranal in northern Chile using a “service observing” scheme is due, in no small measure, to the connection with the Hubble observatory via the ECF.

During the early part of its history, the ECF was an entirely European affair. Its role was purely the support of European astronomers, and its existence was neither acknowledged nor accounted for in the ESA/NASA MoU.

Early History

Following the conclusion of the ESA/ ESO MoU, the ST-ECF started operating in early 1984. The staffing level was set at fourteen: seven ESA and seven ESO employees. However, with ESO proving the infrastructure and operational support the effective level was somewhat larger. The head of the ST-ECF was to be the European Hubble Project Scientist. Available resources included one VAX-11/780 and several image processing workstations. The host data-analysis software system was ESO’s MIDAS, then state-of-the-art.

With launch scheduled for 1986, the build-up of the ECF had to be very quick. Staff had to be recruited and things made ready for operations. An important element was the science data archive, which was to be based on emerging optical disk technology. It was apparent early on that the archive would be shared between Hubble and the science instruments at the ESO telescopes. This has been achieved and, while Hubble was certainly the driver during the early days, the science archive is now numerically dominated by the VLT and the growing complement of wide-field imagers at ESO.

Mechanisms for user support were implemented, such as the STDESK service (an email hot line). A number of seminars were arranged at major centres of European astronomical research in order to make the community aware of the opportunities offered by access to Hubble and to lay out the plans for observing proposal preparation and eventual data analysis. A newsletter was started and has continued with an average of around two issues per year. We like to think that this helped pioneer the use of emerging desktop-publishing technology in astronomy. With the exception of the final printing run, the entire process of design, layout and production was carried out in-house, by ECF staff. For many years now, some 3,000 copies per issue have been distributed throughout the world. A series of data analysis workshops was started and, in the mid-1990s, merged with the regular Astronomical Data Analysis Software and Systems (ADASS) events. Steps were taken to ensure that software developed at the Institute could be used at the ECF and, if required, elsewhere in Europe.

Operational Era

The Challenger accident in January 1986 introduced a long delay. When the telescope was finally launched in 1990, however, the archive was ready and much work on instrument calibration had already been done. The spherical aberration bombshell prompted a lot of hard thinking and, until the situation was effectively recovered in hardware as a result of the first servicing mission in 1993, software was the only option. At that time, Leon Lucy was a member of the ESO scientific staff, and he quickly appreciated that, provided that a good knowledge of the aberrated point spread function (PSF) was available, an iterative, non-linear deconvolution method could do a pretty good job of recovering the Hubble resolution, if not its limiting sensitivity. Leon subsequently joined the ECF staff as an ESA staff member and stayed with us until he returned to the UK in 1998. While, in retrospect, those three years of aberrated Hubble operation have been largely eclipsed by the superb optical performance subsequently achieved, the mathematical infrastructure triggered by those early needs has matured and grown into many areas of astronomical data analysis, including spectroscopy as well as imaging.

One of the more endearing or annoying aspects of Hubble, depending on your viewpoint, is the undersampling of the PSF by the pixels on most of the detectors. While this was done for good design reasons, including the scarcity of pixels in early CCDs and the need to minimize the effects of detector noise, it led inevitably to some creative operational and data reduction strategies. Thus did the terms “dither” and “drizzle” enter the Hubble vocabulary. Whilst the ECF should not take all the blame for these lexical aberrations, it should be said that there was considerable attention given in Garching to these undersampling problems long before they became an integral part of Hubble operations. The application of these techniques came to full public attention as a result of the monster data reduction efforts needed for the Hubble Deep Fields. It has led to a continuing collaboration between the ECF and the Institute on the development of new image combination tools.

The calibration of the instruments on a space observatory is a time-consuming and consequently an expensive process. The advantages of basing the calibration strategy on a firm foundation of knowledge of the physics and design parameters of an instrument, although not generally part of astronomical tradition, are now manifestly apparent. The construction of “instrument physical models” has been of great benefit to Hubble and VLT spectroscopy. The retroactive application of these methodologies to the FOS has led to a greatly improved calibration accuracy, which is now available to users through the archive. This modelling has been extended to the echelle modes of STIS and enables much improved wavelength calibration of spectra. The availability of the scheme to recalibrate Hubble data “on-the-fly” – a concept developed jointly by the ECF and the Canadian Astronomy Data Centre (CADC) – has enabled the benefits of such calibration improvements to be realized in a manner convenient to the user and has also simplified and reduced the size requirements of the archive.

Another example for this “added value” approach is the introduction of so-called “WFPC associations”, whereby multiple observations with the same instrumental configuration of the same target or field can be identified in the archive and grouped for subsequent automatic co-addition.

In many ways these efforts typify the way in which the ECF operates. Given the level-of-effort funding, it was clear that not every aspect of the Hubble project could be covered equally by a small European team. Instead it was decided only to offer rather basic services, i.e., user support and data-analysis assistance, across the board. Beyond that, the strategy has been to focus on selected topics, the choice of which has been agreed with the Institute and, where appropriate, the instrument teams. Where possible, these topics follow common themes that allow the building up of significant expertise within the group. A current example is the provision of slitless spectroscopy software and expertise for the ACS. This started with an agreement to work on the grism capability on NICMOS, for which the ECF provided a quick-look tool and a calibration capability. This accumulation of spectroscopic know-how has, e.g., been applied to the studies and the design assessment of the JWST spectrograph. The Wide Field Camera 3, which will be deployed on HST during Servicing Mission 4, also has a slitless spectroscopy mode which will be fully supported by the ECF in a similar manner to ACS slitless spectrscopy.

Being relatively small in number carries both advantages and disadvantages for the group. It means that resource limitations require a continual re-assessment of priorities and impose a severe limit on the number of projects that can be pursued at any one time. On the other hand, it enables excellent internal collaboration. With all ECF staff around the table once a week, we have been able to maintain a high level of communication between the different activities being carried out at any time.

Members of the ECF have, over the years, played their part in many Hubble administrative and oversight affairs. ECF has helped in the selection of, and travel support for, European members of the TAC and its associated panels, membership of science review and study teams.

The ECF maintained its traditional role of data analysis, user support, and maintenance of the science data archive until about 1994, when, as a result of the Internet revolution, it became apparent that an evolution was warranted. While it was still necessary for ESA to have an independent European source of expertise on Hubble, a data-analysis capability, and ensured access to the science data, the ECF should do more than just mirror for Europeans what the Institute was doing so effectively for the entire user community.

Recognising these changes in emphasis, ESA called a mid-term review of the ECF, which was carried out in 1996 by a team comprising Len Culhane, Rolf-Peter Kudritzki, and George Miley. This process endorsed a more direct role for the ECF in the Hubble project, with selected activities becoming part of the renewed ESA/NASA MoU, which ran originally for only eleven years following launch. Some specific tasks resulting from these recommendations were the pre-processing of the plate-scan data (from Turin Observatory) for the second Guide Star Catalog (GSCII), the establishment of a postoperational archive calibration team to work on refined calibration of de-commissioned instruments, and the creation of a small Public Relations team to work with ESA on the European exposure of results from Hubble and other ESA spacecraft.

The ESO Connection

ESO has provided much more than an operational infrastructure for the ECF. It has created a rich and active scientific environment for the staff and enabled the exchange of ideas on a wide range of technical issues. Any ECF member, past or present, would be delighted to recognize the vital role that the host organization has played in the life of the group. Thanks to the positive attitudes of a series of ESO Directors General, both ESO and ESA ECF staff have enjoyed all the privileges our host can offer. While maintaining its individuality, the ECF is very firmly integrated into ESO and participates fully and without prejudice in many of the host’s activities. This involvement, of course, carries its own responsibilities, and we think it is fair to say the ECF has pulled its weight in tasks associated with the ESO Faculty as well as a number of management and technical activities. Indeed, for a while the previous head of the ECF, Piero Benvenuti, was also leader of the ESO data management division. Partly because of its close day-to-day connection, through the Institute, with developments in the US, the ECF was always concerned with computer-support issues and also with the collection and distribution of astronomical software which led to the “Scisoft” initiative.

Perhaps the greatest advantage offered by this stimulating environment is the ability it gives to participate in major scientific programmes, some of which involve both the ground-based telescopes operated by ESO and Hubble. The Great Observatories Origins Deep Survey (GOODS) is an example of such a cooperation. Since the ECF has been present in Garching for essentially the entire construction phase of the VLT, it has been very rewarding for us to see this project reach completion and earn such worldwide respect as one of the major observatories of our age.

Closure of the ST-ECF

Following decisions made by both ESA and ESO during the last few years, the ST-ECF closed and ceased operations on 31 December 2010. Established as a joint venture between these two organisations in 1984, the ST-ECF has worked in close collaboration with the STScI in Baltimore to support European users of Hubble and to provide significant contributions to the operation of the observatory and its instruments. The history of the activities of the group in Garching is captured in the 48 issues of the ST-ECF Newsletter1, of which this will be the final edition.

Information resulting from the group's instrument science activities has been transferred to STScI and is available from their website. Of topical interest, this includes all the work concerned with slitless spectroscopy obtained with NICMOS, ACS and WFC3.

The Hubble outreach activities, started in response to a request from NASA more than a decade ago and delivered through www.spacetelescope. org, will continue at a substantial, but somewhat more modest, level with the continued involvement of the ESO education and Public Outreach Department (ePOD) and a financial contribution from ESA.

Until very recently, it was intended to keep the European Hubble Data Archive at ESO, where it was originally the model for the development of the ESO Science Archive Facility. The intention now, however, is to create a European access point for Hubble data at ESAC in Spain. ESO will cooperate as far as possible in achieving this transfer in a way that avoids an interruption in public availability in Europe.

Maintained by Britt Sjoeberg <bsjoeber@eso.org>