Reflections on a Decade of ADASS

Since its inception ten years ago, the ADASS conference series has been the premiere forum for software development related to the acquisition, calibration, analysis, and dissemination of astronomical data. We review the history of the conference itself, and highlight a few of the most important advances in the software and systems, and ponder what future advances might bring. We conclude with a description of the new adass.org Web site, and the services it offers.

1. Introduction

The past ten years have witnessed extraordinary changes in the way astronomers pursue their science. These changes in the way they build and operate the national observatories, the way they obtain, reduce, and analyze their data, and the way they communicate their results to their peers and the public, have all been profoundly changed, and changed primarily, through the revolution in software, networks, and information technology. The Astronomical Data Analysis and Software Systems (ADASS) Conference Series was established ten years ago as a forum on the development of software and systems for the acquisition, calibration, analysis, and dissemination of astronomical data. This annual conference has given those of us on the forefront of the astronomical information age a chance each year to pause and reflect on the trends and techniques of our craft, and on the application of the myriad of advances in the computer industries to our profession.

2. ADASS Past and Future

The landscape of astronomical software development prior to the 1980s was typified by small, heroic, but generally isolated groups that built highly specialized software for the support of specific missions or observatories. Later, as major observatories took on the role of supporting the data reduction and analysis needs of broader and growing communities of observers, more general reduction and analysis systems such as AIPS, IRAF, and MIDAS came into wide use. Yet it was generally difficult for non-experts to add major applications and otherwise extend these systems. As a result, software development for many projects continued its divergent course.

The ADASS conference series began in 1991 as somewhat of an experiment: to see whether astronomers and the developers of software systems for data reduction and analysis, archiving, etc. could be brought together (even once) to share ideas, experiences, and techniques for the collective good of astronomy. The hope was that by raising the awareness of the various approaches to common problems across systems and wavelength domains, a willingness to cooperate and even collaborate would result. This first conference was hosted in the fall of that year by the National Optical Astronomy Observatories (NOAO), and was attended by more than 300 people. Shortly after its inception, ADASS became the premiere, world-wide conference on astronomical software; it has been hosted by a number of major astronomical institutions in the US, Canada, and Europe (see Table 1). ADASS conferences have generally been well attended, although the attendance is somewhat lower when the venue is outside the US (see Figure 1).

The typical three and one-half day ADASS program consists of a few dozen oral presentations (including invited talks), numerous display presentations, software demonstrations, ``Birds of a Feather'' (BoF) sessions on special topics, and one or more tutorials on important new technologies. The legacy of ADASS is captured in the proceedings volumes, which have been published each year by the Astronomical Society of the Pacific in hardcopy and (since 1993) electronic form. The combined volumes comprise nearly 1200 papers, which include the oral and display presentations and summaries of some demonstration sessions. Perhaps not surprisingly, many of these papers have been cited elsewhere in the astronomical literature (some with enviable citation rates).

**Figure 1:** Annual totals of ADASS attendance ( *left bar*), presentations ( *middle bar*), and papers that appeared in the proceedings ( *right bar*).
$\begin{figure} \plotone {O10-02a.ps} \end{figure}$

3. Demographics

The ADASS Conference series was conceived with astronomers and software developers (who are often one in the same) in mind. So, too, were technical leads and managers, though the program content did not explicitly target this demographic group in the early years. The largest segment of ADASS attendees received formal, post-graduate education in astronomy; few attendees have a formal background in computer science or software engineering. Surveys of the ADASS attendees in 1993, 1998, and 1999 yield an interesting insight: Figure 2 shows a breakdown by formal professional education or training, and Figure 3 by primary job responsibility. Evidently, most developers (and managers) of astronomical software came in to their current careers as astronomers.

**Figure 2:** Professional background of ADASS attendees.
$\begin{figure} \epsscale{0.9} \plotone {O10-02b.ps} \end{figure}$

**Figure 3:** ADASS attendees by professional duties.
$\begin{figure} \epsscale{0.9} \plotone {O10-02c.ps} \end{figure}$

4. Projects that Changed Astronomy

Looking back on a decade of ADASS, it is interesting to search for the best or most compelling papers, or those projects that were closest to the cutting edge of technology. We have created a database of authors, subjects, and paper titles from the first decade of ADASS to enable such searches, which will be kept up to date as a resource for the community. We selected for this review those papers that describe projects which arguably have had the most profound impact on astronomy and astronomical data analysis during the last ten years. Regrettably, time and space limitations prohibit us from highlighting more than a few papers.

The 1980s and 90s witnessed the introduction and maturation of the major data reduction and analysis systems, with which most astronomical data are processed. ADASS has always been an excellent forum for developers to hear of the latest enhancements and plans for these systems, e.g., AIPS++ (Croes 1992) and IRAF (Tody 1992). These and other systems are routinely featured in ADASS BoF sessions. The increasing importance of space-based observatories during that time frame, particularly in 1990 with the launch of the Hubble Space Telescope, drove the development of sophisticated software in a number of areas. These included the application of AI for astronomical scheduling problems (Johnston 1992), and observation planning tools (Koratkar & Grosvenor 1999).

The archiving of digital data has also been a major focus of space missions for a few decades now, but it is fair to say that the supporting technologies only reached maturation in the 1990s. More than 100 papers have been presented at ADASS on the technologies behind digital archives and archive science. In the past few years, larger institutions have been leveraging their archiving expertise and infrastructure to build multi-mission archives. This is not to say that the generic problem of archiving ground-based data has been ignored, as discussed by Crabtree (1996). The SDSS planned perhaps the most ambitious electronic archive of ground-based data for its era (Brunner et al. 1996).

If space missions and electronic archives have ushered in a golden age in astronomy, then the extensive surveys that populate the archives are a significant portion of the precious ores. The science goals and the technologies employed in many of these surveys have been presented at ADASS, including FIRST (Becker 1994), the Sloan Digital Sky Survey (Kent 1994; Szalay et al. 2000), the Hubble Deep Fields (Ferguson 2000), and 2MASS (presentation by Cutri). What has become abundantly clear, though, is that it is now easier to populate archives with vast quantities of survey data than it is to mine and refine the best science from them. This very point was addressed in the recent Decadal Survey of Astronomy (NRC 2000), which advocated the construction of a National Virtual Observatory. The NVO will be based on a distributed model of data repositories and of computational resources, and will provide the necessary infrastructure for astronomers anywhere on the Internet to access any public archive to resolve sophisticated queries (Szalay 2001).

The ``killer app'' of the 1990s, NCSA Mosaic (the precursor of Netscape), was presented at ADASS III by Joseph Hardin, before most of the world was even aware of the Internet. This single application revolutionized the way in which individual astronomers interact with observatories, professional journals, and each other. The metamorphosis of the Astrophysics Data System (ADS) into the virtual repository of astrophysical literature is a prime example of the profound effect of Web browser technology: compare, for example, the initial presentation on ADS by Good (1992) to that by Accomazzi et al. (1995). Web browsers also made possible the sharing of data, information, and the results of research more rapidly and effectively than ever before.

The technology behind some of the most profound science results of the past decade have also been presented at ADASS. For example, perhaps the most widely used (and cited) tool for photo-ionization modeling, CLOUDY, was described in detail by its author and curator, Gary Ferland at ADASS VI (Ferland, Korista & Verner 1997). Also that year Paul Butler reviewed the Lick Observatory planet search, which was one of the most successful techniques behind the then-new discovery of Jupiter-sized planets around nearby stars. Results such as these, and the supporting technologies, have helped to shape the direction of the NASA ``Origins'' program as well as components of the Decadal Survey (NRC 2000). Looking to the future, ADASS X attendees learned of the objectives and techniques of the Search for Extra-Terrestrial Intelligence (SETI) (presentation by Tarter), including the familiar distributed computing technologies used to process the accumulating data (presentation by Werthimer) on an amazing number of idle workstations and PCs around the world.

5. A Look Ahead

What can we expect to be presented at future ADASS conferences? Given the extraordinary and largely unanticipated advances in the last decade, offering a serious answer to this question invites, at worst, comparisons to salesmen, politicians, and fools; more likely, it will provide punch-lines for the ADASS XX review speaker.

Some developments are not too hard to foresee: the continuing development of XML as a de-facto standard of astronomical data and meta-data exchange, for example would be quite complementary to its use in commercial applications. The design and construction of NVO (though we are confident it will be a ``GVO,'' or Global Virtual Observatory) will also almost certainly come to pass. What is less clear is whether NVO will be successful at first (given the cost and public profile, we hope so), or whether the fundamental GRIDS technology involved will be stable for the duration of the project (we're not taking any bets here). Concomitant with NVO will come data mining software, and other middle-ware that will provide generic methods to access data, and to map it to/from one of a few ``standard'' models. We are already seeing applications that can access data from multiple archives and display images, catalogs, etc. on areas of the sky. We suspect that this is only the first wave of ``net-savvy'' applications that will be built or modified from other analysis software. This trend certainly has precedent: consider the evolution of simple word-processing software from simple text-entry to include WYSIWYG presentation, spell-checking, graphics and spread-sheet importation, Web-page generation, etc. Simple file browsers for local file selection will not be able to compete with VO-aware applications.

NVO is an example of a system where alternative computing models are critical to success. But we have already seen examples of Beowulf and other parallel or at least distributed systems for specialized applications in astronomy. We expect this trend to continue in the next decade. We also suspect that wireless applications will find interesting uses in astronomy--beyond the portable computer: telescope control, data taking, and instrument control are potential applications. Beyond that, we expect to see more (semi-) autonomous systems used in observatory operations. Studies of automated scheduling and data acquisition and calibration are underway for the Next Generation Space Telescope and other missions. Can self-calibrating detectors, self-constructing data pipelines, and automated data analysis be far behind?

6. Implications of the Virtual Observatory

The Virtual Observatory has the potential to change astronomy as profoundly as the major, space-based archives have done individually. But as proposers for observing time at major facilities know, programs are unlikely to survive the peer-review process unless the proposals demonstrate that archival data has been used to its fullest extent. In this sense, observational astronomy is changing in a very profound way: proposers are being held responsible for every photon in every public archive! We suspect this point is under-appreciated in the community, but we believe it will drive the development of observation planning and data analysis applications that can access archives transparently.

7. ADASS On-Line

In view of the continuing role of ADASS to the professional community of astronomers and software developers, we have begun developing an ADASS Web site with the rather ambitious goal of ``one-stop shopping for astronomy software.'' The electronic version of the conference proceedings and the database of authors and papers were described above. Beyond that, we plan to provide resources such as centralized conference registration services, presentation and publications policies, and templates and style files for proceedings authors. We also plan resources that will benefit a somewhat broader community, such as master author and subject indexes for the combined proceedings, links to other conferences of potential interest to ADASS attendees, and career services for professionals in astronomical software. Links to these resources will be found on the ADASS home page at http://www.adass.org/.

Acknowledgments

The ADASS conference series would not have been possible without substantial support from the sponsoring institutions, whose material, financial and in many cases staff support have helped make ADASS a success each year. Table 2 lists the past sponsors to whom we are all indebted.

References

Accomazzi, A., Grant, C. S., Eichhorn, G., Kurtz, M. J., & Murray, S. S. 1995, in ASP Conf. Ser., Vol. 77, Astronomical Data Analysis Software and Systems IV, ed. R. A. Shaw, H. E. Payne, & J. J. E. Hayes (San Francisco: ASP), 36

Brunner, R. J., Csabai, I., Szalay, A., Connolly, A. J., & Szokoly, G. P. 1996, in ASP Conf. Ser., Vol. 101, Astronomical Data Analysis Software and Systems V, ed. G. H. Jacoby & J. Barnes (San Francisco: ASP), 493

Croes, G. A. 1993, in ASP Conf. Ser., Vol. 52, Astronomical Data Analysis Software and Systems II, ed. R. J. Hanisch, R. J. V. Brissenden, & J. Barnes (San Francisco: ASP), 156

Good, J. C. 1992, in ASP Conf. Ser., Vol. 25, Astronomical Data Analysis Software and Systems I, ed. D. M. Worrall, C. Biemesderfer, & J. Barnes (San Francisco: ASP), 35

Johnston, M. 1993, in ASP Conf. Ser., Vol. 52, Astronomical Data Analysis Software and Systems II, ed. R. J. Hanisch, R. J. V. Brissenden, & J. Barnes (San Francisco: ASP), 329

NRC 2000, Astronomy and Astrophysics in the New Millennium, Astronomy and Astrophysics Survey Committee, Board on Physics and Astronomy--Space Studies Board, Commission on Physical Sciences, Mathematics, and Applications, National Research Council (Washington, D. C.: National Academy Press)

Szalay, A. S., Kunszt, P., Thakar, A., Gray, J., & Slutz, D. 2000, in ASP Conf. Ser., Vol. 216, Astronomical Data Analysis Software and Systems IX, ed. N. Manset, C. Veillet, & D. Crabtree (San Francisco: ASP), 405

Tody, D. 1993, in ASP Conf. Ser., Vol. 52, Astronomical Data Analysis Software and Systems II, ed. R. J. Hanisch, R. J. V. Brissenden, & J. Barnes (San Francisco: ASP), 173