The HST Proposal Selection Process

• How the New Cycle 9 Review Process Worked , December 1999, pp. 21-22.

• Evolu tion of the HST Proposal Selection Process: Changes for Cycle 9 , Meg Urry, June 1999, pp. 9-10.
• The HST Proposal Selection Process, Mike Shara, January 1999, pp. 3-5.

How the New Cycle 9 Review Process Worked

There were five key changes in the proposal review process for Cycle 9. With the review now completed, we have received extensive feedback from the Cycle 9 panels and TAC (Telescope Allocation Committee) indicating the changes were very successful, hence we plan to repeat the new process (with minor changes) in Cycle 10. Here we summarize the process changes, followed by some advice for proposers in future HST cycles.

Goals, Changes, and Outcome

• Goal 1: Increase the fraction of large observing programs, to the 10-30% levels that have been recommended by external advisory committees.
  Changes: The role of the TAC was redefined to be review of Large proposals (100 orbits or more), for which up to 1000 orbits were available. Medium-sized proposals (15-99 orbits) were encouraged via orbit subsidies in the review process. To encourage more submissions, these new opportunities were heavily advertised in the Call for Proposals, the STScI Newsletter, and elsewhere.
  Results: More than 15 times as many Large proposals were submitted as in Cycle 8, constituting 1/4 of proposed orbits rather than a few percent. The TAC recommended a total of six Large programs, for ~1/4 of the total allocated orbits. Requests for medium-sized allocations doubled, constituting ~60% of the requested orbits and a similar percentage of the selected orbits. Thus the acceptance rate was largely independent of proposal size.
• Goal 2: Better determine the balance among scientific sub-disciplines.
  Changes: We greatly broadened the scientific focus of each review panel and charged them with allocating 100% of the Regular (<99 orbit) proposals, so that science balance could be achieved within individual panels. In preparation for this change, proposers were explicitly told to address the overall importance of their program to astronomy (the "big picture").
  Results: All proposals were evaluated via in-depth discussions by experts, rather than one third (the gray-area proposals) being discussed by a much broader and overly large TAC. The Cycle 9 panels were highly supportive of this change. Interestingly, quite a few panelists admitted to premeeting apprehensions about the breadth of expertise required, but by the end they embraced the new process. With redundant panels for each broad area (see next goal), there were effectively two independent determinations of science priorities per cycle; in a few cases, mirror panels came up with quite different mixes of science, making the new process more robust against the vagaries of peer review.
• Goal 3: Minimize the impact of conflicts of interest while still allowing experienced HST users to participate in the review.
  Changes: We assigned two panels to each broad science area (with the exception of the Solar System panel, for which there are too few proposals), so that a panelist's proposals could be reviewed by the other panel. Results: The acceptance rate was the same for PIs who served as reviewers and those outside the process. There were also dramatically fewer instances of panelists having to recuse themselves because of conflicts of interest, leading to a more consistent review involving a larger fraction of panel expertise.
• Goal 4: Enable exciting science with fast ToOs and multiwavelength HST+Chandra proposals.
  Changes: We increased the number of fast ToOs allowed, and reduced the minimum activation time proposers could request (with actual implementation to be driven by the proposed science). We instituted the first joint multiwavelength opportunity, for Chandra and HST, awarding up to 400 ksec of Chandra time and giving the Chandra project 100 orbits of HST time for their next review.
  Results: Several of the top rated Cycle 9 proposals involved fast ToOs and Chandra-coordinated science, attesting to the importance of these opportunities.
• Goal 5: Reduce the burden on the astronomical community.
  Changes:We halved the number of panels, and thus panelists (and cost). We implemented a triage process to keep the burden on individual reviewers manageable, after verifying, via tests on the review databases from previous cycles, that it accurately identified the bottom 1/3 of proposals.
  Results: Only ~75 reviewers served in Cycle 9, compared to ~150 in Cycle 8. The panels completed their work just as quickly, in part because of the triage approach, which they overwhelmingly supported.

Advice for Future Cycles

The new HST review process having been highly successful, we plan to continue it in future cycles. This means proposers should keep in mind several key points.

• First, don't be shy about submitting Large proposals; they have the same chance of success as any other proposal (a 1 in 5 chance). The TAC discusses each Large proposal in considerable detail, so winning proposals are not only of compelling scientific interest, they are well thought out and involve expert teams.
• Second, make the effort to address the non-expert. Most successful proposals set the context with sufficient background information, and clearly described the importance of the investigation to all astronomy (the "big picture"). Since this was an explicit criterion for evaluation, proposals could be downgraded for failing to include it.
• Third, don't pad your request for time: fewer than 10% of the approved proposals were cut (and those largely to avoid duplicate observations), so proposers either got what they asked for or were rejected.

The main advice?

Start from the science, write a compelling story for your fellow astronomers, ask for the resources you really need, and submit those proposals whatever their size. With such a high oversubscription (1/6 by orbits), many truly excellent proposals had to be rejected. Remember that rejected PIs are in good company, and that many of these proposals, possibly revised according to comments from the panel or TAC, may succeed in future years.

Cycle 10 Announcement Page

Copyright Notice