Документ взят из кэша поисковой машины. Адрес оригинального документа : http://www.stsci.edu/institute/atlast/documents/ServicingRFI_stsci_jhu_jpl_final.pdf Дата изменения: Unknown Дата индексирования: Tue Feb 5 12:50:38 2013 Кодировка: Поисковые слова: earth's atmosphere

FEASIBILITY OF USING HUMAN SPACEFLIGHT OR ROBOTIC MISSIONS FOR SERVICING EXISTING AND FUTURE SPACECRAFT RFI # NNG10FC43-RFI Authors: M. Postman1, M. Stiavelli1, L. L. Whitcomb2, J. Tumlinson1, H. Ferguson1, D. C. Redding3, J. Green3, M. Mountain1, G. Hager2, A. Okamura2, P. Kazanzides2, N. Cowan2, R. Kumar2, I. Iordachita2, R. Taylor2, G. Chirikjian2, J. Grunsfeld1, R. Brown1 Responding Organizations: Space Telescope Science Institute 3700 San Martin Drive Baltimore, MD 21218 Contact: Dr. Marc Postman, (410) 338-4340, postman@stsci.edu
1

Laboratory for Computational Sensing and Robotics The Johns Hopkins University 3400 North Charles Street Baltimore, MD 21218 Contact: Prof. Louis Whitcomb, (410) 516-6724, llw@jhu.edu Jet Propulsion Laboratory ­ California Institute of Technology 4800 Oak Grove Drive Pasadena, CA 91109 Contact: Dr. Joseph Green, (818) 354-8403, Joseph.J.Green@jpl.nasa.gov Categories: Science Mission enabled by (or benefiting from) on-orbit servicing or assembly: o HST, JWST, Starshade spacecraft, ATLAST Facilities to enable telescope technology using the ISS Enabling Technologies for on-orbit servicing: o Software environment for remote manipulation with time delay
3

2

· · ·

Format: White Paper All organizations listed above are willing to a) participate and present our ideas at follow-up workshop, b) showcase our capabilities to the NASA study team at locations, and c) engage in and contribute to developing/enhancing technologies for orbit servicing both with NASA and with other partner organizations. JPL has controlled information that might be useful for this exercise and would be wil to discuss it if proper arrangements can be made to protect the information. Submitted: January 14, 2010 the our onling

1


Table of Contents
Executive Summary ........................................................................................................ 3 1. Introduction................................................................................................................. 4 2. Servicing Existing and Future Space Observatories ..................................................... 5 2.1 The Scientific Benefits of Servicing the Hubble Space Telescope .......................... 5 2.1.1 The Future of the Hubble Space Telescope ..................................................... 7 2.2 Servicing and JWST .............................................................................................. 8 2.3 The Virtual Mountain Top: Servicing Space Observatories Beyond JWST .......... 11 2.3.1 Additional Benefits to Servicing Future Space Observatories ........................ 13 2.4 An ISS Observatory Testbed................................................................................ 14 3. Enabling Technologies .............................................................................................. 16 3.1 Active Optical Systems for Large Space Telescopes ............................................ 16 3.1.1 Active Lightweight Mirrors........................................................................... 16 3.1.2 Wavefront Sensing and Control Systems....................................................... 17 3.1.3 Laser Truss ................................................................................................... 17 3.1.4 ISSO as the First Step on a Flexible Path ...................................................... 18 3.2 Remote Manipulation With Time Delay .............................................................. 19 3.2.1 Information-rich immersive software environment for planning and executing remote fine telemanipulation tasks ............................................................. 19 3.2.2 Virtual and physical simulation environments for servicing telemanipulation operational plan development, training, and execution ............................... 20 3.2.3 Fine telemanipulation with time delay through predictive model-based telemanipulation......................................................................................... 20 3.2.4 Semi-automated telemanipulation behaviors for safe remote manipulation.... 21 3.2.5 Semi-automated manipulation "macros" for safe performance of fine telemanipulation tasks in the presence of time delay................................... 22 3.2.6 Real-time computer vision scene understanding and tracking of complex telemanipulation environments................................................................... 22 REFERENCES ............................................................................................................. 23

2


Executive Summary This is a joint STScI ­ JHU ­ JPL response to the RFI entitled "The Feasibility of Using Human Spaceflight (HSF) or Robotic Missions for Servicing Existing and Future Spacecraft." We focus on addressing the following questions posed in the RFI: Q1: How does servicing using currently planned and future human spaceflight or robotic systems make a previously impossible mission technically feasible? Q2: How does servicing using currently planned and future human spaceflight or robotic systems reduce mission risk? Q3: How does servicing using currently planned and future human spaceflight or robotic systems offer a significant reduction in the cost of accomplishing the mission? Q4: What life-extension or performance enhancements are enabled by servicing using currently planned and future human spaceflight elements or a robotic system? Q6: What core technologies enable servicing for more than one architecture? How could the ISS be used to further the development of this technology? These questions are addressed here, in part, by identifying the scientific and technical benefits of servicing the Hubble Space Telescope, the James Webb Space Telescope, and future large UVOIR mission concepts such as ATLAST. On-orbit servicing provides three key functions that directly extend or magnify the scientific impact of space observatories: · Extension of observatory lifetime through replenishment of expendables and replacement of limited-lifetime items; · Restoration of observatory operation through the replacement or repair of degraded or failed components; · Expansion of observatory capability by upgrading to newer technology. There are clear scientific benefits to a longer lifetime and upgraded instrumentation: the observatory is able to perform observations not possible with its initial complement of instruments, undertake investigations not conceived of when it was initially built, and maintains our long-term access to space-based coverage of a broad set of wavelengths, keeping the observatory at the forefront of research well beyond the nominal mission life of most unserviceable facilities. This is particularly critical in an era where we can afford to launch only one flagship class mission per decade. Furthermore, a serviceable and modular design can reduce cost and risk through lower margins on expendables, simplification of the I&T phase, and reduction in component redundancy. Given this, we believe future large space-based observatories should be designed to be serviceable and upgradeable. In the case of JWST, we identify a limited set of lifetime extension servicing options. We further address the above questions by presenting a suite of possible facilities and development projects based on HSF infrastructure to test and validate software environments for remote control of robotic systems and to mature the key technologies needed to assemble and operate large lightweight telescopes in space. We recommend a near-team first step; an ISS observatory that will demonstrate key technology elements that support future serviceability and on-orbit assembly.

3


1. Introduction Great partnerships flourish when mutual interests are not just simply aligned, but when partners find ways together to build on their respective strengths, making the whole greater than the sum of the parts. Even though the Hubble Space Telescope (HST) was not a prominent part of the original rationale for human spaceflight after the Apollo program, the fact that this science facility was adapted to the shuttle program, which in turn developed key capabilities to service HST, jump-started an amazing two decades of space science. Decades from now, HST's impact on science and on the American public may well be regarded as the most significant outcome of the shuttle era. In this spirit, it is critically important to consider opportunities to use NASA's nextgeneration human-space-flight system to enable frontier space science. This requires communication, collaboration, and compromises between NASA's Exploration Systems Mission Directorate (ESMD) and Science Mission Directorate (SMD). It requires creativity in designing spacecraft, and it requires broad thinking about how to fund the development of infrastructure that can best support a wide variety of missions and destinations. Indeed, this approach is at the core of the "Flexible Path" scenario suggested by the Augustine Committee1. We see this RFI as the beginning of a new era of collaboration between ESMD and SMD. In this response to the RFI, we provide views of how NASA's human spaceflight program and its associated infrastructure enable scientific investigations that would otherwise be infeasible and maximize the scientific impact of large space telescopes. Our RFI has two segments. First, we layout a number of opportunities where NASA's human spaceflight (HSF) capabilities and/or robotic technologies can be used to substantially extend observatory lifetimes, augment their scientific capability via instrument upgrades, potentially simplify integration and testing (I&T) and reduce development costs. We discuss some potential scenarios for servicing HST and the James Webb Space Telescope (JWST). We also layout the key benefits that serviceability provides for future astronomy mission concepts such as ATLAST2. Second, we describe some of the key enabling technologies that need to be advanced to facilitate such activities and identify the human space flight infrastructures that both benefit from and facilitate their development. We view all of the above servicing concepts as well integrated with and beneficial to NASA's future exploration initiatives, as highlighted in Figure 1. While our focus in this RFI is on existing and future UVOIR space observatories, much of what we discuss is applicable to facilities that operate beyond the UVOIR spectrum (e.g., IXO, SAFIR). Combined with the right technologies, astronaut and robotically assisted telescope servicing and assembly will create the future facilities for exploring the structure of the Cosmos and for characterizing exoplanets in the search for life.
1 2

See http://www.nasa.gov/pdf/384767main_SUMMARY REPORT - FINAL.pdf ATLAST = Advanced Technology Large Aperture Space Telescope. Further information is available at http://www.stsci.edu/institute/atlast/. 4


Figure 1: The capabilities needed to service existing and future astronomical observatories are well aligned with the needs of NASA's Human Space Flight program. Th e resulting astronaut and robotic technical capabilities will have broad applications for manipulation and assembly of structures in space. An approximate timeline for the v arious activities is suggested here.

2. Servicing Existing and Future Space Observatories 2.1 The Scientific Benefits of Servicing the Hubble Space Telescope HST is widely regarded as the most successful scientific facility in history ­ and it owes that rank in large part to its serviceable design. HST epitomizes a natural synergy between humans and telescopes in space, particularly for telescopes of modular design. The 2009 servicing mission has made the observatory more powerful than at any time since launch in 1990. The science community eagerly competed for observing time on the refurbished observatory, with an oversubscription rate of 6:1 in the normal review cycle and an oversubscription of 18:1 for the latest Multi-cycle Treasury program call, the first since the new instruments were installed. The following brief history of HST servicing and its impact on science provide our first set of answers to questions Q1, Q2, and Q4 (see page 3). The chief advantage of a telescope in space is access to smooth wavefronts of 5


astronomical light, undistorted by passing through Earth's turbulent atmosphere. These wavefronts are collected and combined by the near-perfect optical surfaces of "diffraction-limited" telescopes and focal-plane instruments. The greater the dimension of the collecting optics, the more accurately the source can be located, or the higher the "resolution." Also, larger apertures collect more light in a shorter time. Because light equals information, the amount and quality of astronomical information from telescopes determines the size and precision of their optics. To the limit of a 2.4-meter aperture, HST fully achieves the advantages of space astronomy over four octaves of wavelength from the far ultraviolet to the near infrared. In this regime, astronomers can detect deeper sources and weaker signals, separate more closely spaced sources, record fainter detail on complex objects like galaxies and nebulae, and measure smaller displacements and tinier variations of brightness and color--all with greater precision than any telescope, ever. The first HST servicing mission in 1993 restored its optical performance (from an undetected prelaunch mirror polishing error) and gave it a much more powerful camera, the Wide-Field / Planetary Camera 2 (WFPC2). The servicing also corrected a flaw in it solar panel design that resulted in excessive jitter in the telescope when the thermal loads changed rapidly across the day/night terminator. One of the most important achievements of the refurbished HST was the resolution of a decades-old controversy about the expansion-rate of the universe. These measurements had been planned well before launch, but needed a well-focused telescope to succeed. Entirely unexpected was the discovery in 1998 that the expansion of the universe is accelerating, a result which was arrived at by two separate teams using careful HST and ground-based measurements of very distant supernovae. The result has now been confirmed by a variety of independent techniques, and indicates that there is a huge gap in our understanding of fundamental physics. Either our understanding of gravity is incorrect, or there is an unknown repulsive force ­ dubbed "dark energy" ­ counteracting gravity on cosmic scales. Astronauts visited HST again in 1997 and installed the Space Telescope Imaging Spectrograph (STIS) and the Near-Infrared Camera and Multi-Object Spectrograph (NICMOS). STIS observations have provided some of the most important measurements of the masses of black holes in the centers of galaxies, and revealed a strong correlation between the black hole masses and the masses of the surrounding stellar systems. This correlation provides very strong constraints on theories for how super-massive black holes form in the centers of galaxies ­ a problem that remains unsolved. STIS also provided the first measurements of the atmospheric constituents of a planet orbiting a nearby star. Among the highlights from NICMOS were exquisitely detailed images of disks of material surrounding nearby stars, which revealed both the remnants of the processes that form stars and the material that could evolve into planetary systems. A servicing mission in computer, a new solidthe gyroscopes needed had forced Hubble into 1999 enhanced many of HST's subsystems, including the central state data recorder to replace its aging magnetic tape drives, and for pointing control. A month prior to launch, a gyroscope failure "safe mode," with no ability to observe astronomical targets.

The next servicing mission in 2002 installed the Advanced Camera for Surveys (ACS) ­

6


providing a tenfold improvement in survey speed over WFPC2. Astronauts also brought NICMOS back to life. A design flaw in its original cooling system resulted premature loss of coolant. NASA engineers developed an innovative mechanical cooling system to resolve the problem and tested it on the space shuttle in 1998. The cooler is still in operation in 2009. Cosmologists have used the combined power of ACS and NICMOS to confirm and greatly improve the measurements of cosmic acceleration, and to detect and study galaxies so distant that the light reaching us today left when universe was less than 1 billion years old. By 1998 NASA had selected two fourth-generation instruments for HST: the Cosmic Origins Spectrograph (COS), which would provide an order-of-magnitude boost for ultraviolet spectroscopy, and the Wide-Field Camera 3 (WFC3), which would provide a similar boost for infrared and ultraviolet imaging. The Columbia failure threw the servicing mission into question and delayed the installation of these instruments for about seven years past the original plan. By January 2007, both the ACS and STIS had failed due to problems with power supplies. While both instruments had exceeded their design lifetimes, engineers were able to pinpoint the problems well enough to design replacement electronics. Engineers were also able to respond quickly to the failure of a command and data-handling computer in 2008 and have a replacement system ready for installation on the May 2009 servicing mission. As of January 2010, both COS and WFC3 are returning ground-breaking science that would have been impossible with HST's previous suite of instruments.
2.1.1 The Future of the Hubble Space Telesco pe

With the installation of the new instruments and repairs to the electronics, as well as replacement batteries and pointing-control system components, HST is at the all-time height of its capabilities, and is expected to continue returning spectacular results until at least 2014. NASA has no immediate plans to return to HST. With the retirement of the Space Shuttle, any future servicing would need to use the Constellation system or a comparable commercially provided system. Regardless of the means of transportation, a final mission to HST is necessary to install a propulsion system for de-orbit. Fixtures were installed in the last servicing mission to facilitate doing this robotically. It is conceivable that new instruments could also be installed during the same mission, but NASA has no definite plans to do so. Over the last decade there have been several studies of possible fifth-generation instrumentation to address some of the highest priority science goals on the NASA astrophysics agenda: finding and characterizing extrasolar planets, and studying dark energy. Incorporating advanced adaptive optics into a coronagraph would make it possible for HST to measure the orbits and obtain information on the physical properties of giant planets orbiting nearby stars. Existing HST instruments do not suppress the glare of the starlight enough to make this possible. HST could be equipped to provide a starlight suppression capability comparable to what would be provided by a dedicated 1.5-m

7


coronographic telescope. This kind of observation is an important precursor to the more ambitious goal of obtaining images of earthlike planets. By replacing two fine guidance sensors (FGSs) with wide-field near-infrared cameras, HST's imaging survey speed could be boosted by about a factor about 50. This could be useful for measurements of distant supernovae and weak gravitational lensing, both of which are key techniques for studying dark energy. A wide-field imager would also open a wealth of studies of stellar populations in nearby galaxies, accomplishing in one year what it would take 50 years to accomplish with current imaging capabilities. One of the most interesting results of the last decade has been the discovery of ultra-faint dwarf galaxies and giant streams of stars orbiting the Milky Way and the Andromeda galaxy. These streams are thought to be the remnants of satellite galaxies that have been disrupted by the gravitational interaction with their larger neighbor. Measurements of such structures around more distant galaxies provide key tests of current models of galaxy formation, as well as insights into the nature of dark matter. Whether or not it is worth pursuing new instrumentation is a question that should be addressed both by the scientific community and the exploration community. On the scientific side the question comes down to whether it is more efficient (in terms of survey speed) and cost effective to use an existing, well-tested but aging, space-telescope in lowearth orbit for these measurements, or to design and build special-purpose telescopes (which could go into different orbits) to accomplish the science goals. Costs, risks, and science capabilities would all need to be weighed. This kind of trade study was not part of the charter of the 2010 Astrophysics Decadal Survey. From the expertise would be some cost exploration perspective, a key question is whether or not the experience and gained by servicing HST in low-earth orbit using the Constellation system valuable for other human-exploration goals, and whether that value justifies sharing with SMD. 2.2 Servicing and JWST The James Webb Space Telescope (JWST) is a 6.5-meter, deployable, near-infrared space telescope with a lifetime that is ultimately limited by the amount of fuel required for station keeping. JWST is not explicitly designed to be serviced on orbit. However, a robust servicing infrastructure that can operate at SE-L2 or EM-L1 could still be beneficial to JWST to handle certain repairs, extend its lifetime and its scientific capabilities. This section provides answers to questions Q2 and Q4 (see page 3) as they relate to JWST. The capability to repair telescope and instruments on JWST is more limited than it is for Hubble for a variety of reasons including the fact that Hubble was designed for human access and that the JWST Integrated Science Instrument Module (ISIM) does not grant easy access to the instruments after integration. Nonetheless there are possible repairs that one can consider such as repairing micrometeorite damage on the sunshield. The cold Region 1 of the ISIM includes the optics and the detectors and it is very hard or

8


impossible to work on those (see Figure 2). However, the instrument electronics is for the most part in the intermediate temperature Region 2 and it's more easily accessible. The spacecraft bus resides on the warm side (Region 3) and is also more easily accessible than the ISIM.
Region 1: Science Instru ment Op tics Assemb lies Nea r In frared Camera (NIR Cam) Nea r In frared Spectrograph (NIRSpec) Mid Infrared Instrument (MIR I) Fine Guidance Sensor w/Tunable Filter (FGS /TF) Optical Bench Structure Radiators and support s truc ture (NGST-supplied) Region 2: ISIM Electronics Compa rtment Focal P lane Electronics (FPE) Instrument Control Electronics (ICE, MCE ) ISIM Remote Services Unit (IRSU) Region 3 ISIM Command & Data Hand ling (C&DH ) Electronics

Figure 2: An illustration of the James Webb Space Tel escope thermal regions. Th e sunshield lies between Regions 2 and 3.

Lifetime extension is a very promising opportunity. JWST will be placed on a biased Sun-Earth L2 orbit that requires regular maintenance burns to keep the observatory on the desired trajectory. After orbit insertion, these burns are the main driver of fuel consumption for JWST. The spacecraft would abandon the meta-stable L2 locale in a few months if its orbit were not properly maintained. The fuel tank is sized to enable the 10year lifetime goal. Assuming the observatory is fully functional (or repairable) at the end of its nominal 10-year lifetime one could, in principle, refuel the telescope for an extended mission. Refueling in low earth orbit has been demonstrated in 2007 by the DARPA Orbital Express spacecrafts (Figure 3). In addition to refueling, a lifetime extension mission could include repairs, if needed. An extended lifetime would enable additional science. JWST with an external occulter (a.k.a. starshade, see Figure 4) would be capable of finding terrestrial planets in the habitable zone of nearby stars (~ 10 pc). With low-resolution spectra, it can assess planet habitability (by searching for water vapor and a sizeable atmospheric column density), and search for biomarkers (e.g., molecular oxygen, ozone, methane, carbon dioxide). It can characterize known radial velocity planets with imaging and spectroscopy, and has the capability to detect exozodiacal disks as faint as our own solar system zodiacal disk. Although JWST was not initially designed for observations with a starshade, its large diameter and versatile instruments are appropriate for these science goals (wavelength range, available filters, spectral resolution, sensitivity). Without the starshade, JWST would not be able to perform the above observations.

9


Starshade missions are typically limited by the amount of fuel needed to maneuver the occulter spacecraft from target to target. While a starshade matched to JWST could accomplish many exoplanet research objectives at significantly lower cost than other starshade/telescope/coronagraph combinations, the starshade itself carries considerable mission risk if JWST only lasts for its minimum 5-year lifetime. Even if a starshade could be ready for launch in 2016-2017, a JWST that launches in 2014 and then lives for only 5 years would result in a limited starshade mission science return. Extending the lifetime of JWST would offer a comfortable window of opportunity to build, launch, and operate a starshade with JWST. An extended JWST life would also dramatically extend the number of stars that could be observed with the starshade because of the overhead associated with starshade maneuvers.

Figure 3: Illustration of the Orbitral Express Astro and NextSat rendezvous.

A starshade with JWST may be the only avenue to obtain a direct image of a terrestrial planet, characterize its habitability and search for evidence of life in the 2020-2030 time frame under realistic budget constraints. In a scenario where all of JWST's science instruments have failed (or have become seriously degraded) but where its main optics and actuators are still functional, a more extreme option, both in cost and complexity, would be to create a Nasmyth focus for JWST by connecting a 45-degree mirror to the tertiary mirror support. This mirror would need to be actuated for alignment reasons. The best location for the Nasmyth instruments would be close to the sunshield to avoid tightening the field of regard restrictions. The instrument at the Nasmyth would need to include a steering function and could reside on booms connected to Region 2 or to the ISIM structure. The fact that the booms would not need to withstand the acceleration and vibrations of launch would reduce their mechanical requirements.

10


Figure 4: A starshade shado ws J WST fro m the star, while light from a terrestrial exoplanet passes the edge o f the starshad e (Fro m Cash et al . 2009 submission to the Astro2010 survey "The New Worlds Probe: A starshad e with J WST").

2.3 The Virtual Mountain Top: Servicing Space Observatories Beyond JWST The most exciting frontiers in space astronomy in the next two decades will involve resolving sources on or below angular scales of 20 milli-arcseconds, obtaining very high resolution spectra on similarly small angular scales, and obtaining photometry and spectroscopy of nanoJansky sources across the 0.1!2.5 micron wavelength range. These challenging requirements demand large apertures in space - at least 8 meters in diameter and, by the 2030 era, up to 25 to 30 meters. Fortunately, the technology needs of several communities are aligning to make large space-based optical systems feasible and affordable. Yet even with new cost-breaking technologies, future large space-based observatories will be too costly and valuable to dispose of after a nominal 5-year lifetime. To recognize the full potential of these large public investments, they should be designed to be serviceable and upgradeable. Indeed, a paradigm shift in the way observational space science is done is in order. We ultimately need to operate orbital platforms for astronomy the same way we operate mountaintop observatories ­ as facilities where new scientific instrumentation can be deployed without having to provide new primary optics, power, attitude control, thermal control, and data handling systems each time. In this context, architectures that enable in-space servicing are most compatible with the long-range goal of building larger and more capable observatories in the future. We highlight here some potential advantages to mission design and cost (i.e., partial answers to Q2, Q3, Q4, and Q6 on page 3). There are four specific examples of on-orbit servicing that will benefit the next generation of large UVOIR space telescopes: 1. Replenishment of expendables. The two most familiar expendable commodities are fuel for maneuvering or station-keeping, and cryogenic fluids or solids for cooling. Refilling the tank periodically will extend mission life. 2. Replacement of limited-lifetime hardware. Many of the "utilities", such as solar arrays, batteries, gyroscopes and reaction wheels tend to wear out after several years of operation in the space environment. Their demise may be predictable, or can be anticipated by observing a deterioration of their performance. Replacing them with fresh units restores the full capabilities.

11


3. Replacement of degraded or failed components. Components may also fail because of isolated events such as radiation damage to electronics, failure of a solder joint or printed circuit, particulate interference with a mechanism or contamination of a thermal control surface. Again, replacing the troublesome items may restore full performance. 4. Upgrading with newer technology. Large space observatories are conceived many years before being built. Many of the critical subsystems contain technologies that are considered adequate and low risk now, but whose performance will be surpassed (even before launch!) by subsequent generations of devices. Examples of opportunities for on-orbit upgrades include entire instruments, focal plane arrays, cryo-coolers, guidance sensors, computers and memory modules.

Removable instruments in rear science module b ay

Figure 5: Concept for the design of a serviceable larg e space telescope taken fro m the ATLAST concept study. The details sho wn here are fro m the ATLAST monolithic 8-meter teles cope design. LEFT: Accessible science instrument bay located at the rear of the OTA. RI GHT: Servicing spacecraft replacing a scien ce instrument module. Individual instruments can be removed for rep air or replacement. Th e entire module can be replaced fo r major upgrades.

Figure 6: Orbital Replacement Units (ORUs) in a s erviceable spacecraft bus contain all key flight avionics and communications subsystems. Each compart ment can be remov ed for repair or replacement by robot or astronaut .

Servicing will be most productive and cost-effective if it is part of the design philosophy from the beginning (e.g., Figures 5 and 6). To enable future servicing, either by human crews or by robotic agents, the interfaces between the service provider and recipient need to be defined, documented and agreed to, and built into the system from the start. 12


Designing for serviceability will have benefits during the pre-launch development and testing. The system will feature modularity, accessibility and clean interfaces. A system designed to be a cooperative recipient of servicing will have clear markings, navigation and metrology aids, hand-holds and safe attachment points. In addition, it will also be desirable to have externally accessible connections for data, power, and fluid couplings with servicing spacecraft, and a modular system design to facilitate on-orbit replacement of spacecraft subsystems. While intended to enable in-orbit replacement, this partitioning will also provide flexibility in procurement strategies, assembly sequences and systemlevel integration and test. Modularity has several facets: (a) subsystems that may need replacement should be easily accessible; (b) geometrically it must be possible to easily insert and remove components; (c) fasteners should not fuse to any of the substrates that they are meant to join after deployment on orbit; (d) the heads of screws should not strip during removal; (e) fasteners should remain connected to one of the substrates after they are loosened from the other. Since many space observatory components are similar in nature to those for human space exploration vehicles, such modularity can benefit multiple architectures. As has been demonstrated in servicing of the Hubble Space Telescope, spacecraft components that are not specifically designed to be serviced on orbit require a tremendous amount of planning and effort when they eventually do need servicing. By properly designing spacecraft from modular components, the manipulation requirements for servicing become much lower. They can be reduced to the point where teleoperation (even with significant time delays) becomes feasible, as does autonomous robotic replacement of modules. Furthermore, an observatory that is serviceable in space may be simpler to design, build and test. For example, it may be acceptable for expendable resources that will be replenished to have less margin to begin with. Subsystems that will be replaced after n years may use a different approach to reliability and redundancy than those required to last 2*n years. Single string systems may be acceptable in cases where normal practice would require dual string architectures. The assurance of servicing after launch may enable savings in mass, complexity, testing, development time and cost for high-value, long-lived space observatories (c.f., Q2, Q3 on page 3).
2.3.1 Additional Benefits to Servici ng Future Spa ce Observatories

In an era when only one major flagship mission is flown each decade, opting to fly unserviceable observatories will mean that there will likely be intervals of 20 years or more where access to specific wavelength regimes will not be available with the sensitivity limits, angular resolution, PSF stability, or field of view enabled exclusively by a large space-based astronomical observatory. Such long gaps can stagnate key areas of astrophysics. Designing flagship observatories that are serviceable ensures near continuous access to the full spectrum of data needed for the full interpretation of astrophysical phenomena.

13


A suite of serviceable space-based observatories does not preclude new "small mission" scientific investigations. A balanced science instrument development program would ideally allow investigators weigh the merits of proposing a new small mission versus proposing the construction of a new instrument for an existing flagship facility. In some cases, the higher sensitivity or resolution provided by a flagship facility will enable far better science within the context of a specialized field of study than could be achieved by a smaller explorer class mission. 2.4 An ISS Observatory Testbed In this section we present a more specific answer to question Q6 on page 3: How can the International Space Station (ISS) be utilized to further the development of technologies ultimately needed to deploy, build, and service the next generation of large space telescopes? The ISS provides a unique platform for an externally mounted active optics telescope and camera system to test and mature the enabling technologies for future highperformance optical systems. We hereafter refer to such a facility as the ISS Observatory (ISSO). From its vantage point on the station, the system can image astronomical objects as well as the Earth surface; making it an ideal testbed for future instrument concepts and measurement techniques. The objectives we envision for the ISSO are: · · · · Advance the TRL of active lightweight mirrors, laser metrology trusses and wavefront sensing and control technologies. Characterize active mirror performance and validate system and component models on-orbit. Provide an upgradable facility to the ISS that may be used to test and demonstrate future instruments and measurement techniques. Test servicing and assembly methods, software, and technologies, especially if coupled to an advanced robotic manipulator.

The main components of the ISSO (Figure 7) are an actuated hybrid mirror (AHM), a wavefront sensing and control system, a laser truss to maintain high optical performance during long exposures, and a focal plane detector. Additionally, the telescope will be on a gimbaled mount for pointing control. After the observatory has been successfully mounted on and connected to the ISS and after it undergoes basic commissioning, it can begin its technology demonstration activities. In its technology demonstration phase, the ISSO would collect images to test and characterize system performance and validate different wavefront sensing and control techniques. The ISSO as envisioned here would allow one to fully characterize a) the onorbit mirror actuator responses, b) the thermo-optical response of the system using wavefront sensing, c) the system point spread function using stellar targets, and d) ground-resolved distance using established resolution-targets on the Earth3. In addition, the ISSO will allow focus shifts to be correlated with environmental loading and laser truss tracking and will demonstrate wavefront correction limits using established methods (e.g., stellar and extended scene Shack-Hartmann tests; stellar focus diversity, etc.).
3

For example, the Edge-Targets at Stennis Space Center 14


These ISSO experiments will fully validate the system performance and stability. They will also validate the models and modeling approaches used to predict the performance of the system. Taking measurements of stars and of resolution targets provides independent verification of the wavefront sensing and control performance. Ultimately it is the characterization of the wavefront performance that will demonstrate the viability of AHM technologies for future space missions.

Figure 7: The ISS Observatory concept.

The ISSO will initially employ a visible band (450 ­ 900 nm) camera system with a 1arcmin field of view. While the focus of the concept is for technology demonstration, the ISSO also has potential for astronomical and Earth science applications. In its science operations mode, the pointing system can track out vehicle motion to maintain fixedpoint imaging, which enables multi-frame imaging over periods of several minutes. When continuously observing a fixed point on the Earth during a pass, the images will be at a high spatial resolution and come from a variety of elevation angles. Since the ISS is not sun-synchronous, these data sets can be collects at a variety of Sun angles. Potential science applications include: Astronomy Near-Earth Object Discovery and Characterization High Speed Stellar Photometry Precision Stellar Photometry High-contrast imaging UV emission line surveys (requires UV sensitive detector) Observe natural and man-made events and disasters Use multi-angle images to recover atmospheric model parameters Use multi-angle images to recover high-resolution elevation maps Observe short-term dynamical characteristics of Mars, Jupiter and Saturn atmospheres

Earth Science Planetary Science

Beyond the baseline instrument, we envision the back-end of ISSO supporting one or more additional instruments that can be swapped in over the facilities lifetime. This flexibility can be used to demonstrate new human or robotic servicing techniques and 15


systems in a space environment while also helping to prove out new instruments and measurement techniques. 3. Enabling Technologies Two of the key technology areas that need to be matured in order to realize large-aperture serviceable telescopes, such as ATLAST, are active optical systems (needed for realizing large, lightweight segmented apertures) and software environments and computer-aided vision systems for remote manipulator systems operating with a time delay (as would be the case for robots operated by astronauts or ground personnel at a distance). The latter technology, in particular, directly addresses Q6 (page 3) as it is a core technology that enables servicing for multiple architectures. Many of these technologies can be tested and flight qualified either using existing facilities (e.g., the International Space Station) or as part of the near future plans for human space exploration at ever increasing distances from the Earth. 3.1 Active Optical Systems for Large Space Telescopes The ISSO (section 2.4) provides an example of a servicing technology test platform that makes use of our existing human space flight infrastructure. We describe here the three key telescope system technologies that the ISSO is initially designed to test and validate: active lightweight mirrors, advanced wavefront sensing and control systems, and a laser truss.
3.1.1 Active Lightweight Mirrors

Actuated hybrid mirrors (AHM) can be manufactured four times faster than conventional means at only one third of the mass. The highly controllable mirror figure relaxes the need for precise optical fabrication and alignments while ensuring even strictest requirements can be achieved and maintained throughout the mission. For both conventional missions as well as the post-launch assembled concepts, this technology enables a much shorter integration and test phase of a telescope implementation, greatly reducing the overall program expense. AHM technology is well suited to the construction of very large segmented aperture telescopes and, hence, the ISSO testbed is an excellent example of existing human space flight infrastructure being used to enable a key future space observatory technology. JPL has recently sponsored the successful development of two active lightweight mirror technologies at sizes > 1 meter that are appropriate for NASA space telescopes: AHMs, which utilize replicated nanolaminate metal foils bonded to a near net shape silicon carbide substrate with embedded solid-state actuators; and glass mirrors, with force actuators supporting an ultra-lightweighted ultra-low expansion (ULE) glass facesheet. Both the mirror technologies have demonstrated that they can meet launch load environments, with a mass areal density of between 10 and 15 kg/m2. Both have sufficient correctability that they could easily compensate for a Hubble-class optical error (reducing risks associated with prelaunch mirror figure testing). And both can be manufactured at production rates of 2 months/segment. The AHMs were developed by 16


Northrup Grumman, Xinetics, LLNL and JPL. The ULE mirrors were developed by ITT, working with JPL. The current technology readiness level (TRL) for these mirrors is 6 and 5, respectively. The AHM technologies are being pursued through f of our ongoing effort; smaller AHM segments have to qualification levels at JPL. There is the potential class AHM and its associated control electronics for ull space qualification at JPL as part already been environmentally tested to use an existing residual 0.5 meterthe ISSO (Figure 8).

3.1.2 Wavefront Sensing and Control Systems

Active optics requires that the optical performance of the telescope must be precisely measured while on orbit. To do this, JPL has developed Wavefront Sensing and Control (WFSC) techniques, which utilize images and spectra made by the telescope cameras to determine figure, phasing and alignment errors, and which compute the necessary actuator motions that correct them in a closed-loop fashion. One of these sensing techniques is called prescription retrieval, which was first used by JPL to successfully diagnose the error in the HST primary mirror (Redding et al. 1993). It involves taking and analyzing a series of point source (star) images as the telescope is scanned through nominal focus and across the field. JPL developed many WF control techniques as the original WFSC architects for the NGST/JWST project in support of NASA GSFC (Redding et al. 2002). They have been demonstrated at TRL 6 on more than 17 platforms, including the Keck II telescope, the TPF High Contrast Imaging Testbed, the JWST/JPL/GSFC WFC Testbed (Figure 9), and the JWST/Ball Aerospace Testbed Telescope. JPL was awarded the 2007 NASA Software of the Year Award for its imagebased WF sensing software. From stellar measurements, ISSO establishes and maintains an excellent system wavefront using its science camera as the sensor.

Figure 8: (Left) Front-side of a 49cm AHM and (Right) Si C backplane with embedded actuators. 3.1.3 Laser Truss

To achieve and preserve high optical performance during long-duration scientific observations, the JPL Space Interferometry Mission (SIM) developed extremely highprecision Laser Metrology technology (Laskin 2006). The SIM Laser Distance Gauges (LDGs) provides 10 picometer accuracy over a 10 m range in their relative mode; in their absolute mode they provide cm-to-micron dynamic range. The SIM laser truss system is operational on the 9 m SIM testbed and was demonstrated at TRL 6 two years ago. Key 17


elements were further matured to brass-board, prototype, or engineering model levels. JPL has continued to develop LDG technology for application to space telescopes, exploiting the basic principle shown in Figure 10. The accuracy required for a segmented primary mirrors is only at the nanometer level, compared to the 10 picometer performance required for SIM, enabling use of much smaller components. The result is an extremely lightweight, nanometer-accuracy laser truss, capable of continuously measuring all rigid-body motions of the optics without reducing observational efficiency of the telescope. As a proof of concept, the ISSO will demonstrate the laser truss principles and coarse performance by maintaining the secondary mirror alignment during its operation.

Figure 9: Th e NGST/J WST Wav efront Control Test bed is shown in a 3-seg ment configuration (a). Defocused measurements of a point-source/star (b) enable the recov ery of a map of the system optical path differen ces (c). This map of errors is used to minimize the system wav efront error that is rev ealed by a s econd round of measurements (bottom ro w) and con firmed by examining the in-focus star measurements (d ).

Figure 10: Sch ematic of a 2-dimensional laser truss, showing 3 measurements L that can be used to determine 3 rigid-body degrees o f freedo m dx , dz an d ". 3.1.4 ISSO as the First Step o n a Flexible Path

Ultimately, the technologies, tools and techniques demonstrated through ISSO will carry forward to larger scale systems. A potential follow-on could be a 3-segment facility where the primary mirror is assembled on station. With the post-launch assembly achieving only mechanical tolerances, the WFSC systems will be exercised to achieve precision alignment and figuring of the telescope segments using only stellar observations. There is no need to employ large-scale test optics to verify performance. The completed system would then be boosted to a higher orbit to enter its science operations phase.

18


This new path for large space telescope construction breaks through the fundamental scaling limitation in the current paradigm. As such, we can provide yet another suite of answers to Q3. By relocating the final stages of I&T to an on-orbit facility, many requirements are transferred into much more tractable ones on the small, separate subsystem modules. The designs themselves no longer have to consider functioning in 1-g environments. More minimalistic, active structures may be built to take full advantage of the fact that they will have never need to operate on Earth. Finally, the optical testing of the large apertures becomes a much more straightforward exercise. A major challenge with the current approach is that with growing aperture sizes (such as on JWST) there is an equal growth in the scale, complexity and cost of the test-optics, fixtures and facilities needed to verify system performance prior to launch. A telescope that is born in space can be tested using only itself, its instruments and a star for calibration. 3.2 Remote Manipulation With Time Delay Whether one services or assembles an astronomical satellite with humans or robots will depend on the complexity of the task, the time available to perform the task, and the location of the work site. As more and more astronomical satellites choose high Earth or Sun-Earth L2 orbits as their optimal environment, it is clear that robots controlled remotely by humans can provide an optimal solution for a broad range of in-situ servicing and assembly functions at these more distant locales. Spacecraft repair and servicing inherently requires manipulation, and thus necessitates contact between a robot and the spacecraft. Purely autonomous robotic manipulation systems typically fail unless the environment is carefully designed for robotic servicing. In the case of most existing spacecraft, we need to design robotic manipulation systems that can work with environments not originally designed for robotic servicing. In this section, we provide some more explicit answers to Q6 as we identify the algorithms and 3D visualization capabilities needed for in-orbit telerobotic manipulation and servicing of space hardware. The software environment and toolkits identified in the following subsections have the potential to work under a variety of architectures but they do not yet exist in its spacequalified form. The ISS and ISSO could prove to be ideal laboratories to test and validate some of the key algorithms and mechanical components of such a system.
3.2.1 Information-rich i mmersive softwa re enviro nment for planning a nd executi ng remote fine telema nipulation tasks

In telemanipulation systems, a human operator controls a remote robot using monocular or stereoscopic visualization, and sometimes force (haptic), feedback. We recommend the use of telemanipulation for in-orbit servicing of spacecraft and contend that technology and methods developed for other applications, principally telesurgery, can be applied. In telesurgery, JHU's robotics department has developed the Surgical Assistant Workstation (SAW) open-source software framework (Vagvolgyi et al. 2008), which is an information-rich immersive environment for planning and executing telesurgical procedures, such as those performed with the da Vinci® robot system (Intuitive Surgical, Inc., Sunnyvale, CA).

19


The underlying principle of SAW is that the telemanipulation master console should be used not only to control the remote robots, but also to provide a three-dimensional (3D) interface that enables the user to interact with other sources of information and to provide high-level control parameters to the system. To support this, the system provides a "masters as mice" mode, where the master manipulators are (temporarily) disconnected from the remote manipulators and used as 3D input devices to manipulate graphical objects in the 3D environment. For example, in a surgical application, the user may wish to view a 3D model (such as a preoperative CT or MRI scan) of the patient. This model could appear in a small "3D window" within the field of view, and can be overlaid on the stereo video. The latter case requires a registration between the coordinate systems of the 3D model and the stereo video (Su et al. 2009). In a spacecraft servicing application, the 3D model could consist of the CAD model of the spacecraft. By registering and overlaying it on the stereo video, it is possible to give the user "x-ray vision" to see the internal structure of the spacecraft in much the same way that a CT or MRI scan enables the surgeon to see the internal anatomy of the patient. The "masters as mice" mode also enables the user to provide high-level control parameters, such as the definition of virtual fixtures (further discussed in section 3.2.4 below). As a simple example, the user can define safety boundaries (boundary virtual fixtures) in the 3D view, which are then enforced by the system during telemanipulation. This is especially useful when there are time delays because the virtual fixture definitions can be uploaded to the in-orbit robot control system and therefore not subject to the effects of time delay.
3.2.2 Virtual a nd physical si mulation environments for servici ng telemani pulation operational plan dev elopment, training, and ex ecution

Planning and training for robotic servicing of spacecraft will require both real (neutral buoyancy facility) simulators and virtual (computer simulation) simulators for development of operational plans, personnel training, and online support of advanced telemanipulation algorithms. The development of such computer simulations will require significant advances in the modeling and simulation of complex multi-body interactions (Son et al. 2004) and the modeling of manipulation grasping mechanics (Ciocarlie et al. 2009). A natural next step would be to augment the SAW architecture with an advanced simulation capability. In this simulation system, it will be possible to instantiate physical models of in-orbit hardware using prior CAD data and physical parameters. These simulations will be dynamically synchronized with in-orbit hardware through computational vision as further discussed in section 3.2.6. Physical models of this type are familiar to use as we use them extensively in our recent work in sampling-based planning (Plaku et al. 2010).
3.2.3 Fine telemani pulation with ti me delay through predictive model-based telemanipulation

Teleoperation enables remote robotic manipulation, while circumventing the need for autonomous robotic behaviors. However, large time delays results in a disconnect between action and reactions, requiring human operators to adopt a move-and-wait strategy. This not only increases operation time, but can also result in major errors if an overzealous "move" or too short "wait" causes an unexpected error. In addition, large 20


time delays make it difficult to provide a remote sense of touch (haptic or force feedback) to the human operator. Researchers have shown in many manipulation contexts that haptic feedback improves teleoperation performance, but haptic feedback is unstable under large time delays. One way to enable more natural teleoperation (with or without haptic feedback) under large time delays is model-based teleoperation, which uses models of the environment acquired from models developed a priori and updated in real time during manipulation (Hirzinger et al. 2009; Sheridan et al. 1995; Sayers et al. 1995). Preliminary work has shown that model-based teleoperation with haptic feedback improves user performance under very simple conditions with delays of up to 4 seconds (Mitra et al. 2008), but manipulation has not yet been accomplished under this paradigm. For reference, the round-trip light travel time between Earth and SE-L2 is 10 seconds; the round-trip travel time between Earth and the moon is 2.5 seconds. We recommend development of a model-based teleoperation control system that is seeded with an a priori model of the manipulator, the environment (the spacecraft to be serviced), and the specific task to be performed. The human operator performs a small action (sub-task) in the virtual environment (Funda et al. 1992; Tzafestas et al. 2001), and the remote robot manipulator attempts through local autonomous control to replicate that sub-task. While the remote manipulator is performing the sub-task, sensors are used to acquire an updated model (Dupont et al. 1999), which is then transmitted to the human operator. An important aspect of this research will be the characterization of appropriate sub-tasks that can be accomplished safely by an autonomous manipulator and used to acquire useful data for model updates.
3.2.4 Semi-a uto mated telema nipulation behaviors fo r safe remo te manipulation

Haptic virtual fixtures are software-generated force and position signals applied to human operators via robotic devices (Bettini et al. 2004; Abbott et al. 2007). Virtual fixtures help humans perform robot-assisted manipulation tasks by limiting movement into restricted regions and/or influencing movement along desired paths. By capitalizing on the accuracy of robotic systems, while maintaining a degree of operator control, humanmachine systems with virtual fixtures can achieve safer and faster operation (Abbott et al. 2007b). Although often thought of as operating within the context of haptic feedback, virtual fixtures can be developed that combine multiple sources of sensory feedback, such as computer vision (Bettini et al. 2004; Kumar et al. 2000; Kapoor et al. 2008) and extremely complex task and environment models (Li et al. 2006). Since they fundamentally involve humans sharing control of the robot with a computer, virtual fixtures can also be considered as a path toward autonomy. As models of the environment and/or task improve, the virtual fixtures may increase their constraint over the teleoperation, eventually to the extent that that the system is operating autonomously. Alternatively, the human operator may position the robot's end effector and then explicitly invoke a separate function (such as turning a screw) under sensor control. We recommend further investigation and development of the use of virtual fixtures is a promising way to both improve human performance of model-based teleoperation (see 2.1.1) and develop semi-autonomous behaviors.

21


3.2.5 Semi-a uto mated ma nipulation " macros" for sa fe perfo rmance o f fi ne telema nipulation tasks in the presence of ti me delay

Carrying out repetitive fine-manipulation tasks (e.g., removing 111 screws on an access panel) is challenging for a ground-based operator. We recommend the development of methods for support semi-automated execution of commonly occurring and/or repetitive tasks. It is generally well accepted that human fine manipulation makes use of a relatively small and constrained set of hand postures and hand motions for manipulation (Klatzky et al. 1990; Thakur et al. 2008). Focused manipulation tasks can be modeled using the timeseries methods developed for speech and language processing (Lin et al. 2006; Varadarajan et al. 2009). As a result, it is possible to view complex manipulation tasks as a combination of a small set of motion primitives, much as speech is generated from a small set of phonemes and phonetic variants. In most cases, these motion primitives can be tied to specific task objectives (e.g. gross motions for transport, fine motions for positioning, and force primitives for loosening, tightening, and so forth). Thus, the problem of performing a semi-automated manipulation task can be posed in terms of finding the appropriate combination of motion primitives, with the choice based on the immediate objectives of the task at hand. We further recommend the development of a system for semi-automated manipulation by incorporating these results into a recently developed hybrid motion planning methodology (Plaku et al. 2007, Plaku et al. 2008). The planning system will employ motion primitive-based models learned from expert task performance to develop a discrete, high-level plan sketch. This high-level motion plan will be refined into a control input using a sampling-based planning that uses the discrete plan as a heuristic (Plaku et al. 2010). This system will be integrated within a model-based teleoperation environment that will run both a ground-based dynamic simulator and an in-orbit operational (Hirzinger et al. 1994; Hirzinger et al. 2009). The system will monitor the motion of the ground-based system (driven by the human operator) and "parse" the operator motions into the underlying discrete manipulation gesture language. These gestures will be supplied as hints to the planner, which will operate in-orbit to continually refine and carry out a motion plan. If the motion planner is unable to succeed in finding a solution, or if the computed plan significantly contradicts the input direct control be turned over to the operator until automated planning again succeeds.
3.2.6 Real-ti me co mputer vision scene understa nding and tra cking of co mplex telemanipulation enviro nments

Much of the work described above depends on accurate synchronization of earth-based simulations with in-orbit activities. We recommend the use of visual tracking to provide that synchronization. There have been numerous recent advances for tracking in environments containing structured, known objects (Yilmaz et al. 2006). In particular, our group has developed several techniques that are capable of real-time visual tracking known articulated structures such as robot manipulators (Hager et al. 1998; Pezzementi et al. 2009; Rother et al. 2009). Other groups have developed techniques for tracking rigid objects that are based on the detection of specific structures using optical intensity or range information (Yilmaz et al. 2006). We further recommend the development and combination of these methods to create an orbital scene understanding methodology. The proposed methods will provide a rapid framework for "learning" about in-orbit structures

22


using both manual and automated techniques, and methods for "compiling" this information into an online scene understanding and tracking system. Specifically, we will assume that prior CAD models of most in-orbit hardware will be available prior to encounter. Upon first contact, an optical survey of the in-orbit hardware will be performed. In a video record of the survey, stable visual landmarks will be detected using recently developed vision algorithms such as SIFT (Lowe et al. 2004), SURF (Bay et al. 2006) and related methods (Mikolajczyk et al. 2004). These visual cues will be manually related to the CAD model, providing a known image-to-model relationship that can be later used to detect and track object pose. During operations, these visual cues will be redetected and matched to the cue library. Robust regression methods will be used to compute a reliable object pose (Wang et al. 2008). Further, we will make use of manipulator tracking (either through kinematics or through video tracking) to account for occlusions in the scene due to manipulation. The result will be a sequential scene understanding system that is rapidly customizable to a specific application, and which is highly robust to complex interactions in the viewed scene. The advanced semi-automated manipulation "macros," environmental feedback algorithms, and dynamic 3D visualization utilities described above will enable robots, under human guidance from afar, to perform many complex servicing and assembly tasks in space. This capability will be critical to broaden the reach of such missions to many orbital locations and will also be of use in a broad range of human spaceflight activities (e.g., robotic exploration of NEOs or Martian moons guided by nearby astronauts). These capabilities are not strongly architecture specific. But they do need to be developed, tested, and flight qualified. The ISS is a good initial platform for this technology development to be performed. REFERENCES
Abbott, J.J. and Okamura, A.M., " Pseudo-admittance Bilater al Teleman ipulation with Gu idance V irtu al Fixtures", The Intern ational Journal of Robotics Research , V ol 26, pp. 865-884, 2007. http://articleworks.cadmus.com/doc/761015 Abbott, J.J., Mar ayong, P., and Okamur a, A.M., "H aptic Vir tual Fixtur es for Robot-Assisted Manipulation," in Robotics Resear ch, Spr inger Tracts in Adv anced Robotics, Vol. 28 (Proceed ings of the 12 th International Symposium of Robo tics Research, 2005), S. Thrun, H . Durran t-Whyte, and R. Brooks, Eds., 2007, pp. 49-64. [Abott et al. 2007b] Bay , H., Tuytelaars, T., and V an Gool, L., "Surf: Sp eeded up robust featur es. Lecture no tes in computer science," 3951:404, 2006. Bettini, A ., Marayong, P., Lang, S., Okamura, A.M., and H ager, G .D., "V ision A ssisted Control for Manipulation Using Vir tual Fixtur es," IEEE Transactions on Robotics, Vo l. 20, No . 6, pp. 953-966, 2004. Cio car lie, M. and A llen, P., Design and Analysis Too l for U nderactuated Comp liant Hands , I EEE/RSJ International Conferen ce on Intellig ent Robots and Systems, October 11-15, 2009 St. Lou is, pp. 52345239. Dupont, P. E.; Schu lteis, T. M.; Millman, P. A .; Howe, R. D., 1999, "Automatic id entification of h aptic environmen t properties", Presence, Vol. 8, No. 4 , pp. 394-411. Funda, J., Lindsay, T.S., and Paul, R.P., "Teleprogramming: toward delay- invariant remo te manipulation", Presence: Teleoperators and Virtual Environmen ts, vol. 1, p p. 29-44, 1992. Hager, G. and Toyama, K . The XVision System: A G ener al- Purpose Substr ate for Por table Real-Time V ision Applications (w ith K. Toyama). In Computer Vision and Ima ge Understanding 69(1) pp. 23 ­ 37, 1998.

23


Hirzinger, G., Brunner, B., D ietrich, J., and Heindl, J., "ROTEX - The First Remotely Con trolled Robot in Space." ICRA 1994: 2604-2611. Kapoor, A. and Tay lor, R., "A Constrained Op timization Ap proach to Vir tual Fixtures for Multi-Handed Tasks" , in IEEE In ternational Conference on Robotics and A utomation (I CRA), Pasadena, 2008, pp . 3401-3406. Klatzky, R.L. and Lederman, SJ and Pellegrino, JW and Doh erty, S. and McClosky, BP Procedures for haptic object exploration vs. man ipulation In Vision and action: The con trol of gr asping, M.A . Goodale, Ed . pp. 110--127 1990. Kumar, R., Hager , G .D., Barnes, A., Jensen, P., and Taylor, R.H ., "An Augmentation System for Fin e Manipulation", in Medical Image Co mputing and Computer- Assisted Interv entions ( MICCAI), Pittsburgh, 2000, pp. 956-965. Lask in, R.A., "Su ccessful co mpletion of SI M-PlanetQuest technology," SPIE 6268, 23 (and ref eren ces therein) (http://p lan etquest.jp l.nasa.gov/do cumen ts/SPIE_06- rev6_small.pdf) (2006). Li, M., Ish ii, M., and Taylor, R.H., "Spatial Motion Constr aints in Med ical Robot Using V irtual Fix tures Generated by Anatomy," IEEE Transactions on Robotics, vo l. 2 , pp. 1270-1275, 2006. Lin, H.C., Sh afran, I., Yuh, D ., and H ager, G.D ., "Towards Automatic Sk ill Evaluation: Detection and Segmen tation of Robot-A ssisted Surgical Mo tions," Computer Aided Surgery, 11(5):220-230, September 2006. Lowe, D., "Distinctive imag e featur es from scale-inv arian t k eypoints. International Journal of Compu ter Vision ," 60(2):91­110, 2004. Mikolajczyk, K . and Schmid t, C., "Scale & affin e invariant interest po int detectors. Intern ational Journal of Computer V ision ," 60(1):63­86, 2004. Mitr a, P., and N iemey er, G., 2008, "Mod el-mediated Telemanipulation", In ternation al Journal of Robo tics Resear ch, Vol. 27, No . 2, 253-262. DOI: 10 .1177/0278364907084590 Pezzementi, Z., Voros, S., and H ager, G.D ., "Ar ticulated Object Tr ack ing by Render ing Consisten t Appearance Parts," in International Conferen ce on Robotics and Automation (ICRA), May 2009. Plaku, E., Kavrak i, L.E., and Vardi, M.Y., "D iscr ete Sear ch Lead ing Continuous Exploration for Kinodynamic Mo tion Planning" Robotics: Science and Systems, Atlan ta, Georg ia, 2007, MIT Press, pp. 326--333. Plaku, E., Kavrak i, L.E. and V ardi, M.Y ., "Impact of Workspace Decompositions on D iscr ete Sear ch Lead ing Continuous Exploration (DSLX) Motion Planning." IEEE Intern ational Conference on Robotics and Au tomation, Pasad ena, CA, 2008, pp. 3751--3 756. Plaku, E., and H ager, G.D ., " Samp ling-based Mo tion and Sy mbolic A ction Planning with G eometric and Differen tial Constraints." I EEE International Conferen ce on Robotics and Au tomation, Anchorag e, AK, 2010, to app ear. Redd ing, D ., Du mont, P. and Yu , J., "Hubble Space Telesco pe Prescrip tion Retrieval," Applied Op tics, V.32, No.10, (1993). Redd ing, D ., Basing er, S., Coh en, D ., Green, J., Lowman, A., Ohar a, C., Shi, F., Bower s, C., Burg, R., Burns, L., D ean, B., "Next G enera tion Space Telescope Wavefron t Sensing and Control," Proc. SPI E 4 8 5 0 (2 0 0 2 ). Roth er, D . and Sap iro, G . Seeing 3D Ob jects in a Single 2D I mage. (Accep ted for publication at the Proceed ings of the 12 th IEEE Intern ational Conference on Computer V ision), 2009. Sayers, C.P., Pau l, R.P., Catipov ic, J., Whitcomb , L.L., and Yoerger, D ., "Teleprogramming for Subsea Teleoper ation Using Acoustic Communication," IEEE Journal of Ocean ic Engineering , 23(1):60-71, January 1998. h ttp ://robotics.me.jhu.edu/~llw/ps/00659450.pdf Sheridan , T.B., "Space teleoperation through time delay : r eview and prognosis," Robotics and Automation, IEEE Transactions on , vol.9 , no.5, pp.592-606, Oct 1993 U RL: http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=258052&isnumber =6545 Son, W., K im, K ., Amato, N.M., and Tr inkle, J.C., "A gen er alized framework for interactive dynamic simu lation for multi-r igid-bodies," IEEE Transactions on Sy stems, Man , and Cybern etics, 34(2):912924, April 2004. Su, L.- M., Vagvolgy i, B., Agarwal, R., Reiley, C.E., Tay lor, R.H ., and H ager, G.D ., "Augmented reality during robot-assisted laparoscopic par tial nephrectomy : Tow ard real-time 3D- CT to stereoscopic video reg istration," Journal of Uro logy, 73(4), pp. 896-900, Apr. 2009. Thakur, P., Bastian, A ., and H siao, A ., "Mu ltidig it movemen t synerg ies of th e human hand in an unconstrained hap tic exploration task," J Neurosci, 28:1271­1281, 2008.

24


Tzaf estas, C. S. (2001). Teleplanning by human demonstration for VR-based teleoper ation of a mob ile robotic assistan t. Proceedings I EEE Workshop on Robot and Human In ter-active Commun ication (RO MAN), pp. 462­467. Vagvolgyi, B., DiMaio, S., Deguet, A., K azanzid es, P., Ku mar, R., H asser, C., and Taylor, R., "The Surgical A ssistant Workstation: a sof twar e framework for telesurgical robotics r esearch ," In MI CCAI Workshop on S ystems and Arch. for Computer Assisted Interventions, (onlin e at Midas Journal, http://hdl.h andle.net/10380/1466), Sept. 2008. Varadar ajan, B., Reiley , C.E., Lin, H., Khudanpur, S., Hager, G.D., "Data-D eriv ed Models for Segmen tation with Application to Surgical Assessmen t and Train ing," MICCAI (1) 2009: 426-434. Wang, H ., Miro ta, D., Ish ii, M., and H ager, G., "Robust Motion Estimation and Stru cture Recov ery from Endoscopic Image Sequences With an Adap tiv e Scale Kern el Consensus Estimator. In Co mputer Vision and Pattern Recognition ," 2008. CV PR 2008. IEEE Conference on, pag es 1­7, 2008. Weismuller , T. and Leinz, M., ``GN&C technology demonstrated by th e orbital express autonomous rendezvous and capture sensor system,'' in Proc. 29th A AS Guidance and Con trol Conf., Breckenridge, CO , 2006, pp. 1­9. Yilmaz, A., Jav ed, O., and Shah, M., "Object Tracking: A Su rvey", A CM Journal of Compu ting Survey s, Dec 2006.

25