The Pixon and Bayesian Estimation

To statistically model the measurement process, a number of issues must be addressed. First, there is the physics describing the measurement process. This, however, is assumed to be set down in Eq. (1). Second, a number of details related to the measurement process itself, must be properly incorporated. These include the manner in which the data was collected, e.g., as a rectangular grid of counts as is the case for astronomical imaging with a solid state detector, as well as the characteristics of the noise, etc. Finally, a number of decisions must be made regarding how Eq. (1) is to be inverted mathematically. Common assumptions here are that the image can be represented by a grid of numbers and that the integral represented in Eq. (1) can be approximated by a discrete sum over this grid. Each of these aspects of the problem, i.e., the physics, the particulars of the measurement, and the mathematical assumptions are part of the model, , which links the image and the data.

The Bayesian approach to inverting Eq. (1) is to use Bayes' Theorem to develop a formula for the most probable value of . This begins by factoring the joint probability distribution of the data, image, and model triplet, , and , i.e., :

where is the probability of given that is known. [Bayes' Theorem states that .] Equating the various terms in Eq. (2), we find

From Eq. (4) we can find the M.A.P. (Maximum A Posteriori) image using

where is the model from the M.A.P. image/model pair. (The proportionality in Eq. (5) should really be an ``approximately-proportional'' symbol since we have dropped some terms [e.g. ] and we have assumed that the peak in the probability distribution is representative of a typical sample from the posterior probability distribution. While this is usually the case, this is not absolutely guaranteed.)

Eq. (3) is the typical starting point of Bayesian image reconstruction in which one wishes to determine the M.A.P. image, i.e. the image which maximizes . (The M.A.P. image, of course, is only one of several choices for the ``best image''. Another sensible choice might be the average image, .)

Eqs. (4) and (5) are our preferred prescription for Bayesian image reconstruction in which both the image and the model are varied simultaneously to obtain the best combined image/model solution, e.g., the M.A.P. image/model pair. Eq. (4) might be used directly to find the optimal image, i.e., the image paired with the model in the M.A.P. image/model pair. Alternatively, the M.A.P. image can be found using Eq. (5).

The significance of the terms in the above equations are well known. The first term, , is a goodness-of-fit (GOF) quantity. A typical choice for is to use , where is the chi-square of the residuals. This form for the GOF statistic allows the best unbiased estimate for the image to be determined. However, it produces a value for which is uncharacteristically small (i.e., ). Another approach that we have explored is to use where is the actual chi-squared distribution. This choice for the GOF criterion recognizes that the most probable value for is not zero, but depends on the number of random variables. This GOF criterion also seems to produce quite good image reconstructions and to avoid problems associated with ``over-fitting'' the data.

The terms and are ``priors''. Since they do not depend on the data they can be decided a priori. The first of these, i.e., , is normally termed the image prior and expresses the a priori probability of an image given the model. The second, i.e., , we have termed the image/model prior, and expresses the a priori probability of both and . In GOF image reconstruction the prior is ignored or is effectively set equal to unity, i.e., there is no prior bias concerning the image or the model. In Maximum Entropy (ME) image reconstruction, the image prior is based upon ``phase space volume'' or counting arguments and the prior is expressed as , where is the entropy of the image and is an adjustable constant that is used to weight the relative importance of the GOF and image prior.

The final quantity, , is termed the ``Evidence'' for the model. Actually, we normally refer to as the Evidence, but is proportional to this quantity under the common assumption that and (there generally is not an a priori manner for choosing between valid data sets nor between ``sensible'' models.) Choosing models on the basis of the Evidence allows one to ``peak-up'' on more favorable solutions.