Документ взят из кэша поисковой машины. Адрес оригинального документа : http://www.adass.org/adass/proceedings/adass96/reprints/verkhodanov2.pdf Дата изменения: Thu Jan 15 01:14:43 1998 Дата индексирования: Tue Oct 2 16:37:19 2012 Кодировка:

Astronomical Data Analysis Software and Systems VI ASP Conference Series, Vol. 125, 1997 Gareth Hunt and H. E. Payne, eds.

Non-parametric Algorithms in Data Reduction at RATAN-600
V. S. Shergin, O. V. Verkhodanov, V. N. Chernenkov, B. L. Erukhimov, and V. L. Gorokhov Special Astrophysical observatory, Nizhnij Arkhyz, Karachaj-Cherkessia, Russia, 357147 Abstract. Non-linear and non-parametric algorithms for data averaging, smoothing and clipping in the RATAN-600 flexible astronomical data processing system are prop osed. Algorithms are based on robust methods and non-linear filters using an iterative approach to smoothing and clipping. Using robust procedures to detect faint sources is prop osed also. This detector is based on the ratio of two statistics, characterizing the noise and signal, in the given interval. These methods allow us to accelerate the process of the data reduction and to improve the signal/noise ratio. Examples of op eration of these algorithms are shown.

1.

Introduction

Obtaining a reliable result on the background from different typ es of interferences is one of the main problems for the observational astronomy. The question "what is useful signal and what is noise?" is esp ecially essential when we b egin "dumb" data processing. The ordinary way to obtain a good signal/noise ratio is to apply the standard average for vectors of data observed on the same sky strip. This way is the most optimal and realizes the maximum likelihood estimation with n improvement. But the real observational data have a distribution far from normal. This is caused by the presence of p ower spikes, "jumps," and slow trends. The source of this interference is human activity, atmosphere, and p ossible instability of the receiver. Therefore, observers prefer a manual method of data reduction, b ecause a single bad record sp oils the resulting sum when doing "dumb" standard averaging. To automate the users' procedure of data quality checking, sp ecial algorithms have b een worked out. The background problems are absent for ordinary average. But when we use the robust (non-parametric) average, the correct background subtraction (or smoothing) is the main problem. Therefore, the first problem to b e solved is to find the correct (from the viewp oint of an observer) smoothing. Moreover, the knowledge ab out the background around the sources is very imp ortant for some problems in radio astronomy. The use of standard procedures (fitting with splines) very often can not help us in this situation. Thus, sp ecial non-linear algorithms for smoothing were develop ed. Another typ e of processing where similar algorithms can b e used is data compression (when we have a surplus of p oints p er b eam and can compress the data keeping useful information) and searching for sources. 182

© Copyright 1997 Astronomical Society of the Pacific. All rights reserved.


Non-parametric Algorithms in Data Reduction at RATAN­600 2. Smoothing

183

The history of algorithms, based on the robust methods and non-linear filters using an iterative approach to smoothing and clipping, b egan twelve years ago in the data reduction at the RATAN-600. Since then they have b een further develop ed and are used in modern computer systems in the RATAN-600 data reduction system FADPS (Verkhodanov et al. 1993; Verkhodanov 1997) and in the MIDAS (Shergin et al. 1995). The algorithm uses several start parameters: noise disp ersion, smoothing interval, iteration count, and typ e of smoothing curve. Practically there is no effect on a real signal in a given interval, and the background is accurately calculated. The detailed description of the smoothing and clipping (SAC) algorithm is given in Erukhimov et al. (1990) and in Shergin et al. (1996). Briefly, the SAC algorithm consists of the following steps: ( i) calculation of an input vector C i for i-th smoothing iteration as C i = F (S, W i-1 ) where W i-1 is a vector of weights calculated in the previous iteration. In the simplest case the function F is calculated as a product of each vector comp onent Fk = Sk Wk , where k is k-th comp onent; (ii) smoothing of C i : B i = () C i , where i () is the smoothing op erator, and is the input parameter; (iii) subtraction Di = S - B i , where S is the input data vector; (iv ) calculation of a new function of weights as a non-linear transformation i ( ) which is a function of input noise and numb er of iteration in a general case: W i = i ( ) D i ; and (v ) transition to the next iteration. There are several p ossible smoothing and weighting methods. In practice we use the following methods for smoothing: simple b oxcar average; convolution with Gaussian profile (see Figure 1 in Verkhodanov 1997), median average (Erukhimov et al. 1990) (see Figure 2 in Verkhodanov 1997); weighted least squares p olynomial approximation of 1/15 . For calculation of weights some empirical transformations are used. They may b e sp ecialized to cut emission, absorption or b oth. 3. Averaging

After background subtraction the procedure of robust averaging is applied for records. Usually we do it by the Hodges-Lehmann (HL) method (Hub er 1981, Erukhimov et al. 1990) with estimation of middle value: X = med Xi + Xj 2

Robust averaging with the HL method is illustrated in Figure 1. 4. Compression

Another typ e of data reduction procedure is data compression. The result of data compression on the base of Hodges-Lehmann (Hub er 1981) estimates is illustrated in Figure 2. A similar robust algorithm was prop osed and applied in the RATAN-600 data registration system by Chernenkov (1996).


184

Shergin et al.

Figure 1.

Average of 20 records. The record at zero level is a result.

Figure 2. 5. Detection

Compression with factor 5.

Based on robust procedures, a detector for faint sources (extremal-median signal detector: EMSD) in a series of synchronous scans (Verkhodanov & Gorokhov 1995). is prop osed also. This detector is based on the ratio of two statistics, characterizing the noise and signal, in the given interval. For this algorithm several statistics are calculated: zij = mint {Di,j +t }, vij = maxt {Di,j +t }, where i = 1,m; j = 1,n; t = 1,r ; D is the matrix of m vectors, m is the numb er of scans, n is the numb er of p oints in a scan, r is the numb er of p oints in a signal search interval. Using these two extremal statistics the medians and the following statistics are computed in each interval: vj = medi vij - medi zij , zj = medi zij , where i = 1,m; j = 1,n; then their ratio is used as a test statistic: qj = vj /
z
j


Non-parametric Algorithms in Data Reduction at RATAN­600

185

Figure 3. Results of EMSD op eration for the different input intervals: g--interval of two b eam patterns, h--one b eam pattern, for real data of five synchronous scans (a­e). To compare results the scan with robust average of these data is shown (f ). The antenna temp erature is for the first six scans, and ratio (5) is for the two last scans plotted along the Y-axis. where j = 1,n. In this case the hyp othesis of ob ject detection is adapted when the quantity qj exceeds a certain quantity c. The threshold c can b e either set by user or computed in the program. The example of EMSD op eration is illustrated in Figure 3. Acknowledgments. O. Verkhodanov thanks ISF-LOGOVAZ Foundation for the travel grant, the SOC for the financial aid in the living exp enses and the LOC for the hospitality. References Chernenkov, V. N. 1996, Bull. of SAO, 41, 150 Erukhimov, B. L., Vitkovskij, V. V., & Shergin, V. S. 1990, Preprint SAO RAS, 50 Hub er, P. J. 1981, Robust Statistics (New York: Wiley) Shergin, V. S., Kniazev, A. Yu., & Lip ovetsky, V. A. 1996, Astron. Nach., 317, 95 Verkhodanov, O. V., Erukhimov, B. L., Monosov, M. L., Chernenkov, V. N., & Shergin, V. S. 1993, Bull. of SAO, 36, 132 Verkhodanov, O. V., & Gorokhov, V. L. 1995, Bull. of SAO, 39, 155 Verkhodanov, O. V. 1997, this volume, 46