Äîêóìåíò âçÿò èç êýøà ïîèñêîâîé ìàøèíû. Àäðåñ îðèãèíàëüíîãî äîêóìåíòà : http://hea-www.harvard.edu/AstroStat/astro193/VLK_slides_mar30.pdf Äàòà èçìåíåíèÿ: Mon Mar 30 23:17:46 2015 Äàòà èíäåêñèðîâàíèÿ: Sun Apr 10 11:59:04 2016 Êîäèðîâêà: Ïîèñêîâûå ñëîâà: ï ï ï ï ï

Astro 193 : 2015 Mar 30
·

Follow-up
·

Gibbs sampler
· ·

The meaning of

(t-1) -j

Why all draws are accepted

· ·

Ockham's Razor Reading: Chapter 5 of Modern Statistical Methods for Astronomy, Feigelson & Babu

Non-parametric Tests

·

Signal Processing: FFT


Ockham's Razor
Numquam ponenda est pluralitas sine necessitate. (Plurality is never to be posited without necessity.) William of Ockham Entia non sunt multiplicanda praeter necessitatem (Entities must not be multiplied beyond necessity) (attr.) p(D|M1) = d p(|M1) p(D|,M1) [1/] L() = L() [/] Maximum Likelihood x Ockham Factor compare to model M2 with 1, L(1,2) [1/1][2/2] M1 is preferred unless improvement in likelihood for M overwhelms the penalty of shrunk parameter space
2 2


Non-parametric tests
·

Comparing distributions Kolmogorov-Smirnov Cramer-von Mises Anderson-Darling Mann-Whitney U test Kruskal-Wallis ANOVA Wald-Wolfowitz Higher Criticism ²

·

Correlation tests Pearson's coefficient Spearman's Kendall's


Comparing distributions
·

One-sample test X = {x, i=1..N} Is the sample drawn from a specified distribution F0(X)?

·

Two-sample test X
(1)

= {x i, i=1..N}, X
(1)

(1)

(2)

= {x j, j=1..M}

(2)

Are X
· ·

and X

(2)

drawn from the same distribution F0(X)?

Null hypothesis (H0) is that the sample is drawn from the same distribution A statistic is calculated from the data and compared to a critical threshold value which is set by requiring a certain significance level . If the statistic exceeds the threshold, then the null hypothesis is rejected. Standard p value disclaimers apply
· ·

·

you can say "no", but you can't say "yes" there is no confidence interval on p


Kolmogorov-Smirnov
·

test statistic is the supremum distance between cdfs, D = supxX|F(xi)-F0(xi)| or D = supxX|F (xi)-F (xi)|
(1) (2)

·

not very powerful, but robust, easy to implement and understand, and the test is distribution-free, i.e., the distribution of D does not depend on F() p-value, p =1 - P(Dd|H0), for d [0,1]
·

·

for N p = 1 - P(DNz|H0) = 1 - 2k=0.. (-1) exp[-2 k z ]
k 2 2

·

for d0 or d1, d 1/2N : p = 1,
N N

1/2N < d 1/N : p = 1 - N!(2d-1/N) , 1-1/N d < 1 : p = 2(1-d) ,
·

1d: p=0 Beware the Kolmogorov-Smirnov test! by Feigelson and Babu
http://asaip.psu.edu/Articles/beware-the-kolmogorov-smirnov-test


Kolmogorov-Smirnov
·

DO NOT USE Kolmogorov-Smirnov test for X categorical data, but only for variables that can be written as real numbers X discrete distributions (like Poisson) X multi-dimensional datasets X determining differences that dominate in tails of distributions X comparing data with model derived from same data. Goodness-of-fit for best-fit model will not have the standard P(Dd|H0). Compute it empirically via bootstrap or Monte Carlo simulations.


Cramer-von Mises / Anderson-Darling
·

test statistic is based on weighted square of distance between cdfs, T = N (2 2 ,)

dF(x) (x) [FN(x)-F0(x)]
2

2

T = N k=1..N (xi) [k/N - F0(xi)]
­1

(x) = 1 for Cramer-von Mises (x) = [F0(x) (1-F0(x))] for Anderson-Darling · No unique distribution P(T t|H0). Distribution of T available only for some specific F0(X) (Normal, lognormal, exponential, etc). Critical values (thresholds) must be calculated separately for each distribution. · Don't try this at home. e.g., for fully specified Normal (Anderson & Darling 1952, Ann. Math. Stat., 23, 193): p(T z|H0) = (1/z)
2 2 j=0.. 2

(-1) [(j+1/2)/(1/2)j!] (4j+1)
2

j

1/2

exp[-(4j+1) /16z] K1/4((4j+1) /16z), where K1/4(x) is the modified Bessel function · Strongly affected by ties (leads to high false rejection rates)


Mann-Whitney U / Wilcoxon Rank
·

Order the data values (min=1, max=N1+N2), compute the mean of the ranks in each subsample, and ask: what is the probability that the mean rank of two samples is as far apart as they are? Say Wk = sum of all ranks from sample k. Uk = Wk - Nk(Nk+1)/2 U = min{U1,U2} Under the null, E[U] = N1N2/2 var[U] = N1N2(N1+N2+1)/12 Sensitive to changes in the median Gehan's test generalizes to include left-censored data
·

· ·

Ranks assigned as Uij = +1 if x < x j, Uij = ­1 if x i > x j, Uij = 0 if x i=x or indeterminate, then summed over all i,j, WG = i=1..N1
j=1..N2

1 i

2

1

2

1

2 j

· ·

U

ij

var(WG) = N1N2

i=1..N1+N2

U /[(N1+N2)(N1+N2-1)]

2 i


non-parametric ² test
·

Any statistic that is distributed as the ² distribution is called a ² test ² = i (Oi - Ei)²/Ei

·

examples:
· ·

variability of a source tests for independence


Kruskal-Wallis H
· ·

Extension of Mann-Whitney/Wilcoxon to k>2 samples Test statistic is H is a normalized sum of squared differences in rank of each group H = (N-1) g=1..k ng ( r g- r ) / D D = g=1..k
j=1..ng 2

(rgj- r ) , N=g=1..k ng, = average rank in sample g

2

r g = (1/ng)

j=1..ng rgj

r = (N+1)/2 = average of all ranks
·

When there are no ties, H = [12/N(N+1)] g=1..k ng r g - 3(N+1)
2

· ·

Asymptotic distribution is ²k-1 Useful for
· ·

testing population differences in contingency tables comparing means of samples


Wald-Wolfowitz Runs test
·

Checks whether a sequence that can take on binary values is drawn from a random process. Asks whether the number of runs of one value is consistent with the expected number Test statistic Z = (R-R)/R, where R are the observed number of runs, R = 1 + 2 n1n2/(n1+n2), R = 2n1n2(2n1n2-n1-n2)/(n1+n2)2(n1+n2-1) and is distributed as N(0,1)

·


Higher Criticism
· · ·

Takes advantage of large fluctuations HC* = max Recipe:
· · ·
0<<1N

[(fraction at )-]/[(1-)]

compute p-values pi for all x sort p
i

i

compute z-scores as HC(N,i) = N [(i/N)-pi]/[pi(1-pi)] HCN* = max HC(N,i)

· ·

1iN

critical value h(N,) = (2 log log N) such that P(HCN*>h(N,))


Pearson's Correlation Coefficient
· Given (xi,yi), i=1..N, x= xi/N, y= yi/N, sxy= [1/(N-1)]
2 x 2 y i=1..N

(xi-x)(yi-y)
i=1..N

sxx= =[1/(N-1)] syy= =[1/(N-1)]

(xi-x)² (yi-y)²

i=1..N

cor(x,y) = sxy/(sxxsyy) · For simple linear regression, cor(x,y) = r = R² is a measure of how much of the variance in the regression is explained by the relationship between y and x. · To determine significance, do bootstrap. For bivariate normal samples, -1 distributed as N(tanh (r),²=1/(N-3)) · sensitive to outliers, hence not robust


Spearman's
· Pearson's correlation for ranked lists · Recipe: 1.sort on xi to determine ranks Rxi 2.sort on yi to determine ranks R
yi

3.compute Pearson's coefficient r for (Rx,Ry) · Less sensitive to outliers than regular Pearson's coefficient, hence more robust


Kendall's

· Rank correlation coefficient, checks the number of concordant pairs vs discordant pairs · Measure of the difference in probability that the data are in the same order vs that they are not in the same order · = (N
concordant

-N

discordant)/[N

(N-1)/2]

· For large N, ~ N(0,²=2(2N+5)/[9N(N-1)])


What correlation method to choose?

· If data can be turned into frequency table, use ² · If one of the datasets can only be written as a sequence, or ranked, use Kendall · If relation between the two datasets is linear, use Pearson · If monotonic at best, use Spearman