Документ взят из кэша поисковой машины. Адрес оригинального документа : http://hea-www.harvard.edu/AstroStat/astro193/VLK_slides_Feb4.pdf Дата изменения: Thu Feb 5 02:19:54 2015 Дата индексирования: Sun Apr 10 11:32:42 2016 Кодировка:

Astro 193 : Feb 4
·

Data (contd)
· · ·

Fundamental equation of astronomical data Propagation of errors Multiple Imputation

· ·

R Homework 2 via python


The fundamental equation of astronomical data

(x',E',t';) = dt dE dx f(x,E,t;) A(E; x',t,f) P(x,x'; E,t,f) R(E,E'; x',t,x) (t,t')

Y(x',E',t';) ~ Poisson() Y(x',E',t';) ~ Normal(,І)


The fundamental equation of astronomical data
Expected intensity photon flux at! telescope aperture

(x',E',t';) = dt dE dx f(x,E,t;) A(E; x',t,f) point spread function P(x,x'; E,t,f) spectral response matrix R(E,E'; x',t,x) (t,t')

effective area

frame time! dead time! light speed

Y(x',E',t';) ~ Poisson() Y(x',E',t';) ~ Normal(,І)


(x',E',t';) = dt dE dx f(x,E,t;) A(E; x',t,f) P(x,x'; E,t,f) R(E,E'; x',t,x) (,t,t')

Statistics Notation: Estimated intensity
!

(x',E',t';)

Not a measurement
!

In the absence of perfect knowledge, construct a model function and ask how close it got to the observed data


(x',E',t';) = dt dE dx f(x,E,t;) A(E; x',t,f) P(x,x'; E,t,f) R(E,E'; x',t,x) (,t,t')

Estimates and uncertainties in the Normal case ±
!

Suppose you have many instances of {1, 2, ..., N} = sample mean = 2 = sample variance =
i=1..N

/ N

i=1..N

(-)2 / (N-1)

is standard deviation


Propagation of errors
In many cases, we need to know the error on combined, or transformed, parameters g=g(ai)
!

g=+ g/ai (ai-Бi)
!

І(g) = jk g/aj g/ak
!

jk

when jk = 0 for jk, І(g) = i (g/ai)2 І(ai)


Examples


Examples

g = C a : g = C

a


Examples

g = C a : g = C

a

g = 1/a : g/g = a/a


Examples

g = C a : g = C

a

g = 1/a : g/g = a/a g = a + b : Іg = Іa+Іb square-adding


Systematic errors and Multiple Imputation
Suppose we have the same quantity estimated under different conditions, where we know different systematic errors are present. Say {, } Compute group estimate as mean of individual estimates, = (1/N) Compute within-variance as average of individual variances, W = (1/N) І Compute between-variance from individual estimates, B = (1/N-1) (-)І


Systematic errors and Multiple Imputation
Total variance T = W + (1+1/N) B Described by a t-distribution, inflated wrt Gaussian. Approximated by Gaussian when degrees of freedom is high.
!

di = (N-1) (1 + (N/N+1) (Wii/Bii))

2


R


·

What is R?
· ·

Statistical computing environment programmable/scriptable C-like language (similar to matlab, IDL) Huge number of analysis packages, written for and by statisticians

·

·

Where to get it from?
·

http://www.r-project.org/


Recommended reading: Tutorial on AstroStatistics and R
http://hea-www.harvard.edu/AstroStat/Tutorial2014/
Some functionalities of R
arithmetic & linear algebra bootstrap resampling empirical distribution tests exploratory data analysis generalized linear modeling graphics robust statistics linear programming local and ridge regression max likelihood estimation multivariate analysis multivariate clustering neural networks smoothing spatial point processes statistical distributions statistical tests survival analysis time series analysis

(slide from Eric Feigelson)


Recommended reading: Tutorial on AstroStatistics and R
http://hea-www.harvard.edu/AstroStat/Tutorial2014/
Selected methods in Comprehensive R Archive Network (CRAN)
Bayesian computation & MCMC, classification & regression trees, genetic algorithms, geostatistical modeling, hidden Markov models, irregular time series, kernel-based machine learning, least-angle & lasso regression, likelihood ratios, map projections, mixture models & modelbased clustering, nonlinear least squares, multidimensional analysis, multimodality test, multivariate time series, multivariate outlier detection, neural networks, non-linear time series analysis, nonparametric multiple comparisons, omnibus tests for normality, orientation data, parallel coordinates plots, partial least squares, periodic autoregression analysis, principal curve fits, projection pursuit, quantile regression, random fields, Random Forest classification, ridge regression, robust regression, Self-Organizing Maps, shape analysis, space-time ecological analysis, spatial analyisis & kriging, spline regressions, tessellations, three-dimensional visualization, wavelet toolbox

(slide from Eric Feigelson)


R quick help
at command line help.start()! ?functionname! help(toolname)!
!

R reference card: http://cran.r-project.org/doc/contrib/Short-refcard.pdf!
!

R cheat sheets: http://devcheatsheet.com/tag/r/! http://www.amaynard.ca/computing/R_Cheatsheet.pdf! http://cheat-sheets.org/saved-copy/R-refcard.pdf!


Demo