The Big Picture
Our hometown rag (the Boston Globe) runs an occasional series of photo collections that highlight news stories called The Big Picture. This week, they take a look at the Sun: http://www.boston.com/bigpicture/2008/10/the_sun.html
The pictures come from space and ground observatories, from SoHO, TRACE, Hinode, STEREO, etc. Goes without saying, the images are stunning, and some are even animated. The real kicker is that images such as these are being acquired by the hundreds, every hour upon the hour, 24/7/365.25 . It is like sipping from a firehose. Nobody can sit there and look at them all, so who knows what we are missing out on. Can statistics help? Can we automate a statistically robust “interestingness” criterion to filter the data stream that humans can then follow up on?
hlee:
I believe you already know what I’ll ask.
How a computer understands your “interestingness”? How a computer measures such criterion? I’ve been asking you training data sets so that a computer can learn those “interestingness” criteria or be trained to acquire that interestingness as solar physicists recognize, from which statistics, as a consequence, can provide uncertainties. Without learning, or knowing definite models (not a model from physics), one cannot assess the degrees of uncertainty.
I talked about this automatized computer vision problem without a training set and with no rules of learning, to a friend and the advice was “don’t go further. It’s not that easy as you think without training sets and a priori knowledge. You must insist for training sets or develop your own vision learning strategies (Neither I succeeded). The problem is, if you succeed in what you’ve been asked to do, grad students and postdocs, including you, will lose jobs. Nothing seems welcome. Take it easy.”
10-13-2008, 1:45 pmvlk:
How does a computer understand interestingness? Yes, that indeed is the crux of the problem. I see the task of astrostatisticians as defining that quantity in a statistically meaningful way. I don’t mean to imply that this is a tractable, or even a well-posed question. If I knew how to do it, I would be famous (and certainly not out of a job, I don’t think!) The analogy I would make is that for point sources in X-ray images, we have solved that problem by appealing to the significance of background fluctuations (cf. celldetect, wavdetect, etc). Not perfectly, and there is considerable room for improvement, but there is a solution that works pretty well. But so far we have been unable to generalize the statistical lessons learned in solving that simpler problem to the case of extended sources (which is what the solar images are).
10-13-2008, 4:42 pmaneta:
Thanks Vinay for posting these images. They are incredible! One way to think about interestingness is to define a “stable” and uninteresting situation and then look for some deviations from that uninteresting situation. Has someone generated a boring pictures that are assumed to be normal for the Sun? Somehow each time I look at the images of our closest star I’m amazed by the variety of pictures and the simple beauty of the nature and the physics displayed in front of us. However,
10-13-2008, 8:30 pmto a computer program some images can be defined as boring and some as interesting, so there may be a way to be able to search for interesting data in the vast amount of images.