Thursday, December 3, 2009

What Is Brewing at the CRU?

There is a quietly unfolding event known as the CRU Hack. The CRU is the UK's East Anglia University's Climatic Research Unit and coordinates climatology research.

An unknown person recently broke into the CRU's computer system and removed a number of documents. The mysterious cracker then published the data on the Internet. Anthropegenic global warming (AGW) skeptics claim those e-mails and files prove the researchers are attempting to skew the results of their studies to show there is global warming. The CRU asserts that nothing of the sort is going on — the documents from the CRU Hack are being taken out of context and they have sound reasons for what is going on behind the curtain.

I suspect the out of context defense will hold in some cases. For example, there is one e-mail where the author uses the word "trick" and skeptics claim this amounts to a nefarious plot. Not true say the researchers, the trick was just an obscure but legitimate mathematic or statistical method. I go with the later, I recall many discussions on how to solve a problem and one person saying "the trick is…". The infamous "hide the decline" is harder to account for without knowing more details of what was being discussed. However, the plain and simple reading of the note is not good for the CRU crew.

A more serious problem is the deletion of data from the CRU's database(s). In my previous post I go over the notion of data outliers. My central point is researchers frequently and rightly exclude certain collected data from analysis. However, generally accepted standards dictates researchers report all raw data and justify data exclusions from further analysis. The inclusion or exclusion of data may be open to debate and further experimentation and that makes good science.

I am aware of a number of justifications for discarding this data. One of them is the raw data went through quality assurance procedures and combined with other sets of data therefore that raw data is no longer needed. That does not pass the smell test, what about those quality processes? Sometimes that justification is combined with the need to save storage space, which does not improve it.

Phil Jones (currently embattled CRU Director) back in October of 2009 states:
The research unit has deleted less than 5 percent of its original station data from its database because the stations had several discontinuities or were affected by urbanization trends. When you're looking at climate data, you don't want stations that are showing urban warming trends, so we've taken them out. Most of the stations for which data was removed are located in areas where there were already dense monitoring networks. We rarely removed a station in a data-sparse region of the world.
So, in essence Mr. Jones asserts the missing data tends to register warmer temperatures than what is warranted or that excessive reports from a given area are removed. Those are real justifications open to debate, if only we had the data, we do not and so we have to take Mr. Jones at his word.

This event does not prove or disprove anthropogenic (or otherwise) global warming. It just calls into question the research the promoters of anthropegenic global warming use to support their contentions and policies they believe we need to implement. Further and more rigorous research may indeed uphold the view the earth is warming or it may not.

7 comments:

rc said...

"This event does not prove or disprove anthropogenic (or otherwise) global warming. It just calls into question the research the promoters of anthropegenic global warming use to support their contentions and policies they believe we need to implement. Further and more rigorous research may indeed uphold the view the earth is warming or it may not."

There is one more very big issue with the CRU models...they have utterly failed to make any accurate predictions. For a scientific theory to survive, that is the simplest and most straight forward test. So far...I think alchemy has a better track record.

Marcus Aurelius said...

I'm not addressing the models in this -- simply the data they have collected.

Modeling is a completely different affair. The hockey stick chart is NOT based on models it is based on data. Data (seemingly very reluctantly) was recently made available and by reports I have read the data does not support the hockey stick. Even though the hockey stick is bull-puckey we are stuck with it.

People pushing AGW must demonstrate the global temperature is rising. If they can not do that, no one gives a rat's wetpu what their models say.

rc said...

"Modeling is a completely different affair."

I guess I don't understand...the models are based on the data..the algorithms which predict a catastrophic climate...used by politicians and the media...are extrapolated from the data. I see the two as one and the same. If you gather data, then develop and theory and models to predict the future...a very bleak future if we are to believe the AGW disciples and the CRU and IPCC...then those models are assumed to be based on the data from which the theory is based, no? Why this distinction between the models and the data they are based on then? Especially since the two originated from the same organization?

rc said...

Wow...you know, the more you read them, the more you find all kinds of quotes in these e-mails that can be taken out of context...like this one:

“We need to show some left to cover the costs of the trip Roger didn’t make and also the fees/equipment/computer money we haven’t spent otherwise NOAA will be suspicious.”

rc said...

Science!

http://www.washingtonexaminer.com/local/Former-NASA-climate-scientist-pleads-guilty-to-contract-fraud-8613137-78268862.html

Marcus Aurelius said...

RC,

The main thrust of what I discuss in the post is solely based on the data. Yes, the data is used to set initial conditions to the simulation models but again, if the models says the average temp in Appleton Wisconsin is supposed to be 50 ° F (10 ° C) and it turns out to be 15 ° F (-9.5 ° C) and this happens repeatedly no one will take the models seriously.

Seems to me the chips are being put on the horse named "Establish correlation between atmospheric carbon dioxide, industry, and global temperatures" and the case is made. Never mind the difference between correlation and causation. The models are an attempt to establish causation but they only need correlation.

What they are doing with the data is understandable to most people and can be explained to people with a high school science education, whereas the models would mystify even most undergraduate science students (excepting perhaps advanced physics, math, and meteorology students).

The other point I am trying to make is not too get too excited about the content of the CRU Hack. It is a huge find and is having a noticeable effect, but I believe it is too possible to overplay it.

In the end the question comes down to science.

rc said...

"The main thrust of what I discuss in the post is solely based on the data. Yes, the data is used to set initial conditions to the simulation models but again, if the models says the average temp in Appleton Wisconsin is supposed to be 50 ° F (10 ° C) and it turns out to be 15 ° F (-9.5 ° C) and this happens repeatedly no one will take the models seriously."

And this is what has happened, no? And aren't all models based upon 'initial conditions' and historical trends? I still don't understand your distinction...the models are what the IPCC bases the hysterical claims on. The models may have many problems (and evidence is emerging that the do), but their singular inability to predict anything seems to me to be the biggest red-flag of all...the models are simply mathematical representations of the theory...formed from the 'data'.

Well, as far as 'overplaying'...there is truth and there is untruth. You call them as you see them. The stakes in the AGW debate are enormous...not just politically, but economically.

Now there is strong evidence (actually, it's been there for a couple of years, but it's been ignored by the press) that AGW has very little to do with science...and a lot to do with politics.

Every day now, we get more revelations about AGW fraud globally...for instance, we now know that NASA refuses to release documentation explaining it's bizarre adjustments to historical data...

http://www.washingtontimes.com/news/2009/dec/03/researcher-says-nasa-hiding-climate-data/

The original data set said 1934 was the warmest year on record...last year, they abruptly adjusted the data, with no explanation to show that the last 2 years were the warmest in the century...they are being sued to release that...but, why do they have to be sued?