A Global Warming Data Set

The UK Met office has a temperature observation data set that you can download here. I am not sure, but I believe that this is in response to the recent Climate-Gate controversy. In any case, check out the "Questions About the Dataset." I will highlight the juicy parts:

1. Is the data that you are providing the “value-added” or the “underlying” data?

The data that we are providing is the database used to produce the global temperature series. Some of these data are the original underlying observations and some are observations adjusted to account for non climatic influences, for example changes in observations methods or site location.

The database consists of the “value added” product that has been quality controlled and adjusted to account for identified non-climatic influences. It is the station subset of this value-added product that we have released. Adjustments were only applied to a subset of the stations so in many cases the data provided are the underlying data minus any obviously erroneous values removed by quality control. The Met Office do not hold information as to adjustments that were applied and so cannot advise as to which stations are underlying data only and which contain adjustments.

2. What about the underlying data?

Underlying data are held by the National Meterological Services and other data providers and such data have in many cases been released for research purposes under specific licences that govern their usage and distribution.

It is important to distinguish between the data released by the NMSs and the truly raw data. e.g. the temperature readings noted by the observer. The data may have been adjusted to take account of non climatic influences, for example changes in observations methods, and in some cases this adjustment may not have been recorded so it may not be possible to recreate the original data as recorded by the observer.

3. Why is there no comprehensive copy of the underlying data?

The data set of temperatures, which are provided as a gridded product back to 1850 was largely compiled in the 1980s when it was technically difficult and expensive to keep multiple copies of the database.

For IT infrastructure of the time this was an exceedingly large database and multiple copies could not be kept at a reasonable cost. There is no question that anything untoward or unacceptable in terms of best practices at the time occurred.

This is really much of the controversy in a nutshell. The original data has been adjusted, and no record of those adjustments exist. We are supposed to spend trillions of dollars on this basis? I wouldn't claim that there was anything untoward done here, but to say this is acceptable "in terms of best practices at the time" is a bit hard to swallow. In what other field of science would such a process be acceptable, and considered "best practice?"

Imagine testing a new blood pressure drug. The drug is administered in a test population, then follow up blood pressure readings are taken. Next, each reading is adjusted in some non-specific way to account for vague external factors, e.g. stress on the patient. These adjustments are then discarded along with the original data. Statistical analyses are run on the adjusted data and a manuscript is written and submitted to a journal for publication. Good luck with that.

In criminal trials, if evidence has been tainted, it is appropriate to throw it out. This can happen even in the most severe criminal cases, regardless of the consequences to the prosecution's case, and regardless of the intent of the police. The same should be done here. Tainted evidence is essentially worthless.

Ann says: Just think how small an error could be, and yet still throw one station, or if the error is more systematic, the whole data set out of whack. We just adjusted our clocks for daylight savings, what used to be 7AM is now 6AM, and the temperature difference between those to times can be several degrees--it's just when the sun is coming up this time of year. Imagine if a station starts reporting the old 7AM temperature in a way that makes it look like the 6AM temperature is a couple of degrees warmer. Voila, global warming! The simplest thing such as time of day for the recording can vastly change the outcome of these records.

These are very complicated datasets, with an enormous number of variables that need to be controlled: station siting, time of day, frequency of recording, growth of heat islands, etc. The fact that the researchers are basically saying "trust us!" at the same time they are screaming that the sky is falling, and at the same time they are being daily shown to be incompetents, doesn't exactly promote a belief in their science and theories.