Climate Change Scandal: Raw Data Tossed
The controversy over the hacked climate change emails continues to gain steam, forcing the East Anglia team to reverse course and promise to release their raw data.
The U-turn by the university follows a week of controversy after the emergence of hundreds of leaked emails, “stolen” by hackers and published online, triggered claims that the academics had massaged statistics.
In a statement welcomed by climate change sceptics, the university said it would make all the data accessible as soon as possible, once its Climatic Research Unit (CRU) had negotiated its release from a range of non-publication agreements.
Alas, the extent to which damning evidence remains available is in question.
SCIENTISTS at the University of East Anglia (UEA) have admitted throwing away much of the raw temperature data on which their predictions of global warming are based.
It means that other academics are not able to check basic calculations said to show a long-term rise in temperature over the past 150 years.
The UEA’s Climatic Research Unit (CRU) was forced to reveal the loss following requests for the data under Freedom of Information legislation.
The data were gathered from weather stations around the world and then adjusted to take account of variables in the way they were collected. The revised figures were kept, but the originals — stored on paper and magnetic tape — were dumped to save space when the CRU moved to a new building.
In a statement on its website, the CRU said: “We do not hold the original raw data but only the value-added (quality controlled and homogenised) data.”
The CRU is the world’s leading centre for reconstructing past climate and temperatures. Climate change sceptics have long been keen to examine exactly how its data were compiled. That is now impossible.
Roger Pielke, professor of environmental studies at Colorado University, discovered data had been lost when he asked for original records. “The CRU is basically saying, ‘Trust us’. So much for settling questions and resolving debates with science,” he said.
Jones was not in charge of the CRU when the data were thrown away in the 1980s, a time when climate change was seen as a less pressing issue. The lost material was used to build the databases that have been his life’s work, showing how the world has warmed by 0.8C over the past 157 years.
He and his colleagues say this temperature rise is “unequivocally” linked to greenhouse gas emissions generated by humans. Their findings are one of the main pieces of evidence used by the Intergovernmental Panel on Climate Change, which says global warming is a threat to humanity.
The “space” explanation is plausible enough — the amount of data in question is massive — but it is nonetheless, as the Church Lady would say, “convenient” under the circumstances.
Christopher Booker, a leading climate skeptic, calls this “the worst scientific scandal of our generation.”
A week after my colleague James Delingpole, on his Telegraph blog, coined the term “Climategate” to describe the scandal revealed by the leaked emails from the University of East Anglia’s Climatic Research Unit, Google was showing that the word now appears across the internet more than nine million times. But in all these acres of electronic coverage, one hugely relevant point about these thousands of documents has largely been missed.
The reason why even the Guardian‘s George Monbiot has expressed total shock and dismay at the picture revealed by the documents is that their authors are not just any old bunch of academics. Their importance cannot be overestimated, What we are looking at here is the small group of scientists who have for years been more influential in driving the worldwide alarm over global warming than any others, not least through the role they play at the heart of the UN’s Intergovernmental Panel on Climate Change (IPCC).
Samizdata’s Michael Jennings offers a few thoughts as well.
I have written one or two computer models of physical systems. And as it happens, they are hard. It is possible to use all the computational power you have in simply modelling something tiny: the vortices around the tip of an aircraft wing, say. As the systems you have become larger and larger you make more and more approximations and more and more assumptions that particular terms in equations are small and will be small in the future because they have been small in the past. There comes a point where models more from theoretical to empirical. You end up basically extrapolating from the recent past to the near future. In systems containing a lot of nonlinearity, factors that have not had macroscopic impacts in the past can suddenly flare up and become dominant in the future. Sometimes it is possible to figure out just when this will happen and add these effects to your models at the right time. Sometimes it isn’t. The earth and its climate is a huge system. In places it is highly nonlinear. It is horribly hard to model.
Recall that chaos theory was developed in response to the vagaries of weather forecasting, wherein even seemingly minor rounding decisions can have massive long tail effects.
A second thing one learns from working as a research scientist is that people in research labs resemble people in other workplaces. Petty fiefdoms exist. People stab one another in the back. Some people do better work than others. People will have different levels of respect for the work of other researchers. Some people rise to the top through doing good work. Others rise to the top through playing good politics (good researchers generally hate such people, but they none the less manage it). People at opposite ends of the corridor hate one another. If one is going to work in a particular team, one must work within the culture and beliefs of that team. In one’s work, you often have to start with whatever the person before you left behind.
In scientific research involving computer modelling and data analysis, this often leads to computer models consisting of layer on layer of code crufted on top of lower layers that are not well (or at all) understood. Data does get lost, or assumed to be correct because the previous person used it and there is no real way to verify it. Supposedly impartial journals do become captive of a particular point of view. People’s whole careers do become dependent on a particular interpretation of the results, and it then becomes very hard for them to back down. People become more and more certain of their results when the personal cost of abandoning them gets greater and greater.
However, once again as in any other workplace, good work still happens amongst all this. If there are six different cliques in different places, they will compete with one another until the truth comes out. If there are six different journals, then they won’t all become captive to the same clique, and eventually the one with the best and most meaningful results will become the most prestigious. There will be enough ability to move between teams that younger scientists will not necessarily be caught in a particular viewpoint because of who they work with. Politics will be horrible. Much bad work will be done. It will be a messy process, but it will generally be understood who does the best work, and the truth will come out. Really good researchers will be able to figure out what in the crufty codebase is good and what is not, and get meaningful results anyway.
This, fundamentally, has been my problem with the science of global warming – the denial of the messiness of it all. We have been told that “The Science is Settled” by men in white coats in ivory towers, and that we are “denialists” and unworthy of being listened to, if we dare to question the process or to state the obvious – that science is a messy and uncertain process and that as a consequence of being a very hard problem, modelling the climate is going to give answers with huge margins of error and huge unpredictability. (Nicholas Naseem Taleb would say it’s a system highly susceptible to Black Swans, and he would be right).
A blogger whose spouse works in the field emails to second Jennings’ analysis.
My initial take on this mess — that conspiracies are near impossible to pull off and that science ultimately rewards truth rather than consensus — remains strong. But the evolution of this story shows it to be more than it initially seemed — the misunderstanding of terms of art by laymen and some “boys being boys” on a geek listserv. Whether all of this rises to the level of “scandal” or merely unfortunate sloppiness remains an open question.
When all you have is massaged data (and they won’t release even that), they effectively exclude those who disagree with you from the professional journals and on the basis of such faulty science you have politicians declare the debate over and the time to spend trillions on, you have a scandal. The best analogy is global warming has become like a religion (and see in history what can be done in the name of religion before you dismiss ‘conspiracy’) with the primary difference that religion asks you to have faith about the supernatural while global warming asks you to have faith about what should be reviewable and understandable to others if proper scientific debate were allowed.
James, time will provide an answer to your final question, I am sure. Yet, you make no judgment on the fact that the levers of power in government, the people we entrust with protecting us, have casually accepted the claims of Mr. Gore and have passed legislation impacting every American.
Does this naive acceptance of “settled science” by supposedly intelligent people not bode poorly for other legislation promoted by the same levers of power? I’m glad you have started to pay attention to this issue. Perhaps you will stand back and consider a bigger picture; that noble intentions (cleaner air) have been corrupted by self serving “levers of power” and that it is somewhat arrogant to conclude that the human race is capable of drastically changing the Earth’s climate in such a short period of time.
Now I have Monty Python’s “I am the Bishop of East Anglia” sketch in my head. I couldn’t find the link. It would be a good chaser.
1) If data was from “around the world” then it is unlikely that this computer had the sole copy.
2) I read a nice piece on this scandal in which a scientist made an entertaining “of course we’re jerks” argument. He said that this improved science, because the guy you savaged on peer review is lying in wait at your next conference presentation.
3) If these guys are really weak, and really corrupt, then it should be easy for someone to climb over their bodies and win at the science.
So far I haven’t seen anything that is clear scientific misconduct. However, the attempt to evade FOI requests seems illegal.
What Jennings says is true, but I think that usually the issue is not so politicized, and there are sufficient checks and balances to keep science moving in at least somewhat the right direction. In this case, however, I think the system is working even less well than usual.
The whole thing with the data does seem odd, and I don’t really understand it. Who funded these weather stations?
Conspiracies can take many forms. The classic preplanned scheme hatched in some darkened room is not likely here but a casual acceptance of similar goals accompanied by a failure to apply historical standards of science is easy to accomplish. People who should have spoke up didn’t in order further their own goals and support their shoddy work. It became more of a classic conspiracy once the concerted effort to undermine the opposition began. The truth will come out but the headlong rush into climate change legislation could have been a disaster.
Al Gore deserves most of the blame for the rush. His culpability in this scandal is based upon his cheer leading and political activism without a sound scientific basis or reasonable caution. Remember, he refuses to debate anyone on the topic. That is now a very irresponsible position to take. Gore should step up and defend his views properly.
Think of the scandal if you replaced Climate Change with Iraq and East Anglican with President Bush….there would be yet more calls for impeachment and war crimes. Of course, those committed to global climate change wants to change the subject….
Many, many moons ago when I was in engineering school my thesis professor (a man of note who worked on the Manhattan project) just about threw me off the case for violating what he considered a rule no scientist could ever violate: you never, ever get rid of any of the raw data or test measurements/setup. Ever.
This was a practice I similarly found to be true for the time I practiced engineering and interacted with other scientists and engineers.
So I find the notion of this original data being lost for “space” reasons to be closer to absurd than plausible.
These guys claim the fate of the world is in the balance. Call me crazy, but I’d kindly suggest you don’t throw that kind of data in the trash. Further, as I understand it, a central issue in the whole debate is that the data had to be “value added” (snicker) to arrive at the claimed temperature rise.
As someone observed in a comment, we can only hope the original data is elsewhere available. If not, start from zero, AGW’s.
And as for this crew of “scientists,” its settled, “the debate is over.” They are frauds.
I agree, anyone who has ever worked in a laboratory or research institute of any kind would say its highly implausible … the researchers in question are either lying, or hopelessly incompetent.
I don’t think some of the commenters here understand that the data in question wasn’t generated at CRU, it was compiled from sources around the world. The only thing they discarded was their compilation of the raw data, not the raw data itself – it’s still available to be recompiled by someone else with the time and funding to do so.
I was curious, so I went over to the ExxonMobil website. They write:
So what’s the conspiracy drill here, that ExxonMobil, the most capitalized firm in America has such scaredy cats that they are pretending?
But anybody who remembers their high school or college science knows it is all wrong?
Should that possibly make sense to me?
I would be surprised if 1 in 500 of the people making strong statements about this understand much of the science.
I’m curous, do folks of the Plunk persuasion wish to deny tout court that climate change is taking place?
Let’s all just sign our names to one of three lists about our opinion on AGW, and we’ll check who was right in 50 years. Maybe we cede power to the winners or something as a prize. Does that sound good? The three lists should be Yes, No, and Don’t Know. I’ll be signing “Don’t Know”.
From 2000 to today, the CO2 content of the air has risen. The temp has gone down. Explain Sam! The leaked emails allude to the fact calling it a travesty. Why? Because thier little earth changing lie is being exposed. I wonder how the IRS would deal with me if I told them I lost the original data but have some adjusted figures to submit? You have been lied to by your high priest, liberals, and the truth has peeked out. It will save millions of lives Gaia demanded and you were so willing to sacrifice to her. If you lefties realy want to cut CO2 emissions, stop breathing.
That’s not what I asked, moron. I asked if you folks were denying climate change, period. You know, Zels, you really are beyond parody. Idiot.
Sam. F you. I am a parody and you are a fool. The climate changes over time. Has for eons. What is discussed here is man made change. You know, Sam, I will supply you with information as to where I live and we can get together to discuss, in any manner you like, to settle who is a fool and who is not. First, I live in California but if it is too far for you, think of me a Palidin.
What about their manipulations of data of Australia and New Zealand temperatures that are basically flat to make it look like the temperatures were rising? Yep nothing suspicious there. We are just ignorant of science.
The one who are ignorant of science are the ones that swallow the Manmade global warming B.S. whole and without question. Talking about religious fanatics.
Dumped data? Sounds like Hillary and that Whitewater story again.
You know the Manmade Global Warming crowd is losing when they try to switch the argument from Manmade Global Warming to climate change. Then they pretend if there is climate change that proves Manmade Global Warming. They fail in scientific and in logical thinking.
You are so right Wayne. Those caught will deny, until tried in court. This may meet RICO standards. I know enough about scientific method to know for there to be any scientific proof of any premise, results must be reproducible. Lost data means there is no possible way to reproduce the results. After reading about altering data it becomes obvious to all but those who can no longer think and only worship Gaia, something was amiss. If Obama goes to Copenhagen and the idea put forth there is shown to be a fraud. Once again the President will look, to the world as well as to this nation, to be the novice that he is. This is vastly more important than the Olympics. Control of every aspect of human activity by a world communist dictatorship is at stake.
Sam, Climate change happens daily, weekly, by season, and year to year. What impact does man made carbon have on it? I don’t know for sure and it is increasingly evident scientists don’t either.
There are studies linking climate change to solar cycles. Some studies show CO2 rising after temperatures rise, not before.
I do understand the difficulty of modeling complex systems with computers. A tiny difference in input can create large differences in outputs. Something as complex as the earth’s climate can hardly be modeled at all given the number of inputs necessary. Computer modeling can also be easily manipulated.
So people of my persuasion are simply skeptics. Especially about things that could ruin our economy with only speculative beneficial results. People like me do not take as fact what ever the newest study might say. Science takes time.
What do people of your persuasion think?
I can only hope the data is still available. However, that begs the question of how scientists outsider CRU would “value add” it. Let’s see, especially now that the fraudulent “peer review” process has been exposed. Follow the money.
John persona –
Sorry, JP, dopey argument. Many of these large companies have economic interests in the AGW debate. They know they have oil, and it must ultimately be used. So the economic potential of that asset is safe. They also know there are dopes with money who want to spend it on alternatives………and they want to tap those funds. Win. Win.
You seem like a good guy. But you are publicly embarrassing yourself.
So you are saying Drew that back when ExxonMobil did try to beat the science, and did fund opposing studies, they hadn’t figured out your argument?
That seems to fail Occam’s Razor. Simpler to think that they tried, and failed.
From earlier this year:
Easier to say Heartland failed.
Links to data and source code can be found here:
As for the discussion by ZR and his fellow wing nuts, it’s beyond parody. None of you people understand atmospheric science, and it seems apparent that very few of you know any physics, even at the undergraduate level. I include Joyner in this group. There are problems involved in global modeling – the treatment of cloud processes, the coupling with the ocean, particularly in terms of the ice modeling, the difference between the spin-up times of the oceans and the atmosphere, the parameterizations of the atmospheric boundary layer and the mixed layer in the ocean – none of which are mentioned in any of the comments.
That’s because those making the comments, ZR in particular, don’t know dick about atmospheric physics and are too stupid to know their own ignorance. There are a few people who know the science and who doubt the more dire forecasts made by Hansen and colleagues, for example Dick Lindzen, Roger Pielke, and, I think, Jim O’Brien. If you want to attack the concept of global warming, try linking to these people instead of repeating propaganda from the Wall Street Journal.
Historians will note this Global Warming/Climate Change movement as the greatest hoax of the 20th century. Raw data is not lost when the consequences are this serious. This is not about global warming or climate change. It is about controlling people. CO2 has nothing to do with the climate change. It has to do with taxes. If you can tax CO2, it is the mother lode. CO2 by definition cannot be a pollutant. We exhale it. Come on. How much more do you need?
There is a big post at Real Climate on the data. It looks as though the data is not lost. It is all available at the original stations. It has been available for those who want to go thru the work. It sounds more like the skeptics want the information compiled and collated for them. There is also more than one data set. They are providing links, which were apparently available to those who wanted them all along. Will await McIntyre’s response.
Since we are now all about transparency, what say we have the last year of AEI’s emails published?
Top scientists have been caught red handed manipulating and falsifying data. They can’t claim it was simply a honest mistake like Hansen tried to claim on his manipulation to make 1998 look like the hottest year in recorded history.
So what do the Gaia worshippers do? Deny everything and repeat over and over the skeptics don’t know science.
The biggest shame in this type of blind following is it brings all science study into undue doubts. All science study should be scrutinized but now many will simply be discarded as scientist falsifying research once again. After all the believers are not holding them accountable. It will be considered nothing more than another religious belief.
It is beyond a religious belief. It is criminal fraud.
Well, people of my persuasion, a class of one person, think along the following lines. We know that human activity can, at the least, have a profound effect on the upper atmosphere — see, CFCs and the ozone layer — I don’t think anyone denies that, right? And we do see evidence of a warming trend in the retreat of glaciers, diminution of the Antarctic ice sheet, opening of the Northwest Passage, and so on. It’s not a wild leap to hypothesize that, given our experience with CFCs, the observed warming might, I say “might”, be connected with human activity. I’m inclined to think it does, but this is controversial . But what is not controversial, I believe, is the warming trend itself, and this trend, whatever its etiology, might have a profound and detrimental effect on us.
What I find distressing in the debate is the tendency, on the part of some, to argue in the following way: The hypothesis of anthropogenic global warming is false, therefore the claim that there is global warming is false. In my opinion, this is dangerous. Causes aside, we ought to be asking ourselves what the effects of this warming trend will be on us, and, if the effects are less than happy, how we can best deal with them. I haven’t seen much discussion of this.
And BTW, Zelsdorf, you really are an idiot. Threatening someone with a firearm:
is really not a smart thing to do. Someone less knowledgeable of your mental deficiencies than I might track you down via your isp and lodge a complaint with law enforcement.
We can go back and forth on melting and increase of ice areas and how the GW crowd likes to cherry pick their areas. I can point out many areas where scientist prediction were wrong including their predicting the oceans were going to warm and they got cooler.
However like many point out, the fact is the Earth goes through warming and cooling phases. The debate is how significant human is contributing to it, is it economical feasible to try and control it and for that matter what is the ideal temperature.
Many of the skeptics think man contribution is a very insignificant. Wipe out the whole human race and little in global warming or cooling would change.
What is the ideal temperature? Maybe being a little warmer would help. Al Gore’s outrages claim of massive sea level rising have been debunk even by some of his supporters. What happen if the Sun’s activities go down further and we go into an ice age? Are we going to start pumping more greenhouse gases in the air? Doubtful. Perhaps we need to adapt to Earth changes instead of regulated it because it will change one way or another.
Your comments are very interesting Wayne. I could pick some of them that acknowledge AGW as “contributing”, but then some that confuse outcomes.
Climate has ranged from ice age and back over many thousands of years. We don’t usually worry about a thousand years from now, it’s true. The question is how much we should care about, or how much we should feel responsibility for, things 50 years from now. Al Gore’s sea level rise is not about today, after all.
Are you over or under 50 yourself?
How does that affect your outlook?
How many AGW deniers plan on being safely dead?
Personally I think AGW is a significant contribution, but that people are too short-term focused to deal with it. From a policy standpoint, 50 years from now might as well be a thousand years. And so we’ll see what it’s like.
Not because that’s smart, but because we’re dumb.
BTW, I mention “deniers” that might be kind of a hot button. In truth “supporters” aren’t doing that much more. The big climate conferences aren’t really about true solutions, just doing enough to be seen doing something. Another reason we’ll get to experience if for ourselves, if we are young enough.
BTW, I don’t think this is a reasonable question: “What is the ideal temperature?”
The reason is that global average temperature is a useful metric for only one thing, tracking changes in global temperature.
As we live our lives we don’t really care what the weighted average temperature is in the ocean, in the deserts, and in our front yards. We care about our front yards, and probably where our food comes from. Those regional temperatures have a non-linear relationship with the global average.
You say “Maybe being a little warmer would help.” Maybe, but then again maybe it would make Kansas a dust bowl. The conservative position would probably be “I like Kansas the way it is, don’t change it” … to say “go ahead, take a chance” seems not very conservative to me.
But in this case “conservative” isn’t about the climate, is it?
Sam you are very right pointing out this has multiple facets to discuss.
The AGW hypothesis will remain a hypothesis until proper experimentation with results that can replicated. The problem is how to replicate when the authors will not share data and code? And is replicating complex computer models actually sound science? The observation of ice and glaciers seems to make sense until we put it into context of previous warming periods not linked to man made CO2. Remember these were problems for the AGW proponents.
I agree we should be discussing the next step of whether or not we should even be doing something about warming. Instead we are only looking to stop it (which we may not be able to do) by controlling emissions. Maybe a warmer planet is a better planet but it’s impossible to have that discussion.
The issue has never been really handled properly. The science was immediately politicized and charlatans like Gore looked to it as a vehicle to greater things for themselves. Now the media has turned the debate into all together different by inflaming passions in what should be an impassionate scientific debate.
I look forward to the raw data, code, and other parts of the recently released documents to undergo examination and evaluation. We might finally get some trustworthy facts.
“The problem is how to replicate when the authors will not share data and code.”
See my earlier post. The code and the data are freely available. And if you’d like to compare the code with other models, you can get the NCAR Community Climate Model and the GISS Model. Let us know what you find out.
Re “Al Gore’s sea level rise is not about today, after all.â€
Do you believe his estimate?
His estimate is so far overstated that it is a great lie. Even GW scientist supporters admitted it was false. The sea level will not rise 20 feet in the next 50 years.
I been hearing for decades that the world going to end in such and such year. Newsflash, it hasn’t. The national debt that we are leaving our children and grandchildren are for more dangerous than manmade global warming except for how politician will use the B.S. to take power.
Re “it would make Kansas a dust bowlâ€
It is now. Maybe it would be a tropical paradise.
Liberals are for change so why not in this case? See that game is easy to play.
If the data is so readily available, why are there lawsuits under UK freedom of information act to get it. Also why have the IPCC claim that some of it was accidently destroyed? Haven’t you been paying attention to the controversy lately or are you just ignoring it?
I know that there was a 10 meter estimate before Gore made his. I consider it a “possible ballpark” rather than a “written guarantee.”
Huh? We went from sea level rise to ends of the world? That doesn’t sound too rational.
Huh? You think you won that, after I said “I like Kansas the way it is, don’t change it”?
I am a lifelong Republican (now more an independent), and I’m asking you why you are gambling? Because you think liberals would too?
You know, Nassim Taleb is famous for disbelieving predictions, but he makes the same argument I do here. We know we can live in this climate. No prediction required. When you voluntarily change the climate, suddenly you need a prediction. Why take the chance?
See the problem with people who want to boldly change the climate while disbelieving climate models? It means they want to play roulette.
Wayne, the data is available, but not from the University of East Anglia. Go to this web site,
and page down to get links to the data.
You believe that 10 meters (over 30 feet) sea level rise in the next 50 years is a reasonable possibility. Enough said.
Stop with the red herring ploy. Linking to some data is not the same as producing ask for data. There are lawsuits requesting certain data that the IPCC says support their conclusion and the IPCC won’t give it up and claim some of it was destroyed. Giving another set of data just doesn’t cut it since the IPCC can claim their original set says whatever they wanted it to say.
In case yo want some links to destroyed raw data.Perhaps you think the only relevant data is their modified data?
“We do not hold the original raw data but only the value-added (quality controlled and homogenized) data.”