More Questionable Global Warming Data Hijinks

Steve McIntyre at Climate Audit appears to have caught NASA and James Hansen quietly changing data that is available on NASA’s website.

Shortly after, NASA published their source code on Sept 7, we started noticing puzzling discrepancies in the new data set. On Sep 12, 2007, I inquired about the changes to Hansen and Ruedy, observing that there was no notice of the apparent changes at their website:

I’m sorry but this simply looks bad. This doesn’t address directly the validity of the various hypotheses about global warming, but this certainly does look dubious. It goes without saying that data is extremely important in science and when you are doing research and advocating policies that could cost trillions of dollars and have a major impact on the world economy being at least up front when you change the data strikes me as the best policy. For somebody so smart, James Hansen’s actions are incredibly dumb.

By this time, we’d figured out exactly what Hansen had done: they’d switched from using the SHAP version – which had been what they’d used for the past decade or so – to the FILNET version. The impact at Detroit Lakes was relatively large – which was why we’d noticed it, but in the network as a whole the impact of the change was to increase the trend slightly – enough obviously to make a difference between 1934 and 1998 – even though this supposedly was of no interest to anyone.

I work in a regulated industry and if I were to pull this kind of shenanigans with state regulators I could very well end up in some big trouble.

McIntyre provides this graphic for the impact of this change in the data:

update42.gif

One commenter noted that if you were to smooth this graph out you’d get what amounts to a “reverse” hockey stick. This refers to the “hockey stick” for temperature anomalies found in the research of Mann, Bradley and Hughes. In that case, the hockey stick has the blade at the end of the data series with the most recent observations showing a very steep warming trend. This “reverse” hockey stick that Hansen has applied to the NASA data lowers earlier temperatures thus making the later years relatively hotter. So, not only do we have an un-announced change in the data, but also one that “helps” the global warming hypothesis.

This has lead McIntyre to wonder if perhaps researchers in global warming shouldn’t be required to adhere to a principle similar to one of the principles of the Generally Accepted Accounting Principles: the Principle of Permanence of Method. Basically, methods used by the account are always the same and are coherent when doing the type of work. That is, an account can’t use one method when working at one corporation and using an entirely different method at another corporation even though the work at both corporations are the same.

In response to the objection that accounting and GAAP and science are different subjects and hence such an expectation is unreasonable McIntyre has this response:

Now you may say that this is “science” and accounting principles don’t apply. And my response would be that I’d expect GAAP principles to be a minimum standard for the type of climate statistics being carried out by NASA. Even if NASA climate statisticians are unaware of GAAP per se, they should be adhering to the principles. Sharp practice is sharp practice, however it is gussied up.

I have to agree. One of the bedrock principles of science is that research can be replicated. By quietly changing the data makes it virtually impossible for another researcher to replicate previous results raising a whole bunch of rather unpleasant questions. Any scientist should make his data and source code freely available whenever possible, and given that Hansen is working on the public’s dime and pursuing a research program that could be very, very expensive this seems like a minimal standard to hold him to.
research of Mann

FILED UNDER: Climate Change, Environment, Science & Technology, US Politics, ,
Steve Verdon
About Steve Verdon
Steve has a B.A. in Economics from the University of California, Los Angeles and attended graduate school at The George Washington University, leaving school shortly before staring work on his dissertation when his first child was born. He works in the energy industry and prior to that worked at the Bureau of Labor Statistics in the Division of Price Index and Number Research. He joined the staff at OTB in November 2004.

Comments

  1. Bithead says:

    Same topic, more or less…

    Did you note Boortz the other day?

    He points out that there is a good deal of research that suggests we’ve had global warming for some time, now… 55 million years ago, in fact. The cause? Methane from wetlands. The plant and animal decay, that invariably is attached to such plaes.

    Boortz observes that the simple solution would be to dry up the wetlands and build stuff there. I guess we know how that’s gonna go over, huh?

    It does seem to raise an interesting question, however; we have many studies that show we actually have more trees on the planet now than we did in the time of Christopher Columbus. Assuming that’s true, would it also stand to reason that we also have more in the way of wetlands, which in turn leads to the methane production which in turn leads to global warming?

    Do you suppose we’ll ever see Gore screaming that we should dry up the wetlands to stop global warming? No? Well, I suppose he doesn’t take the problem very seriously, then.

  2. Grewgills says:

    NASA is in the process of updating its code which should be released to the public in a few months.
    Some station data were removed by NOAA.

    September 2007: The year 2000 version of USHCN data was replaced by the current version (with data through 2005). In this newer version, NOAA removed or corrected a number of station records before year 2000. Since these changes included most of the records that failed our quality control checks, we no longer remove any USHCN records. The effect of station removal on analyzed global temperature is very small, as shown by graphs and maps available here.

    The purpose of FILNET:

    Estimates for missing data are provided using a procedure similar to that used in the homogeneity adjustment scheme in step three. This fourth adjustment uses the debiased data from the third adjustment (SHAP) and fills in missing original data when needed (i.e. calculates estimated data) based on a “network” of the best correlated nearby stations. The FILNET program also completed the data adjustment process for stations that moved too often for the SHAP program to estimate the adjustments needed to debias the data.

    The actual change for 1834 and 1998: 0.01oC, temperatures that are still not significantly different, no difference in study conclusions, and a lot of commotion about what amounts to nothing.

    He points out that there is a good deal of research that suggests we’ve had global warming for some time, now… 55 million years ago, in fact. The cause? Methane from wetlands. The plant and animal decay, that invariably is attached to such plaes.

    So, there is research that the globe has been continuously warming for 55 million years? Can you link to any of that? I haven’t seen anything even close to saying that.
    The data I would guess Boortz is misrepresenting is that wetlands do typically produce methane (and sulfates while absorbing CO2). It is quite possible that this produces a net increase of greenhouse gas strength though I haven’t seen any good data showing the greenhouse gas balance of any wetland habitat much less a comprehensive set.

    It does seem to raise an interesting question, however; we have many studies that show we actually have more trees on the planet now than we did in the time of Christopher Columbus. Assuming that’s true, would it also stand to reason that we also have more in the way of wetlands…

    How exactly does that follow?
    BTW as far as the impact of trees on global warming is concerned it would be total biomass that mattered more than number of individuals. As far as habitat is concerned it is more about the type of forest (old growth or new growth, monoculture or polyculture, etc.).

  3. Hal says:

    Lambert takes out this garbage in the proper fashion

    The McIntyre factor is the amount that you have to multiply the size of an adjustment in the GISS US temperatures by to get the number of words in the resulting Steve McIntyre post. Empirical evidence puts the McIntyre factor at 125,000.

    I leave it as an exercise to the readers to calculate an estimate of the Verdon factor, which appears to be a secondary trailing signal tracking, with a phase difference of .3 radians, the loons that he seems to think have any credibility what so ever.

  4. Tano says:

    “…we actually have more trees on the planet now than we did in the time of Christopher Columbus. Assuming that’s true, would it also stand to reason that we also have more in the way of wetlands”?

    I dont know Bithead. Seems like a total non sequitor to me. Please explain the logic which leads you to believe that having more trees = also having more wetlands.

    And what is the relevance of any of that to anthropogenic global warming? In a “natural” state, there is effectivly a finite amount of carbon that cycles through the ecosystem – in the atmosphere, in plants, in animals, and back into the atmosphere (much more complex than that of course). Doesnt really matter all that much how much of it is locked up in trees or wetlands at any given moment. The underlying problem with AGW is that humans are digging up massive amounts of carbon that has been sequestered underground for millions of years and burning it, thus adding it to the total flux.

    Boortz is, obviously, retarded. How does drying up a wetland lessen the carbon flux? The carbon emmitted from a wetland was sequesterd there in plants in the first place – its just a stop-off point in the global carbon flux. Drying up the wetland means no more carbon emitted, but none sequesterd either. Makes no difference in the grand scheme of things.

    Of course, a nice office tower and a thousand respiring humans might generate a few carbon emmissions….

  5. davod says:

    I am not a scientist (but I did stay at a Holiday Inn last night) but I do recall my lab clases in school. The lab note book is where I had to detail every experiment and the results. My teachers made it clear that the notes were needed so others could track how the work was done. The reason – because, in the sciences, others must be able to verify your work.

    It seem to me that some have forgotton the need for verification.

  6. Grewgills says:

    It seem to me that some have forgotton the need for verification.

    How so?
    The code is being updated and will be released later this year.
    The complaint seems to rest on NASA releasing updates piecemeal on their website without fanfare prior to releasing the fully updated product.
    All of this is readily available for independent verification, as McIntyre’s overlong post indicates.

  7. Hal says:

    While not a scientist either, I did work for some years on the acid rain deposion modeling project while at NCAR. Also my wife, a chemical engineer, not a scientist, spent many years doing emissions modeling for regional planning. Both of us, her in particular as she was directly involved in the process of input conditioning to the models, can find nothing unusual here. This is a part of science and as many have pointed out, this is an ongoing process that the ignorant have latched upon in another of their long string of failed attempts to raise questions.

    Steve’s title suggests this is another in a long series of unanswered wuestions that have been raised. The reality is that it’s another question raised by the ignorant which has trivial answers that they apparently can’t understand.

  8. davod says:

    The ignorant ones seem to be getting an increasing number of experts on their side.

  9. Hal says:

    Yes, McKitrick and Essex claim there’s no such thing as global average temperature. McKitrick and McIntyre confuse radians with degrees.

    These are experts? These are people who have an agenda who are desperately doing anything they can to generate smoke to confuse those who don’t know any better.

    Steve should know better, but he keeps shilling this stuff because, I guess, he’s aligned with the same agenda.

    Like the dufus, Kane, Steve keeps shilling who’s embarrassing lack of expertise in anything remotely required to criticize the Lancet study (read the comments, it’s a hoot), these guys keep coming back like zombies from B grade sci fi movies. Even after repeated and brutal embarrassment when shown how incompetent they are, they seem to be incapable of shame.

    Experts? In what?

    Can’t wait until Steve jumps on the DDT wagon.

  10. Gary says:

    The real story of McIntyre’s work on the GISS data is that it has adequately shown that the most trusted Global temperature records in the World are little more than a collection of fudge factors and bad data.

  11. Steve Verdon says:

    Hal,

    I see you got nothing other than insults. Guess you can’t deal with the actual issue. Same for Lambert. Instead of dealing with the issue of quietly changing data he resorts to insults. Of course, if John Lott had pulled a stunt like this Lambert would be screaming it from the mountain tops. Intellectual honest is not something I expect from Lambert.

    Tano,

    And what is the relevance of any of that to anthropogenic global warming? In a “natural” state, there is effectivly a finite amount of carbon that cycles through the ecosystem – in the atmosphere, in plants, in animals, and back into the atmosphere (much more complex than that of course).

    Interesting, I hadn’t thought about that, but so what. Man isn’t changing that other than possibly releasing some that is sequestered. That is suppose the upper bound on CO2 is X and in the natural state the actual amount of CO2 is a(t)*X where a(t) is a number between zero and one. All man’s impact would be is to cause a change in a(t) not in X. And as you say, this is actually far more complex than how I’ve represented it.

    Boortz is, obviously, retarded. How does drying up a wetland lessen the carbon flux?

    As far as I know it wouldn’t. But if wetlands are producing methane, then reducing methane output would, in theory, help since it is, IIRC, a “more potent” GHG than CO2.

    Grewgills,

    How so?
    The code is being updated and will be released later this year.

    If this is true, then why change the data available to the public now? Why not make an announcement of it? Why not make the code available now? They are obviously changing the data so the code must be use currently.

    Hal again,

    While not a scientist either, I did work for some years on the acid rain deposion modeling project while at NCAR. Also my wife, a chemical engineer, not a scientist, spent many years doing emissions modeling for regional planning. Both of us, her in particular as she was directly involved in the process of input conditioning to the models, can find nothing unusual here. This is a part of science and as many have pointed out, this is an ongoing process that the ignorant have latched upon in another of their long string of failed attempts to raise questions.

    A classic case of wanting to have your cake and eat it too. If a corporation did something like this it would be bad…very bad. But when a scientist does it, no big deal. Sorry, that is pretty hypocritical, IMO. And that is McIntyre’s objection by the way. That we expect something from a firm that may reach a few tens of billions of dollars, but a multi-trillion dollar government policy…who cares about verifying the data and best practices.

    Yes, McKitrick and Essex claim there’s no such thing as global average temperature. McKitrick and McIntyre confuse radians with degrees.

    Yes you mean like how Daniel Davies and Tim Lambert don’t understand confidence intervals. As for Kane and his views on the Iraq study that is way off topic, any more comments on it and I’ll delete them. Take it to the appropriate forum.

  12. Zelsdorf Ragshaft III says:

    Steve, you have attacked the holy religion of the left and they will attack you. Fortunately, the attack is similar to scabies. Small and irritating but curable. What does one expect from those who believe the lies of a proven liar with whom they agree and call liar, an honest man with whom they do not.

  13. Dear Editor, Sept. 23/07

    Recent research by Henrik Svensmark and his group at the Danish National
    Space Center points to the real cause of the recent warming trend. In a
    series of experiments on the formation of clouds, these scientists have
    shown that fluctuations in the Sun’s output cause the observed changes in the
    Earth’s temperature.

    In the past, scientists believed the fluctuations in the Sun’s output were
    too small to cause the observed amount of temperature change, hence the need
    to look for other causes like carbon dioxide. However, these new
    experiments show that fluctuations in the Sun’s output are in fact large
    enough, so there is no longer a need to resort to carbon dioxide as the
    cause of the recent warming trend.

    The discovery of the real cause of the recent increase in the Earth’s
    temperature is indeed a convenient truth. It means humans are not to blame
    for the increase. It also means there is absolutely nothing we can, much
    less do, to correct the situation.

    Thomas Laprade
    480 Rupert St.
    Thunder Bay, Ont.
    Canada

    http://discovermagazine.com/2007/jul/the-discover-interview-henrik-svensmark

    http://environment.newscientist.com/article.ns?id=dn11462

    http://mypetjawa.mu.nu/archives/188993.php

  14. Hal says:

    Steve, ignorance is a condition – stupidity is a strategy. Sorry if you find the term “ignorant” offensive, but we’re all ignorant about most things. As has been repeatedly pointed out, there is no there there. If a Corp did this I’d us the same response.

  15. joated says:

    ANyone else notice we have our second named “Subtropical” Storm of the year? Like Andrea back in early May, Jerry has formed out in the Atlantic due east of the mid-Atlantic states.

    You don’t think the folks at NOAA are trying to gin up the numbers of named storms to suit their agenda do you? You know, name every little dip in the pressure that looks like it might have circular rotation so they can meet their predicted numbers?

    Nah, me neither.

  16. Tano says:

    “…suppose the upper bound on CO2 is X and in the natural state the actual amount of CO2 is a(t)*X where a(t) is a number between zero and one. All man’s impact would be is to cause a change in a(t) not in X.”

    What is your point here? No one is claiming that humans are manufacturing carbon that isnt present somewhere on earth in the first place. So if X = total carbon on earth, then yes, it is effectivly a constant. But what is the relevance of that? Some of it is buried underground, outside the loop of the carbon cycle operating on land, sea and atmosphere. Until we dig it up.

    If you are talking about CO2 specifically, then you are just wrong. Carbon in the form of coal or oil is not in the form of CO2 – so when we burn these fossil fuels we are adding to the total amount X, of CO2.

    Thats why, of course, the burning of biofuels can be, at least in theory, a partial solution to the problem. It basically uses the carbon that is already present in the surface flux – taking it from plants, burning it into the atmosphere, where it can then be reabsorbed by plants – thus not increasing the total supply, as is done when you burn stuff that had been buried deep underground.

    “…if wetlands are producing methane, then reducing methane output would, in theory, help since it is, IIRC, a “more potent” GHG than CO2.”

    Aye, there is the ticket. There oughtta be a law against all this bacterial degradation of plant material. If only them plants wouldnt die, or if they did, they wouldnt degrade, then we could sequester away all the excess carbon in plant material. As a side benefit, think what it would do for the rake industry!

  17. Hal says:

    Steve, as is hopefully plainly obvious from the comments of your supporters on the right, they clearly believe there is a conspiracy. Apparently, the entire scientific establishment is in collusion with the “left” and – for some as yet unexplained reason – this grand conspiracy is trying to pull a fast one on the rest of the planet.

    The problem, Steve, is that you *only* post about these bizarro like this one of McIntyre’s. You don’t post about anything else wrt global warming. Now, your blog and you can post about anything you like, but it’s quite telling that this is what you select.

    Quite frankly, it’s got all the earmarks of confirmation bias – i.e. the tendency to search for or interpret new information in a way that confirms one’s preconceptions and avoid information and interpretations which contradict prior beliefs. It’s no big deal as everyone has it in spades (experiments by those evil lefty scientists show this). But the problem is you have to do something to actually deal with it or it simply rules your perceptions.

    What’s amazing is the level of conspiracy your fellow commenters here – and you, apparently – believe is going on. Seriously. According to the world view that you seem to buy into, the amazingly competitive world of scientific research has failed to produce any serious challenge other than the rag tag fleet of McKitrick, Essex and McIntyre – all conveniently funded by people with a vested interest in confusing the issues surrounding global warming.

    I mean, really.

    We’re supposed to believe that no one would find it to their advantage to poke holes in global warming? I mean, one of your commenters above is even claiming that NOAA is faking evidence to make sure their predictions come true. And no one – except the jokers previously mentioned – is able to take on this conspiracy. Everyone is colluding to stage global warming for? what? Because the left just wants to stick it to capitalists? Because we’re all just a bunch of commies (where are they these days, except Venezuela?)

    Seriously, what kind of massive conspiracy theory are y’all buying into?

    Maybe you’re right and you and the M&M team are the only ones who see the truth and are tirelessly fighting the fight so that the conspiracy will be revealed. Or maybe you think the entire scientific community doesn’t understand such things as confidence intervals and M&M (and cohorts) are going to set them right. Or maybe you think the entire scientific community has such amazingly shoddy practices such as not releasing their code and/or not providing sufficient notice that they are updating your code and with this campaign you can perhaps do your part to make science whole and trustworthy again.

    Whatever the explanation, I think that the application Occam’s razor would tell you that you’re unlikely to be right and that almost everyone else in the scientific community is right.

    What’s more likely is that the pitiful attacks which have proven over and over to be the products of people who don’t even understand the basic science and tools of the trade which understandably results in blatantly wrong and trivially countered gibberish. And that these misunderstandings and molehills into mountains magnifications confirm your own biases and so you get your shoulders behind them and push.

    What it looks like from the outside (to this admitted lefty who’s in collusion with all the climate researchers and other “liberal” academics) is that you’ve bought into at least part of the massive conspiracy theory that your fellow commenters seem to have bought into in spades. Like I said, your blog, your posts, your interests, your rules. But if you don’t want to look like a raving Bircher you might want to broaden your horizons to counter your own confirmation bias and start posting something other than these choreographed fainting spells you have over the fact that NASA didn’t personally inform McIntyre that they were updating their code.

  18. Grewgills says:

    Interesting, I hadn’t thought about that, but so what. Man isn’t changing that other than possibly releasing some that is sequestered.

    and your answer is bolded for you

    Re: Wetlands
    Wetlands are generally a source of methane (and sulfates) and a sink for CO2. I haven’t (with a quick and lazy search) been able to determine the magnitude of the CO2 sink relative to the CH4 source. This would be necessary to calculate any climate forcing by wetlands.
    Wetlands on a global scale are and have been shrinking in number and size, so whatever this forcing may be it is and has been decreasing for quite some time now.

    If this is true, then why change the data available to the public now? Why not make an announcement of it? Why not make the code available now? They are obviously changing the data so the code must be use currently.

    It is a work in progress, due out later this year. They are updating the website piecemeal as they work on updating the entire project.
    The earlier code is available now and the new coding will be released later when it is finalized.
    As shown and linked to the data source change is not a source change, but an additional process (FILNET) to account for missing data points. Data on this process is also available from NASA.

    A classic case of wanting to have your cake and eat it too. If a corporation did something like this it would be bad…very bad.

    If a company put in place a process to account for missing data and made that process available for review would this really be bad…very bad? Really?
    Once again, the data is the same. It is still the SHAP data processed by FILNET to account for missing data. What about this is so difficult to process?

    Thomas,
    The solar and cloud data break at about 1996. If solar activity were the answer we would be cooling now. A layman’s explanation is available here (graphs near the top).

  19. davod says:

    Gregills: Why didn’t NASA announce what they were doing?

    Hal: Bizzaro? There you go again. You make arguments but undercut them by your abject refusal to accept that others may be commenting from a reasonable basis. Have you recovered from finding out the world is not flat.

  20. davod says:

    PS:

    Hal:

    The difference between now and in the past is that in the past it would have taken fifty years of ruinous government policies before the shallowness of the man-made global warming “evidence” was challenged.

  21. Grewgills says:

    Why didn’t NASA announce what they were doing?

    They announced they were making updates.

    September 2007: The year 2000 version of USHCN data was replaced by the current version (with data through 2005). In this newer version, NOAA removed or corrected a number of station records before year 2000. Since these changes included most of the records that failed our quality control checks, we no longer remove any USHCN records. The effect of station removal on analyzed global temperature is very small, as shown by graphs and maps available here.

    This blanket announcement was simply not explicit enough for McIntyre et al.
    McIntyre was easily able to see that the data were processed through FILNET, so I fail to see how NASA was covering up anything.

    You make arguments but undercut them by your abject refusal to accept that others may be commenting from a reasonable basis. Have you recovered from finding out the world is not flat.

    The flat earthers and creationists are overwhelmingly on the no AGW side of this argument. The hypothesis then, theory then proven fact (to the agreement of almost all) that the earth is a globe rather than a plane either resting on the back of a tortoise or somewhere above hell works far better as an analogy for those denying the AGW hypothesis than for those accepting it. (That also holds true for the “ID” issue.) Perhaps you should find a different analogy.
    Do you really think that Gary, Ragshaft, and joated were “commenting from a reasonable basis” in this thread?

  22. Hal says:

    Davod, you seem to be a true blue Bircher. Keep up the inane commentary! It’s a hoot.

  23. Wayne says:

    Now that they have change the data and created a reverse hockey stick, couldn’t one come to the conclusion that the global temperature is stabilizing? Maybe because it hits it’s saturation point or maybe because manmade industrial contributions has help stabilize the temperature?

  24. Hal says:

    Yes, I’m sure the polar bears will be counting their lucky stars they dodged that bullet.