So Much for Scientific Consensus

One of the things about the global warming/climate change debate that annoys me is the claim that there is a scientific consensus about global warming/climate change, hence it must be true. The problem is that scientific theories/hypotheses can fall with new data. Take for example the idea of six degrees of separation.

Six degrees of separation is the hypothesis that anyone on Earth can be connected to any other person on the planet through a chain of acquaintances with no more than five intermediaries.

The hypothesis was first proposed in 1929 by the Hungarian writer Karinthy Frigyes in a short story called Chains. The concept is based on the idea that the number of acquaintances grows exponentially with the number of links in the chain, and so only a small number of links is required for the set of acquaintances to become the whole human population.

[snip]

In the 1950s, Ithiel de Sola Pool (MIT) and Manfred Kochen (IBM) set out to prove the theory mathematically. Although they were able to phrase the question (given a set N of people, what is the probability that each member of N is connected to another member via k1, k2, k3…kn links?), after twenty years they were still unable to solve the problem to their own satisfaction.

In 1967, American social psychologist Stanley Milgram (see Small world phenomenon) devised a new way to test the hypothesis, which he called “the small-world problem”. He randomly selected people from various places in the United States to send postcards to one of two targets: one in Massachusetts and one in the American Midwest. The senders knew the recipient’s name, occupation, and general location. They were instructed to send the card to a person they knew on a first-name basis who they thought was most likely, out of all their friends, to know the target personally. That person would do the same, and so on, until it was delivered to the target himself/herself.

Although the participants expected the chain to include at least a hundred intermediaries, 80% of the successfully delivered packages were delivered after four or fewer steps. Almost all the chains were less than six steps. Milgram’s findings were published in Psychology Today, and his findings inspired the phrase six degrees of separation.

However, now there is this story in at the BBC.

The phrase was coined by an American academic, Stanley Milgram, after experiments in which he asked people to pass a letter only to others they knew by name. The aim was to get it, eventually, to a named person they did not know living in another city.

The average number of times it was passed on, he said, was six. Hence, the six degrees of separation.

It is a seductive idea.

Films have been made about it, there are parlour games based on it and mathematics has begun to propose theories for why it should be true. But is it?

Judith Kleinfeld, a professor psychology at Alaska Fairbanks University, went back to Milgram’s original research notes and found something surprising.

It turned out, she told us, that 95% of the letters sent out had failed to reach the target.

Not only did they fail to get there in six steps, they failed to get there at all.

Milgram was a giant figure in his world of research, but here was evidence that the claim he was famously associated with was not supported by his experiments.

[snip]

And when she looked for other studies, none of those matched up to the claim either.

In the most recent, two years ago, only 3% of letters reached their target.

“If 95 or 97 letters out of 100 never reached their target, would you say it was proof of six degrees of separation? So why do we want to believe this?”

Take a look at the quote from Wikipedia, note that it refers to 80% of the letter that made it to the target. To claim there is six degrees of separation when 95% or more of the letters never arrive at all is evidence against the claim, not evidence in favor of it. By looking just at the letters that arrive the sample is severely biased.

And this isn’t even new evidence, but a more complete look at the initial evidence in favor of the hypothesis. Now this doesn’t mean that the global warming/climate change hypothesis is not true, but it does highlight that scientific consensus is not a fact and it can change when new evidence comes available, or even with a better look at old evidence.

Via Climate Audit.

FILED UNDER: Climate Change, Science & Technology, , , , , , ,
Steve Verdon
About Steve Verdon
Steve has a B.A. in Economics from the University of California, Los Angeles and attended graduate school at The George Washington University, leaving school shortly before staring work on his dissertation when his first child was born. He works in the energy industry and prior to that worked at the Bureau of Labor Statistics in the Division of Price Index and Number Research. He joined the staff at OTB in November 2004.

Comments

  1. Lorraine says:

    The business networking tool LinkedIn (www.linkedin.com), or even MySpace, is based on this concept too. You sign up and list the people you know and invite them to link to you. Then they list the people they know, etc. It doesn’t take many levels or many contacts to generate 10s of thousands of links.

    Even if it isn’t true, it’s still an interesting exercise.

  2. TTT says:

    There was no “scientific consensus” about the six degrees assertion. There was one published article 40 years ago, which has now been refuted by new research.

    Anyone who tries to conflate a single article about statistics from 1967 with thousands of articles on global warming now, saying that each of those two categories equally represent a “consensus,” is just dancing the creationist two-step. See, all those scientists believed in the Piltdown Man, so evolution isn’t true!

  3. Steve Verdon says:

    Actually, TTT there was at least another paper on it that used e-mail instead of the postal service. So your claim that it was a single article is not quite accurate. Also, there is the “small world” phenomenon that is getting scientific scrutiny that is closely linked to the six degress of seperation. So your portrayal of this topic is…well very much like a creationist.

  4. Tano says:

    What utter nonsense you package this story in Steve.

    First of all, no serious person claims that because a consensus of climate scientists agree on global warming that therefore it is TRUE. The claim is, rather, that since a consensus of those who spend their lives studying the climate hold one position, that position is most likely to be true. Relative to other positions. Beleive it or not, scientists are familiar with the concept of new data undermining previously accepted conclusions. That is the world that they swim in everyday. That you can present this notion as if you were uncovering some little insight makes you seem pretty clueless about what goes on in science.

    Ditto for this six degrees story. Proposed by a writer in a short story? You think this a scientific hypothesis? Then the notion is tested by (what seems to be) some statisticians, and after twenty years they cannot resolve the question? And then a psychologist claims to have demonstrated the idea in one, uncontrolled experiment? And from this you claim that there is a “scientific consensus” about it? Are you on drugs or something?

  5. george says:

    You’re not really equating the position of the National Academy of Sciences on climate change to a single paper on six degrees of separation. Certainly scientific consensus can and ultimately usually is wrong, but up to now it’s the best we’ve been able to do for understanding the nature of the physical universe. If you need absolute proof you have to throw out everything in science and take up math.

  6. Steve Verdon says:

    George and Tano,

    First, there isn’t just one paper. Second, the issue with consensus is most people trot it out as if it ends all debate. A scientist should know that any theory can be and someday probably will be undermined, the problem is what scientists should know, and how they behave are not always the same.

    You want more examples? How about he atom, scientists were convinced it couldn’t exist. A couple of them (Ernst Mach and Welhelm Ostwald) went out of their way to make sure such research didn’t go very far. Of course, turns out they were wrong, the consensus was wrong. There are other instances of where the consensus turned out to be wrong.

    The notion of consensus shouldn’t be the talking point in science, but the evidence. Funny, you guys aren’t making that case.

  7. Anderson says:

    Steve is of course right that “consensus” doesn’t equal “certainty” … another straw man, demolished!

    Btw, Milgram was not the single most ethical guy in the world, & had rather a hard time getting a job after his notorious electroshock experiment for which he’s best known. (IIRC, anyway; Alfred McCoy discusses Milgram in A Question of Torture.)

  8. Steve Verdon says:

    TTT, Tano and George,

    Here is an article that made it into Science.

    They started out sending over 24,000 e-mails and only 384 made it. The biggest reason for failure, the authors surmize, is that of apathy/disinterest as the length of the chain grows. The research, to me at least, refutes the claim of short social networks linking people together across the globe. Yet, the New Scientist claims this research is proof the six degrees of seperation.

    Note, that Watts has been working on the “small world phenomenon” for some time.

    Oh and as for other examples of “consensus” being wrong look at the work fo the guy on viruses and stomach ulcers. The consensus was wrong there as well.

    Now, as I’ve already noted, this doesn’t mean Global Warming/Climate Change is a crock, it just means that using consensus is risky and not very good science.

  9. Tano says:

    Steve,

    You continue to miss just about all the points. You dont need to pepper us with examples of how scientific consensus can be wrong. Most everyone in the world who has spent as much as a semester in a science course understands that.

    Nontheless, at any given point in time, the scientific consensus (if there is one) on any given subject, represents the most widely held interpretation OF THE AVAILABLE EVIDENCE, by those who make their living studying that evidence.

    So your dichotomy between evidence and consensus is an absurd one. Consensus forms around evidence.

    We all know that new evidence can undermine, or further reinforce the existing consensus. Duh.

    In the global warming world, there is a consensus BASED ON THE AVAILABLE EVIDENCE. To say that new evidence might overturn that is really to say nothing remarkable. The GW skeptics need to go out and find some of that new evidence if they have any hope of advancing their arguement.

    In the meantime, some of them seem to be trying to make the case that we should reject the consensus because it is a consensus, and consensuses can be wrong. How mindless is that?

    Why not just reject the skeptics, since skeptics can be wrong? Plus of course, they dont have the evidence on their side either!

  10. Steve Verdon says:

    Tano,

    You continue to miss just about all the points. You dont need to pepper us with examples of how scientific consensus can be wrong. Most everyone in the world who has spent as much as a semester in a science course understands that.

    I think you are far too optimistic here.

    Nontheless, at any given point in time, the scientific consensus (if there is one) on any given subject, represents the most widely held interpretation OF THE AVAILABLE EVIDENCE, by those who make their living studying that evidence.

    Again, I think this is overly optimistic and simplistic.

    So your dichotomy between evidence and consensus is an absurd one. Consensus forms around evidence.

    Ideally, yes. However, the ideal isn’t always the case.

    The GW skeptics need to go out and find some of that new evidence if they have any hope of advancing their arguement.

    I agree.

    In the meantime, some of them seem to be trying to make the case that we should reject the consensus because it is a consensus, and consensuses can be wrong. How mindless is that?

    To some extent I agree, however, the consensus isn’t always representative of the available evidence. Take for example the atom and the ulcer forming stomach virus. In both cases there was damn little evidence in support of the hypotheses, but at the same time that lack of evidence does not imply that those hypotheses are wrong. However, the scientific community took a rather dim view of these two hypotheses and needless to say there was a consensus (based on no evidence I might add). So, while it is true that simply rejecting the consensus because it is consensus isn’t a valid complaint, but the idea that because there is a consensus we must act in dramatic fashion isn’t very persuasive either.

  11. Steve Verdon says:

    Oh and Tano, I got your points, I just reject them as naive and simplistic. In case it isn’t clear to you.

  12. Tano says:

    Steve,

    If you have responsibility for a problematical situation, and the consensus of the relevant scientific field points in one direction, then you either base your decision on that consensus, or find some other justification (non-scientific) for your decision. You cannot decide against the consensus and then declare that your decision is based on science.

    You seem to have a non-science-based preferance for a decision on GW issues that is at odds with the scientific consensus. Why not be straightforward about it? “I don’t care what the science says, this is what I want us to do”.

    I know. In a world in which rational argument has at least some compelling sway, this is not a very persuasive argument. So y’all try to construct an argument that a scientific consensus does not offer a rational basis for action.

    It comes down to an argument for ideology over reason.

    As for me, I make my living in the world of science (biology, not climate), so I dont find your charges, that I am naive or simplistic about how science works, to be very compelling.

    I don’t think you know what you are talking about, and are just doing PR work for a political agenda. In case it isn’t clear to you.

  13. george says:

    Sure, things should just be discussed based on the evidence. Let’s take atoms since you mention them … I’m skeptical they exist. How would you convince me of it without appealing to authority? I suspect that unless you’re willing to go through an awful lot of background (including electromagnetics and quantum mechanics … again not using any authority) you’ll find eventually you’re going to have to say the equivalent of “they’re the best theory available according to people I trust”.

    There are journals of fundamental physics (often a mix of physics and applied math but there you go) which examine the proofs for much of what we take for granted (atoms and other subatomic particles for instance) … most of them are very heavy reading indeed, and very few people have the background to make any headway in them. Even most practicing chemists and physicists (in fact probably all but specialists) “believe” in subatomic particles because we were taught in university that some people did some experiments and came up with certain conclusions.

    In practice, at a certain point we have to trust in a consensus of science or we’d get nothing done … reinventing every theory gets you nowhere. That’s not to say no one should challenge those consensuses (consensei? should have taken more English classes I guess), but it’s hard to avoid the feeling that people are cherry picking when they believe in the consensus and when they don’t.

    Both the left and the right applaud scientific consensus when it backs their view, condemn it when it doesn’t. Fine, except very few of the people making the condemnations have more than a superficial knowledge of the science involved, and then get upset when the science community doesn’t take them seriously.

    I sometimes wonder if this ever happens in other fields … if I demonstrate a complete lack of understanding of finance, would accountants feel obligated to take my views into consideration?

  14. Steve Verdon says:

    Tano,

    If you have responsibility for a problematical situation, and the consensus of the relevant scientific field points in one direction, then you either base your decision on that consensus, or find some other justification (non-scientific) for your decision. You cannot decide against the consensus and then declare that your decision is based on science.

    You know for somebody who claims to be in a scientific field Tano you sure can’t reason very well. I haven’t indicated that I’m hostile to the GW/CC hypothesis. While I was more skeptical in the past, I’m becoming much less so. Still, my problems with the GW/CC issue aren’t so much whether its happening, but the proposed solutions. So the entire paragraph above is irrelevant since it doesn’t apply here. I haven’t decided on the issue of GW/CC on non-scientific reasons and then claimed it is science, especially in this post.

    I know. In a world in which rational argument has at least some compelling sway, this is not a very persuasive argument. So y’all try to construct an argument that a scientific consensus does not offer a rational basis for action.

    No, my view is that simply pointing to the consensus isn’t very persuasive and that pointing to some of the evidence is also a good idea. Have you followed the ID/Creationism debate? My guess is no since that is precisely what most of the anti-IDers do. Not only do they invoke the consensus, they also point to the evidence in the debate. Just about any post over at the Panda’s Thumb will verify this.

    With GW/CC debate the problem is you got a lot of hucksters and politicians out there saying stupid crap. Al Gore runs around the country saying that just about any unusual weather phenomenon is due to GW/CC. Its too cold? Global Warming! Scientific Consensus! It is too hot? Global Warming! Scientific Consensus! More hurricanes? Global Warming! Scientific Consensus! Fewer hurricanes? Global Warming! Scientific Consensus! Wild fires, flooding, tornados, and earthquakes? Global Warming! Scientific Consensus! But that is just crap.

    I know pointing to the data/evidence can be boring, but it sure beats the heck out of “blame every bit of bad weather on global warming.”

    As for me, I make my living in the world of science (biology, not climate), so I dont find your charges, that I am naive or simplistic about how science works, to be very compelling.

    What a suprise. Are all you biology guys such jackasses or is it just you and PZ Myers who ruin it for the rest in the bio field?

    Oh, and since you didn’t get my points in my last comment it wasn’t that you are naive and simplistic about how science works, but about the interplay of science, public opinion, and politics. I’d think that as a biologist you’d get this, what with all the ID/Creationist stuff, but I guess not.

    I don’t think you know what you are talking about, and are just doing PR work for a political agenda. In case it isn’t clear to you.

    And I think you’re an idiot.

    George,

    In practice, at a certain point we have to trust in a consensus of science or we’d get nothing done … reinventing every theory gets you nowhere.

    Whoa, who said we have to reinvent every theory. Using consensus with evidence is one thing. As I noted above, the ID/Creationism debate mixes both the evidence and appeals to consensus/overwhelming evidence pretty well. It doesn’t cover everything and doesn’t get into all the really hardcore stuff, but it gets the point across in many instances.

    By the way my big beef with GW/CC isn’t so much that it is/isn’t happening, but that many seem to use the scientific consensus that it is happening to arrive at conclusions about what we should do. That is, while it might be happening (or not), what to do about it a rather different question entirely…one in which the consensus isn’t nearly as settled, IMO.

    Just to give you some food for thought on that last one, Enron was a big backer of doing something about GW/CC. They probably thought there was a way to make tons off of whatever scheme was going to be slapped together. If that doesn’t give you at least a momentary pause, then I don’t know what to say.

  15. Uncle Pinky says:

    Don’t want to rain on anyone’s parade here, but from what I have seen the “consensus” of scientists on GW/CC is made up of very few qualified climatologists. It is one thing to offer an opinion on a matter in which one is an expert, but to have Sociologists, Psychologists and various other “soft-scientists” listed as authorities undermines, to my way of thinking, the concept of consensus. I am reminded of the astounding, statistically, number of dentists who believe in hypnotherapy for past life regression. As to Science self modifying based on the best and most recent evidence: yes that is its nature. But the nature of statements by alarmists lead one to believe that the consensus is not based on factual studies. To wit: “we have to offer up scary scenarios, make simplified,dramatic statements, and make little mention of any doubts we may have. Each of us has to decide what the right balance is between being effective and being honest.”—Dr. Stephen Schneider. That is not the Science that I learned in school. Call it Quasi-religio-consensual-Sci.

  16. Tano says:

    Well Steve,

    You certainly have the ad hominem-insult part of rational discourse down cold.

    Its good to see that you are open to be convinced of the scientific basis of GW. But it is rather odd to blame me for missing the point that your real concern is with the interface with politics. That was not the issue you raised in your original post. Rather, you focused on attacking the notion that a scientific consensus should lead anyone to conclude that GW is likely true.

    And yes, since my training was in evolutionary biology, I have been very involved in the ID wars. I don’t see the relevance of your allusion though. I have never found climate scientists to be reluctant to engage on any aspect of the evidence base from which the consensus was built.

    Then you launch into an over-the-top, ridiculously distorted account of the arguments that Al Gore is making. Which makes me think that maybe I did in fact peg you wrong. MAybe I should have said: “you don’t know what you are talking about and you don’t really care about the issue itself at all, but are simply a pure partisan scrounging around for any excuse to rant about a political opponent”. Is that a little closer?

    “Are all you biology guys such jackasses or is it just you and PZ Myers who ruin it for the rest in the bio field?”

    No, its all of us. Thats what grad school was all about – turning little sheep into raging jackasses.

  17. DMonteith says:

    No, my view is that simply pointing to the consensus isn’t very persuasive and that pointing to some of the evidence is also a good idea.

    What, pray tell, do you think the consensus is based on, if not evidence? Since you’re so worked up about evidence, is there any evidence that scientists have been ignoring relevant contradictory data? Have they formed an anti-capitalist cabal dedicated to hoodwinking the masses?

    Also, why do you insist that Tano or any other poster have a detailed understanding of the evidence in question? I don’t hold the fact that you don’t seem to know all the details of climate science against you. Isn’t that what experts are for? They look into things so that we don’t have to.

    And finally, I recommend that you read Thomas Kuhn’s “The Structure of Scientific Revolutions” before you take up any more critiques of the scientific method. The Luddite/empiricism challenged community continues to attack science in ways that anyone with a passing familiarity with Kuhn’s arguments should be embarrassed to repeat. Assuming, of course, that embarrassment is part of your repertoire…

  18. Brian S. says:

    Interesting to hear that Steve Verdon is becoming less skeptical. How much less skeptical? Last fall your blog was one of a number that I challenged to bet me over global warming, and I never heard anything.

    If you’re not interested in betting, you’re applying the precautionary principle – you’re basing your behavior on the assumption that global warming is right. Good for you, and now we should make the same type of decision for national policy.

  19. george says:

    By the way my big beef with GW/CC isn’t so much that it is/isn’t happening, but that many seem to use the scientific consensus that it is happening to arrive at conclusions about what we should do. That is, while it might be happening (or not), what to do about it a rather different question entirely…one in which the consensus isn’t nearly as settled, IMO.

    I think that’s a fair statement. Climate (and global industrial society)are very complex, and I don’t know of any climate scientists who pretend to have any kind of scientific based certainty on what the best solution is. Science is about understanding and describing nature, not deciding upon the best way of modifying it. You can know a lot about quantum mechanics, for instance, and still have no idea on how to design and make an integrated circuit. The same is true for global warming, and the climate scientists I’ve talked to don’t pretend to know what the solutions is (they’re really not simplistic enough to believe everyone is going to stop polluting). Only politicians have that kind of certainty.

    My complaints are with those who argue that the scientific consensus isn’t complete and thus is meaningless. Even among climate experts there’s as good a consensus on global warming as you’ll find on any scientific issue … try reading a journal on particle physics if you want to see real disagreement between different theories. Since the same could be said for every theory in physics (even the most basic theories have conflicting views … and in most cases a range of views), I can only imagine they mean we might as well stop doing physics, since the certainty they desire will never come from any science.

  20. Steve Verdon says:

    You certainly have the ad hominem-insult part of rational discourse down cold.

    It isn’t an ad hominem when it is true Tano, you’re a prick.

    And yes, since my training was in evolutionary biology, I have been very involved in the ID wars. I don’t see the relevance of your allusion though. I have never found climate scientists to be reluctant to engage on any aspect of the evidence base from which the consensus was built.

    Try reading Climate Audit. One of the problems is getting data from the scientists on this issue. Turns out they are very reluctant to hand it over. Of course, they try to hide behind, “They are energy industry shills,” but I find that completely unsatisfying in that that is a type of ad hominem. Letting people look at the data with a different perspective isn’t something bad, in fact, as a Bayesian I’d say it is a requirement. Also, this problem isn’t specific to just climate science, but to quite a few areas of academic inquiry. Making data and models available to those who want to look at them should be pretty easy in the digital age. Oddly enough, it seems to be really hard to get data.

    Then you launch into an over-the-top, ridiculously distorted account of the arguments that Al Gore is making. Which makes me think that maybe I did in fact peg you wrong. MAybe I should have said: “you don’t know what you are talking about and you don’t really care about the issue itself at all, but are simply a pure partisan scrounging around for any excuse to rant about a political opponent”. Is that a little closer?

    No, and now I know you are an ignorant prick as well. Several years ago, whe Gore was still VP and pushing Kyoto he was running around the country pointing to various things like wild fires, hurricanes and so forth and then launching into his song and dance about global warming. Frankly, the similarity between Gore’s antics then and some of Dembski’s antics is pretty sad.

    What, pray tell, do you think the consensus is based on, if not evidence?

    Please go back and read previous comments.

    And finally, I recommend that you read Thomas Kuhn’s “The Structure of Scientific Revolutions” before you take up any more critiques of the scientific method. The Luddite/empiricism challenged community continues to attack science in ways that anyone with a passing familiarity with Kuhn’s arguments should be embarrassed to repeat. Assuming, of course, that embarrassment is part of your repertoire…

    You know, reading some of my past posts on how scientists evaluate data might disabuse of this false belief of yours.

  21. Tano says:

    Steve,

    Ad hominem is Latin for “toward the person”, which means that a characterization of a statement (in this case, an insult) as ad hominem has nothing to do with whether it is true or not.
    It simply refers to the target – a person (jackass, prick), rather than the idea they are presenting.

    I have Climate Audit in my bookmarks, along with about a dozen other sites that focus on the issue. I don’t find their arguments persuasive, nor does, apparently, very many other scientists.

    I wonder what it means to consider oneself a “bayesian” – if by that you mean anything more than that you find the theorem, and the statistical tools based on it, to be useful sometimes. Perhaps you mean that you are, by nature, someone who is uninterested in questioning your own prior assumptions, and only interested in how they evolve?

  22. Ray says:

    Is it me, or do most “intelligent’ people seem to miss the obvious. This being the fact that the Earth is a dynamic system in constant change and every time we try to use static models for prediction the results never match the observed environment and when we try to apply solutions based on that static model we end up making things worse.

    Suppose we do everything we can to stop global warming and actually succeed. Do you really think that such a condition will last for any appreciable period of time? Do you really believe that stopping global warming will actually have a positive effect on nature? Will nature even allow us to interfere? Can anyone reasonable answer these questions? Anyone? Anyone?

    It seems to me that every time we try to artificially stabilize a dynamic system we end up actually speeding up change and causing more damage then if we just left that system alone or adapted our response to include that dynamic change. People, we can’t stop entropy, nature will not allow that. It would be more prudent to adapt to a changing environment than to force a changing environment to adapt to us. That what evolution is all about, isn’t it?

  23. Tano says:

    Yes Ray, it is you.

    Believe it or not, climate scientists have figured out, even before you have, that the Earth and its climate is dynamic. And therefore, they construct, wait for it…., dynamic models.

    The goal is not to somehow stop the dynamic Earth in its tracks. It is to stop introducing a human factor that is driving an unusually rapid temperature change, one that would so quickly change the environment that the natural processes of adaptation would not be able to keep up with – indeed our own very rapid human technological adaptation might not be able to keep up with. Leading to enormous economic dislocation for humans, and devastation to many other species.

    You seem to have a fundamental misunderstanding here. There is not some natural change that we are trying to control or stop. The natural changes that you refer to when you speak of a dynamic earth, are changes that happen gradually – on a scale that dwarfs a human lifetime. The change we are seeing now is man-made, and extremely rapid relative to natural dynamics.