Zogby Numbers Cooked for Clients?

Mark Tapscott points to a Washington Times piece by Joel Mowbray that implies pollster John Zogby’s recent survey showing results unkind to Wal-Mart is suspect because Zogby has done work for anti-Wal-Mart interests. Tapscott observes that, “Zogby developed his survey questions in consultation with Wake Up Wal-Mart and that the advocacy group even wrote the news release announcing the results of the survey.”

I would note, however, that survey research firms often have conflicts of interest but still produce good numbers. My wife works for Public Opinion Strategies, the top Republican polling firm in the country. The company also does polling for several companies. But, while POS has an interest in their clients’ causes, they have an even stronger interest in getting the numbers right. If they don’t, the clients stop soliciting their services and they go out of business.

Zogby gained a reputation for getting the numbers right even while others were getting them wrong. Ironically, it was the likes of Fox News and Rush Limbaugh that helped tout his successes, because they were weighting raw numbers in a way that–correctly–predicted Republican wins in close elections while others were going the wrong way. As Tapscott notes, though, Zogby has been wrong in the opposite direction of late, including predicting Gore and Kerry victories. If he gets it wrong often enough, people will quit paying him any attention.

The partners at POS love this line from Glengarry Glen Ross:

“As you all know, first prize is a Cadillac Eldorado. Anyone want to see second prize? Second prize’s a set of steak knives. Third prize is you’re fired.

In polling, you’re only as good as your last set of numbers. Zogby damned well knows that. I can’t imagine he’d put out bad numbers for one client and jeopardize his company.

Update: I do agree with Tapscott and Bill Nienhuis, though, that it’s inexplicable that Zogby didn’t disclose his affiliations. That was stupid.

FILED UNDER: General, , , , , , ,
James Joyner
About James Joyner
James Joyner is Professor and Department Head of Security Studies at Marine Corps University's Command and Staff College and a nonresident senior fellow at the Scowcroft Center for Strategy and Security at the Atlantic Council. He's a former Army officer and Desert Storm vet. Views expressed here are his own. Follow James on Twitter @DrJJoyner.

Comments

  1. Anderson says:

    Zogby actually had my hopes up on Kerry; I had resigned myself to his loss until 2 or 3 days before the election, then Zogby gave me hope.

    We are so through.

  2. Sgt Fluffy says:

    I’ve gotten to the point were I don’t trust any polls, even the ones that are positive towards the president. I really don’t trust something that only asks say 1000 people out of millions their opinion and tout it as accurate.

  3. James Joyner says:

    Sgt Fluffy: When you’re cooking up some chili, do you have to taste the whole pot before you know whether it needs more Tabasco?

  4. Anderson says:

    DeLong goes off on a fine rant vs. his bete noir Donald Luskin on the subject of statistics.

    The whole purpose of the science of statistics is to tell us that … as long as you can take a random sample of your population, you can find out an enormous amount about the population from a relatively small number of observations. You can find out what proportion of rich people had poor paretns, or what proportion of twenty year olds think they will graduate from college, or pretty much any other average proportion that you want.

    Now the “random sample” part of this is very important. But if your sample is random–if the fact that the yes-no pattern of observations so far makes it no more (or less) likely that you next observation will be a “yes”–then the law of large numbers tells us that the sample average you compute will converge to the true population average at a frighteningly rapid speed.

  5. spaceman says:

    Zogby has been showing his true colors for a few years, a Bush-hater creeping out of the closet.

    So it’s not surprising he’s associated with the latest liberal fetish, Wal-Mart whipping. Ya know, I have yet to see anyone forced against their will to go Wal-Mart.

  6. Herb says:

    Anderson:

    You have confirmed that your comments and Bush hatred is tainted by you “personal” loss of the 2000 and 2004 elections. You were led to falsly think that Kerry was actually going to win while we in the red states mid west knew all along that it would be a Bush Victory

    Your problem Anderson is that you look at your Liberal political views from a left coast point of view and don’t know a thing about the thinking of people from the center of our country.

  7. bryan says:

    Of course, it’s also possible to produce “good numbers” with loaded questions, like the bi-polar question that leads the CNN article.

    “I believe that Wal-Mart is bad for America. It may provide low prices, but these prices come with a high moral and economic cost for consumers.”

    vs.

    “Wal-Mart is good for America. It provides low prices and saves consumers money every day.”

    Compare that with the results elsewhere in the survey which seem to indicate a much less clear picture:

    The Zogby poll also questioned consumers on whether they thought that Wal-Mart was becoming too powerful an economic force in America. Some 33 percent were very concerned, while 20 percent said they were not at all concerned.

    Thirty-three percent strongly agreed that Wal-Mart was a retail monopoly that threatened the future health of the U.S. economy, but 35 percent did not agree at all.

    It seems that the number actually concerned about Wal-Mart is closer to 33 percent than 56.

  8. I’m rather skeptical of his online Zogby Interactive polls that he’s been releasing and that makes me a little critical of Zogby. He knows that no one will realize that they are online polls, and that this methodology is still somewhat suspect.

    He’s been releasing polls in the Texas gubernatorial race, which flavor the coverage. Polls are now news items in themselves, which help influence how journalists and voters see races. I’m not impressed that he doesn’t more clearly label his online polls.

  9. Anderson says:

    You have confirmed that your comments and Bush hatred is tainted by you “personal” loss of the 2000 and 2004 elections. You were led to falsly think that Kerry was actually going to win while we in the red states mid west knew all along that it would be a Bush Victory

    Backwards, Herb; my Bush hatred led me to take the loss of the elections personally. 2000 not so much, since how bad Bush would be was not yet evident.

    Your problem Anderson is that you look at your Liberal political views from a left coast point of view and don’t know a thing about the thinking of people from the center of our country.

    Pretty funny, since my part of the left coast is called “Mississippi.” Not the center, you’re right.

    As I admitted, I only kidded myself into seeing a Kerry victory in the last few days of the campaign, based not least on Zogby. As it was, a few more votes (or fewer Diebold machines?) in Ohio, and we’d have had our second pop-vote loser in the White House.

    (Don’t mistake me for a Kerry fan, btw; I hope he doesn’t run again.)

  10. Carol says:

    I mistrust 100% of the polls highlighted in the news.

    If a poll is “on-line” you have no idea who filled in the answers.

    I can think of only one research company that recruits for on-line polls in an expensive but trustworthy way to ensure the people on-line are exactly who they say they are. No one else goes to any substantial effort.

    It is the easiest thing in the world to bias the results. Wording of a question. Wording of the answers. Choice of answers. The way the questions are offered, ie. what’s asked first, second and so on.

    It is very, very hard to do good quantitative research and eyewateringly expensive.

    Just saying.

  11. Herb says:

    Anderson:

    Wow, you are from Miss. ? I didn’t think that Miss had any democrats left down there. Perhaps
    you took my comment out of context, I said you look at your liberal political views from a left coast point of view. And that still applies wherever you are from. Although you may be from Miss. you still don’t think and act like you know a thing about the central part of the country, Being from Miss, you are from the South and while it may be central, it is still South. And while Miss is a Red State, it makes me wonder where you went wrong.

    Oh well, I guess there some are dinosauers that still remain in the red states in contrast to the herds of them in the Blue States. While you are surrounded in Red, I guess there may still be some hope for you to see the light and not to look at everything through dark sunglasses.