Legitimate Reasons To Question September’s BLS Household Survey
While the conspiracy theory is nuts, there are legitimate reasons to be skeptical of some elements of September's Jobs Report.
As my posts yesterday (see here and here) should make clear, I tend to agree with Steven Taylor regarding the absurdity of the reaction to yesterday’s jobs report from many on the right. The idea that the numbers were consciously manipulated is, quite simply, absurd and it betrays the kind of conspiracy theory mindset best left to the people who think that there are aliens being kept at Area 51 and that Elvis and JFK are running a diner in Montana. The fact that it’s being pushed by people like Jack Welch, the former CEO of one of America’s most prestigious corporations, is all the more troubling. Say what you will about Welch, but he’s no Donald Trump, and I’m not sure whether or not he simply tweeted something without thinking (believe me, it happens) or whether he actually believes that the Obama campaign is manipulating economic data. On the other hand, I’m not at all surprised to see people like Allen West pushing this theory, his time in Congress to date has not exactly been auspicious on a number of counts and he seems to attract most of his support from the most radical wings of the Tea Party movement.
All of that said, though, that doesn’t mean that there aren’t legitimate reasons to question some of the numbers that we get from the Bureau of Labor Statistics. As I noted in my posts yesterday, there are actually two completely separate surveys that make up the jobs report that gets released every month from the BLS. As Harvard Economics Professor Greg Mankiw explains, there are issues with both surveys that need to be taken into account:
One might expect these two measures of employment to be identical, but that is not the case. Although they are positively correlated, the two measures can diverge, especially over short periods of time. A particularly large divergence occurred in the early 2000s, as the economy recovered from the recession of 2001. From November 2001 to August 2003, the establishment survey showed a decline in employment of 1.0 million, while the household survey showed an increase of 1.4 million. Some commentators said the economy was experiencing a “jobless recovery,” but this description applied only to the establishment data, not to the household data.
Why might these two measures of employment diverge? Part of the explanation is that the surveys measure different things. For example, a person who runs his or her own business is self-employed. The household survey counts that person as working, whereas the establishment survey does not because that person does not show up on any firm’s payroll. As another example, a person who holds two jobs is counted as one employed person in the household survey but is counted twice in the establishment survey because that person would show up on the payroll of two firms.Another part of the explanation for the divergence is that surveys are imperfect. For example, when new firms start up, it may take some time before those firms are included in the establishment survey. The BLS tries to estimate employment at start-ups, but the model it uses to produce these estimates is one possible source of error. A different problem arises from how the household survey extrapolates employment among the surveyed households to the entire population. If the BLS uses incorrect estimates of the size of the population, these errors will be reflected in its estimates of household employment. One possible source of incorrect population estimates is changes in the rate of immigration, both legal and illegal.
(…)
More important than the specifics of these surveys or this particular episode when they diverged is the broader lesson: all economic statistics are imperfect. Although they contain valuable information about what is happening in the economy, each one should be interpreted with a healthy dose of caution and a bit of skepticism.
That last point is an important one, I think. Much like polls are merely a snapshot in time and any individual poll can be subject to any number of variables that could turn a reliable poll in to an outlier, the twin employment surveys can be flawed for one reason or another in particular months.This is one of the reasons that the BLS routinely revises it’s figures for at least two months after making the initial report. So, to some extent, we shouldn’t put too much import on a single month’s report, especially if it is inconsistent with other, more reliable, economic statistics.
All of this brings up the question of just how reliable some of our economic statistics actually are. For example, for years economists have been criticizing the model that the Commerce Department uses to calculate the Consumer Price Index, arguing that it doesn’t reflect reality anymore largely because it’s based on a model developed decades ago that doesn’t reflect the budget of an average family in the 21st Century an that it doesn’t account for the fact that increases in cost often come with increases in product quality. The issues are different with regard to the BLS employment survey’s, of course, because they deal with questions of the methodology of the survey, but as Jazz Shaw points out, making the survey more accurate could be problematic:
First of all, the survey is not just done by BLS workers. It’s conducted in conjunction with the census bureau. And how many people would they need to interview to produce more statistically meaningful results? Ten times as many? Polling professionals will tell you that it’s hard to get people to complete even a moderate size survey. Take a look at the current survey being used. The labor statistics section alone is more than 20 pages long. If you wanted to get a vastly larger sample, the government would need to be calling millions of people every month. It could turn an already bloated bureaucracy into an unmanageable behemoth.
Jazz seems to question the need for the unemployment reports altogether. I’m not sure I agree with this sentiment. In a modern economy, there’s some importance in the government, and specifically institutions like Congress and the Federal Reserve, having at least some idea of what the state of the economy might be. While it’s true that there are private surveys done by entities like Gallup and ADP that also survey the jobs market, they suffer from their own methodological problems and may not be any more reliable than the BLS survey. Taking the information the BLS provides away, as flawed as it might be, doesn’t strike me as the best response to this situation. Instead, I’d suggest that we should find ways to make the survey more reliable while at the same time making it more clear that a single month’s results aren’t nearly as definitive as the media makes them out to be.
With regard specifically to the September report, economists John Ryding and Conrad DeQuadros of RDQ Economics comment on the supposedly large increase in jobs reflected in the Household Survey portion of today’s jobs report:
This report is a tale of two labor markets. The establishment survey (payrolls) painted a picture of moderately growing employment over the last three months but at a marginally slower pace than over the last year. At this pace of job creation, the unemployment rate should be barely drifting lower given underlying demographic trends. In contrast, the household survey painted a picture of a sharply falling unemployment rate—down 1.2% points over the last 12 months. Such a rapid decline in the unemployment rate would be consistent with 4%-5% real economic growthhistorically but much of the decline is accounted for by people dropping out of the labor force (over the last year the employment-population ratio has risen to only 58.7% from 58.4%). We believe part of the drop in the unemployment rate over the last two months is a statistical quirk (the household data show an increase in employment of 873,000 in September, which is completely implausible and likely a result of sampling volatility). Moreover, declining labor force participation over the last year (resulting in 1.1 million people disappearing from the labor force) accounts for much of the rest of the decline. With this report, the ISMs, and vehicle sales, the September economy is off to a better-than-expected start but nowhere near as good as suggested by the decline in the unemployment rate.
(Emphasis in original)
As we already know, we aren’t in an economy where the economy is growing at a real rate anywhere near 4% to 5%. Indeed, the final revision of second quarter GDP ended up revising growth downward to 1.3% and the estimates for third quarter GDP, the first report of which comes out later this month, has it in the same general range but in any case no higher than 1.5%. So either the GDP numbers are way, way off, which seems unlikely, or there’s some statistical flaw in the Household Survey. Of course, once you take into account the fact that a large number of the new hires were apparently part-time and seasonal jobs, the numbers start to make more sense. More importantly, though, the last time we actually saw anything close to 800,000 jobs created in a single month was in an era when GDP was growing at a mind-blowing 9.3%. Based on all of this, it seems more likely than not, the Household Survey numbers for September are a statistical outlier that we are likely to see adjusted downward, with a subsequent upward tick in the Unemployment Rate, in the coming months.
If we accept that there are flaws, the real question is are these flaws unique to the Obama era, or have they existed across previous administrations?
If there have been changes introduced simply to make Obama look good, that is news. If these are just the same old flawed numbers we have known and loved all along, that is something else entirely.
The weird thing is that no one used to dealing with these numbers was going to get very excited by just one of them. In all the data series, in all the world, one moved from 8.1 to 7.8. As I’ve noted, that isn’t even a big change, just 3.7% variation.
No one would normally spend as much energy as you’ve just done worrying about whether the change will stick. They’d maybe shift outlook slightly, and then wait for next month’s data.
As I’ve been emphasizing, this is all a drama only because the leading digit changed. 7.8 is below 8 and that is big news, even though it is only below by 0.2 and 0.2 is not much.
So the bottom line is to wait a month or two, and don’t get all nuts about the leading digit. If you want to round to a single digit, 7.8 still rounds to 8.
I do not think there is a conspiracy or some major fault with the figures as given( Actually there were some things that happened at 51 that the government will keep classified forever, and they are not related to ufo’s) What are resources that give more information about the jobs statistics? I would want to know breakdowns by age, salary, benefits, and type of work. Where do the numbers come from? How do they compare to previous years? What trends do we see? How is the math done? When there is more data, there can be a clearer picture of what is going on.
@anjin-san:
I am not suggesting that the flaws are unique to the Obama years.
@john personna:
I don’t disagree with you all that much, but the fact remains that these month-to-month results get a lot of play in the media and the political world.
@Doug Mataconis:
Only because of the timing, right? U3 spiked from 9.5 to 9.8 in November 2010, and then dropped to 9.4 the following month. That was a one month 0.4 drop, but no one cared. Oh, and all that variation in 2010 was within “the nines,” with the same leading digit.
@Doug Mataconis:
Watch it Doug, or else everybody is going to come in here and go “Oh jeez, he’s doing his ‘both sides do it again’ drivel.”
@Carson:
The raw numbers are here.
All that really matters here is the trend. It shows that the economy is slowly recovering.This is just one of the clearest indications of that, which is why conservatives are freaking out about this.
“After all we’ve done to wreck the economy and prevent it from recovering, the economy is STILL showing some signs of life. That can’t be POSSIBLE!!!!”
Well, boo effin’ hoo. Had the Presidents Jobs Act from last year been passed, unemployment would be lower still.This is what Obama should be saying when be comments on this.
Had the trillions not been spent on the “stimulus”, the recovery would have been in 2009.
@Carson:
That’s 9000 year old earth stuff. No economic basis. No reasonable model. It is just some meme traded by the faith based universe.
@Carson:
The stimulus was 2/3rds tax cuts. Are you saying that we would have recovered faster if we were all paying more taxes?
The household survey tends to show greater enthusiasm than the establishment one, which is why the leftstream media were so adamant about using the establishment one back during Bush 43. The drop in the U3 simply does not jibe with the number of jobs created unless the U6 took a similar jump. Did it? You’re basing your statistics on a survey of people that may not properly understand the parameters, just so you can say “oh look, U3 is under 8!” Is it merely coincidence that this is just in time for the election?
Blame the media. The BLS provides much of the nuance in the supporting tables that accompany the jobs release.
I’ll explain. The BLS says there were 114.000 jobs created in Sept. That is not true. The 114,000 figure is a statistical estimate within a 90% confidence interval. In other words, based upon the BLS models, the “real” number of jobs created lies within plus or minus one standard deviation of 114.000. How large is the standard deviation? The tables suggest it is about plus or minus 100.000. A more accurate way of describing it would be, “The BLS has determined with 90% confidence that the number of jobs created in Sept. was between 14.000 and 214.000”. Note that also implies there is a 10% chance that the “true” job creation number was less than 14,000 or greater than 214.000. The only thing we know for sure is that the 114.000 number that makes every headline is a fictional number. I am not saying it is dishonestly calculated or there is anything nefarious going on. It’s arrived at using sound statistical science. But it’s simply not accurate to say there were 114,000 jobs created. As the saying goes, “there are lies, damn lies, and statistics”.
It borders on journalism malpractice for reporters to present the number with a false sense of precision and make no effort to explain part of the science and the numerous assumption that go into its calculation. For example, the raw data are adjusted for seasonal adjustments. And as Mankiw correctly points out, the BLS uses something called the “birth/death” model to estimate the number of jobs created by new startups and the number of jobs lost from small business closures. People have been criticizing the “birth/death” model for probably at least a decade. And I’m not talking about internet cranks. I’m talking serious people who analyze this stuff for a living like John Williams at Shadow Statistics, investment analysts, and serious financial bloggers.
I’ve been fascinated by this debate the last few days. I think JP has good perspective.
All of the various purely random statistical problems, sampling statistical problems and the hoary birth death estimate have been well hashed out.
The elephant in the room is the poor absolute jobs number, or in the case of the big one, it’s part time component. This is consistent with an abysmal absolute number of jobs created pattern for months now. Forget the statistical mumbo jumbo of a single month. It’s all political posturing, and pure analytical crap. This economy is nowhere.
@Carson:
Just take a look at the countries that chose austerity…
Yes, Republicans have very good reasons to deny the reality of every piece of news or information that does not conform to their alternative reality. That’s all they do …..
1) Obama? Not American, not legitimate.
2) Polls? Skewed, biased toward Democrats, not legitimate.
3) Labor Statistics? Manipulated by Democrats, not legitimate.
4) Major Media? Biased … blah blah blah … not legitimate.
5) Higher Education? Biased toward liberal ideas, not legitimate.
6) Insurance Mandates? Conservative idea, valid until adopted by Democrat, not legitimate.
7) Carbon Emission Tax Credits (Cap & Trade)? Conservative idea, valid until adopted by Democrats, not legitimate.
@Mr Evilwrench:
Wow! Clever! Kudos!
Jack Welch may be softening his rhetoric a bit, but is not backing away from his original tweet: Welch: ‘I should have put a question mark’ on tweet
Welch is another billionaire in the tank for Romney. So in an attempt to save his boy he said something transparently stupid. He’s an old man, and obviously he made a mistake. I hate to see him lower himself to Donald Trump status.
It makes me nervous — I’m hoping not to suffer major IQ drops as I age. But the fact is you do start seeing little bits of decay around the edges. My night vision ain’t what it used to be. As far as work goes I’m compensating for loss of stamina with the efficiency that comes from experience. But yeah, age comes for us all, and it’s come for Jack Welch.
One thing I hope to avoid is the stubbornness of old age. I’ve never been stubborn, but who knows?
Why is it assumed that a Tea Party individual such as Allen West, or even Jack Welch, for that matter are necessarily biased? I find this position insulting. Both men are raising valid questions. And why does a survey that seeks to ascertain who is working and who is not require 20 pages? There should be an investigation of the methodology and the specific results of this month’s survey need to be looked into. Someone not associated with the DLS should look at the calls made, what was said and compare those to the stats. If there is no audit trail, that should raise some suspicions. Secondly, what are the party affiliations of the people working there? If they are all Obama supporters then there should be some interviews of these people. Thirdly, how many people have been hired here over the last few years and what are the party affiliations of those people? And finally, why are part time jobs counted as jobs? Why aren’t they counted as half a job? Why is it partisan to question this organization? It seems to me that a statistical analysis that leaves open the possibility of mere assumptions has to be suspect. It is hardly scientific. The possibility of swinging an election that is as important as this one, either by means of deliberate cooking of the numbers or even by means of chance requires that we ask questions. I’m not persuaded by a mere assertion that everything is on the up and up.
@Bobby V:
They are career bureaucrats. They are not party hacks. There may be reason to doubt their methodology but not their honesty. It’s well known that Nixon tried to get to them and failed. There has never been so much as a whiff of scandal associated with these public servants, so the reason Welch is being called out is that he made a crude insinuation without any evidence whatsoever. It was the equivalent of me saying, “I don’t know what’s going on with Bobby V, but there sure are some dead hookers. I’m just sayin. . .”
I’m being kind suggesting Welch is senile. The alternative is that he’s an assh0le who trashed hardworking people without any any evidence and solely in order to further his selfish political agenda.
Why are you assuming that the people at the DLS are dishonest? Just because you don’t like the numbers? Should anyone who tells you something you don’t want to hear be investigated?
In Aug to Sept 1980 it fell from 7.7 to 7.5
In Oct to Nov 1984 it fell from 7.4 to 7.2
In Oct to Nov 1993 it fell from 6.8 to 6.6
OMG unemployment going down in election years! It has to be a conspiracy! This is BS!! lol Gee it dropped from 9 to 8.6 in Q4 2011 – WTF people? Where were you all then?
Man, WTF did people do before 24 hour “news” shows and 24 hour internet? Before BLS figures and reports were delivered to your email at exactly 8:30 on the first friday of every month.
My question would be – has the internet age enhanced our intelligence or made us more prone to idiocy?
@anjin-san:
As I have suggested repeatedly, those flaws were pointed out when the left was chanting “Jobless recovery” under the Bush administration., and charging that the use of U3 numbers was deceptive. So, no, they’re not particularly unique, of themselves. What is fairly unique is the way the Obama WH and the press are touting these numbers as if Obama worked some kind of miracle… just when he needed to to save his butt after the (Lack of ) performance in Denver.
And career bureaucrats, whose job depends on governmental largess, tend to lean left, no?
In any event, funny how now suddenly the bets get hedged and there’s now suddenly legit reasons to question the reports.
I suppose that a jobs report that more conforms to the pattern of the previous 44 months of Obama is in the offing, to be released mere hours before the polls open… At which point, we’ll hear charges from the left abot how the whole thing is rigged against Obama.
But the economy still stinks. Witness:
2009 “This 7.8% unemployment rate Bush left me is horrible
2012: “This 7.8 unemployment rate I made happen ROCKS….
@Carson:
Fixed that for you.
If this were not simply crap that you made up, you might have a point. But that quote only exists between your ears. Back here in the real world, Obama has said more times than I can count that we have made progress, but times are still hard and we have a long way to go. That you are unable to give Obama any credit for the modest gains that have been made and lie about how they present them is consistant with your pathology.
Personally I will take tepid job growth over the 500K a month job loss rate that Obama inherited.
Ah, so government workers depend on “lagresse”?
So, cops, firefighters, Marines, teachers – all inferiors who live off the gift of your tax dollars?
What happened to you to make you such a bitter, deluded creature?
To even think something like that requires a complete ignorance of math and economics.
anjin-san said,
What happened to you to make you so hopelessly naive about how government spending really works?
When they want your money it is all about roads, cops, fireman. When they spend your money it is all about some crony on K Street.
@Scott: WRT ordinary citizens understanding statistics and science: This is precisely why it matters that the state of Texas is deliberately dumbing-down our children. This is why it matters that “Honey Boo-Boo” is on the Learning or History Channel or whatever the capitalist substitution for PBS is this week.
When our population is stupid, they are easily manipulated. It’s our modern equivalent of the Southern Strategy.
@Tony W:
Worth noting. TLC was originally The Learning Channel. TLC hosts Honey Boo-Boo.
Maybe the lesson is “go read a f*cking book.”
john personna,
The only way to be sure is to …
Kill Your Television
If this is so easy, name the cuts…
I have a relative who is severely disabled. The government actually does stuff for people like that. I have a mother and a mother in law who are elderly. They actually need Social Security and Medicare.
I went to a public college and university. Got me out of a dead end job and into a well paying career.
Why don’t you get back to me when you have a little more life experience under your belt?
Sorry, but I call bullshit on “Harvard Economics Professor Greg Mankiw”. I seem to recall he had a more prominent role a few years ago. President Dush? Gush? Something like that. I read that line and went “right. Done now. Its a Mataconis piece full of bad faith arguing’
@swearyanthony:
Mankiw is an interesting combination of things. He’s a bright economist, who likes to lie now and then, and runs a blog without comments for that reason. When he’s reasonable (carbon tax) that’s fine, when he goes full partisan he doesn’t want push back. Luckily other econ blogs bring it.
An Interested Party said,
Sure…
Close the following departments:
Department of energy (roll weapons projects over to the pentagon), Department of Ag (roll food stamps to Health and Human services), DEA, TSA, import export bank. End ag subsidies, end ethanol mandates, reduce EPA, OSHA, and department of labor head counts by half.
Military procurement is a disaster streamline rationalize that process and use the saving to help pay for personnel costs.
Means test entitlements allow people to get out what they put in + a reasonable interest rate.
That would be a start off the top of my head. That may not be enough.
In the end cuts are going to be made because Washington spending is outrunning the economies ability to fund the politicians spending addiction.
I can easily see why people could suspect that an administration which is directing companies to explicitly ignore worker protection laws requiring layoff notices be sent out (and offering to pay the legal costs and fines these companies may incur from ignoring the law) might also put their thumb on the scale to influence employment reports to help their re-election campaign.
Defense Companies Hold Off On Pink Slips, For Now – Layoffs Look Bad In Politics
anjin-san,
Vast amounts of federal spending does nothing but take taxes paid by people who have less money and give it to other people who have more money. Ag Subsidies are a good example of this and they represent a tiny amount of that type of federal spending.
The college education that helped you is burying the youth of today in crushing, inescapable, life ruining debt.
How Liberal Arts Colleges Are Failing America
I don’t subscribe to the notion that GDP needs to grow by 5% for unemployment to fall .3%.In fact it has a whiff of totally made up statistic. I will note that when the GDP fell by 8% unemployment grew by 4%.
So let’s end corporate welfare and send the money to people that actually need it. There are a lot of them.
Anyone who expects a “guarantee” of anything in life is a fool. Look, the world has changed. For much of my life, we had 5% of the worlds population consuming roughly a third of it’s resources. Pretty sweet if you were an American in the post WW2 world. Well, that world is gone, and Americans can’t simply pass Go and collect $200 anymore..
Don’t blame it on the liberal arts, and don’t blame it on the government. History is trying to resume its normal course. 1-2% of the population owns pretty much everything, there is a small elite that serves the owners/rulers and does well, and everybody else is a peon. The question is, will Americans just sit back and let it happen.
@Duracomm: So…closing a few federal agencies and departments, changing how military items are procured, and means testing (it would help if you could be a little more specific with that last one) will help to significantly balance the budget? Hmm, these things alone don’t seem to be nearly enough…meanwhile, we have all these people like you telling us that cuts are definitely going to be made…and yet, we’ve been living with significant debts since Reagan, and somehow, we are still here…
Yes, keep telling yourself that it is all a conspiracy…when the President wins reelection, that will be part of the conspiracy too…
@Duracomm:
No cuts made can balance the budget, assuming that is the goal. You forget there are two components to the federal budget, the discretionary component and the cyclical component. The discretionary component is what government chooses to fund during a given fiscal year (the military, EPA, Department of Justice, Forest Service etc.). The cyclical component is sensitive to other factors such as tax revenues and welfare payments. Our current deficits are primarily a product of the cyclical component.
Attempts to reduce the deficit by adopting a contractionary discretionary stance will simply exacerbate the cyclical components by reducing revenues, generating additional unemployment and causing welfare payments to rise. Even if government were to make cuts to welfare payments this would not solve budget deficits because the cuts would drive down revenues as spending contracts.
Ben Wolfe,
I never mentioned welfare
You are correct that discretionary spending is only a portion of the budget. Mandatory spending on entitlements is a big and increasing part of the budget and that also has to be cut.
When fiscal times are good cuts are opposed because hey times are good and there is plenty of money. When fiscal times are bad cuts are opposed because they will be contractionary.
Cuts are going to happen because the amount of money needed to fund the desired spending is outrunning the ability of the productive economy to pay for the spending.
An Interested Party ,
The comment you have in blockquotes was not by me.
You said
Greece’s economy was just fine till it blew up. Lesson there is that fiscal situations can change very rapidly once the bond buyers see increased levels of risk in government bonds.
The government debt that you show no concern over is going to eat an increasingly large and unsustainable percentage of government revenues.
The Problem with Debt Is That It’s Expensive
That which can’t be sustained won’t.
@Carson: “Had the trillions not been spent on the “stimulus”, the recovery would have been in 2009. ”
With no proof whatsoever.
@Herb: “The stimulus was 2/3rds tax cuts. Are you saying that we would have recovered faster if we were all paying more taxes? ”
No, but if the ‘looters and moochers’ were taxed more, then it stands to (econtheological) reason that we’d have less, right?
end of the day, obama’s still 2% higher than he promised. what’s a promise anyway, most of us knew he’d fail. can’t wait for the next debate!