It’s Hard to Audit the Rich, So We Go After the Poor

The IRS is outgunned.

I love taxes police money Euros
Image under CC0 Public Domain license from pxhere.

A month ago, I blogged extensively on Jesse Eisinger and Paul Kiel’s ProPublica report, “The IRS Tried to Take on the Ultrawealthy. It Didn’t Go Well.” My takeaway was taxing the rich is harder than it looks.

Eisinger and Kiel are back with a piece in the NYT explaining “Why the Rich Don’t Get Audited.” Essentially, they argue, the IRS is simply outgunned.

The hot policy in Democratic circles these days is raising taxes on the rich. Senator Elizabeth Warren has a plan to tax “ultramillionaires,” as she calls them. Senator Bernie Sanders wants to expand the estate tax. Representative Alexandria Ocasio-Cortez has floated raising the top income tax rate to 70 percent for those making over $10 million a year.

But before this country raises taxes, it should grapple with something much more prosaic but equally important for tackling inequality: saving the Internal Revenue Service.


Slowly and quietly over the past eight years, the I.R.S. has been eviscerated. It’s lost tens of thousands of employees. It has fewer auditors now than at any time since 1953. In real dollars, the agency’s budget has dropped by almost $3 billion since 2010.

Businesses and the wealthy benefit the most from this state of affairs. The largest corporations in America used to be audited every year. That started to change when the cuts began, and today, the audit rate has fallen by half. It’s a similar story for individuals making $10 million or more a year: With twice the chance of escaping I.R.S. scrutiny, the ultrarich are much less likely to lose at the game of audit roulette.

While Eisinger and Kiel don’t say so directly, that’s surely no accident. While Americans love to hate the IRS, it’s obviously those who owe the most in taxes who benefit from these cuts.

The ultra-affluent — with the help of legions of tax professionals — make domestic income disappear overseas or hide it in a pyramid of partnerships. It’s like trying to take on a modern army while armed with spears and clubs.

The I.R.S. has difficulty tracing the income of the superwealthy or countering their sophisticated arguments about why what appears to be one type of income is actually something else. The agency also has trouble valuing their assets (a problem that, as The Times revealed, dates back at least to when Fred Trump was misleading the I.R.S. about how much his buildings were worth). By the public admission of numerous I.R.S. officials, it has long done a poor job of scrutinizing complicated partnerships to understand who owns what portion of what stock.

The top 0.5 percent of highest-earning Americans account for about a fifth of the income that’s hidden from the I.R.S., according to a University of Michigan study, or more than $50 billion a year in today’s dollars.

That was the predictable outcome from a weaker IRS. Less predictably, perhaps, was the perverse corollary.

It’s much easier to enforce the tax laws for the bottom 90 percent of earners. Wages are reported straight to the I.R.S., and computers can easily check that tax returns accurately report that income. This means that inadequate enforcement of the tax laws necessarily has a regressive effect, liberating those at the top from scrutiny while the masses continue to be tracked by machines.

With its budget slashed, the I.R.S. has pulled back across the board — except for one area where it’s been easier to keep the numbers from falling so much: audits of the poor. More than one-third of all audit targets claim the earned-income tax credit, one of the nation’s largest anti-poverty programs. By the hundreds of thousands, I.R.S. computers spit out letters that require low-income taxpayers to prove their eligibility. The counties with the highest audit rates aren’t found in the hedge fund precincts of Connecticut or the lobbyist enclaves of Northern Virginia. No, they’re rural, mostly African-American counties in the Deep South.

Given that the cutbacks began when Barack Obama was President and Democrats had a majority in the Senate, one presumes siccing the IRS on poor blacks wasn’t the goal. No, this is almost surely just bureaucracy doing its thing. An algorithm finds an anomaly. An audit gets triggered. It’s low-hanging fruit.

Going after those being more creative in their financing, meanwhile, is painful, exhausting, and expensive.

There’s nothing easy about auditing the rich. Even when the I.R.S. has evidence of flat-out cheating, such as when it discovers a hidden Swiss bank account, it can take years of work to prove its case. Taking that kind of time clashes with one of the core ways the I.R.S. evaluates its managers and agents: how efficiently they open and close audits. As one senior manager put it to us, “If you’re not churning out the exams, you have to explain why you’re not.”

But more often, the affairs of the superwealthy are infernally complicated and the I.R.S. doesn’t have black-and-white proof of evasion. Instead, the agency runs up against what’s called “aggressive” planning in the tax world. To the rich, tax returns are just opening offers to the government. If they don’t get audited, that’s great. If they do, well, they’re ready for a fight.

The focus of my previous posting on their work, the failure-to-launch of the “audit Delta Force,” consumes the next part of their analysis here and I won’t repeat my thoughts on that.

They close with something of an irony:

So, what to do? One 2020 candidate already has a bold proposal to resuscitate the I.R.S. It’s a plan to pump tens of billions into the agency, enough to fund a second army of agents. That candidate’s name is Donald Trump.

Of course, it was Mr. Trump’s party that gutted the agency, costing the country, we estimate, more than $100 billion in lost revenue. The president’s 2020 budget, however, did propose adding $15 billion over 10 years to the I.R.S. enforcement budget, a move that would more than double the agency’s overall budget. The administration estimated this would produce revenue of at least $47 billion. The question arises: Has the administration genuinely had a change of heart? Or is this merely a neat trick to make deficit projections appear smaller?

So far, this is all just talk. It is, though, one of the few areas where Mr. Trump and Democrats are in substantive agreement. Democrats in the House recently proposed a $400 million I.R.S. budget increase next year.

As I noted earlier, it’s simply not true that Republicans are solely responsible for gutting the IRS. By Eisinger and Kiel’s own telling, the cuts started eight years ago—when Democrats controlled the White House and Senate. But of course Trump’s talk of beefing up the IRS now is just talk. Had he wanted to do it, he’d have done so in his first year in office when he had maximum “mandate” and Republican control of both Houses of Congress.

Among the Democratic presidential candidates, Ms. Warren does mention the I.R.S. in her wealth tax proposal. She calls for an unspecified but “significant increase in the I.R.S. enforcement budget” and a minimum audit rate for taxpayers subject to her “ultramillionaire” tax.

Raising taxes on the rich is a more compelling cause than increasing the operating budget of a bureaucracy most American voters probably hate thinking about. But the boldest tax plan of all starts with salvaging the I.R.S.

Any of the Democratic contenders would surely be better than Trump on this issue. Warren would likely be the most effective in this regard simply because she’s an honest-to-goodness expert on the subject who brings both detailed knowledge and passion.

Still, while fixing the IRS is an obvious step, it’s a secondary one. As OTB regular Michael Reynolds observed early in our last conversation on the topic, “No amount of high-priced legal help can hide your money unless politicians have first written laws creating hiding places.” We need to radically simplify the tax code to make it harder to be “creative.”

Ironically, this was always a Republican Party talking point—albeit one aimed at middle-class voters rather than the wealthy. Those of us whose financial situation is sufficiently complicated that we can’t fill out the 1040-EZ* and yet not sufficiently toney to afford an army of lawyers and accountants, tax season is a giant pain in the ass. But, for us, the only place to “hide” our money is in tax-deferred retirement accounts.

Alas, those who have enough money to hide from the IRS have the means and incentive to lobby Congress for carve-outs. It’s unlikely, indeed, that we’ll ever see tax laws that aren’t written for their benefit.


*The -EZ has actually gone away with the 2018 filing season, per the most recent changes in the tax law. I’m using it here as a shorthand for “extremely simple filing.”

FILED UNDER: Taxes, , , , , , , , , , , , , , ,
James Joyner
About James Joyner
James Joyner is Professor and Department Head of Security Studies at Marine Corps University's Command and Staff College. He's a former Army officer and Desert Storm veteran. Views expressed here are his own. Follow James on Twitter @DrJJoyner.


  1. Kit says:

    Another pain point is the obligation of US citizens living abroad to file US taxes. Every year, there goes another weekend lost. Most civilized countries simply require people to file where they live, but the US wants its cut, too. My guess is that most of the small fish lose time and money (if they decide to pay for an accountant), but otherwise escape the net. And I rather doubt that the big fish are ever troubled. So it’s all just useless bureaucracy.

    Oh, and try opening a bank account abroad: many will no longer accept American clients as the reporting obligations are too damn onerous. I guess the day will come when China forces financial institutions all over the world to dance to it’s tune. Cue American outrage.

  2. Mikey says:

    Ironically, this was always a Republican Party talking point

    Lots of things were Republican Party talking points. Deficit control, not selling out to Russia, you know, that sort of thing. That’s all gone by the wayside in the age of Trump.

    But one thing that is always GOP-consistent: fucking over the poor to benefit the rich. The pattern of who gets audited and who doesn’t is just a manifestation of that central Republican principle.

  3. Tyrell says:

    New rules are needed: no audits what so ever for anyone who makes less than $100, 000 annual gross income.
    Anyone who has an annual income of less than $50, 000 does not have to file.

  4. OzarkHillbilly says:

    The same can be said for the entire criminal justice system too:

    It’s Hard to Convict the Rich, So We Go After the Poor

    is a title waiting for a blog post.

  5. I would note, too, that tax simplification rhetoric from the GOP was usually linked to the flat tax, which would have been a huge boon to the wealthy as well.

    But yes: the tax code is out of control. Indeed, our entire system needs a massive overhaul, but that seems rather unlikely to happen.

  6. Guarneri says:

    Well, Reynolds is basically correct. What lefties fail to understand is that this problem is pervasive. It’s called regulatory capture. But they only like to talk about it wrt things like taxes, but not some new regulation inevitably destined to be twisted in favor of the wealthy and connected regulated.

    Just look at the Boeing situation. How did regulation do there? The answer is to stop writing dense and manipulateable prose and start prosecuting fraud and gross negligence. (And stop recasting it as extremely careless, but I digress).

    As for the NYT authors, I stopped listening at pyramids of partnerships. They are pass through entities, pure and simple. It’s not the structure, it’s the intent to not report, or to fraudulently claim expenses. Again, deal with the fraud or you are not a serious person on the subject.

  7. Hal_10000 says:


    For once, I’m going to agree with you … a bit. The insane complexity of laws — tax and regulatory — is a huge benefit to wealthy powerful interests. Warren was saying something the other day about why the big banks never got prosecuted for their role in the financial crisis. It’s very simple: the laws governing banking are complex and the banks can hire lots of lawyers to drag it out (fun fact: the Exxon Valdez lawsuits were still unwinding 25 years after the spill). It was just easier to get them to cough up a tiny amount of money. Of course, that only increased the moral hazard.

    The tax code is the same way: it’s been 30+ years since our last real reform (Republican BSery about Trump’s tax cut not withstanding). That provides enormous room for the rich and powerful to litigate it out because it’s cheaper to pay lawyers than to pay taxes.

    A massive simplification of the system would help. It provides fewer shadows for the cockroaches to hide.

  8. Claire Matthews says:

    I just experienced this first hand. I gave my employees raises increasing my employer tax withholdings to where I would need to make payments monthly instead of quarterly. Didn’t know this tax policy until I received the fine. Since I have been completely up to date on taxes outside of this I filed for a first time penalty abatement. Got denied and now I have to pay over $300. Leave us little guys alone or at least show us some grace all I did was give raises to my employees!

  9. DrDaveT says:

    By Eisinger and Kiel’s own telling, the cuts started eight years ago—when Democrats controlled the White House and Senate.

    Well, except that this turns out not to be true. The IRS budget grew for the two years that Democrats still controlled the House, then began its sharp decline. The Democrats only controlled the Senate for four months out of those 2 years; they used those four months to pass Obamacare. It’s hard to complain about their priorities.

    Perhaps more importantly, the IRS workforce (full-time equivalents) peaked in 1992. Yes, automation has something to do with that, but the current workforce is only 2/3 as large, dealing with a population 27% larger.

  10. Just nutha ignint cracker says:

    @Kit: Filing my taxes while I was working in Korea was always both labor intensive and simple. But I had the advantage of working for Korean employers in whose service I had no US tax liability, so all I needed to do was certify who I was working for and subtract my income from line 4 (?) on line 25 to net 30 or 40 dollars (in a good year) in interest income as my AGI. Had I gone to Korea 5 or 10 years earlier, I wouldn’t have needed to even do that. I think the change started a few years after Chevron started paying Gazprom Russia to hire a cousin of mine to work in Russia on a joint project so that she could say she had “no US income.”

    ETA: @ Ozark: NO KIDDING! I wonder when we’ll see THAT post.