All of America’s Secrets on Amazon Cloud
The US intelligence community is gambling that it can be more efficient through a public-private partnership than going it alone.
The US intelligence community is gambling that it can be more efficient through a public-private partnership than going it alone.
Government Executive (“How the CIA Dismissed the Status Quo, Partnered With Amazon and Changed Intelligence“):
The intelligence community is about to get the equivalent of an adrenaline shot to the chest. This summer, a $600 million computing cloud developed by Amazon Web Services for the Central Intelligence Agency over the past year will begin servicing all 17 agencies that make up the intelligence community. If the technology plays out as officials envision, it will usher in a new era of cooperation and coordination, allowing agencies to share information and services much more easily and avoid the kind of intelligence gaps that preceded the Sept. 11, 2001, terrorist attacks.
For the first time, agencies within the IC will be able to order a variety of on-demand computing and analytic services from the CIA and National Security Agency. What’s more, they’ll only pay for what they use.
The vision was first outlined in the IC Information Technology Enterprise plan championed by Director of National Intelligence James Clapper and IC Chief Information Officer Al Tarasiuk almost three years ago. Cloud computing is one of the core components of the strategy to help the IC discover, access and share critical information in an era of seemingly infinite data.
Obviously, this was a contentious issue in a business that values secrecy above almost all else. But they came up with a solution that satisfied their security needs while giving them a better information platform than they could have built in-house.
Money was a factor, according to the intelligence official, but not the leading one. The government was spending more money on information technology within the IC than ever before. IT spending reached $8 billion in 2013, according to budget documents leaked by former NSA contractor Edward Snowden. The CIA and other agencies feasibly could have spent billions of dollars standing up their own cloud infrastructure without raising many eyebrows in Congress, but the decision to purchase a single commercial solution came down primarily to two factors.
“What we were really looking at was time to mission and innovation,” the former intelligence official said. “The goal was, ‘Can we act like a large enterprise in the corporate world and buy the thing that we don’t have, can we catch up to the commercial cycle? Anybody can build a data center, but could we purchase something more?
“We decided we needed to buy innovation,” the former intelligence official said.
After wasting almost two years in a legal battle between Amazon and IBM over the contract, the deal was done. So, how was the security issue resolved?
The Amazon-built cloud will operate behind the IC’s firewall, or more simply: It’s a public cloud built on private premises.
Intelligence agencies will be able to host applications or order a variety of on-demand services like storage, computing and analytics. True to the National Institute of Standards and Technology definition of cloud computing, the IC cloud scales up or down to meet the need.
The upside is huge:
As with public clouds, the IC cloud will maximize automation and require standardized information, which will be shared through application programming interfaces, known as APIs. Amazon engineers will oversee the hardware because AWS owns the hardware and is responsible for maintaining it just as they do in the company’s public data centers.
Whenever Amazon introduces a new innovation or improvement in cloud services, the IC cloud will evolve. Company officials say AWS made more than 200 such incremental improvements last year, ensuring a sort of built-in innovation to the IC cloud that will help the intelligence community keep pace with commercial advances. Wolfe said AWS’ capacity to bring commercial innovation from places like Silicon Valley to the IC is one of the contract’s greatest benefits. Whenever AWS introduces new products, the CIA will be able to implement them.
“The biggest thing we were trying to do—the visionary folks a couple years ago—was answer the question, ‘How do we keep up?'” Wolfe said. “The mission we have is important. The pace and complexity is really not [diminishing], in fact, it may be increasing. We feel it is very important to deliver the best IT and best products and services we can to our customers in the IC.”
That simply wouldn’t be possible with an in-house solution. Every upgrade would require massive bureaucratic wrangling, especially since it crosses 17 agencies.
Yet, security concerns remain:
The IC cloud “will be accredited and compliant with IC standards,” says a senior CIA official familiar with the IC cloud. It will, for example, be able to handle Sensitive Compartmented Information, a type of classified information. “Security in the IC cloud will be as safe as or safer than security on our current data centers,” the senior CIA official says.
Because the IC cloud will serve multiple tenants—the 17 agencies that comprise the IC—administrators will be able to restrict access to information based on the identity of the individual seeking it. The idea is to foster collaboration without compromising security. Visually, the IC cloud can be thought of as a workspace hanging off the IC’s shared network—a place where data can be loaded for a variety of tasks like computing or sharing. The IC cloud gives agencies additional means to share information in an environment where automated security isn’t a barrier to the sharing itself. This could prove vital in situations reminiscent of 9/11, in which national security is an immediate concern.
Cloud vendors, including Amazon, have argued that cloud infrastructures can be more secure than traditional data centers because there are fewer points of entry, but the leaks by Snowden illustrate the potential threat from inside an organization. Snowden was able to access and download classified information intelligence officials said he shouldn’t have been able to access.
To access information within the IC cloud, analysts must have the proper permissions. In addition, the standardized environment and automation means all activity within the cloud is logged and can be analyzed in near real-time.
Some government officials view cloud computing as inherently less secure than computing on locally controlled servers, but the CIA’s acceptance of commercially developed cloud technology “has been a wake-up call” to those who balk at it, according to John Pirc, a former CIA cybersecurity researcher who is now chief technology officer at NSS Labs, a security research firm.
There’s no such thing as perfect security and in any case security is in direct tension with availability. After the 9/11 attacks, we came to believe that stovepiping—the siloing of information in the agency or bureau that collected it, with very guarded sharing with others—was tremendously harmful. Yet, as Wikileaks and the Snowden case demonstrate, the more information that any individual has, the more he can share illegally without clearance.
I don’t have the technical knowledge to assess the tradeoffs here but the intelligence community has access to some of the best experts. If they’re confident that the risks are minimized, the gains in infrastructure certainly seem impressive.
Presumably the next Snowdenesque whistleblower will be an Amazon employee. Going out-of-house has the detrimental side effect of exposing your entire information network to people who technically don’t have any clearance.
@Tillman: I doubt that’s true. That is, I’d imagine those who are actually working on the IC’s cloud have in fact gotten clearances. We make contract employees who even work in buildings where classified information is handled, even though they themselves would never have a need to know to get authorized to see said information, to be either cleared or personally escorted.
A cousin who works in IT pointed out that these guys can rely on Amazon’s experts, or their own, to fight the entire internet.
What I find quite interesting is that this means that Amazon can guarantee that all servers and networking is under physical/legal US control. I was told that Google docs and such can’t be used in healthcare because Google could not guarantee that the data would stay on US soil (once it’s off US soil, other legal systems can grab it). Considering the fact that Google was painfully aware of that last fact (NSA cooperation), and I was surprised by them not dealing with it.
There’s an unusual amount of tracking software used on this website.
@Cletus: I presume it’s a function of the ad networks which, ironically enough, I’ve turned over to an outside contractor. Is there something in particular that’s of concern?
@James Joyner:
It’s not a big deal but just something I noticed when I updated my Ghostery software. I realize the site doesn’t run itself and ad dollars help you guys operate, but it seemed a little bit excessive.
@Cletus: No, I get it. We’re running into the same issues that the big boys are: the bottom has fallen out of web advertising and so we’re making up for it with more units trying to find something that works. But we need to do better at eliminating the ones that serve bad ads, especially if they’re not generating much revenue.
@James Joyner:
I’m getting loud invisible audio commercials. It’s gotten to the point that I have to turn off the sound in my browser if I open more than one OTB article at a time.
@Grewgills: That shouldn’t be happening, either. I’m afraid one or more of our networks are occasionally sneaking ads with audio. I’ve even hit on an interstitial–the ones that suddenly transport you to a new page—a few times. We’re trying to run those down.