Outside the Beltway

Search Engine Optimization Dirty Secrets

JCPenney used black hat SEO to game Google. But Google's penalties are arguably just as bad. And what about HuffPo?

James Joyner · Sunday, February 13, 2011 · 7 comments

Retail giant JCPenney has been caught using “black hat” SEO techniques to improve their rankings within Google and other search engines.

The company bested millions of sites — and not just in searches for dresses, bedding and area rugs. For months, it was consistently at or near the top in searches for “skinny jeans,” “home decor,” “comforter sets,” “furniture” and dozens of other words and phrases, from the blandly generic (“tablecloths”) to the strangely specific (“grommet top curtains”).

This striking performance lasted for months, most crucially through the holiday season, when there is a huge spike in online shopping. J. C. Penney even beat out the sites of manufacturers in searches for the products of those manufacturers. Type in “Samsonite carry on luggage,” for instance, and Penney for months was first on the list, ahead of Samsonite.com.

[…]

The New York Times asked an expert in online search, Doug Pierce of Blue Fountain Media in New York, to study this question, as well as Penney’s astoundingly strong search-term performance in recent months. What he found suggests that the digital age’s most mundane act, the Google search, often represents layer upon layer of intrigue. And the intrigue starts in the sprawling, subterranean world of “black hat” optimization, the dark art of raising the profile of a Web site with methods that Google considers tantamount to cheating.

Despite the cowboy outlaw connotations, black-hat services are not illegal, but trafficking in them risks the wrath of Google. The company draws a pretty thick line between techniques it considers deceptive and “white hat” approaches, which are offered by hundreds of consulting firms and are legitimate ways to increase a site’s visibility. Penney’s results were derived from methods on the wrong side of that line, says Mr. Pierce. He described the optimization as the most ambitious attempt to game Google’s search results that he has ever seen.

“Actually, it’s the most ambitious attempt I’ve ever heard of,” he said. “This whole thing just blew me away. Especially for such a major brand. You’d think they would have people around them that would know better.”

Essentially, JCPenney got itself linked by thousands and thousands of spammy pages throughout the web solely for the purpose of achieving high search rankings, gaming the system.

Exploiting those hovels for links is a Google no-no. The company’s guidelines warn against using tricks to improve search engine rankings, including what it refers to as “link schemes.” The penalty for getting caught is a pair of virtual concrete shoes: the company sinks in Google’s results.

Often drastically. In 2006, Google announced that it had caught BMW using a black-hat strategy to bolster the company’s German Web site, BMW.de. That site was temporarily given what the BBC at the time called “the death penalty,” stating that it was “removed from search results.” BMW acknowledged that it had set up “doorway pages,” which exist just to attract search engines and then redirect traffic to a different site. The company at the time said it had no intention of deceiving users, adding “if Google says all doorway pages are illegal, we have to take this into consideration.”

J. C. Penney, it seems, will not suffer the same fate. But starting Wednesday, it was the subject of what Google calls “corrective action.”

Last week, The Times sent Google the evidence it had collected about the links to JCPenney.com. Google promptly set up an interview with Matt Cutts, the head of the Webspam team at Google, and a man whose every speech, blog post and Twitter update is parsed like papal encyclicals by players in the search engine world.  “I can confirm that this violates our guidelines,” said Mr. Cutts during an hourlong interview on Wednesday, after looking at a list of paid links to JCPenney.com.

[…]

That said, he added, “I don’t think I could do my job well if in some sense I was not offended by things that were bad for Google users.”  “Am I happy this happened?” he later asked. “Absolutely not. Is Google going to take strong corrective action? We absolutely will.”

And the company did. On Wednesday evening, Google began what it calls a “manual action” against Penney, essentially demotions specifically aimed at the company. At 7 p.m. Eastern time on Wednesday, J. C. Penney was still the No. 1 result for “Samsonite carry on luggage.” Two hours later, it was at No. 71. At 7 p.m. on Wednesday, Penney was No. 1 in searches for “living room furniture.” By 9 p.m., it had sunk to No. 68.

In other words, one moment Penney was the most visible online destination for living room furniture in the country. The next it was essentially buried.

Now, this seems like fitting punishment. Penney gamed the system to rank ahead of its competitors for months and it’s now paying the price.

But here’s the thing: If the rationale for banning black hat SEO techniques is that artificially inflated rankings are “bad for Google users” — which they undoubtedly are — then doesn’t the reverse equally apply? Surely, JCPenney is not the 71st most significant retailer of Samsonite luggage but rather something like, say, the 9th. And, while it’s surely not the number 1 place to buy living room furniture, there surely aren’t 67 better, either.  So, Google’s “manual” penalty is arguably just as injurious to users — indeed, probably more so — that JCPenney’s own gaming.

I honestly don’t know what the solution is. It’s obviously in everyone’s interest to have honest search engine rankings, particularly on Google given its market dominance. But it’s also very much in the interest of website owners to rank as high as possible. Being ranked in the top 5 for something on Google and being ranked 25th or 125th can be the difference between selling your crappy website to AOL for $315 million and going out of business.

While it’s in Google’s interests to have brilliant folks like Matt Cutts keeping an eye out for people gaming the system, it’s even more in their interests to fix their algorithm to make spam techniques less fruitful.

For example, HuffingtonPost has become a laughingstock among insiders for their hamhanded keyword stuffing, with the infamous “What Time Does The Superbowl Start” spam page often cited in recent days. But guess what? It works!  And that’s spawned a cottage industry of SEO “experts” charging vast sums of money, promising to put various sites on the top of Google’s rankings. And, since Macy’s and Bloomingdale’s and Sears and dozens of other companies are competing with JCPenney in that endeavor, it’s a virtual certainty that some or all of their “experts” are using black hat techniques to outdo the others.

Cutts insists this isn’t what Google wants and I have no reason to doubt him. But until Google stops rewarding bad writing and various annoying tricks, they’ll proliferate on the Web. Because, if nobody is reading the page, it doesn’t matter that it’s actually better written and contains more useful content than the pages that the search engines are sending readers to.

via Liz Heron

UPDATE: Stacy McCain weighs in and suggests that other bloggers join Arianna Huffington in gaming Google, Memeorandum, and other aggregators for fun and profit. The gang at Prairie Weather disapproves.

Stacy neglects to mention Techmeme, another of Gabe Rivera’s aggregators, that has even more links to people discussing this story.  The tech bloggers tend to agree with me that search is broken. See Laura June, Matt Rosoff, Adrian Chen, Frank Watts, and especially Michael Arrington. Frank Oliveira dissents, saying we’re all expecting too much from Google, which is really awesome.