Anyone who has pushed the bleeding edge of search engine optimization has had a site go greybar, or just totally tank from the SERPs. It really sucks. Sometimes it’s worth a reinclusion request, and sometimes it’s best to keep walking and never look back. Bannings are a very interesting phenomena that don’t get discussed often except by those who are proud to be blackhat. Fortunately, it’s not a problem I’ve had to deal with for personal sites, but I seem to have been getting requests for consulting from a lot of folks who managed to get themselves banned lately, and I wanted to put together some information on the subject. I would like to try to open up a discussion for a few of my own pressing questions including:
I don’t blame the search engines for being ambigious about bannings. It is a sticky subject in terms of mainstream media, and offers no benefits for revealing to those trying to manipulate their results. They want to maintain the impression that all the decisions are made algorithmically by computers. The simple fact is that there will always be some level of human interaction since a human programmed that machine to begin with. If the SE’s told SEO’s EXACT guidelines for most of the criteria, we would all push it right up to that limit. That’s why there is gray territory that they allow us to flounder in, and have to give somebody a good slap in public once they stray too far into darker territory.
How do you know when a ban is a ban?
Google
Generally greybarred on the toolbar, and a site:yoursitehere.com returns no results. This is the full on ban. Here’s a checklist from GoogleGuy.
Yahoo
I’m a little more fond of Yahoo’s method as it makes a bit more sense, and is helpful for a user that may still be looking for a banned site…a site:yoursitehere.com returns only one result for the homepage. No other pages will be indexed.
In either case a search for a block of quoted text from the homepage will return no results where it normally would, and there will be no cache of any of the pages on the site. These are the most basic ways to tell if a site has been banned. There are of course other issues why these types of results may occur (hosting problems, technical issues, etc.), but generally these are the tell-tale sites of a “yep your site is screwed” ban. It’s not really that fun breaking this news to a new potential client that calls on the phone.
When is it a ban? Filter? Penalty? When is it just poor optimization or the “sandbox”?
Bannings are often confused with other symptoms of low rankings. It is also tough for someone to admit that they have been banned I recently spent half an hour on the phone re-iterating the above mentioned points and hoping the gentleman on the other end wouldn’t start to cry aloud. The main reason for this is that their subconcious really wants to convince themself that it hasn’t happend. Sometime they haven’t. Technical issues including too many dynamic pages, session issues, miscoding, or other issues can often cause an algorithmic (or potentially even hand edit) banning. Most folks will claim that technical issues are the number one reason for what seems like a banning other than in instances of blatant and egregious spamming (scraper sites, blogspamming etc.)
When it’s only a penalty.
Penalties and filters occur for a variety of different reasons. Filters are generally the band-aid fixes the engines put on the holes that SEO’s exploit. Examples of this would be the misuse and devaluation of meta tags, alt text, H1’s, and anchor text. They all still have value, but the sum total of going to specific on these may incur a filter. Call it a filter, call it a penalty, call it the sandbox, or call it a kick in the teeth, the simple fact is that it is a reason for your site not ranking and there is immense value in figuring out why or at least potential reasons why. In my mind, filters are a bit lighter than penalties, which are much lighter than bannings.
Filters - If you think in terms of the “Price is Right”…you’ve overbid. SEO is like the showcase showdown. Whoever is closest to the retail price without going over wins. If you’ve been a little over-zealous with putting your EXACT kw’s in every known potentially beneficial area for optimization, you’ve most likely tripped a filter. Filters tripping can generally be fixed through analysis in a fairly short period of time.
Penalties - You really went overboard and kept on going. You hit the wall and kept on driving. You paid no attention to varying your anchor text. You put the exact un-naturally occuring phrase 5 times on each page across all 300 of your pages. Penalties can be rectified, but generally take a longer period of time.
Bannings - I’m still examining these. Luckily, I don’t have a lot of experience here, but certainly have been looking at information from others. This post started upon beginning to ponder when a banning is a hand edit vs. algorithmic collateral damage and has evolved from there. Since this is such a grey area it is very difficult to tell. What I *do* know is most sites that are banned are pretty much screwed, and it is generally easier to start over than to try to retain original top rankings.
Sandbox and Poor SEO - I’ll save this rant for another day. There definitely ARE filters for new sites. They are tough. They have been named the “sandbox” by default. The sandbox has also become the cop-out for every poor optimization strategy launched on a website in the last 2 years. Poor rankings could be the result of a variety of different SEO variables. If you’d like to confuse yourself further with sandbox discussion you can read the sandbox roundup.
Examples of potential penalties
On the other hand, don’t be too afraid of penalties, or linking.
What percentage of bannings are hand bans vs. algorithmic collateral damage?
My best guess is that sites that get banned while they are in the top 50 or so sites for decent volume SERPs are much more likely to be a hand edits. If you are only ranking for a few long-tail phrases and you get de-indexed it was most likely algorithmic. The algorithmic bannings are the ideal, and are preventative maintenence. They are also the source of a lot of griping and trouble because of the collateral damage they sometimes cause to quality sites (that of course DESERVE to be in the top 10). The hand bannings are public relations management. They maintain the image of high relevance and reduce the appearance that the engines can be manipulated. They are another band-aid fix, and the number of them that occurs will most likely remain a tightly guarded secret.
Will the search engines admit to doing hand editing for quality control?
If I were the search engines, this would be released on a “need to know” basis. They definitely don’t want to admit how often it is done currently. I think the whole Eval.google.com fiasco proved this. There is a whole team of folks dedicated to human editorial control (and this is not necessarily a bad thing). I wouldn’t be real excited, however, about the mainstream media twisting around my words and getting the mass public upset if I was the (other) Goog PR team either.
Yahoo certainly has a level of hand manipulation which is quite visible in high profile searches for things like “viagra” where special “h” tags show up in the listings. A lot of discussion has been held on the subject of yahoo hand editing results, and the spam spin of user reported spam for both engines. A bit more information on Yahoo hand editing. If SEO is the Wheel of Fortune, I’d like to skip the vowels and buy the letter H.
A search for Jew reveals the closest either search engine has come to admitting to hand manipulation with their offensive result explanation. Even this only came after quite a bit of poor media coverage on some search results that were algorithmically in favor of dissenting religious groups to Judiasm and Scientology.
I’m really not trying to be a a wild-eyed alien-probbed conspiracy theorist here. I’ve seen it in action, and don’t blame the engines a bit for editing by hand. It would certainly help to improve relevancy, and there is always going to be some level of human decision making involved in the process (even the decision to not make a decision is a decision ;)) There is certainly some level of philosophy that is being ignored and sweeped under the rug. This will most likely be one of the larger issues that they will continue to face in the future. Yahoo has already made the decision to hand edit news, and it has seemed to work well for them. It really shouldn’t be a problem to admit to this, but the I would guess the fear of potential rampant corruption (similar to DMOZ) is a big issue in deciding to embrace and fess up to editing search results. Editorial decisions are made with paid search all the time. The longer this issue is ignored, the more likely it will become a bigger issue with those outside the world of SEO.
How can you best prevent a site from incurring the wrath of a banning while still pushing the bleeding edget of search engine optimization?
Follow the webmaster guidelines from google and yahoo. They are incredibly ambigous most of the time, and there will always be examples of folks that rank well and DON’T follow them. If you don’t follow them, be prepared for the reprocussions that may eventually occur. Establish your risk tolerance, and have a backup plan in case the worst case scenario occurs.
What are the best practices for reinclusion if your important or branded domain gets popped
Reinclusion is a tough proposition. If you do get back in the index, from all I’ve seen and heard it is tough to ever re-attain your original rankings. Most time you will be stuck with some type of “dampening effect” that will allow you to be in the index, but not back at full power. There are of course exceptions to this, however. If you do manage to get re-included, you had better keep your site squeeky clean because you are now definitely above the radar, and the engines do not look kindly on recidivists.
List of discussions on search engine reinclusion:
Tags: Search Engine Banning, Search Engine Optimization, Google, SEO, Yahoo
« Hide it
