June 5, 2013

Penguin Recovery: Does Google's Disavow Tool Work? A Case Study

Published: 5 June 2013 

Since Google released the Disavow Tool last year a question many SEOs have been asking is - should I disavow links for sites affected by Penguin? Will it help me recover?

In a comment to a recent Search Engine Watch article a reader asks - where is the proof that the link disavow tool has worked for anybody?

A few days ago Marie Haynes published an interesting example of a likely Penguin recovery assisted by the disavow tool. I thought I'd share an additional example with some thoughts on the process.

The site described below was primarily a test site, one of several similar websites and was ranking for its primary keyword at #1 for several months before the initial Penguin update in April 2012. The links had been built, not acquired naturally and the site's anchor text had been heavily focused around keyword-rich phrases. While the anchors were diversified the site had a very small number of branded, URL-based or generic anchor text links. At its peak there were probably about 100 linking root domains pointing to the site, as reported by Google Webmaster Tools.

The site was hit by the the first Penguin update after which it was left unattended until February 2013 when I submitted a disavow file with over 50 domains that were likely hurting the domain's rankings.

Nothing happened until 9th May (10th in Australia) when the site's ranking soared:

You might say - Wait, wait, wasn't Penguin 2.0 (a.k.a. Penguin #4) rolled out on 22nd May not 9th? Yes, however from what we've seen and from what others have noticed it seems that Google either pre-released the new Penguin or refreshed a portion of the data used for its calculation earlier, around the 9th of May. Interestingly, some of the other sites which had similar link disavowals completed (I'll get to them in a moment) recovered on the 9th of May while others on the 22nd.

Were singular pages or whole domains disavowed?

The "domain:" parameter was used for ALL submitted links. The reason for this approach was that after inspection it turned out many of the links were being duplicated across several pages on their host domains (mainly on tag pages, category pages and the like), so being extra careful I decided to err on the side of deleting too many links than missing toxic ones.

To be sure that all potentially problematic links are disavowed I included in the disavow files even those domains which no longer hosted links to the penalised site, but were reported in Google Webmaster Tools or of which I knew that had hosted such links in the past.

Which links were included in the disavow file and which were omitted?

Links were classified as good or bad solely based on their anchor text - if the anchor text was not the URL, brand name or a generic phrase it was labelled unnatural and the link was included in the disavow file. Links with branded or URL-based anchor text were preserved in most cases even if they came from a potentially spammy domain.

Was there any link-building performed since the initial Penguin penalty?

Not really. After the site was hit by Penguin #1 in April 2012 one relatively insignificant link was pointed to the domain in December 2012. No significant, longterm change in rankings was observed after the link was published.

Was a reconsideration request filed?

No. Since the site was clearly hit by an algorithmic penalty and nearly all of the site's links were far from kosher, I decided not to submit a reconsideration request.

Has the site recovered fully?

While the site has not regained exactly the same rankings as before the penalty, I do believe it has recovered fully. Why? Consider the following time-line:

  1. the website was ranking #1 for its main keyword for several months before it got hit by Penguin
  2. after Penguin hit, the site dropped to position #8-#10 for several months. The positions kept gradually dropping, likely due to link decay, to a low of ~#30 in February 2013
  3. at that point the majority of the site's links were disavowed. Over the next couple of months the rankings kept dropping at an even larger pace, maybe due to the disavow file being processed or due to further link decay
  4. finally on 9th May 2013 the site bounced back to position 10, a position last seen over 9 months ago. Over the next couple of weeks the site's positions crept up to as high as #4, significantly higher than the initial positions directly after Penguin initially hit.

So with the majority of the links disavowed, a year of link-decay and nearly no new links the site improved to rankings not seen after Penguin 1. The simplest explanation seems to be that the Penguin penalty has been either completely lifted or very significantly reduced.

Was this a fluke or can it be replicated?

While I've shared here only rankings of one site I went through a similar disavow process for a number of sites. Overall after disavowing links for 14 sites, 10 have shown signs of clear improvement after the latest Penguin update.

Why did not all of the sites recover? Here is some speculation on potential reasons:

  • too cautious with disavow: not enough links were included in the disavow file
  • site is too weak to rank: the site recovered from the Penguin penalty, BUT so many links were disavowed that it no longer has enough links to rank even after the penalty is lifted
  • disavowal was too late: not enough time had passed between submitting the disavow file and a Penguin recalculation, as the majority of the links will need to be re-crawled before Penguin is rerun. This is a possible reason but not likely in the above cases as the disavowals were performed in February. In general though at least a gap of several weeks is likely needed
  • there's something else going on: site has been hit by a different penalty on top of Penguin. This was the case with at least one of the sites, which had been previously affected by the EMD update.

One More Experiment: Will a 301 Redirect Shake-Off a Penguin Penalty?

As all the links to the site were linking to the www version of the home page, in May 2012 I redirected the site to its non-www version, so that http://www.site.com became http://site.com.

A similar change was made at that time on a number of domains, none of those sites recovered during the following Penguin 2 or Penguin 3 updates in May and October 2012.

Lessons learned

#1 The Penguin penalty can be lifted by simply disavowing a significantly large number of offending links.
#2 Submitting a disavow file on its own will not always bring the rankings back.
#3 A reconsideration request is not required to lift a Penguin penalty (you should consider it though if you think a manual penalty has been applied to your site).
#4 A simple 301 redirect within the same domain will not shake-off a Penguin penalty.

What to do if your site has been hit by a Penguin penalty

  • Collect all the the links you can. A good idea is to collate links from several sources like Open Site Explorer, MajesticSEO or Ahrefs, so that you end up with a comprehensive list. That said, a GWT backlinks export should suffice in most cases.
  • Check each link's anchor text and mark as unwanted all incoming links with unnatural anchors (usually keyword-rich). Take note also of any links coming from extremely poor quality neighbourhoods, but focus primarily on finding unnatural anchor text.
  • Include your list of links in the disavow file. Use the domain: parameter heavily. It's better to be a bit more drastic than too conservative when dealing with a Penguin penalty.
  • Submit the disavow file as soon as possible, so that Google has time to recrawl all these links before the next Penguin refresh.


Good luck!

Since Google released the Disavow Tool last year a question many SEOs have been asking is - should I disavow links for sites affected by Penguin? Will it help me recover?

In a comment to a recent Search Engine Watch article a reader asks - where is the proof that the link disavow tool has worked for anybody?

A few days ago Marie Haynes published an interesting example of a likely Penguin recovery assisted by the disavow tool. I thought I'd share an additional example with some thoughts on the process.

The site described below was primarily a test site, one of several similar websites and was ranking for its primary keyword at #1 for several months before the initial Penguin update in April 2012. The links had been built, not acquired naturally and the site's anchor text had been heavily focused around keyword-rich phrases. While the anchors were diversified the site had a very small number of branded, URL-based or generic anchor text links. At its peak there were probably about 100 linking root domains pointing to the site, as reported by Google Webmaster Tools.

The site was hit by the the first Penguin update after which it was left unattended until February 2013 when I submitted a disavow file with over 50 domains that were likely hurting the domain's rankings.

Nothing happened until 9th May (10th in Australia) when the site's ranking soared:

You might say - Wait, wait, wasn't Penguin 2.0 (a.k.a. Penguin #4) rolled out on 22nd May not 9th? Yes, however from what we've seen and from what others have noticed it seems that Google either pre-released the new Penguin or refreshed a portion of the data used for its calculation earlier, around the 9th of May. Interestingly, some of the other sites which had similar link disavowals completed (I'll get to them in a moment) recovered on the 9th of May while others on the 22nd.

Were singular pages or whole domains disavowed?

The "domain:" parameter was used for ALL submitted links. The reason for this approach was that after inspection it turned out many of the links were being duplicated across several pages on their host domains (mainly on tag pages, category pages and the like), so being extra careful I decided to err on the side of deleting too many links than missing toxic ones.

To be sure that all potentially problematic links are disavowed I included in the disavow files even those domains which no longer hosted links to the penalised site, but were reported in Google Webmaster Tools or of which I knew that had hosted such links in the past.

Which links were included in the disavow file and which were omitted?

Links were classified as good or bad solely based on their anchor text - if the anchor text was not the URL, brand name or a generic phrase it was labelled unnatural and the link was included in the disavow file. Links with branded or URL-based anchor text were preserved in most cases even if they came from a potentially spammy domain.

Was there any link-building performed since the initial Penguin penalty?

Not really. After the site was hit by Penguin #1 in April 2012 one relatively insignificant link was pointed to the domain in December 2012. No significant, longterm change in rankings was observed after the link was published.

Was a reconsideration request filed?

No. Since the site was clearly hit by an algorithmic penalty and nearly all of the site's links were far from kosher, I decided not to submit a reconsideration request.

Has the site recovered fully?

While the site has not regained exactly the same rankings as before the penalty, I do believe it has recovered fully. Why? Consider the following time-line:

  1. the website was ranking #1 for its main keyword for several months before it got hit by Penguin
  2. after Penguin hit, the site dropped to position #8-#10 for several months. The positions kept gradually dropping, likely due to link decay, to a low of ~#30 in February 2013
  3. at that point the majority of the site's links were disavowed. Over the next couple of months the rankings kept dropping at an even larger pace, maybe due to the disavow file being processed or due to further link decay
  4. finally on 9th May 2013 the site bounced back to position 10, a position last seen over 9 months ago. Over the next couple of weeks the site's positions crept up to as high as #4, significantly higher than the initial positions directly after Penguin initially hit.

So with the majority of the links disavowed, a year of link-decay and nearly no new links the site improved to rankings not seen after Penguin 1. The simplest explanation seems to be that the Penguin penalty has been either completely lifted or very significantly reduced.

Was this a fluke or can it be replicated?

While I've shared here only rankings of one site I went through a similar disavow process for a number of sites. Overall after disavowing links for 14 sites, 10 have shown signs of clear improvement after the latest Penguin update.

Why did not all of the sites recover? Here is some speculation on potential reasons:

  • too cautious with disavow: not enough links were included in the disavow file
  • site is too weak to rank: the site recovered from the Penguin penalty, BUT so many links were disavowed that it no longer has enough links to rank even after the penalty is lifted
  • disavowal was too late: not enough time had passed between submitting the disavow file and a Penguin recalculation, as the majority of the links will need to be re-crawled before Penguin is rerun. This is a possible reason but not likely in the above cases as the disavowals were performed in February. In general though at least a gap of several weeks is likely needed
  • there's something else going on: site has been hit by a different penalty on top of Penguin. This was the case with at least one of the sites, which had been previously affected by the EMD update.

One More Experiment: Will a 301 Redirect Shake-Off a Penguin Penalty?

As all the links to the site were linking to the www version of the home page, in May 2012 I redirected the site to its non-www version, so that http://www.site.com became http://site.com.

A similar change was made at that time on a number of domains, none of those sites recovered during the following Penguin 2 or Penguin 3 updates in May and October 2012.

Lessons learned

#1 The Penguin penalty can be lifted by simply disavowing a significantly large number of offending links.
#2 Submitting a disavow file on its own will not always bring the rankings back.
#3 A reconsideration request is not required to lift a Penguin penalty (you should consider it though if you think a manual penalty has been applied to your site).
#4 A simple 301 redirect within the same domain will not shake-off a Penguin penalty.

What to do if your site has been hit by a Penguin penalty

  • Collect all the the links you can. A good idea is to collate links from several sources like Open Site Explorer, MajesticSEO or Ahrefs, so that you end up with a comprehensive list. That said, a GWT backlinks export should suffice in most cases.
  • Check each link's anchor text and mark as unwanted all incoming links with unnatural anchors (usually keyword-rich). Take note also of any links coming from extremely poor quality neighbourhoods, but focus primarily on finding unnatural anchor text.
  • Include your list of links in the disavow file. Use the domain: parameter heavily. It's better to be a bit more drastic than too conservative when dealing with a Penguin penalty.
  • Submit the disavow file as soon as possible, so that Google has time to recrawl all these links before the next Penguin refresh.


Good luck!

Ben Maden

Read more posts by Ben

7 comments on “Penguin Recovery: Does Google's Disavow Tool Work? A Case Study”

  1. Thanks for the mention in your article Nemek. I saw you in my referrer stats and popped over for a look. It's an interesting case study. Is it a Penguin recovery? It's tough to say!

    No one seems to know what happened on May 9. My gut instinct is that it was a minor Panda refresh. Google is no longer announcing Panda refreshes because they are happening so frequently, but they still are pushing them out. The reason why I say that I think it is connected to Panda rather than Penguin is that I have one site for which no link building was done. This site was getting 5000 visitors per day and has a whack of completely naturally earned links. There is no way that Penguin could affect it. However, it has had a couple of dips and then recoveries with some of the Panda updates. This particular site saw a significant traffic dip on May 9 as well.

    But Panda probably wouldn't explain what is happening with your site so who knows...perhaps it was one of those other random algo tweaks that Google is constantly doing. It does definitely seem that disavowing your links had a positive effect.

    Thanks for reporting. Even if we don't know what is happening, the more information we have from actual sites the better.

    Marie

  2. Thanks for the comment Marie. I was wondering if it could have been a release from a different penalty, but find it hard to piece a different theory together. Looking at the rankings the site was very clearly hit by Penguin in April 2012 and has recovered at some point from the penalty (because rankings are now higher than at any point after the initial Penguin demotion). There was no improvement after Penguin update #2 or #3, hence the idea that some kind of (partial?) Penguin refresh must have happened around the 9th of May. Appreciate your input!

  3. Penguin april 2012 was so near to a Panda Update.

    Matt tell us that hadn't penguin update on 8may

    The most probably was Panda

  4. Thanks Carlos for the comment. I have considered it being a Panda hit, but the original drop date (24/25 April) matches the Penguin 1.0 spot on. Since some of the sites mentioned recovered on the 9th and some on the 22nd of May (and all looked like Penguin victims) I might gather the data together and revisit the topic in a separate blog post.

  5. Great case study example Nemek. I also often perform a full link audit and disavow file submission for new clients that have low quality, over optimised link profiles from previous SEO work. I also have the links pinged and crawled as this seems to speed up the recovery process.

    My theory is that the disavow file is honored only once Google has recrawled the disavowed links, regardless of any Penguin updates. This has been my experience with many of the sites I have tested. Since it can take Google months to recrawl such low quality links, it makes sense to force them to crawl them to speed up the entire process.

  6. Well, penguin is dangerous, specially for those who are not aware of Good Link building strategies. They can get hit by this algorithm update quite easily and recovery for them is also very difficult. Being faced this scenario in my blogging career, I can easily understand the pain of recovering a blog from Penguin penalty, as it can months to remove bad backlinks and then, wait for the penguin refresh, so that our blog gets free from penalty. Anyways, I must say, the post is quite good, well written. Thumps Up for this.

Leave a Reply

Your email address will not be published. Required fields are marked *

Shares