Google's 'obey or perish' rules of the internet

Christmas was officially cancelled for heavily VC-backed music lyrics site Rap Genius.

The site, backed by prominent venture capitalist organisation Andreessen Horowitz, managed to irk Google with its search engine optimisation tactics enough to spark an official investigation on Christmas Eve. Rap Genius basically has written lyrics for popular music songs of all genres, but is considered by many to be superior to other lyrics sites due to its use of notations to explain meaning and intent behind lyrics.

When conducting the investigation Google didn’t at all like what it saw and lowered its search engine ranking position significantly, removing Rap Genius from the front page of Google search results for prominent music lyrics and placing it in the wasteland of page 6 and beyond.

The result? Rap Genius traffic fell off a cliff – the re-ranking came into effect on Christmas Day and caused a 60 per cent drop in traffic according to Quantcast.

According to Valleywag, Rap Genius had “effectively been kicked off the internet.”

So what happened? And what does it all mean?

It all started when a blogger named John Marbach posted an email he had received from one of Rap Genius’s co-founders, Mahbod Moghadam, on his blog. Moghadam was spruiking the Rap Genius blog affiliate network and asked the blogger to embed links to Rap Genius’ results for the new Justin Bieber album if he was going to post about it on his blog. Moghadam’s promise was “massive traffic”.

Marbach’s post raising concerns of Rap Genius’ “growth hacking” (their term, others called it shady SEO) strategy made its way around the internet, and was picked up Y Combinator’s ‘Hacker News’ forum by Google’s Matt Cutts.

Cutts was concerned that Rap Genius was engaging in a manipulative linking scheme designed to trick search engines like Google into thinking that the site was potentially more relevant and trusted than it actually was. These sorts of shady SEO tactics have been around for more than a decade, but Google is growing less and less tolerant of them and is taking actions to demonstrate that if you want Google visibility you need to play by Google’s rules.

The traffic drop for Rap Genius was immediate and significant. A 60 per cent drop within 24 hours. It demonstrated two things. For one, it showed just how reliant the majority of websites are on Google to provide traffic. Without Google referrals traffic drops, with less traffic there’s less pages, with less pages there’s less advertising impressions, with less advertising there’s less revenue.

Most sites generate between 40-65 per cent of their traffic directly from Google. For some it’s as high as 80-90 per cent. Over time so-called ‘front door’ traffic (where a user enters a sites url in the address bar) has dropped significantly, as has most publishers efforts to build it. This strategy works well when Google is delivering you traffic, but if there’s a change in Google’s approach to relevance or if you do something malicious to boost traffic that angers the company you can effectively be made invisible within hours. Second, due to its incredible power and consumer trust, Google can effectively create the ‘rules of the internet’. Obey them and you remain visible to their consumers disobey them and you better find a new way to generate traffic.

Rap Genius, which has received $15 million in investment, realised the error of its ways and quickly began apologising to Google. Over the Christmas period the site initiated changes to allow it to better comply with Google’s rules and as of 6 January the site has started to return to the front page. The sites founders even threw their co-founder Moghadam, the person responsible for the scheme, under the bus in a public blog post calling his tactics “lazy”, “ineffective” and “over-the-top”.

As of yesterday, Rap Genius is slowly but surely becoming more visible on Google – a search for current worldwide hit, Drake’s ‘Hold On We’re Going Home’, shows Rap Genius at number two on the search engine results page.

While avoiding dubious SEO gaming tactics disguised as growth hacking is the most public lesson Rap Genius has learned over the past two weeks, the more important lesson – and magnitudes more difficult – is how the site can exist as a brand and destination people access directly as opposed to exclusively via Google.