The Emperor’s New Clothes

Search Engine Optimization (SEO) is becoming much more difficult. And it will get much, much worse.

Six years ago you could easily count the firms doing SEO work. The number of sites competing for each search term were fewer, and the state of the art spamming tricks centered on white-on-white text and multiple title tags. Getting a top search engine ranking out of the 50,000 results returned was relatively easy to accomplish because those 50k results were most often poorly optimized. Getting into the top 10 meant being in the top 0.02% of the 50k results, which is certainly not trivial, but easily accomplished when facing naive competition. A little effort went a long way, and site owners could even do it themselves.

Now the tip of the iceberg is larger. Time has caused two things to happen: the SEO competition is trickier than 6 years ago, and the population of web sites is larger. There are now around 250,000 results for most 2-word searches, and the new sites are often tuned by SEO practitioners. Instead of needing to be in the top 0.02%, you now need to be in the top 0.005% of a more competitive group of pages. Such rankings are still possible, but beyond the ability of most site owners.

Now we see that the rules regarding spam have tightened, and find many of the top ranked sites are violators. As spamming sites are being removed we find that a disproportionate number of the top pages are going away. These “taking the easy way” pages have been in vogue for so long that many newer SEO firms know nothing else and are deciding to “get out of the SEO business” before they go broke. Top SEO rankings are again becoming the realm of those editing web sites instead of creating external pages. Site owners, should they care to take the time to learn a new profession, can still tune their sites, but it is difficult. After all, you still need to be in the top 0.008% of all sites to be top 10 after the spammers are gone.

But now we see that it is truly the tip of an iceberg. What we have never before
seen is a massive amount of hidden content that previously resided
behind the barriers inherent to dynamic content web sites. It has
been estimated that the web is actually 500 times larger than the
number of pages spidered to date.

For the search engines the good news is that the 90:10 rule applies, and that by adding only 50 times more pages, instead of the 500 times predicted to exist as hidden pages, they will account for 90% of the frequently used content on the web. The remaining content is seldom used. The consequence is that the iceberg is no longer hidden. The search engines will need to adjust their algorithms a few times to compensate for the increase in indexed content, but they will love the fresh topic-specific and relevant pages.

For SEO practitioners, it means that rapid growth in the number of indexed pages is imminent (we knew it was coming anyhow). As a result, getting a top 10 organic SEO ranking will soon mean getting a site ranked in the top 0.0002% of the results. Obviously this is not work for lesser SEO practitioners, and certainly beyond the capabilities of most site owners. In fact, we suspect that most search engine optimization firms will be unable to satisfy their client’s SEO ranking requests (close, but no top 10) and there will be significant client dissatisfaction with SEO results in general within six months. We are already finding that many companies have tried three or more SEO practitioners without success, and it will only get worse.

For web design firms, they will find that the rush to the web will come to a crawl. Many web designers will be hard pressed to find new business as prospective clients find top search engine ranking beyond their financial reach. If so, we expect there to be renewed focus on alternate promotion activities such as PPC search engines, ads, and yes, maybe even resurrected banners for the initial nine months or so for each SEO project.

Now the numbers game starts. Suppose there are 200k results today to a 2-word query, and suppose there are only 50 times as many pages for each query once dynamic sites are added to the engines. We believe it is realistic to find ourselves with ten million pages in response to each inquiry within one to two years. Now suppose the average page starts in the middle (5,000,000), and that the average ranking can always be increased 90% with each consecutive “improvement” [aka: tweak] (certainly not always the case, but stay with me). That means that after tweak 1 the ranking improves by 90% to 500k, tweak 2: ranking is now 50k, tweak 3: 5k, tweak 4: 500, tweak 5: 50, and tweak 6 may result in a top 10 ranking. Nobody can get 6 consecutive 90% gains facing increased competition from optimized sites without the right analysis tools and methodologies, and many tweaks may prove ineffective. In fact, the effectiveness of consecutive tweaks diminishes, such that tweak one is 90%, two is maybe 75%, three is maybe 60% and so on. Remember, your competition is tweaking at the same time, and the search engines are constantly refining their algorithms to keep pace with this new content. And each consecutive tweak is more difficult than the previous due to the quality and quantity of the competition. This means that only the top SEO practitioners can ever attain a top 10 result for a meaningful keyword. The rest will just fade away.

Aside from the additional number of tweaks, each with a submission and spidering cycle of over three weeks (making this a much longer process than today, probably doubling the project schedule a year from now), the precision needed to rank a page well in one search engine will certainly disqualify that same page from a simultaneous top natural SEO ranking in many search engines. Pages that rank well today in many engines will find that their aggregated ranking will erode and sites will have to settle for ranking in only the three or four engines at a time. If a client wants other engines to rank their site, then they must tune additional pages, thus they will have to expand their content and organic Search Engine Optimization base to include many more pages within their site. (Remember – doorways and cloaking are considered spam, so editing honest pages will be necessary). Optimizing more pages is certainly the way to go, but it doubles or triples the work involved by the SEO practitioner.

As a result, SEO practitioners have a much more difficult battle, they require much more sophisticated and integrated tools, projects require more time, they must optimize more pages, and thus they must inherently charge much more than today. If this is done top SEO rankings are still very possible, but this becomes the realm of only the exceptionally competent (or very lucky) SEO practitioner.

And any SEO firm that offers guarantees has got to be kidding!

As such, expect a significant growth in the number of indexed pages and expect a fallout of those SEO practitioners taking the easy way to ranking by using spamming techniques that are no longer viable. You can also expect longer natural Search Engine Optimization schedules, expect fewer top rankings per optimized page (necessitating larger projects), and expect an increase in pricing of at least triple that paid today.

What this does to the entire web industry is to scare off those without the funds to participate in a competent SEO effort. What was once thought of as free now has a high cost-of-entry. And this will kill the web as a golden goose. It will cost much more to make money on the web, just like in a “real business”.

The rewards are getting larger, but so is the cost. And as with the Emperor’s New Clothes, many SEO practitioners have been reluctant to discuss this transformation for fear of getting hurt (scaring off customers). We are not afraid to roll up our sleeves and do the hard work for the just rewards. But not everybody is as capable. Many in the SEO industry are standing there naked, and it isn’t a pretty picture. But it is time to expose the difficulties associated with SEO.

The moral of the story is that if the SEO practitioner isn’t well ranked themselves for major industry keywords (Search Engine Optimization, search engine ranking,…), then they probably can’t get you into the top results no matter how much you spend or time you allow. But those that are well ranked will probably do well for any customer given time and resources. Be patient, plan on the expense, and choose well.

View our search engine rankings, and compare it to other SEO firms you are considering.