The Economist over Search engines

Search engines: Google is engaged in an elaborate dance with firms determined to keep their websites high up in its rankings

Afgelopen week is er een aardig stuk geschreven in de Economist over Search…toch wel stoer om naast Matt Cutts te worden vermeld: THE industrious spider “bots”? that crawl around the web on behalf of Google, the world’s biggest search engine, evoke both fear and reverence. Late last year the bots swept through the world’s web servers to scrutinise some 8 billion web pages and determine their new rankings in Google’s search results.

As Google tweaks its mighty ranking algorithms, and applies them to the constantly changing pages of the web, different sites shuffle up and down wildly in its search rankings , repeatedly gaining and losing ground. This operation, which takes place two or three times a year, is known as a “Google Dance”.

Like hurricanes, Google Dances are given names, such as Bourbon, Gilligan and Florida, often by commentators at a website called webmasterworld.com. Jagger, the two-week dance that began late last October, moved through servers in three waves, eliciting chatter in internet forums. “Been kicked off the face of the index by jagger1,? reads a typical entry. “Will try not to panic til after jagger3″.

All search engines test and implement new algorithms constantly, though not in such a dramatic fashion. Because achieving a high search ranking is crucial to success for many online businesses, a huge industry has sprung up devoted to the pursuit of website visibility. In 2005 worldwide spending on such “search-engine optimisation? (SEO), grew 125%, to $1.25 billion, according to SEMPO, a Boston-based trade association. It predicts spending will grow by 150% this year.

The Google Dance can rattle companies, since precipitous—but often temporary—falls down the pecking order are common. But Paul Aelen of Checkit, a large SEO firm based in the Netherlands, admits that algorithm updates are “very exciting? for SEO experts, who can then test an array of tricks to figure out what works, and what doesn’t.

SEO firms boost their clients’ online rankings by tinkering with their websites to enhance the characteristics that search engines consider positive. This involves techniques such as simplifying complicated page addresses, rewriting copy to produce single-theme pages with accurate titles, adding extra keywords to the invisible page descriptions (or “metatags?) read by indexing software, and putting product information stored in databases directly on to fixed pages, so that search engines’ bots can read it.

The quest to understand how search engines rank websites has become an obsession for some. Web forums devoted to the topic hum with debate, and many SEO experts comb through patent applications to find out what new algorithms are in the pipeline. That ruse, however, is becoming less effective. Matt Cutts, a senior engineer at Google who is assailed with algorithm questions at industry conferences, says his firm, like its competitors, carefully controls access to its secrets. “A lot of our best ideas don’t get filed as patents because patents eventually become public,? he says.

The best SEO firms can do wonders for a site’s lees het hele artikel op de website van de Economist