Internet Anthropologist Think Tank

The biggest search engine ?

Menlo Park based Cuil will launch later this evening with an index of 120 billion web pages, making them arguably the most comprehensive search engine on the web (Google doesn’t disclose the size of their index, although they claim to know about a trillion unique web pages) (Update: see our very early testing here). They’ve also dropped one of the “l’s” from their name - previously the company was “Cuill.” Either way, it’s pronounced “cool.”

The super-stealth search project was founded by highly respected search experts. Husband and wife team Tom Costello (CEO) and Anna Patterson (VP Engineering) were joined by Russell Power. Patterson and Power are also ex-Google employees, and the company has been the subject of intense speculation over the last couple of years.

Much of the secret sauce of Cuil is in the way they index the web and handle actual queries by users. Both are costly to scale, and Cuil claims to have found a way to massively reduce those costs. That allows them to run the search engine a lot cheaper, even at Google-scale should it ever reach that point. By some estimates, Google spends a billion dollars a year to run the back end infrastructure of it’s search business.

Source:

BACKGROUNDER: Billions of pages a day.

G

.

Labels: Anna Patterson, Cuil, google, Search Engines, Searching, Tom Costello, Web search engine, World Wide Web

Share |

Billions of pages a day.

In a blog post today Google says they’ve identified 1 trillion unique URLs on the web. It’s actually more, they say, but some web pages have multiple URLs with exactly the same content or URLs that are auto-generated copies of each other.

What they note way down in the fourth paragraph, however, is that they don’t actually index all of those pages, so you can’t find them on Google. Estimates on the true size of the Google index are a mere 40 billion pages or so.

Why don’t they index all the pages they’ve found? Some of them are spam. But it’s also very expensive to index sites. And the fact that Google indexes many news sites, blogs and other rapidly changing web sites every 15 minutes makes all that indexing even more expensive. So they make value judgment on what to actually index and what not to. And most of the web is left out.

Google also says “But we’re proud to have the most comprehensive index of any search engine.”

Even after removing those exact duplicates, we saw a trillion unique URLs, and the number of individual web pages out there is growing by several billion pages per day.

SOURCE:

xxxxxxxxxxxxxxxxxxxxxx

That means there are
1,000,000,000,000 web pages,
__40,000,000,000 Google indexed web pages.
______11,000,000 Terrorist pages indexed in our Terror web site search engine.

So there are more pages not INDEXED than Indexed.

Our sources say there is a bigger search engine , watch for updates.
Update: Its here.

G

.