AllTheWeb: FAST's search engine, 1999–2011

By Bill Hartzer · Updated 19 August 2026

What AllTheWeb was, and where its results came from

AllTheWeb — originally FAST Search, at fast.no before alltheweb.com — was a general-purpose, crawler-based web search engine built by the Norwegian company Fast Search & Transfer. It ran its own crawler and its own index: its own stored copy of the web, built by software that fetched pages itself rather than relying on anyone else's data. For a period around 2002–03 it was the main technical rival to Google on index size and crawl freshness, and heavy searchers liked it for its speed, its advanced query syntax and its early image, video, audio, FTP and news verticals.

That independence has an end date, and it is not the one most people give. AllTheWeb stopped running its own index around 2003–04, not at its 2011 shutdown. Wikipedia is explicit that shortly after Yahoo's acquisition the AllTheWeb site started using Yahoo's database, and that several advanced functions were removed at the same time. Yahoo had bought Inktomi earlier the same year and already had a crawler and an index of its own; it had no reason to fund a second one.

So the honest structural summary is three phases. Its own index from 1999 to roughly 2003–04. Yahoo's results behind the AllTheWeb interface from then until 2011. Nothing at all after 4 April 2011. The distinctive interface outlived the distinctive index by about seven years, which is why so many accounts describe an independent engine that survived until 2011. It did not.

The crawler and the index

FAST ran its own crawler, identifiable in server logs by the FAST-WebCrawler user-agent family. The recorded strings are a small fossil of the ownership change: one reads FAST-WebCrawler/3.8 (crawler at trd dot overture dot com…) — trd for Trondheim, and an Overture contact address that only makes sense after the 2003 sale.

The documented index milestones:

2 August 1999: around 200 million unique URLs, a record at the time.
June 2002: over 2 billion pages crawled.
August/September 2003: roughly 3.2–3.3 billion documents, per Search Engine Watch's index-size history.

AllTheWeb also powered other companies' search. From launch, per Wikipedia, FAST aimed to provide its database to other search engines, major portals, internet service providers and content sites; the products sold under that model were FAST Web Search and FAST PartnerSite, both named in the 2003 Overture acquisition. In FAST's own framing, the consumer site at alltheweb.com was partly a shop window for the technology it wanted to license. Specific named portal customers are widely asserted but could not be confirmed from a primary source, so they are not listed here.

Who owned it, and how it made money

The ownership chain matters, because skipping a step in it makes the rest of the story unreadable:

Fast Search & Transfer ASA, Norway, from 1999. FAST was founded on 16 July 1997; AllTheWeb grew out of FTP Search, a doctoral project begun by Tor Egge at the Norwegian University of Science and Technology in Trondheim in 1994.
Overture Services, Inc. Overture announced on 25 February 2003 that it would acquire FAST's internet search division — FAST Web Search, AllTheWeb.com and FAST PartnerSite — for $70 million in cash plus up to $30 million in performance earn-outs over three years. FAST kept and refocused on enterprise search. The purchase completed in April 2003.
Yahoo! Inc. Yahoo announced its acquisition of Overture on 14 July 2003, in a cash-and-stock deal reported at the time at $1.6 billion, closing that October. Some later sources give about $1.63 billion; $1.6 billion is the contemporaneously reported figure.

Revenue came from advertising plus technology licensing. The Register reported at the time of the Overture sale that enterprise search accounted for about 75% of FAST's revenue — consumer web search was roughly a quarter of the business it was selling. Under Overture and then Yahoo, AllTheWeb was monetised with paid search listings; Overture's paid-placement business was the thing Yahoo was actually buying.

The index-size race with Google

In June 2002 AllTheWeb announced 2.1 billion indexed pages and claimed to have “out-Googled Google”. Google disputed the comparison, pointing out that it had passed 3 billion documents in December 2001.

Both statements were true, because the two engines counted differently: Google's total included URLs it knew about but had not fully indexed, while AllTheWeb's claim was about pages fully indexed. The honest version is that the two were close and the comparison was contested. Any flat assertion that AllTheWeb had the world's largest index is unsupportable without that qualification, and it is one of the most repeated errors about the engine.

What is not in dispute is that AllTheWeb's crawl was fast and its refresh cycle was better than most rivals of the period, and that its advanced operators and multimedia verticals were ahead of Google's at the time. It did not lose on quality. It lost on distribution and on money — Google had the default placements and the advertising revenue to fund an index race that a Norwegian enterprise-software company was running as a demonstration of its licensing product.

Timeline, from FTP Search to shutdown

1994: Tor Egge begins FTP Search as a doctoral project at NTNU, Trondheim.
16 July 1997: Fast Search & Transfer founded.
Mid-1999: AllTheWeb launches; ~200 million unique URLs by 2 August.
June 2002: the 2.1 billion page claim, and Google's rebuttal.
18 February 2003: Overture announces it will buy AltaVista from CMGI for $140 million — $80 million in stock and $60 million in cash.
25 February 2003: Overture announces the FAST web search purchase, one week later.
April 2003: Overture completes both acquisitions.
14 July 2003: Yahoo announces its acquisition of Overture; the deal closes that October.
Shortly after October 2003: alltheweb.com switches to Yahoo's database and loses advanced functions. The independent index effectively ends here.
8 January 2008: Microsoft announces an offer for the remaining Fast Search & Transfer — by then an enterprise search company only — valued at about $1.2 billion. FAST becomes Microsoft's Norwegian development centre and its technology becomes FAST Search Server for SharePoint. FAST's Norwegian accounts became the subject of an accounting scandal and investigation reported after the deal.
March 2011: Yahoo announces the closure.
4 April 2011: alltheweb.com redirects to Yahoo Search. Yahoo's notice read: “Yahoo! will be closing AlltheWeb on April 4, 2011, as we focus on other features to improve your search experience.”

As of 19 August 2026, www.alltheweb.com still resolves into Yahoo infrastructure, so the domain remains under Yahoo's control — but there is no AllTheWeb service behind it and has not been for fifteen years.

Common misconceptions

“Yahoo bought AllTheWeb.” Only indirectly. Overture bought it in February 2003; Yahoo bought Overture in July 2003. The intermediate step matters, because Overture also bought AltaVista in the same fortnight — the two engines arrived at Yahoo together, from the same buyer.

“AllTheWeb ran its own index until it shut down in 2011.” No. It served Yahoo's index for roughly seven years before the shutdown. April 2011 is when the brand was retired, not when the engine stopped being an engine.

“AllTheWeb and AltaVista were shut down together.” They were bought together but not closed together. AllTheWeb closed on 4 April 2011; AltaVista survived until 8 July 2013. Search Engine Land's March 2011 report mentions AltaVista alongside AllTheWeb in Yahoo's sunset plans, which is where this error usually starts.

“Microsoft bought AllTheWeb.” Microsoft bought FAST in 2008, by which point FAST was an enterprise search vendor only. The web search division had been sold to Overture five years earlier.

“FAST was a search engine company.” Mostly it was an enterprise search vendor. Consumer web search was about a quarter of its revenue by the time it sold that division.

What it is worth citing for today

AllTheWeb was genuinely excellent and is genuinely gone, and both halves of that have to be said together. Nobody can use it: the domain redirects, the index no longer exists, and no successor service carries the brand. Anyone reaching for it as an alternative to Google is fifteen years too late.

What it is worth citing for is the pattern its ownership chain demonstrates. An independent index built over eight years was sold for $70 million plus earn-outs, folded into a rival's index within about two years, and switched off eight years after that. When a search engine is acquired by a company that already runs an index, the acquired index does not usually survive — the interface may persist for years, which is exactly what makes the acquisition look less consequential than it was.

It also had no AI features and never could have; it closed well before generative answers existed. Its advanced query operators and result filtering were unusually good for their era, which is a different thing entirely and should not be conflated with them.

Frequently asked questions

What happened to AllTheWeb?

Yahoo closed it on 4 April 2011, redirecting alltheweb.com to Yahoo Search. The more important date is earlier: the site switched to Yahoo's database shortly after Yahoo completed its acquisition of Overture in October 2003, so the independent index ended around 2003–04. The brand outlived the engine by about seven years.

Did AllTheWeb have its own index?

Yes, from its mid-1999 launch until roughly 2003–04. FAST ran its own crawler, recognisable in logs as FAST-WebCrawler, and its own index — around 200 million URLs by August 1999, over 2 billion pages by June 2002, and roughly 3.2–3.3 billion documents by August 2003. After the Yahoo acquisition the site served Yahoo's database instead.

Who owned AllTheWeb?

Three owners in sequence. Fast Search & Transfer of Norway built and ran it from 1999. Overture Services announced its purchase of FAST's web search division on 25 February 2003 for $70 million cash plus up to $30 million in earn-outs, completing in April. Yahoo then announced its acquisition of Overture on 14 July 2003, closing that October.

Was AllTheWeb bigger than Google?

It was contested, not settled. In June 2002 AllTheWeb announced 2.1 billion indexed pages and claimed to have out-Googled Google; Google responded that it had passed 3 billion documents in December 2001. The engines counted differently — Google's figure included URLs it knew of but had not fully indexed — so the two were close and the comparison was disputed.

Did Microsoft buy AllTheWeb?

No. Microsoft announced an offer for Fast Search & Transfer on 8 January 2008, valued at about $1.2 billion, but FAST had sold its web search division — including AllTheWeb — to Overture five years earlier. What Microsoft bought was the enterprise search company, which became its Norwegian development centre and the basis of FAST Search Server for SharePoint.

Were AllTheWeb and AltaVista shut down at the same time?

No, though they were bought at the same time. Overture announced its purchase of AltaVista on 18 February 2003 and of FAST's web search division a week later, completing both in April 2003. AllTheWeb was closed on 4 April 2011. AltaVista survived until 8 July 2013, more than two years longer.

Can you still use AllTheWeb?

No. The service ended on 4 April 2011 and there is no successor. As of 19 August 2026 the domain www.alltheweb.com still resolves into Yahoo infrastructure, which indicates Yahoo continues to hold it, but nothing operates there. Any site presenting itself as AllTheWeb today has no connection to the original engine.

What was FAST-WebCrawler?

It was the crawler AllTheWeb used to build its own index, and the user-agent string it identified itself with in server logs. Recorded versions include a contact address at Overture in Trondheim, which dates them to after the February 2003 sale. The crawler stopped being the source of AllTheWeb's results once the site moved onto Yahoo's database.

AllTheWeb