How Search Ranking Works

By Bill Hartzer · Updated 19 August 2026

Two stages: finding candidates, then ordering them

A query against a web-scale index does not return twenty documents. It returns millions. The interesting work is not finding matches — the inverted index does that almost instantly — but deciding which handful of them a person sees, and in what order.

Practically every modern engine splits this into two stages. Retrieval pulls a candidate set out of the index cheaply, using methods fast enough to run over the whole corpus. Ranking then applies far more expensive scoring to that much smaller set, because you can afford computation on ten thousand documents that you could never afford on ten billion.

Perplexity's published architecture is a clean illustration: lexical and embedding scorers generate candidates, then cross-encoder rerankers — models that examine a query and a document together rather than scoring them separately — reorder what survives, at both document and passage level. The names differ between engines, but the shape is close to universal. It also explains a common confusion: a page can be perfectly well indexed and never appear, because it never made the candidate cut, which is a different failure from not being indexed at all.

Before links: counting words, and why it stopped working

The first generation of web engines scored documents on the text alone. Broadly, a term that appeared often in a document and rarely across the corpus was treated as a strong signal that the document was about that term — the family of measures usually shortened to TF-IDF, and its better-behaved descendant BM25, which is still a perfectly respectable retrieval function and is still in production use as a candidate generator.

AltaVista, the most capable engine of the mid-1990s, used weighted boolean retrieval over its full-text index. Its achievement was the crawl and the index scale, not the ranking; it had no link-graph analysis at all. That distinction is worth holding onto, because it is exactly what left it exposed.

Text-only scoring has a structural weakness: everything it measures is written by the person who wants to rank. If the score depends only on the words on the page, the words on the page will be chosen to maximise the score, and by 1997 the results pages of every major engine showed it. The problem was not that the algorithms were badly implemented. It was that they relied entirely on evidence the publisher controlled.

Link analysis: evidence the publisher does not control

The fix was to score a document using signals produced by other people. Two research lines arrived at that idea at roughly the same time, and both ended up shipping.

PageRank came out of the BackRub project at Stanford in 1996–97, by Larry Page and Sergey Brin, and treats the web as a graph of votes. A link is a vote; a vote from a page that itself receives many votes counts for more; the score is computed recursively across the whole graph, independently of any query. Because it is query-independent it can be computed in advance and stored in the index, which is what made it usable at scale.

HITS-style analysis took the other route and computed authority at query time, distinguishing hubs — pages that link to good pages — from authorities. Teoma, spun out of the Rutgers DiscoWeb project and launched publicly in 2001, shipped a commercial version it called Subject-Specific Popularity: rather than scoring a page on its link popularity across the whole web, it identified the topical community a query's results belonged to and counted in-links only from inside that community. The argument was that a link from a recognised authority on plate tectonics should count for a geology query and a link from an unrelated high-authority page should not. Ask Jeeves bought Teoma in September 2001 and the lineage eventually surfaced as Ask.com's ExpertRank.

Two things about this era are routinely misremembered. PageRank is not how Google ranks pages today — it is one historical signal among hundreds, and the public toolbar score was retired in 2016. And Teoma's "expert communities" were statistical link analysis, not machine learning in any modern sense; the vocabulary of the marketing has outlived the mathematics of the product.

What a modern engine will admit to using

No general engine publishes its ranking function. What they publish is categories, and Google's acknowledged list is representative of the industry: link-based authority, on-page relevance, freshness, page experience signals including Core Web Vitals, and site-level quality assessments — described as hundreds of signals in total, with the specifics undisclosed. Core algorithm changes are announced on Google's Search Status Dashboard, but as events rather than as explanations.

Other independent engines name their systems without describing them. Seznam ranks with something it calls S-Rank. Naver runs C-Rank and D.I.A., alongside a results page dominated by Naver's own user-generated properties — Blog, Cafe, Knowledge iN, Shopping — which is a ranking decision of a different kind: self-preferencing inside an own index. Cốc Cốc documents its crawlers thoroughly and its ranking not at all; its help pages cover crawl controls and say nothing about how results are ordered.

The honest summary is that ranking is the least transparent part of the stack, at every engine, including the ones whose crawlers are exemplary. Anyone stating a specific ranking factor as fact is either quoting an engine's own general category or inferring from observation, and the difference between those two matters.

Learned ranking, and what it changed

Hand-tuned scoring formulas gave way, over roughly the 2010s, to machine-learned ranking: models trained on large volumes of interaction data to predict which result a person will find satisfactory. This is a real discontinuity, and it has two consequences that are worth being precise about.

The first is that interaction data becomes a competitive asset, not just an analytics by-product. The point was made unusually plainly in the US antitrust remedies against Google: the court ordered Google to supply search-index and user-interaction — "click-and-query" — data to qualified competitors, describing it as "the raw material that Google uses to improve search." The remedies took effect on 3 February 2026; Google filed a notice of appeal in January 2026 and the Department of Justice cross-appealed, with both before the D.C. Circuit and no decision issued as of 19 August 2026. Whatever the outcome, the framing is instructive: a court identified behavioural data as the input that entrenches ranking quality.

The second is that ranking became harder to explain, including internally. A learned model does not have factors in the sense the earlier formulas did. This is part of why engines describe categories rather than mechanisms, and why claims about precise ranking weights should be treated with suspicion regardless of who makes them.

The newest layer is neural reranking of the kind Perplexity documents, where a cross-encoder reads query and passage together. It is expensive per document, which is exactly why it runs only over a small candidate set.

Personalisation, and engines that refuse it

A ranked list can be the same for everyone, or it can be adjusted per person using location, language, device, history and inferred interests. This is the single largest reason two people comparing search results get different answers and conclude one of them is broken.

Mojeek does not personalise at all: results are identical for every user, which makes it unusually useful for anything requiring reproducibility, such as checking whether a page is findable outside the duopoly. Marginalia likewise ranks the same way for everyone. Google personalises, and its default posture is data collection for advertising — which is a description of the business model rather than a criticism, but it does mean a Google results page is not a stable object that two people can compare.

Location is the most consequential form of it, and the least noticed. A query with any local intent is resolved differently in two cities without anything being personalised in the behavioural sense, and an engine with a weak local layer will look inexplicably bad on those queries — one of the concrete places where Bing-derived engines such as DuckDuckGo lag outside major markets, because they inherit Bing's local coverage and cannot fix it.

Ranking is an editorial position

The most useful thing to understand about ranking is that the choice of what to reward is a value judgement, and the engines that make it explicitly are the easiest to reason about.

Marginalia inverts commercial ranking deliberately. Heavy JavaScript, dense advertising, affiliate links, tracking scripts and modern optimised design all push a page down; age pushes it up, under a stated Lindy-effect assumption that surviving content is more likely to be worth reading. The result is that it surfaces personal sites, university pages and unmonetised documentation that a commercial engine structurally will not, because a commercial engine reads the absence of commercial signals as an absence of quality. Its results are not worse; they are differently valued, and that distinction is the whole product.

Kagi sells the ranking layer itself. It crawls into its own Teclis and TinyGem indexes but blends them with anonymised API calls to commercial indexes, and its actual product is the filtering and re-ranking applied on top — with user-controlled site raising and blocking. Its argument is structural: with a subscription and no advertiser, the only party the ranking has to satisfy is the person paying for it.

DuckDuckGo demonstrates the uncomfortable edge of the same point. In March 2022 it announced it was down-ranking sites it identified as Russian state disinformation. Some users read that as inconsistent with a stated position of unbiased results. Both readings are defensible, and the episode is a clean illustration that there is no neutral ranking — declining to intervene is also a choice, and every engine makes one.

Why results differ so much between engines

Given all of the above, it would be surprising if engines agreed. In practice they diverge for three separable reasons, and it is worth telling them apart.

Different indexes. If an engine never crawled the page, no ranking can surface it. This is the largest source of disagreement and the least discussed one.
Different signals. An engine that weights link authority heavily and one that weights text and age heavily will order the same candidate set differently.
Different intent models. Engines interpret ambiguous queries differently, and increasingly rewrite or expand them before retrieval — Google describes AI Mode as using a "query fan-out" technique against its own index.

One corollary matters for anyone comparing engines: Bing, Yahoo and DuckDuckGo broadly agree with one another because they are largely looking at the same index. Treating DuckDuckGo as an independent second opinion on a Bing result is a category error — it is a second interface, not a second index. Genuine disagreement requires a genuinely different crawl underneath, which is why Mojeek and Marginalia are useful as cross-checks in a way that most "alternative" engines are not.

Frequently asked questions

How do search engines rank results?

In two stages. Retrieval pulls a large candidate set out of the index cheaply; ranking then applies much more expensive scoring to that smaller set and orders it. Modern scoring combines text relevance, link-based authority, freshness, page experience and learned models trained on interaction data. No general engine publishes its actual ranking function.

What is PageRank and is it still used?

PageRank scores a page by treating links as votes, weighting each vote by the score of the page casting it, computed recursively across the whole web graph. It came from the BackRub project at Stanford in 1996–97. It is one historical signal among hundreds at Google today, not the ranking system, and the public toolbar score was retired in 2016.

What is the difference between PageRank and HITS?

PageRank is query-independent and computed in advance over the entire link graph, so it can be stored in the index. HITS-style analysis computes hub and authority scores at query time over the pages relevant to that query. Teoma shipped a commercial version called Subject-Specific Popularity in 2001, counting links only from within the query's topical community.

Why do Google and Bing show different results?

Mainly because they crawled different pages. Two engines hold different subsets of the web, so a page one has indexed may simply be absent from the other. On top of that they weight signals differently and interpret ambiguous queries differently. Index differences account for more divergence than ranking differences do.

Do all search engines personalise results?

No. Mojeek returns identical results to every user with no personalisation, which makes it useful for reproducible comparison. Marginalia does the same. Google personalises, and location alone changes results for anything with local intent even without behavioural history. This is the usual reason two people comparing a search conclude one of them is seeing something wrong.

Are search results neutral?

No ranking is neutral, because choosing what to reward is a value judgement. Marginalia deliberately downranks advertising, tracking and heavy JavaScript and upranks old pages. Naver's results page favours Naver's own properties. DuckDuckGo announced in March 2022 that it was down-ranking Russian state disinformation. Declining to intervene is also a position.

What is a reranker in search?

A reranker is a second, expensive scoring pass over a small candidate set produced by cheaper retrieval. Cross-encoder rerankers read the query and the document together rather than scoring them separately, which is far more accurate and far too costly to run over a whole index. Perplexity documents exactly this arrangement in its published search architecture.

Why is search ranking so secretive?

Partly competitive advantage and partly manipulation resistance: a fully published ranking function is a specification for gaming it. There is also a technical reason — learned models do not have explicit factors to disclose. The practical effect is that every specific claim about ranking weights is inference rather than documentation, whoever is making it.

How search ranking works