The Phrase-Based Indexing Patent
In 2017, someone quietly bought a WordPress plugin trusted by roughly 200,000 websites and turned it into a link-injection machine — dropping cloaked links, including payday-loan anchors, into pages that had nothing to do with payday loans. The buyer's identity was deliberately murky; Wordfence attributed it partly to an operator it believed was in Russia. It was industrial-scale nonsense: the right anchor, the wrong page, at volume. And here's the uncomfortable part — variations of it work. So why doesn't all of it work? Because thirteen years earlier, the architect of Google's search index had already designed a system to defeat exactly this. She patented it too — but the patent isn't the reason it fails. The design is. The patent just lets us read how it works.
That patent is US7536408B2, and its inventor is Anna Patterson — the principal architect of TeraGoogle, the index-serving system that grew Google's index more than tenfold in 2006, and later a VP of Engineering at Google. If you've read my analysis of the Reference Contexts patent, that system reads the text immediately around a link. This one supplies the topic model that text is measured against. Together they answer one question Google has been asking for two decades: does this link actually make sense where it sits?
The patent explicitly describes indexing phrases (not just words), measuring which phrases predict each other via co-occurrence and information gain, storing a related-phrase bit vector per document, using that vector as a relevance score, and scoring links by how much the target shares the anchor's related-phrase neighbourhood — a mechanism it says "entirely prevent[s] certain types of manipulations." What we infer: that today's neural topicality signals and leaked attributes like topicalityScore and anchorMismatchDemotion are plausible modern analogues of this 2004 thinking — parallels, not proven descendants. What we don't know: the exact current weights, and — importantly — how many holes remain in practice. This patent was filed in 2004; Google now runs on neural, probabilistic systems, so the patent describes design intent, not necessarily the present-day implementation.
The Honest Hedge
Every analysis has a threshold where certainty ends and inference begins. Here's where that line falls for this patent:
- Google indexes multi-word phrases and models which phrases predict others, via a co-occurrence measure with an information-gain threshold the patent puts at 1.5 (preferably 1.1–1.7).
- Each document carries a related-phrase bit vector; the patent states its value "may be used as the document score."
- A link is scored by how much the target page carries the anchor's phrase neighbourhood: the exact anchor phrase on the target is the strongest evidence ("intentionally related"), but the anchor's related phrases on the target also produce a score when the exact phrase is absent.
- The patent names the attack ("bombed" via artificial anchor text) and states its defense "entirely prevent[s] certain types of manipulations."
- Filed 2004-07-26, granted 2009-05-19, inventor Anna Patterson. Reached full-term expiration on 2025-10-06 — Google maintained it its entire life, and re-filed the concept via continuations through 2018 (US9990421B2).
- The discrete phrase bit vectors of 2004 have been absorbed into neural embeddings and the leaked
topicalityScore/normalizedTopicalityattributes — same job (topical fingerprinting), different math. anchorMismatchDemotionis a plausible modern analogue of the patent's anti-"bombing" logic — a named anchor-vs-page mismatch signal (the leak documents it only as "converted from a mismatched quality boost").- Anchor text carrying topically-related phrases is weighted above unrelated anchor text (supported by BM25 anchor scoring in the leak).
- The exact weight of topical relevance versus other signals in the live ranking stack.
- How many holes remain. The patent describes a system that in principle devalues irrelevant links — but given the absurd complexity of the real algorithm, topically-nonsense links still rank in some niches. This is not a perfect masonry wall. It's cobbled together, and it has holes.
- Whether the production system still uses discrete phrase vectors at all, or has fully replaced them with learned embeddings.
Patent Metadata
Note: This patent was filed in 2004 and expired at the end of its full statutory term in 2025 — meaning Google paid to maintain it across its entire granted life and kept the concept under active protection via continuations through 2018. Its expiration is a sign of a mature, absorbed technology, not an abandoned one. The precise formulas below are the 2004 implementation; the current systems have very likely moved to neural methods.
What This Patent Does (Plain English)
A classic search index maps words to the documents that contain them. This patent adds a second layer: it maps phrases — multi-word units — and, critically, it models which phrases predict the presence of other phrases on the same page. A page genuinely about "President of the United States" will naturally also carry "White House," "vice president," "Oval Office." That cluster of co-occurring phrases is the page's topical fingerprint.
Here's what the system actually does:
- Harvest candidate phrases — slide a window across the text, keeping even stop words. Every candidate starts as a "possible phrase."
- Promote or prune — a possible phrase becomes a "good phrase" if it's frequent enough and predictive; otherwise it's pruned.
- Measure co-occurrence — for phrase pairs, compare the actual co-occurrence rate to the expected rate. When the ratio (information gain) clears the threshold, the two phrases are "related."
- Store a related-phrase bit vector — for each document, flag which related phrases it contains.
- Use the vector as a score — a page carrying the deep, high-order related-phrase cluster reads as genuinely topical; one with the query term but none of the cluster does not.
Here's what that looks like in the patent itself. FIG. 2 is the phrase pipeline — collect possible and good phrases plus co-occurrence statistics (200), classify them (202), then prune on information gain (204), all feeding a phrase-data store that holds the posting lists and the co-occurrence matrix:
Wait. Let me translate that to human.
↓The Related-Phrase Fingerprint (and Its Neural Heir)
The engine underneath all of this is co-occurrence. For any two phrases, the system compares how often they actually appear together against how often you'd expect them to by chance. The patent computes an expected value for a phrase as P(j)/T — the share of documents containing it — and keeps a phrase pair as "related" when the actual-to-expected ratio clears an information-gain threshold it puts at 1.5.
Ignore the notation — it's asking one simple question: do these two phrases show up together more often than random chance would explain? Suppose "White House" appears on 1% of all web pages. If it had nothing to do with "President of the United States," you'd expect it on roughly 1% of the pages that mention the President, too. Instead it shows up on, say, 40% of them. That huge gap — actual usage far above what chance predicts — is the "information gain," and it's the statistical fingerprint of a real topical relationship. "Recipe" shows no such gap against "President," so the two are never treated as related. In the formula, A is how often they actually co-occur, E is how often you'd expect them to by chance, and anything above roughly 1.5× counts as related.
Related phrases are then grouped into clusters, and each document stores a bit vector flagging which related phrases (and higher-order related-phrases-of-related-phrases) it contains. FIG. 4 shows exactly that: identify related phrases by high information gain (400), cluster them (402), and store a cluster bit vector and cluster number (404):
A "bit vector" sounds technical, but it's just a checklist of yes/no boxes — one box for every related phrase in the topic. A box gets a 1 if the page contains that phrase and a 0 if it doesn't. Line the boxes up and you have a fingerprint of how much of the topic a page actually covers: a genuine, thorough guide ticks most of the boxes; a page that just drops the keyword ticks one. Google can compare those fingerprints at a glance — and the diagram below shows two pages doing exactly that.
Wait. Let me translate that to human.
↓This patent was filed in 2004. The related-phrase bit vector described here was the 2004 approach — discrete flags in a list, cheap to compute at web scale. Google's modern systems solve the same representational problem — capturing which concepts a page is genuinely about — but with different, independently developed machinery: neural embeddings out of the broader NLP lineage (word2vec, BERT), not a refined version of this specific patent. Leaked attributes like topicalityScore and normalizedTopicality (the latter "sums to 1" across a page's entities — a zero-sum topical budget) are plausible modern analogues, not proven descendants. The patent describes the architecture and the intent; the implementation moved on by a different road. The precise mathematics don't matter anyway — those are always just sample examples. The principle does.
How a Link Gets Topically Validated
This is where the patent stops being a content-indexing story and becomes a link story — though it's worth being clear this is a downstream application: the core invention is phrase indexing and retrieval, and link scoring is one of the things built on top of it. When Google indexes a document, it doesn't just record the phrases on the page — it also uses the page's outbound links to annotate the phrase fingerprints. FIG. 5 lays out the flow: post the document to the posting lists of its good phrases (500), update the instance counts and bit vectors (502), then — box 504 — annotate the related-phrase bit vectors based on the anchor phrases, and finally reorder the index (506).
The patent's own words make the strongest case concrete. It scans the target of a link and checks whether the anchor phrase shows up there:
"the indexing system … scans URL 1, and determines whether phrase A appears in the body of URL 1. If phrase A not only points to URL 1 … but also appears in the content of URL 1 itself, this suggests that URL 1 can be said to be intentionally related to the concept represented by phrase A."
Read that carefully — but read it precisely, because this is where the piece could easily overstate. A link isn't trusted as a bare signal of significance; it's scored by how much the destination shares the anchor's phrase neighbourhood. The exact anchor phrase appearing on the target is the strongest case — that's when the patent calls the target "intentionally related." But it is not a prerequisite. The patent describes a second path: when the exact phrase is absent, the system checks which of the anchor's related phrases appear on the target and scores the link from those. Its own worked example is a link whose anchor is "Australian Shepherd" — the target earns an inlink score because it carries the related cluster ("blue merle," "red merle," "tricolor"), not because the exact string is hammered in. Either way, the system imports the target's related-phrase bit vector into the linking page's record, which — in the patent's words — "eliminates the reliance of the search system on just the relationship of phrase A in URL 0 pointing to URL 1 as an indicator of significance." The takeaway is graded, not binary: a link's value rises with how much shared phrase evidence exists across it — it isn't switched on or off by an exact-match test.
Wait. Let me translate that to human.
↓How I turn this into safe anchor text — the Search Console method
This is the most practical thing in the entire patent, and it's a method I've used since I started doing SEO, because it works. Two problems show up constantly: a page isn't performing, and the person building links has no idea what anchor text to use without it looking unnatural. You cannot point twenty backlinks at a page with the anchor "roofer in Sarasota." That's not a link profile — that's a footprint. So I don't invent anchors off the top of my head. That has always been a bad idea. I let Google tell me what the page is already allowed to rank for.
- Open the page in Search Console → Queries. Look at the full list of queries the page appears for. It will hand you variations you'd never have thought of — and the richer you've made the page, the more of these random, long-tail strings you'll be ranking for.
- Optimize for what you already rank for. Run those queries through a tool like Page Optimizer Pro and tighten the page for them. Nine times out of ten you'll find you mentioned the term once, if at all — Google is ranking you on the general context of the page, not an exact string. Close that gap.
- Use those strings as your partial-match anchors. For the terms pulling real impression volume where the page is under-optimized: fix the on-page first, then use those Search Console strings as anchor text when you build links. They're the language Google already associates with the page — grounded anchor research, not guesswork. One caveat: a page can rank for a query whose exact words never appear on it, so confirm the destination genuinely supports the phrase (or its related cluster) before you use it. Search Console tells you the semantic territory you own; it isn't an automatic safe harbour.
Here's an actual query Search Console shows this site picking up impressions for:
list us patents on "pagerank" or "web graph" that are classified in g06f16/21, have google as original assignee but are now assigned to a different entity, and have more than 100 forward citations
Eighteen impressions, zero clicks — a string I would never have written as a target in a hundred years. But Google decided this site was relevant to it. That is exactly the raw material the method runs on: hyper-specific queries you'd never brainstorm, handed to you, ready to be optimized for and turned into natural anchor text.
Notice what that last step does. Every anchor you build is one the page genuinely supports — the string, or its related cluster, actually lives there. That's the evidence box 504 looks for. You're not gaming the sense-check; you're feeding it something true.
Sometimes Search Console shows a page ranking for a query it has no business ranking for — it doesn't really answer that query, but Google likes the document for it anyway. Don't waste that signal. Build the dedicated page that does answer it, then link to it from the old page using that exact query as the anchor. You're telling Google, in its own language: "this is the page you should be ranking for this term." The anchor matches the target, and the target finally matches the query. That's the same mechanism working for you instead of against you.
The Anti-Bombing Gate
The patent doesn't leave the motive implicit. It names the attack it's defending against:
"Search engines that use a ranking algorithm that relies on the number of links that point to a given document in order to rank that document can be 'bombed' by artificially creating a large number of pages with a given anchor text which then point to a desired page."
That's the plugin-backdoor story from the top of this article, described two decades early. Someone acquires a large footprint of pages — a network of hacked WordPress installs, a portfolio of plugins bought on a marketplace — and injects a given anchor text at scale, pointing at a page that has nothing to do with it. The 2017 Display Widgets case did it to roughly 200,000 sites; a 2026 supply-chain attack repeated it across 30-plus plugins bought on Flippa, cloaking the spam so only Googlebot saw it. The patent's response is the bit-vector import — and it states the purpose plainly: this "entirely prevent[s] certain types of manipulations of web pages … in order to skew the results of a search."
Here's the retail version I see every week, and it's the same principle at a smaller scale. Someone runs a link-insertion campaign — pay thirty or forty dollars, get an anchor dropped into some existing article. The article is a Stephen King book review on a hobbies site. It mentions a character who happens to walk dogs. And there, exact-match, is a link: "dog walker" → some guy's dog-walking business in Missouri. Why would that article link to that business? It wouldn't. There's no shared cluster, the anchor phrase isn't what the target page is really about, and the sense-check fails. That link is not a weak signal. For relevance, it can be no signal at all.
anchorMismatchDemotion, a named anchor-topic-versus-page-topic mismatch signal.Here's the part most patent write-ups skip: this defense is not airtight. If people run these attacks, it's because they work — often enough, in the right niches, to be worth it. Churn-and-burn affiliate sites in high-payout niches like VPNs get the most questionable links imaginable and still rank long enough to cash lifetime commissions. A patent describing a system that in principle prevents manipulation is not the same as that system being a perfect wall in production. The real algorithm is absurdly complex — it's cobbled together, and it has holes. What follows is how to build for the version of search that lasts, not the exploit that happens to work this quarter.
The Density Trap
There's a symmetric failure mode that the "add all the related keywords" crowd walks straight into. The related-phrase signal runs both ways: too few of the topic's phrases and you read as thin; a statistically improbable number of them and you read as spam, not as super-relevant. Genuine writing lands in a natural band. Stuffing overshoots it.
The modern echo of this is the observedTf versus expectedTf comparison in Google's salient-terms handling — a term-frequency divergence check that flags stuffing. It isn't a one-to-one implementation of the patent's phrase-pair math (it operates on single terms), but it's the same shape: the system knows roughly how often a genuine page uses a term, and it notices when you blow past that. The signal that rewards topical coverage is the same one that punishes overshooting it.
Phrase-Based Indexing SEO Implications: What This Means for Your Link Building
1. Relevance is a multiplier, not a checkbox
The most common mistake I see runs in two opposite directions, and both are wrong. One camp says relevance doesn't matter, just get the highest-authority link you can. The other says only relevance matters. The truth this patent points at is that relevance multiplies authority. An irrelevant link from NASA still helps a dog walker, because of everything else that domain carries. An irrelevant link from a DR-5 blog with no traffic and no place in the link graph does nothing at all. But the NASA link is worth far more when your site is precisely on a subject NASA would sensibly link to, and the linking page is genuinely about your topic. Relevance amplifies. It doesn't stand alone, and it doesn't get to be skipped.
2. Bridge the topical gap — engineer the host page to earn the link
Most link targets don't have an obvious universe of on-topic hosts. If you run a window-tinting business, there aren't many window-tinting blogs lining up to link to you. So we bridge the gap: we build a fresh piece on a host site in an adjacent space — automotive, or even a legal angle — and write the article so it genuinely merges both subjects. It has to rank on the host site without blowing the host's site radius or diluting its site focus, which is why the piece has to be a real contribution, not a thin excuse for a link. Then we go heavy on the related entities and phrases, which earns the right to mention the brand by name and link it home.
3. Brand anchor to the homepage, keyword in the cluster
My default is a brand-name anchor pointing at the homepage, embedded in a paragraph rich with the topic's related phrases. That produces a link whose surrounding cluster is unmistakably on-topic while avoiding the exact-match anchor pattern that trips the mismatch penalty. If you do want a keyword-rich anchor, the same rule holds: the phrase has to belong on the page it sits on, and it has to belong on the page it points to.
4. Cleanup is slower than a clean start
When someone arrives after ten months of insertion-only link building with no results, the honest news is that recovery is often slower than starting from zero. The links did nothing, but the anchor profile is now over-optimized, so we have to build an oversized volume of branded links just to re-equilibrate it — and many of those get discounted anyway, because the Historical Data patent (US7346839B2) watches anchor-text velocity and compresses signals that appear in too narrow a window. Sometimes there's a page-level algorithmic penalty that's genuinely hard to lift. Compare that to a clean canvas — the Miami orthodontist campaign we built at GetMeLinks started from zero, uses only our links, and it's performing exactly as it should. Clean is faster than dirty-then-cleaned.
US8577893B1 — "Reference Contexts" — reads the text immediately around a link. Phrase-Based Indexing supplies the topic model that surrounding text is measured against. The two are halves of one system.
US7716225B1 — "Reasonable Surfer" — weights a link by its probability of being clicked. A link that is both well-placed and topically coherent is doubly validated.
US7346839B2 — "Historical Data" — watches anchor-text velocity over time. The temporal complement to this patent's static topical check.
Google API Leak Cross-Reference: Topical Relevance Attributes
The 2024 Google API leak — first reported by Rand Fishkin and investigated by Mike King at iPullRank — contains attributes that rhyme with this patent's mechanisms. Unlike some link patents, Phrase-Based Indexing has no single field named after it; its fingerprint is architectural. It reads as a conceptual forerunner of Google's topical-relevance thinking, and several leaked attributes are plausible modern analogues — parallels worth noting, not proven descendants.
| Patent Mechanism | API Attribute | Alignment |
|---|---|---|
| Anti-"bombing": anchor topic vs. page topic | anchorMismatchDemotion | 🔶 PLAUSIBLE — a named anchor-vs-page mismatch signal; the leak calls it "converted from a mismatched quality boost" and doesn't define the mismatch |
| Reading the phrase cluster around a link | context2, fullLeftContext / fullRightContext | 🔶 STRONG MATCH — shared with US8577893B1 |
| Related-phrase topical fingerprint of a page | topicalityScore, normalizedTopicality | 🔶 POSSIBLE ANALOGUE — entity↔document topicality signals; not confirmed as a page-level link-validation vector |
| Anchor phrase weighted by topical fit | anchor text / origText + BM25 scoring | 🔶 API EXTENDS |
| Improbable related-phrase density = spam | observedTf vs expectedTf (single-term) | 🔶 POSSIBLE ANALOGUE — same "actual vs expected" shape; the leak doesn't state the gap is a spam penalty |
| Duplicate detection via phrase-ranked description hashing | — | 📜 PATENT ONLY |
The API leak provides attribute names and short descriptions — not scoring formulas, weights, or any confirmation that a field maps to this patent. Treat every row above as a conceptual parallel, not proven lineage: anchorMismatchDemotion's own documentation says only that it was "converted from a mismatched quality boost" (it doesn't define the mismatch); topicalityScore is an entity-to-document relatedness signal, not necessarily a page-level link vector; observedTf/expectedTf exist but aren't documented as a spam penalty. Any numeric ranges seen elsewhere for these fields (e.g. [0–1023]) come from community analysis of the leak, not the fields' own docs. The patent explains a mechanism; the leak shows attributes that rhyme with it. Neither proves the other.
Citation Network
Patent Family & Continuations
US7536408B2 is one of 20-plus phrase-based patents Anna Patterson filed for Google, across several "generations." The concept was kept under active protection long after this filing — continuations run through US9037573B2, US9569505B2, and US9990421B2 (granted 2018), whose claims explicitly rank a document with a plurality of related phrases above one with higher raw query-term frequency.
Related Articles on This Site
- US8577893B1 (Reference Contexts) — the local complement. Reference Contexts reads the words immediately around a link; Phrase-Based Indexing is the global topic model those words are scored against. Same inventor, same year, two halves of one link-quality system.
- US7716225B1 (Reasonable Surfer) — weights a link by click probability (placement, prominence). Phrase-Based Indexing weights it by topical coherence. Well-placed and on-topic is doubly validated.
- US7346839B2 (Historical Data) — watches when links and anchors appear. The temporal layer on top of this patent's static topical check; together they catch both anchor-velocity spikes and topical mismatch.
Phrase-Based Indexing: What Doesn't Matter as Much as SEOs Think
The nature of this patent is simple and permanent: the context that hosts a link has to relate to the subject the link points to. That's all it's really solving for. Everything else — possible phrases, good phrases, co-occurrence matrices, the information-gain threshold of 1.5, the related-phrase bit vector — is flavor.
And flavor is disposable. The instinct behind modern "topical relatedness" — that a page is defined by the network of concepts it carries, not the keywords it repeats — is the same idea Anna Patterson wrote down in 2004. The machinery is completely different (today's embeddings come from a separate lineage of language modelling, not from this patent), but the underlying principle is continuous. The precise mathematics don't matter, because those are always just sample examples. What matters is that for more than twenty years, a link has been worth more when it lives among the concepts it claims to be about. That principle did not get deprecated in a software update. It got upgraded.
Which is also why I've been careful not to oversell it. This isn't a perfect multi-layered wall. It's cobbled together, and it has holes — some black-hat links in some niches work anyway, and pretending otherwise would be dishonest. But you don't build a business on the holes. You build on the part of the machine that's been true for two decades and will still be true in five years.
This is the reason I read patents at all. Working out how the machine actually behaves — from the primary source, before spending a dollar trying to rank in it — is the Theory phase of how I work: the T in TISEL. Everything downstream — the intelligence, the strategy, the execution — is only as good as how honestly you did this part.
So here's the whole thing in one sentence: a link has to make sense before it can matter — that's the gate everything else sits behind. Nonsense is too cheap to generate at scale for it to be otherwise.
Frequently Asked Questions
What does patent US7536408B2 actually do?
It indexes multi-word phrases rather than just individual words, and models which phrases predict each other on a page using co-occurrence and information gain. Each page gets a "related-phrase" fingerprint that acts as a topical relevance score, and the same model is used to validate links: an anchor phrase has to appear on the page it points to for the link to count as a genuine topic signal. It's the documented mechanism behind Google's defense against anchor-text "bombing."
How does phrase-based indexing decide two phrases are "related"?
It compares how often two phrases actually appear together to how often you'd expect them to by chance. The ratio is an information-gain measure, and the patent keeps a pair as "related" when that ratio clears a threshold it sets at 1.5 (preferably 1.1–1.7). "White House" clears the bar against "President of the United States"; an unrelated phrase does not. The set of related phrases is the topic's fingerprint.
What does the Google API leak confirm about this patent?
The leak has no field literally named "phrase-based indexing," but several attributes align with its mechanisms: anchorMismatchDemotion [0–1023] is a named penalty for anchor-versus-page topic mismatch — the patent's anti-bombing logic; context2 and fullLeftContext/fullRightContext store the phrase context around a link; and topicalityScore/normalizedTopicality are the neural heirs of the discrete related-phrase bit vector.
Does an irrelevant backlink still help?
It depends entirely on the source. Relevance is a multiplier on authority, not a standalone requirement. An irrelevant link from a genuinely authoritative, trusted domain still carries weight from everything else that domain has. An irrelevant link from a low-authority, no-traffic site does essentially nothing. But any given link is worth far more when the linking page is actually about your topic — that's the coherence the patent rewards.
Why do cheap link insertions on unrelated pages fail?
Because they fail the sense-check. The patent scores a link by how much of the anchor's phrase cluster the target actually carries — the exact phrase or its related concepts. A "dog walker" anchor dropped into a book review pointing at a random local business shares no cluster either way, so it isn't treated as a topic signal — and the anchor-versus-page mismatch can trigger anchorMismatchDemotion. The link isn't weak; with zero shared topic, its relevance signal can be worth nothing.
Is this 2004 patent still relevant in 2026?
The specific mechanism — discrete phrase bit vectors, an information-gain threshold of 1.5 — is a museum piece; Google now runs on neural embeddings. But the principle is the direct ancestor of modern topical-relevance and entity systems, and the patent's continuations run through 2018. The nature (a link's host context must relate to its target) is permanent; only the flavor (the exact math) has changed.
If Google can devalue irrelevant links, why does link spam still work sometimes?
Because the real algorithm is not a perfect wall — it's an enormously complex system with holes. The patent describes intent, not a guarantee. In low-competition or high-churn niches, topically-nonsense links still rank often enough that black hats keep using them. The honest position is to build for the durable version of search, where relevance and genuine authority compound, rather than for the exploit that happens to work this quarter.