OpenAI's file search tool splits every document into 800-token chunks with 400 tokens of overlap between them, according to OpenAI's own API documentation. Microsoft's Azure AI Search recommends starting at roughly half that: 512 tokens, with an initial overlap of 25 percent. Then in May 2026, Google's Search Central team published guidance stating that publishers do not need to break content into tiny pieces for its AI systems to understand it, contradicting years of advice to fragment pages into micro-sections. Three companies building the infrastructure that decides what gets cited do not agree on the basics. Here is what chunking actually does, and which of that advice applies to your content.
What chunking actually is
Chunking is the process of splitting a page into smaller pieces before a retrieval system indexes it, typically a few hundred tokens each. Every mainstream AI search tool works this way, from OpenAI's file search to the systems behind ChatGPT and Perplexity: none of them retrieve or rank an entire page as a single unit.
Each chunk gets converted into embeddings, a numerical representation of its meaning, as Ahrefs' guide to retrieval-augmented generation explains. When someone asks a question, the AI converts that question into the same kind of vector, then measures how close it sits to every chunk's vector using cosine similarity. The closest match gets pulled into the answer, and often the citation next to it. This is passage retrieval. An engine choosing a citation is not judging your page as a whole. It is judging whether one specific paragraph answers the question better than a paragraph from a competitor's page, which is why a single strong section can get cited even when the rest of the page is weak.
The default chunk sizes OpenAI and Microsoft actually use
OpenAI's file search tool defaults to 800-token chunks with 400 tokens of overlap, meaning every uploaded file gets indexed as overlapping slices instead of clean breaks. Microsoft's Azure AI Search documentation recommends starting at 512 tokens with 25 percent overlap, about 128 tokens, then tuning from there based on the content type being indexed.
The overlap exists so a fact sitting near a chunk boundary does not get orphaned, split across two pieces that neither one scores highly on its own. Neither company publishes a single correct chunk size, because the right size depends on how self-contained your sentences already are. Dense paragraphs that pack several claims into one sentence tend to need smaller chunks to stay retrievable, while looser, narrative writing can survive being split into larger pieces. Redis's engineering team, which builds infrastructure for several RAG products, describes three broad strategies in production use today: fixed-length chunking by token count, semantic chunking that splits at natural boundaries like headings, and hybrid approaches that combine both. The 288-token gap between OpenAI's default and Microsoft's starting point is itself evidence that chunk size is an implementation detail each company tunes for its own index, not a fixed target your writing needs to hit.
Google says the chunking obsession is overblown, and it has a point
Google's Search Central team published official guidance in May 2026 stating that publishers do not need to break content into tiny pieces for its AI systems to understand it. The company's own generative features, AI Overviews and AI Mode, can parse a multi-topic page and surface the relevant portion without that page being pre-segmented by anyone.
The same guidance dismissed llms.txt files just as bluntly, saying Google Search ignores them completely, a point that lines up with what VizibleAI's guide to llms.txt already found: most published llms.txt files see no bot traffic at all. Read narrowly, Google's stance is fair. Its crawler and ranking systems are not built like a vector database, and instructing every site owner to manually slice paragraphs into 300-word blocks is closer to superstition than SEO. Read broadly, the guidance only covers Google's own systems. ChatGPT's file search, Perplexity's index, and most third-party retrieval products still chunk and embed content before retrieving it, and none of them have said otherwise. The practical answer sits between the two claims: you do not need artificial section breaks or an exact token count, but each section does need to already be the kind of self-contained unit a chunking algorithm would produce on its own.
What Anthropic's own research changes about the advice
Anthropic published research in September 2024 showing that adding short contextual summaries to each chunk before embedding it, then reranking the results, cut retrieval failure rates by 67 percent, from 5.7 percent down to 1.9 percent, across codebases, scientific papers, and long-form documents.
That failure rate is measured as the share of relevant passages a system fails to retrieve within its top 20 results, what Anthropic calls one minus recall at 20. The improvement did not come from choosing a better chunk size. It came from making each chunk carry enough of its own context to be understood once it was separated from the page around it, the same principle Google's guidance gestures at from the opposite direction. Anthropic's researchers also named a specific failure mode worth avoiding directly: replacing a company or product name with a pronoun partway through a section measurably reduces how often that section gets retrieved, because a vector built from "it" or "the company" carries almost none of the original entity's meaning.
Combining contextual embeddings, contextual BM25, and reranking reduced Anthropic's retrieval failure rate by 67 percent, from 5.7% to 1.9%, in testing across multiple knowledge domains.
How to write sections that survive being chunked
The practical response to three companies disagreeing on chunk size is to stop guessing a token count and write sections that would hold up no matter how an algorithm cuts them apart. That means each section answers one question completely, names its subject explicitly instead of leaning on pronouns, and keeps supporting evidence next to the claim it backs rather than bundled into a separate paragraph.
Write the heading as the exact question a reader, or an AI system, would ask.
Answer that question in the opening sentence before adding detail.
Repeat entity names across a section instead of substituting "it," "they," or "the company."
Place a statistic or source next to the claim it supports, not in a citation dump at the end.
This is close to the paragraph-level discipline VizibleAI's guide to answer-first content already lays out: state the conclusion, then support it. FAQ sections built the same way, question as a heading, direct answer first, are part of why structured Q&A content held up well after Google's own May 2026 FAQ schema change. Tools that track AI citations, including VizibleAI, can also show which of your existing sections are already being pulled into answers, which is a faster way to learn your content's natural chunk boundaries than guessing at one.
Why the mechanics differ by engine
Chunking assumes a static index sitting behind the AI system, one built in advance and searched against later. That describes OpenAI's file search and most RAG products, but not every engine that cites brands works this way. Claude does not draw its live citations from a pre-built internal index at all. It retrieves current sources through Brave Search's index at the time of the query, then reasons over the fuller pages that search returns, a mechanism VizibleAI's guide to getting cited by Claude covers in more detail. That is one more reason a single chunk-size recommendation cannot cover every engine a brand needs to appear in: the unit an AI system actually judges changes by architecture, sometimes a pre-indexed chunk, sometimes a passage a live search engine just surfaced.
Frequently Asked Questions
Does chunk size actually affect whether my content gets cited by AI?
Indirectly. No AI vendor lets publishers set the chunk size used to index their page, so tuning your writing to hit a specific token count does nothing. What matters is whether each section already reads as a complete, self-contained answer, since that determines whether a chunking algorithm produces a strong or weak slice of your content regardless of the exact size it lands on.
What is the difference between chunking and passage retrieval?
Chunking is the step where a page gets split into smaller pieces before indexing. Passage retrieval is what happens next: the AI system compares a user's question to every chunk's embedding and pulls back whichever pieces score closest, and those are the passages that end up quoted or cited in the answer.
Do I need to manually add section breaks every 300 words for AI search?
No. Google's Search Central team said explicitly in its May 2026 guidance that content does not need to be broken into tiny pieces for its AI systems to understand it. Forcing arbitrary breaks can hurt readability without improving retrieval on the platforms that do chunk content, since their chunking happens automatically regardless of your formatting.
Does chunking work the same way across ChatGPT, Claude, Perplexity, and Gemini?
No. OpenAI's file search and most RAG-based tools index pre-chunked, embedded content ahead of time. Claude's web citations instead come from Brave Search's live index rather than a static internal chunk store, so a technique tuned for one engine's retrieval architecture will not automatically transfer to another.
Why did Anthropic's research reduce retrieval failures by 67 percent?
Anthropic added a short AI-generated summary of surrounding context to each chunk before creating its embedding, then applied a reranking step after retrieval. That combination, contextual embeddings plus contextual BM25 plus reranking, dropped the failure rate from 5.7% to 1.9% in testing published in September 2024, mainly by preventing chunks from losing meaning once separated from the page around them.
Track which sections of your content are actually getting cited
Knowing that a chunk got pulled into an AI answer is different from knowing it happened on your page. VizibleAI tracks brand mentions and citation sources across ChatGPT, Claude, Gemini, and Perplexity, showing which of your pages, and which sections within them, engines are actually pulling from. Start a free trial to see what's currently being cited from your site, and what isn't.