Free Content & Token Analyzer See your page the way AI engines do
Enter a URL to measure code vs. content bloat, estimate the context-window token cost of your page, and preview exactly how AI systems chunk your text for RAG retrieval.
Why the AI Content & Token Analyzer is useful
AI engines don't read your page like a human — they strip the markup, extract the raw text, count it in tokens, and split it into overlapping chunks before storing it in a vector database for retrieval. If most of your page weight is tracking scripts and structural markup, AI crawlers burn their limited context window reading code instead of your actual message.
This tool shows you both halves of that process. The Token Optimizer reveals your content-to-code ratio and how many tokens your real content costs, while the RAG Chunking Simulator previews exactly how your text gets divided — flagging chunks that break mid-sentence and could lose semantic meaning when an AI retrieves them. Together they tell you whether your content is lean, self-contained, and ready to be cited.
Frequently asked questions
How are tokens estimated?
We use the standard approximation of roughly 4 characters per token, which closely matches how models like GPT and Claude tokenize typical English and European-language text. It's an estimate meant to show relative cost and content weight — not an exact billing figure for a specific model.
What is RAG chunking and why does it matter?
Retrieval-Augmented Generation (RAG) systems split your content into small overlapping blocks, embed each one as a vector, and retrieve the most relevant blocks to answer a query. We simulate this with a 400-character chunk size and 80-character overlap. Chunks that end mid-sentence are flagged because they can fragment an idea across two vectors, making it harder for an AI to retrieve and cite your point cleanly.
What does the content-to-code ratio tell me?
It's the share of your page's byte size that is real, AI-ingestible text versus scripts, inline styles, and structural markup. A low ratio means AI crawlers waste context memory parsing code before reaching your message. Lean, content-rich pages are cheaper to ingest and easier for AI engines to understand and cite.
Is it really free, and do you store my site data?
Yes, it's 100% free with no signup. We fetch the public URL you enter in real time, extract the text in memory to run the analysis, and don't store your content or require an account.