For business
What it is
Long documents are split into smaller passages, each about one topic, so the system can later find exactly the part that matters.
Why it matters
Answers become more precise and cheaper: the model reads three relevant paragraphs instead of a two-hour transcript.
Risk without it
Passages cut mid-thought lose their meaning; passages that are too large drown the answer in noise.
For engineers
How it works
Split on structure first (speaker turns, headings, topic shifts), then by size (typically a few hundred tokens), with a small overlap so context is not lost at the edges. Each chunk carries metadata: source, position, speaker, date and client.
Typical stack
- custom splitters
- LlamaIndex / LangChain splitters
- tokenisers
Pitfalls & trade-offs
- Tune chunk size against retrieval quality, not intuition.
- Too much overlap returns the same passage twice.
- Store chunk-to-source offsets so citations can point back.
In this example
In
→ transcript 6 topics
Out
c01 goals c02 income c03 risk … overlap 15%
Fictional sample · not a client system