What is Chunk Overlap?
Chunk overlap refers to the practice of sharing a certain number of tokens or characters between adjacent chunks when splitting documents for embedding and retrieval. Rather than creating completely separate segments, overlapping chunks ensure that information appearing near chunk boundaries is preserved in multiple chunks, reducing the risk that important context or relationships are lost due to arbitrary splitting points.
The overlap size is typically specified as a number of characters or tokens, often representing 10-20% of the chunk size. For example, with a chunk size of 1000 tokens and an overlap of 200 tokens, the last 200 tokens of one chunk would also appear as the first 200 tokens of the next chunk. This redundancy helps maintain continuity across chunk boundaries and increases the likelihood that semantically related information stays together.
While chunk overlap improves retrieval quality by preventing context loss, it also increases storage requirements and processing time since some content is embedded and stored multiple times. Finding the right balance between overlap size and efficiency is an important consideration in RAG system design. Too little overlap may result in fragmented context, while too much overlap creates unnecessary redundancy without proportional benefits to retrieval quality.