13 RAG chunking strategies for better retrieval

A critical step in a retrieval-augmented generation (RAG) system is storing chunks into a vector database. The strategy you use to chunk data before storing it will directly impact the accuracy of the information returned to your users, especially in modern RAG applications. In this article, we'll discuss effective strategies for chunking. Here are the topics we'll cover:

• An explanation of RAG chunking
• Why chunking is important in RAG
• Several chunking strategies that you can use and how to choose the best one for your use case
• Advice for choosing a chunking strategy
• Best practices for RAG chunking
• Evaluating chunking performance
• Common pitfalls and mistakes in document chunking
• Current trends you might see in RAG chunking

What is RAG chunking?

RAG chunking is taking a larger document and breaking it down into smaller components through segmentation. In a text document, for example, the smaller components are phrases from the original document. A chunk can be a few words or a few sentences long. The purpose of chunking is to increase the performance and accuracy of a RAG system. Instead of forcing your system to process one large document to find answers, it can search for smaller bits of information that match a query's data and provide an answer. The chunk should contain a fact or concise information that the RAG system can use to find information. The more concise the chunk, the better performance and accuracy that you can expect from the output.

Why is chunking important in RAG?

The most important aspect of RAG is its ability to return accurate information. Chunking improves performance and the accuracy of responses by combining facts into smaller text segments. These segments are then stored in a vector database to retrieve facts and process answers, thereby improving retrieval accuracy. Chunks must be logical segments, or you could cause hallucinations. For example, if you have a chunk that focuses on two different subjects at the same time, you might get weaker similarity scores than if you separate the two subjects into their own chunks. Using chunks also reduces the number of tokens per query. If you feed an LLM large documents, you can quickly exhaust your token budget. Separating documents into chunks reduces the number of tokens needed for processing, so you also reduce the costs of running a RAG.

What is the best chunking strategy for RAG?

The best chunking strategy depends on your use case, and you should always test it on real-world queries. In general, chunking has four umbrella strategies:

• Fixed size: Split document content into fixed-length phrases regardless of content.
• Sentences: Every chunk represents a sentence in your document.
• Overlapping: Chunks include trailing phrases from previous chunks.
• Semantic: Group chunks and phrases based on logical information. To evaluate whether the strategy you choose is working, base success on precision, recall, and context preservation. You can calculate each of these evaluations.
• Precision: Determines if each chunk is relevant to the query. The equation is TruePositives / (TruePositives + FalsePositives).
• Recall: Of all the relevant chunks, how many did you find? The equation is Number of chunks retrieved with the answer / Total number of chunks containing the answer.
• Context preservation: You can't use an equation for this observation, but you can look at sample chunks to determine if your strategy is sound. Chunks should be self-contained facts, and they should not be cut off in a way that changes facts.

What are the different chunking strategies for RAG?

You have several chunking strategies to choose from, each with its own advantages and disadvantages. We've detailed 13 different chunking methods below:

• Fixed-length chunking
• Sentence-based chunking
• Semantic-based chunking
• Paragraph-based chunking
• Sliding-window chunking
• Recursive chunking
• Hierarchical chunking
• Context-enriched chunking
• Topic-based chunking
• Modality-specific chunking
• Agentic chunking
• Subdocument chunking
• Hybrid chunking

1. Fixed-length chunking

Fixed-length chunking takes a document and splits it into a specified number of characters. Sentence structure, ideas, and formatting are not taken into account. Every chunk is the same length. This approach is sometimes also referred to as fixed-size chunking. Pros:

• Fast
• Easily implemented
• Chunk size is predictable Cons:
• Can cut off facts mid-sentence
• Context can be lost
• Orphaned ideas Best for:
• Log files
• Tabular data like CSV files Example: You have system log files and want to ask questions about a system failure. The log files are set-length data, and context is contained in each column. How to implement:

2. Sentence-based chunking

Sentence-based chunking separates phrases based on punctuation. Rather than separating a document by character count, sentence-based chunking identifies periods, question marks, exclamation marks, and paragraph breaks and uses them to define chunk length. This is a common approach in NLP workflows where sentence boundaries are important. Pros:

• Preserves sentences, so it preserves meaning
• Chunks contain facts without being cut off
• More natural for language Cons:
• A single sentence doesn't always contain all the context for a fact
• Chunk sizes are variable
• Poorly written sentences could interfere with chunking Best for:
• Technical documentation
• Organized, edited stories
• Proofread content Example: You have technical documentation for a software tool and want to allow developers to ask questions about the product. Documents are well-written and edited for structure. How to implement:

3. Semantic-based chunking

Semantic-based chunking groups phrases by ideas rather than by size. A group of sentences that convey the same information is kept together in a chunk. Large documents are chunked by the AI model itself to determine which ones should stay grouped together based on semantic meaning. This strategy depends heavily on the quality of the embedding model used to compare sentences. Pros:

• Chunks contain contextual information
• Provides the best performance for retrieval
• Best for documents with multiple topics Cons:
• Usually uses more tokens than other strategies
• Cumbersome for documents covering a single topic
• Harder to determine chunk length Best for:
• Smaller documents covering multiple topics
• Mixed document types Example: Healthcare documents link symptoms across multiple topics, including pathogens, disease symptoms, recovery, and modes of infection. Chunking each topic can improve performance for each disease subtopic in a single document. How to implement:

4. Paragraph-based chunking

Paragraph-based chunking separates text based on paragraphs. Your chunking process will use blank lines, multiple carriage returns and line feeds, and indentations to indicate that a paragraph ends and another begins. Pros:

• Good at creating chunks from organized thoughts in each paragraph
• Chunks contain complete topics and facts
• Simple to implement Cons:
• Variable chunk size
• Depends on properly formatted documents Best for:
• Technical papers
• Edited stories
• Policy and procedure handbooks Example: Employee handbooks are well-written, proofread, and edited for organization and readability. A document like this would be perfect for paragraph-based chunking. How to implement:

5. Sliding window chunking

Sliding-window chunking uses overlapping data from previous chunks to ensure that whole ideas are encompassed within each chunk. For example, you might have two sentences that say ‘Dogs like food. Cats like food.' One chunk would be ‘Dogs like food.' and the next chunk would contain ‘like food. Cats like food.' It might seem like a silly chunk, but the idea about food is preserved. Pros:

• Chunk sizes are consistent
• Ideas are preserved instead of lost in segmented sentences
• Better coverage for ideas Cons:
• Redundant data is stored, costing more
• Some text meaning can be lost Best for:
• Legal documents with dense information
• Technical documents like white papers, where every sentence has critical information Example: White papers are dense with information, with every sentence offering critical information to the reader. Sliding-window chunking ensures that every sentence and idea is preserved. How to implement:

6. Recursive chunking

Recursive chunking starts with separating paragraphs and then chunking those paragraphs even further based on a hierarchy of separators until each chunk is your target size. First, it separates by paragraphs, then by sentences, then by words, and finally by characters if the final result is still too large. This recursive algorithm is useful when you need a predictable fallback for messy documents. Pros:

• Handles all document types
• Controls chunk size without depending solely on a single format Cons:
• Inconsistent chunk boundaries
• Ideas could get lost
• More complex implementation Best for:
• Documents of unknown size and quality
• Variable document formatting Example: In web scraping and automation, ingestion pipelines pull data from various sources with inconsistent formatting. Recursive chunking will chunk each document regardless of formatting or sentence structure. How to implement:

7. Hierarchical chunking

Hierarchical chunking uses two types of chunks to maintain topics: a parent chunk and smaller child chunks. The parent chunks are sent to the LLM for processing, while the child chunks are used to assess accuracy. Child chunks contain more precise information, and the parent chunk contains the more general information for the overall topic. Pros:

• Preserves accuracy in child chunks but general context in the parent chunk
• Reduces the chance of hallucinations Cons:
• Higher storage costs
• Higher token costs from larger parent chunks Best for:
• Documents with general ideas and context
• Disorganized content with general context Example: Medical content where specific treatments could be applied to various illnesses, so child chunks are good for treatment doses, but the surrounding content adds context to medical questions. How to implement:

8. Context-enriched chunking

Context-enriched chunking adds metadata to each chunk, giving it more context for processing. Storing a chunk with metadata makes it easier to ensure that each chunk contains information that explains its content. It's similar to labeling an image or document so that both computers and humans can understand the information it contains, and it can make each chunk more context-aware. Pros:

• Better accuracy for chunks
• Resolves any ambiguity or unclear ideas Cons:
• Increased storage costs
• Higher token costs
• Over-enrichment can dilute ideas Best for:
• Documents with interlinked or overlapping ideas
• Ambiguous chunks that could have several meanings Example: Customer service chatbots where a store offers multiple products with overlapping data. Metadata ensures that more context can be added when answering general customer queries involving multiple similar products. How to implement:

9. Topic-based chunking

Topic-based chunking groups content based on subject matter. Unlike semantic chunking, which groups surrounding content based on similarity, topic-based chunking pulls data from anywhere in the document and groups it. Pros:

• Organizes scattered content in a document
• Provides more information for a RAG system on a particular topic Cons:
• Chunks can be too large if the same idea is covered multiple times in the document
• Topics must be known before chunking, so automated chunking is much more difficult to implement Best for:
• Organized documents where the content is already well-known
• Queries from users that are predictable and well-defined Example: References for multiple topics that span long, unformatted documents are best for topic-based chunking. Documents such as product information stored in HTML might contain multiple instances of the same facts across the page, and topic-based chunking can extract this data regardless of the page's organization. How to implement:

10. Modality-specific chunking

Modality-specific chunking handles multiple data types (e.g., images, tables, and code) and chunks each document using its own format. Each data type has its own chunking strategy, and non-text document items are preserved. This strategy is useful when preprocessing needs to preserve the structure of different content types before retrieval. Pros:

• Preserves non-textual data
• Handles documents with embedded images and tables Cons:
• More complex handling for multiple document types
• More overhead for developers
• Higher costs for parsers to identify each data type Best for:
• Documents that contain more than simple text
• Code examples
• Financial reports with tables and charts Example: Developers often need their code documented after deployment, and modality-specific chunking can parse code and generate contextual information. Users can then ask questions about the code, which helps with the customer service aspect of code and API deployments. How to implement:

11. Agentic chunking

Agentic chunking lets an LLM decide the best chunking strategy. Instead of using a fixed chunking length, the LLM ingests the document and separates it based on its own evaluation of ideas and content. This is a more LLM-based approach to chunking. Pros:

• Convenient for those who aren't sure of the right chunking strategy
• Handles issues of redundancy, cutting off ideas, or poor formatting Cons:
• More expensive than other strategies
• Unnecessary for smaller, precise documents
• Slower at chunking for large documents Best for:
• Large documents where the content is unknown
• Fast retrieval and when cost isn't important
• Unstructured documents Example: Legal documents can contain thousands of pages for a specific case. Using an LLM to chunk documents saves time and ensures that chunks are properly formatted. Lawyers can then ask questions about the case without spending hours manually reading them. How to implement:

12. Subdocument chunking

Subdocument chunking uses a document's format, like subheadings, titles, chapters, and other sectioning triggers, to determine chunk content. For example, if you have a book with five chapters, subdocument chunking will separate the document into five chunks based on chapter indicators. This can include titles, chapters, and headers. Pros:

• Usually preserves intent and ideas
• Easier to understand and implement Cons:
• Requires structured documents
• Chunks are large and can be an entire chapter if no subsections exist Best for:
• Highly structured documents where you know formatting is done in chapters and sections
• Sections and chapters are named throughout the entire document
• Books, policy documents, manuals Example: A book author might appreciate subdocument chunking to give readers a way to ask questions about story ideas, character details, and world-building. How to implement:

13. Hybrid chunking

Hybrid chunking combines multiple strategies into a single approach. You might have several steps in your RAG pipeline, each using its own chunking strategy. A hybrid strategy is often used in large RAG deployments where multiple document storage locations are integrated, and the automation pipeline spans multiple applications. It is often useful when different chunking approaches are needed for different document sources. Pros:

• Maximum flexibility for complex pipelines
• Handles enterprise systems with multiple applications Cons:
• More complex to maintain
• An error in one strategy can affect the entire pipeline Best for:
• Multiple enterprise documents spanning multiple storage locations
• Variable structured documents in a RAG pipeline **Example: **An internal tool for product managers and product developers to ask questions about products could use corporate web pages, documentation, and user manuals to store information. A hybrid approach could pull all these documents together to answer questions. How to implement:

How do you choose chunk size?

The first step to choosing a chunking strategy is to detail the types of questions that you can expect from your users. Will the questions be general, need precision, or require overlapping information? For example, a subject-matter expert might require more detailed answers than a general user. Your chunk size also depends on the use case, but you can follow a few standards here. Chunks should be large enough to contain ideas but small enough to stay precise. Most LLMs support 512-8,000 token chunks, but you lose precision as you creep into higher token limits. The context window of your model will also affect how much information you can safely include. Document structure will also determine chunk size. If you're using fixed-length chunking, for example, you have a specific chunk size to expect, but other chunking strategies, like modality-specific or paragraph-based chunking, require larger chunks. A tokenizer can help estimate chunk size before the chunks are sent to your model.

How much overlap should chunks have?

For most strategies, 10–20% overlap between chunks is a good amount. If you have too little overlap, you risk losing context and accuracy, and splitting facts into separate chunks, which can create hallucinations. Too much overlap can lead to unnecessary increases in storage and token costs. Just like with too little overlap, too much can also lead to potential hallucinations. While these overlap numbers are ideal, they might not be good for your specific business use case. Test your overlap strategy to find out if it works for you. Use real-world queries to test results and iterate until the output improves.

What are RAG chunking best practices?

You should use a chunking size that fits your use case, but it also needs to contain ideas and stay within your token budget. Here are a few best practices to consider when building a chunking strategy:

• Use chunks that contain a single idea
• Avoid overlap where multiple ideas are contained in a single chunk
• Allow a variable-sized chunk when you have multiple formats and data types
• Use 10–20% overlap unless you have an exception to the rule based on use case
• Base chunk size on queries, such as small chunks for basic facts and medium chunks for longer explanations
• Avoid redundant chunks
• Test your chunking strategy with real-world questions The goal is effective chunking that balances answer quality, cost, and performance.

How do document types affect chunking?

Chunking depends on document formatting and structure, so you first need to evaluate the type of documents you will ingest. Well-structured documents can use more fixed-length or simpler chunking strategies, but coding for your RAG gets more complicated when you include images, scanned documents, tables, charts, and other non-textual elements. Long, unstructured documents like web pages or blog articles usually have several formats. You will need more overlap and a strategy that organizes chunks. You might need context enrichment and modality chunking to add more context and handle other formats. Overall, more structure in documents means you need less overlap. Larger, unstructured documents might require larger chunks and more complex strategies. As usual, always test your strategy with real-world queries to ensure that answers are accurate. For example, structured datasets usually need a different strategy than long-form documents.

How do you evaluate chunking performance?

A human reviewer is necessary to evaluate chunking performance. Metrics like accuracy, performance, recall, and answer coherence are all important to track in a real-world application. You can use code to evaluate some portions of quality, like evaluating chunk size after you've segmented your documents. Every step in the RAG process should be reviewed to avoid inaccuracies and hallucinations. During processing, a chunk is retrieved and assigned a similarity score. Evaluate this score and review the chunks for query-answer relevance. Chunks are then used to query the LLM and its answers should also be reviewed. You might even decide to test multiple chunking strategies to see which one is better for performance, especially if this is your first time implementing a RAG system. One strategy might give you more accurate results, but another strategy might use fewer tokens, saving you money. Testing should also include retrieved chunks to show reviewers whether the system is returning relevant information. Measuring latency can help you understand whether a strategy is too slow for production.

What are common chunking mistakes in RAG?

Most RAG chunking implementations suffer from the following five mistakes:

• Size issues: Chunks that are too small might not capture a complete idea, while chunks that are too large could contain multiple ideas and yield inaccurate results.
• Boundary problems: Poor strategies can cut off ideas from the appropriate chunk or create chunks that lack a complete idea, leading to inaccurate results. These boundaries are sometimes called breakpoints.
• Context issues: Orphaned ideas or excluding overlap when necessary could create chunks with incomplete ideas.
• Pipeline errors: In a hybrid environment, an error in one strategy can propagate to other steps in the pipeline.
• Poor evaluation: A reviewer should be human, and humans make mistakes. If errors are overlooked, the RAG system could produce inaccurate results.

How to choose the right RAG chunking strategy for your use case

RAG systems often involve machine learning, search, ranking, and generation, so choosing the right chunking strategy requires understanding the trade-offs across cost, accuracy, and complexity. You want to balance the simplicity of your strategy with accuracy. Don't forget the costs, too. Unless you're familiar with chunking, you will need to try out multiple strategies before you settle on a single one. Always review the chunks and the LLM's output to identify any issues. Overall, the quality of results should be a priority, and the chunking strategy you choose will directly impact answer quality.

How Meilisearch supports scalable retrieval for advanced RAG systems

Meilisearch combines keyword and semantic search, making it ideal for RAG systems. You can use Meilisearch's built-in embedder to integrate and simplify your RAG workflow. Try Meilisearch