*Day 1 of [Meilisearch Launch Week](https://www.meilisearch.com/launch-week-q1-2026). New releases dropping every day - keep reading to the end to see what else is coming.*

---

Every search deployment eventually hits a wall.

Maybe your index has grown to a size where a single instance starts struggling to keep up. Maybe you have just shipped a product to users across three continents and you are watching latency climb on one end. Or maybe your on-call team gets paged every time you push a Meilisearch update because there is no failover during the restart window.

These are real, painful problems. And up until now, solving them on Meilisearch Cloud meant contacting us and waiting.

Today, that changes. **Sharding and replication are now available on Meilisearch Cloud** -- letting your search infrastructure grow horizontally, stay available, and serve users from the nearest node, all configured and managed by our team so you do not have to handle the infra yourself.


## Two problems, two solutions

Scaling search is not one problem - it is two distinct ones, and they need different answers.

**Too much data** is a sharding problem. When a single instance no longer has the capacity to index and serve your full dataset at the response times you need, you need to distribute the data across multiple nodes. Each node holds a slice (a "shard") of the full index, and queries fan out across all of them before results are merged.

**Too much risk** is a replication problem. When you cannot afford for search to go down - even for the ten seconds it takes Meilisearch to restart during an update - you need multiple nodes holding identical copies of your data. If one goes down, traffic shifts to another instantly.

Meilisearch Cloud now handles both.

## Sharding: horizontal scaling for large datasets

Sharding distributes your data across multiple Cloud nodes. Each node holds a shard - a portion of the total index - and all nodes work in parallel to handle queries and indexing.

**When should you shard?** When your dataset has grown beyond what a single instance can handle at acceptable performance. Not sure where that line is? We help you figure it out. Running scale tests together and determining the right shard count before configuring your cluster is part of how we work with teams today.

**How queries work:** when a search request comes in, it fans out to all shard nodes. Each node searches its slice, returns results, and the leader merges and ranks them before responding. The latency impact is minimal; the parallelism is the point.

**An important nuance on migrations:** sharding does not require a full data migration. When a remote is added, a partial migration is involved - but you are not starting from scratch. This is what allows you to address bigger datasets as you grow, without a forced reindex of everything.

One thing sharding does not solve: availability. If one shard node goes down, you lose access to part of your data. For that, you need replication - or a combination of both.

## Replication: high read availability and geo-distribution

Replication keeps multiple nodes in sync with the same complete dataset. When one goes down, the others keep serving reads seamlessly.

**Architecture:** Meilisearch Cloud uses a static leader-based model. Configuration changes go to the leader node, which proxies them automatically to all remotes. This keeps the cluster consistent and predictable.

A note on scope: replication today covers high *read* availability. *Write* high availability - eliminating the leader as a single point of failure for writes - is not included in this release.

**Three use cases replication covers:**

**1. High read availability - failover during updates**

When Meilisearch restarts during an upgrade, the window is short (typically around ten seconds), but it is enough to interrupt user-facing search. With replication, traffic shifts to another node automatically. Users keep getting results throughout.

**2. Geo-replication - serve users from the nearest instance**

Distribute nodes across regions and route each user to the one that will respond fastest. This is live for replication-only setups today. Optimal geo-routing with sharding combined is on the roadmap.

**3. The combination - scale and availability together**

Sharding and replication are fully composable. If your dataset is too large for a single instance *and* you need high read availability, you can run both simultaneously.

## Composability: running both at once

One of the design principles behind this release is that sharding and replication are not mutually exclusive modes -- they are composable building blocks.

Here is what that looks like in practice:

- **Sharding only:** split a large dataset across multiple nodes, each holding a fraction of the total index. Scale horizontal query and indexing throughput.
- **Replication only:** two or three nodes all holding the same data. High read availability, automatic failover, geo-routing.
- **Sharding and replication:** 5 shards x 2 replicas = 10 nodes. Every shard is replicated. You get scale *and* availability.

Billing follows the same logic: you pay for the number of nodes you run. Two replicas means twice the machines, roughly twice the cost - the same model you would see from most cloud infrastructure providers.


## What this looks like in practice

One enterprise team we worked with recently had reached a point where their index had grown past 100 million documents. A single instance was no longer meeting their performance targets.

Together, we ran a scale test to identify where a single node saturated. Based on those results, we determined the right number of shards - more than our initial estimate, once we saw the actual load profile. We configured the cluster, validated performance, and they were live.

For monitoring, they set up webhooks to track indexing task success across all nodes. When a task failed on one instance, their system detected it and knew to requeue. A single webhook server receives events from every node, giving a natural aggregate view of the whole cluster's health without anything custom.

This kind of setup -- 100M+ documents distributed across multiple shards, monitored via webhooks -- is exactly what our team configures for enterprise customers today. If your needs look similar, that is a good reason to get in touch.

## Who is this for?

**You need sharding if:**

- Your dataset is large enough that a single instance is noticeably slow, or you are approaching index size limits
- You need more indexing throughput than one node can provide
- You are planning for data growth and want headroom without a forced migration later

**You need replication if:**

- You have an uptime requirement that cannot tolerate even brief search interruptions during updates
- Your users are distributed geographically and latency varies noticeably by region
- You want a reliable failover safety net for production

**You need both if:**

- You have a large dataset *and* a high read availability requirement
- You are running an enterprise-grade, always-on search deployment

If you are not sure - starting with a single Cloud instance is completely fine. When the time comes, we will help you figure out the right architecture and handle the setup for you.

## What is coming next

This release is the foundation. Here is what is already in progress:

- **True optimal geo-replication** - combining sharding and replication for globally distributed routing, so each user always hits the nearest node regardless of how the data is sharded
- **Self-serve replication UI** - configure replication directly from the Cloud dashboard, no need to contact us
- **Improved cluster observability** - an aggregated task view across all nodes in your cluster, so you can monitor the health of your sharded or replicated setup without tracking each remote individually

## Part of something bigger

This is Day 1 of [Meilisearch Launch Week](https://www.meilisearch.com/launch-week-q1-2026). Every day this week, we are shipping something new - from features that change how you configure and monitor Meilisearch Cloud, to improvements deeper in the engine. Stay tuned.

## Ready to scale?

Sharding and replication on Meilisearch Cloud are available today as an enterprise add-on, configured by our team based on your specific setup and scale requirements. If your team is hitting data volume or availability limits, get in touch and we will help you figure out the right architecture.

[→ Get in touch](https://meet.meilisearch.com/meetings/cloud/presentation)


*Have questions about how sharding or replication works under the hood? Drop by our [Discord](https://discord.meilisearch.com) or check out the [documentation](https://www.meilisearch.com/docs/resources/self_hosting/sharding/overview).*

Scale without limits: introducing sharding & replication in Meilisearch Cloud

RAG-as-a-Service: what it is, use cases, providers & more

# How Touchless powers instant, scalable vehicle search for large dealer groups

**The company uses Meilisearch to deliver fast, faceted inventory search across multi-location dealer marketplaces - without fighting the tool.**

[Auto Genius](https://www.autogenius.io/) is the market-facing brand of [Touchless](https://www.touchless.io/), a Toronto-based technology company building performance infrastructure for automotive retail. Touchless has two main products powered by its proprietary technology platform. 

- The first, Speed Layer, uses the EXO tunnel system to make any existing dealer website dramatically faster, with no replatform needed, and is currently deployed across 1,000 domains. This product does not interact with Meilisearch at present, though there is potential for stack consolidation in the future.
- The second is the marketplace platform, primarily serving large auto dealer groups. This proprietary product is built on the EXO rendering system (ERS), which powers full vehicle inventory search and discovery. Meilisearch is at the core of that marketplace experience, delivering a modern, fast, and centralized brand experience for large dealer groups.

## **The challenge: large dealer groups need a different kind of search**

Most dealer websites are slow, cluttered, and built on legacy infrastructure - a far cry from the experience car buyers now expect. The Touchless’ marketplace platform is designed to fix that, serving **larger dealer groups** with multiple locations each, and inventories that typically range from 3,000 to 9,000 vehicles.

In the automotive industry, every vehicle is unique, identified by a VIN (vehicle identification number), with its own make, model, year, trim, mileage, fuel type, transmission, color, and price. Buyers navigate that complexity almost entirely through [faceted search](https://www.meilisearch.com/docs/learn/filtering_and_sorting/search_with_facet_filters), and a search experience that is slow, rigid, or hard to filter doesn't just frustrate - it sends shoppers to a competitor.

This is why large dealer groups come to Touchless - for a **modernized, centralized brand experience** across their locations and **higher conversion rates**. That means speed and design are non-negotiable: first page load, interactivity, filter response, and infinite scroll all need to be exceptionally fast.

Touchless had built its own EXO rendering system - a custom stack leveraging Cloudflare edge workers and web sockets for near-instant page delivery. But to make vehicle search itself feel effortless, they needed a search engine that could keep pace and integrate cleanly with that architecture.

## The solution: Meilisearch via the EXO platform

The Touchless team had previously used **Algolia**, but switched to Meilisearch for three core reasons: **API flexibility**, **integration with edge infrastructure**, and **cost-efficiency**.

Stefan Šoć-McLeod, Chief Software Engineer at Touchless, puts it plainly: 

> *"Meilisearch is API-first, which means the team could build exactly the product they wanted on top of it. For a team running a custom rendering engine, indexing strategy, and recommendation logic, that flexibility was essential."*
> 

Meilisearch also proved a natural fit for the EXO stack. The EXO rendering system - a custom rendering and routing layer - runs on Cloudflare edge workers and web sockets. Meilisearch's API topology connects cleanly to that kind of architecture, enabling search queries to be resolved at the edge with low latency. The two technologies are, in Stefan's words, *"married really nicely."*

The pricing model also made Meilisearch a more sustainable foundation for a multi-tenant product at scale. That becomes especially meaningful when the cost of search compounds across dozens of dealer indexes.

## **Implementation: from raw dealer data to live search in 20 minutes**

Onboarding a new dealer group starts with data ingestion. Automotive data arrives from multiple third-party sources - and from dealers who simply drop files onto an FTP server - in inconsistent, poorly formatted shapes. The team runs this through a normalization pipeline that standardizes the data, loads it into [**Sanity**](https://www.sanity.io/), which functions as a database, and then populates Meilisearch from there.

Each dealer group gets its own dedicated Meilisearch index. Spinning up a new one takes around 100 milliseconds. Once the team has the client's data, they can have a live preview with full search and vehicle detail pages ready in **20 minutes**.

Today, Meilisearch powers both the **search and filter experience** and **infinite scroll** across the marketplace sites for 13 dealer groups, with each group having up to 9,000 vehicles in its inventory and totaling about 100 locations. As a buyer navigates, filtering by body style, setting a price range, narrowing by mileage — results respond instantly without a page reload. The scroll never catches up. It's a qualitative step change from what most dealer sites serve today.

> Kyle Mountsier - Co-founder/COO, says:
”*With Meilisearch, you get a modern experience of instant interactivity, and you get an infinite scroll that you can barely catch.”*
> 

![Touchless x Meilisearch powered search for Casa Auto Group car dealershio](https://unable-actionable-car.media.strapiapp.com/Screen_Recording2026_03_19at10_48_04_ezgif_com_video_to_gif_converter_2c3a52d187.gif)
Source: https://www.casaautogroup.com/shop

## **The results: a search experience that sells**

Touchless’ marketplace sites are among the most performant car-shopping experiences available to dealer groups today. The proprietary EXO system as a whole, with Meilisearch as the search layer, delivers the kind of **instant interactivity** that large dealer groups use to drive brand consistency and conversion across all their locations.

The impact is visible in the product: filter latency that feels instantaneous, infinite scroll that moves faster than a shopper can reasonably keep up with, and a search experience that surfaces the right car quickly.

## **Conclusion: the API-first search engine for specialist platforms**

Touchless is building a highly custom product for a highly specific market. That demands infrastructure that stays out of the way and connects where needed, not tools that dictate how you build.

Meilisearch fits that bill. Its API flexibility lets the team at Touchless build a custom indexing strategy, a bespoke recommendation layer, and tight integration with Cloudflare edge infrastructure, without fighting the tool. For SaaS teams building multi-tenant, performance-critical products on modern stacks, that kind of interoperability is the difference between a search layer that limits you and one that frees you to focus on what actually differentiates your product.

How Touchless powers instant, scalable vehicle search for large dealer groups

Self-reflective retrieval-augmented generation (self-RAG) introduces self-reflection into large language models (LLMs) so they can evaluate their outputs and decide how to act on them, such as by retrieving additional context or improving factual accuracy.

This is your complete guide on self-RAG and what it entails. We will cover:

- What self-RAG is and how it is a step-up from standard RAG.
- How the internal workings of self-RAG use special tokens, such as retrieval, reflection, and critique tokens.
- What specific problems self-RAG solves.
- The main benefits of self-RAG, especially in question-answering, fact verification, and generation quality.
- Limitations of self-RAG, including but not limited to cost, complexity, optimization, and efficiency.
- The proper way to implement self-RAG, through embeddings, APIs, and other [RAG tools](https://www.meilisearch.com/blog/rag-tools) like LangChain and Hugging Face.
- Where Meilisearch fits as a high-performance retrieval engine in self-RAG systems.

Let’s start by defining self-RAG.

## What is self-RAG?

Self-RAG is an evolution of [retrieval-augmented generation](https://www.meilisearch.com/blog/what-is-rag) (RAG) that enables models to self-reflect and determine whether their outputs are satisfactory or require additional context.

Self-RAG adds introspective cycles on top of standard RAG to achieve higher factual accuracy and more reliable responses.

## Why was self-RAG introduced?

Traditional RAG relies on a single, fixed retrieval step, even when retrieved documents are incomplete or irrelevant. This leads the AI model to generate inaccurate answers or rely on unreliable citations – resulting in what we call AI hallucinations.

Self-RAG was introduced as a way to combat those hallucinations.

Through self-reflection, the model can evaluate its output impartially and determine what changes are needed to generate a better response.

## How does self-RAG work?

Self-RAG adds an additional loop on top of the standard RAG, similar to [corrective RAG](https://www.meilisearch.com/blog/corrective-rag).

After the initial retrieval of chunks and response generation, the system steps back and evaluates its output.

This evaluation enables the system to determine whether the generated answer meets its requirements.

![A flowchart depicting how Self-RAG works.](https://unable-actionable-car.media.strapiapp.com/How_Self_RAG_Works_dd40ecc945.png)

If the system detects gaps in the answer, a low confidence score, or missing citations, it re-triggers retrieval. The newly retrieved passages are added, and generation continues with significantly improved context.

## What problems does self-RAG solve?

Self-RAG addresses the following problems:

- **Hallucination in LLM outputs:** Standard RAG systems still generate answers when retrieved documents are weak or incomplete. Self-RAG addresses this by retriggering retrieval when grounding is insufficient.
- **Irrelevant or low-quality retrievals:** Traditional RAG often overlooks quality, not by choice, but because the retrieved context is not enough. Self-RAG doesn’t move to the final generation unless the context is deemed healthy.
- **Lack of self-evaluation:** Many RAG systems generate answers without checking accuracy or citation quality. Self-RAG, by contrast, introduces explicit self-critique and self-evaluation mechanisms.
- **Weak handling of complex queries:** Long-form or multi-step questions often require multiple retrieval passes. Self-RAG handles these cases through reiteration and refinement before the final response. This is similar to [speculative RAG](https://www.meilisearch.com/blog/speculative-rag), a smarter approach to retrieving information from knowledge bases and generating responses.

Now, let’s look at the benefits of self-RAG.

## What are the benefits of self-RAG?

The key benefits of self-RAG are the following:

- **Higher factual accuracy:** More retrieval layers improve factual accuracy, as the model can verify whether the answer is supported by the documents or requires fine-tuning with additional context.
- **Improved efficiency:** There’s a common objection related to the ‘added’ retrieval steps and wasted computation in self-RAG. Self-RAG addresses this objection by retrieving additional context only when necessary.
- **Better handling of complex queries:** Multi-step and long-form questions benefit from iterative retrieval and refinement instead of a single-pass response.
- **Greater controllability:** Based on self-reflection signals, developers can control when retrieval occurs and how much iteration is allowed.

## What are the limitations of self-RAG?

Self-RAG improves factual grounding and response reliability, but, as always, there’s a trade-off.

Teams need to account for the following limitations before committing to self-RAG:

- **Higher compute cost:** As the retrieval and generation counts increase, so does the compute cost. Although self-RAG is efficient, it remains more expensive than standard RAG.
- **Increased system complexity:** Adding self-reflection and critique logic introduces more moving parts that require specialized design and maintenance.
- **Latency overhead:** With multiple layers, response time can be significantly delayed.
- **More demanding tuning requirements:** Effective self-RAG often relies on high-quality engineering efforts. Fine-tuning, reflection tokens, and specialized prompt engineering are all a part of these efforts.
- **More challenging debugging:** Multi-step generation paths make it harder to trace errors and explain system behavior.

Keep these limitations in mind when you next want to balance accuracy gains against performance, complexity, and computational cost.

## How is self-RAG different from RAG?

Standard RAG is basically an end-to-end one-pass pipeline. It performs a single retrieval step and generates an answer immediately.

Self-RAG is the opposite. It evaluates its own generations and checks whether the retrieved documents are sufficient. If not, it decides when to retrieve again.

In standard RAG, retrieval is fixed, and errors in context often lead directly to hallucinations or weak grounding.

Self-RAG introduces iterative retrieval, enabling the system to refine context on demand.

## How is self-RAG different from agentic RAG?

Self-RAG focuses on introspection and iterative retrieval. The response is generated only when the system is satisfied with the factual accuracy and relevancy of the retrieved documents.

[Agentic RAG](https://www.meilisearch.com/blog/agentic-rag) is completely different.

It delegates actions to external AI agents, tools, or sub-systems to complete multi-step tasks. In agentic RAG, the system executes a sequence of tool calls or workflows to satisfy user intent, often involving planning, execution, and result aggregation.

Compared with agentic RAG, self-RAG remains within the core retrieval-generation loop and adds self-critique, rather than breaking down tasks or assigning them to external tools.

## How is self-RAG different from modular RAG?

Self-RAG adds self-reflective loops to retrieval-augmented generation, enabling the AI model to evaluate its own output and retrieve additional context when needed.

[Modular RAG](https://www.meilisearch.com/blog/modular-rag) is unique because it breaks the RAG pipeline into interchangeable components that can be independently developed, swapped, and optimized.

Modular RAG decouples retrieval, ranking, and LLM generation into discrete modules, often with separate APIs or microservices.

Compared with modular RAG, self-RAG emphasizes self-reflection, iteration, and critique within the same generation loop.

## When should you use self-RAG?

Whenever you’re in a situation where the overall quality of answers matters more than response speed, self-RAG is the way to go.

To be precise, you should use self-RAG in:

- **Question answering systems:** Self-RAG is useful when there is no compromise on answers being grounded in retrieved documents.
- **Research and analysis workflows:** When models need to synthesize information from multiple sources and refine outputs iteratively, it becomes a no-brainer.
- **High-stakes domains:** When accuracy can be literally life-saving, such as in healthcare or the legal industry, self-RAG is far superior to standard RAG.
- **Long-form or complex queries:** You need self-RAG when questions require multiple retrieval steps or deeper reasoning beyond one context pass.
- **Dynamic knowledge environments:** When training data or the actual dataset changes frequently, models must continually reassess whether the retrieved context is sufficient.

In these cases, self-RAG provides stronger control and more reliable outputs than standard RAG approaches.

## Who should use self-RAG?

The obvious answer is engineering teams that depend on factually accurate and vetted responses. Self-RAG is most relevant for machine learning engineers and AI researchers working on retrieval-augmented generation, long-form question answering, or self-evaluating models.

Platform and infrastructure teams can also benefit from building shared RAG services that require stronger guarantees regarding the grounding and reliability of responses.

Self-RAG is especially useful for teams that are currently experimenting with new RAG architecture, evaluating generation quality at scale, or pushing beyond the limits of standard RAG pipelines.

## How to implement self-RAG

Self-RAG combines standard RAG building blocks with a self-assessment layer.

In practice, the first step is to implement a retrieval workflow. To follow up, add reflection signals and iterate until the output meets the task requirements.

### 1\. Build a baseline RAG retrieval layer

This first step creates the retrieval foundation that returns relevant documents for user queries. You need it because self-RAG can only self-correct when it can retrieve a better context.

A fast retrieval layer also matters because self-RAG may retrieve multiple times per request.

Meilisearch fits here as a low-latency retrieval engine that supports [hybrid search](https://www.meilisearch.com/blog/hybrid-search-rag) and filtering, which helps control which documents enter the loop.

### 2\. Add self-reflection and critique signals

This step adds reflection tokens or critique mechanisms that enable the model to evaluate its own generated outputs.

Self-RAG uses this layer to determine whether the current context is sufficient, whether citations are missing, and whether the factual grounding is strong enough to proceed.

### 3\. Implement a retrieve-critique-regenerate loop

The last major step connects retrieval, generation, self-evaluation, and re-retrieval into a repeatable workflow.

## What is the future of self-RAG?

The future of self-RAG is undoubtedly heading toward a more [adaptive retrieval](https://www.meilisearch.com/blog/adaptive-rag) and self-evaluating AI systems that can reason about their own outputs in real time.

The current research focuses on optimizing self-RAG output, costs, and complexity.

Plus, [conversational search](https://meilisearch.com/docs/learn/chat/conversational_search) and retrieval engines will play a larger role in enabling low-latency, iterative retrieval.

Over time, self-RAG may become a default pattern for building reliable, grounded AI systems rather than a specialized research technique.

## Why self-RAG matters for the future of retrieval-augmented generation

Self-RAG is more than just a new framework in AI and NLP. It is a complete shift in how retrieval-augmented generation systems are designed and evaluated.

Instead of assuming that a single retrieval step is sufficient, self-RAG treats generation as something that should be questioned and improved repeatedly.

With the aim of reducing hallucinations and producing relevant, factually accurate responses, self-RAG is likely to become a standard component rather than an optional enhancement.

Self-RAG explained: how self-reflective retrieval boosts AI outputs

# The Good, the Bad, and the Leaky: jemalloc, bumpalo, and mimalloc in Meilisearch

This post was originally published on [Clément Renault's blog](https://blog.kerollmops.com/patching-lmdb-how-we-made-meilisearch-s-vector-store-333-faster).

---
It's been some time since a couple of open-source users reported that Meilisearch is leaking memory. I spent time reviewing the codebase to see where a leak could have hidden, but I never found anything suspicious enough. I was also very suspicious of those claims, as we are running multiple thousand Meilisearch instances on our Cloud and have never had any leaking issues that caused instances to be killed by the OS...

...actually, yes, we had a leak in v1.25, but it was related to an LMDB feature I was developing back then, and I talked about this in [another blog post](#). This is fixed now, and it was entirely my fault for diving into a complex C codebase and deploying a tricky change without extensive testing.

I also investigated Meilisearch on the Cloud and always found that Meilisearch wasn't leaking memory in any way. As you can see below, this is the memory usage of a customer who is heavily indexing embeddings and keywords. No apparent leak while they have tens of millions of documents on this machine (part of a sharded cluster).

![memory-leak-investigation1.png](https://unable-actionable-car.media.strapiapp.com/memory_leak_investigation1_b7e3f0942a.png)

But one day, I noticed something similar to what was reported: Meilisearch was using more and more resident memory. Resident memory, also known as RSS, is the memory allocated by the OS that resides in RAM, unlike virtual (VRT) memory, which often comes from memory-mapped files and is allocated on demand. RSS memory comes from calls to the `mmap` syscall with the `MAP_ANONYMOUS` option. That's what your allocator does when you `malloc` something if there are no pages already allocated.

![memory-leak-investigation2.png](https://unable-actionable-car.media.strapiapp.com/memory_leak_investigation2_a9fc3ba1ff.png)

That's what I saw when indexing millions of small documents in Meilisearch. So I decided to dive into the rabbit hole and find this leak. In Rust, I thought it was very easy to detect leaks because they often come from explicit calls to the `mem::forget` function. Unfortunately, we don't have any explicit call to it.

## Jemalloc to the rescue

We used jemalloc for a long time before we switched to mimalloc. The main reason was performance and Windows and macOS compatibility. We thought mimalloc would work better with Windows, as it is a Microsoft project, but we were eventually proven wrong and disabled mimalloc on Windows. Still, we were able to use mimalloc on macOS this time. The story repeats itself, and on Windows, Meilisearch uses the default allocator.

However, jemalloc is a very performant allocator – better in every way than macOS's default allocator, for sure 👓 – and, most importantly, it has very helpful tools to track leaks and heap allocations. I changed Meilisearch's global allocator back to jemalloc, enabled the profiling feature, and generated a couple (too many) SVG outputs of memory tracking.

The SVG showed around 100 MiB being leaked in some part of the code I recently modified. New code is often the source of new bugs, so I reviewed my code more thoroughly – and I invite you to do the same, because I am very proud of it. Long story short, there was no memory leak introduced in my PR, yet the report still referenced this code.

I am not a very prominent user of AI agents. I use [Zed](https://zed.dev) – not for its AI-related features, but more because I love Rust, and this editor is quite good. As the Zed team uses Zed to work on the Zed codebase, they've worked hard on the Rust analyzer integration and the debugger. That's a breeze compared to Sublime Text's bad Rust integration and VSCode's browser slowness and resource usage.

End of digression. I still decided to get a little help from an LLM to see where it could lead me. I generated a text report from a jemalloc heap analysis and gave it to my Zed Claude Sonnet 4.5 agent... It impressively found multiple leaks in the codebase – approximately four. Rest assured, only one was an actual leak, yes... AI, you know. Still, the one it found was a real one.

## The trap of bumpalo

You may know about [Bumpalo](https://docs.rs/bumpalo/latest/bumpalo/). It's a bump allocator library. The main goal is to reduce pressure on allocators (e.g., mimalloc, jemalloc) by allocating memory chunks that are freed only when the `Bump` object is dropped. We use this in Meilisearch to perform many small allocations (e.g., document IDs, field names) and retain them throughout indexation, avoiding numerous small, scattered allocations that could lead to cache-unfriendly patterns and slowness.

The main trap with bumpalo is our use of [`bumpalo::Vec::into_bump_slice`](https://docs.rs/bumpalo/latest/bumpalo/collections/vec/struct.Vec.html#method.into_bump_slice). This method takes an allocated vector and converts it into the underlying slice. This slice can be used safely and is not even considered a leak, since its lifetime is the `Bump` one – which means the slice will be deallocated when the underlying memory chunk owned by the `Bump` is dropped. However, there is a catch here. When converting a `bumpalo::Vec` to a slice, the slice never runs the drop glue, and we were storing a global-allocator-backed data structure – i.e., an `std::Vec` – in this `bumpalo::Vec`. What a mistake. You can review [the patch for the AI-assisted-discovered leak](#).

I felt relieved after finally fixing this longstanding bug. It had been there since v1.12 – approximately 1.5 years ago. All that time, and nobody noticed. We wondered whether bumpalo was "at fault" here and whether `bumpalo::Vec` should only accept storing structures that implement a specific `bumpalo::Drop` trait. However, implementing that seemed very complex. So, I started measuring Meilisearch's memory usage again with the clear leak fixed...

![memory-leak-investigation3.png](https://unable-actionable-car.media.strapiapp.com/memory_leak_investigation3_30dd8a6635.png)

What the heck?! I checked whether I was using a pre-patch binary – and no, there was still a huge increase in memory usage. My patch had actually changed nothing. Someone could easily lose their mind over this, but I cooled down and tried using my "super-powered" AI agent again. This time, it found nothing.

## The downsides of using non-Rust dependencies

Meilisearch uses [LMDB](https://www.symas.com/lmdb), a lightning-fast memory-mapped key-value store, to store and operate its internal indexes. We store the inverted index for our search engine to serve search requests, and thanks to its implementation of multi-version concurrency control (MVCC), we can serve search requests while updating the internal data structures. It's probably the most important dependency in the engine, and we put a lot of pressure on it. Note that this library is in C.

In C, you said? Yes – and that's part of the answer to our memory leak. Meilisearch is statically linked with LMDB, and Meilisearch also uses mimalloc as the global allocator. But LMDB doesn't use mimalloc and relies on the default allocator – the system allocator. Long story short, LMDB is allocating and deallocating pages that Meilisearch cannot reuse because both allocators are not cooperating, and that's the main clue to this mystery.

I decided to remove the custom global allocator in Meilisearch and set `LD_PRELOAD` (an environment variable used to load shared libraries before others – in this case, to replace the allocator) with jemalloc and profiling enabled, so I could get a good report to feed to my "super-powered" AI agent. I had already dived into LMDB before, and some allocation-related parts looked quite spooky to me. I tend not to like this kind of unbounded linked list of page-size allocations.

![memory-leak-investigation4.png](https://unable-actionable-car.media.strapiapp.com/memory_leak_investigation4_2d8851cb4b.png)

Wow! The goal wasn't to drastically reduce memory allocations by using jemalloc, but rather to extract a report. This is very interesting – when using jemalloc, we can clearly see that no leak appears. The main change here was to use a single memory allocator for both Meilisearch and LMDB to enable LMDB auditing, since I was only enabling jemalloc profiling for the Meilisearch application before. It seems that the leak was resolved by unifying them. I extracted a report, and it didn't even show the spooky linked list I mentioned before – no actual leak of any kind.

## Using mimalloc everywhere

So, I tried using mimalloc instead of jemalloc to see how Meilisearch behaves when it shares its allocations with LMDB, and... the result wasn't very convincing – the leak is back. The issue seemed to arise only with mimalloc, not with jemalloc. I looked it up and found out that the amazing team behind mimalloc was actually working on a v3. We were using v2, the version available on main.

![memory-leak-investigation5.png](https://unable-actionable-car.media.strapiapp.com/memory_leak_investigation5_adb3432bf7.png)

I compiled mimalloc v3 and `LD_PRELOAD`ed it in Meilisearch, and the result was much, much better. As you can see, it is now close to jemalloc's memory usage. I found out later that [mimalloc's override feature](https://github.com/microsoft/mimalloc#overriding-malloc) existed, and its goal was to force every allocator call to link to mimalloc and enable it. I also used the `nm` command to verify that the different malloc symbols pointed to the `mi_malloc` symbols.

![memory-leak-investigation6.png](https://unable-actionable-car.media.strapiapp.com/memory_leak_investigation6_868d0347e8.png)

According to the documentation, mimalloc v3 is much better at sharing memory between threads and promises a (much) lower memory usage for certain large workloads. I think indexing documents with Meilisearch falls into this category. We also probably benefit from other features, such as the simplified lock-free design and support for true first-class heaps. I don't know what the latter two features do, but they seem very beneficial to our workload.

We ran some benchmarks to see if everything was good, and the results showed very good performance gains. We had some gains of about 13% and some losses of about 9%, but those are necessary tradeoffs. Note that those benchmarks are run on a dedicated NVMe machine. In the Cloud, most of our customers use network disks, and performance gains will be even more visible with higher-latency disks, e.g., AWS EBS.

The most important thing is that the memory usage is much lower than before. Our systems have limited RAM, and the disks are slow (especially network disks). You'd rather not fetch from disk every time you access a page and instead rely on the page cache. That's why reducing our RSS frees up more memory for the page cache and reduces disk reads, ultimately improving performance. There is still a tradeoff between `munmap`ping memory too often or too early versus keeping it, but mimalloc v3 seems very efficient at managing this tradeoff.

![memory-leak-investigation7.png](https://unable-actionable-car.media.strapiapp.com/memory_leak_investigation7_25b9117aad.png)

After testing the engine for some time with mimalloc v3, I still see some spikes in particularly heavy-indexing parts, but it's much easier to see where we can improve the engine without the RSS noise we had before. The story continues – but it will mostly focus on engine-side optimizations to reduce allocation loads rather than "meta" changes to the engine.

*This blog post has been crafted using my own wording and thoughts 🌱*

The Good, the Bad, and the Leaky: jemalloc, bumpalo, and mimalloc in Meilisearch

# Roadmap roundup: where Meilisearch is heading

We’ve been thinking a lot about where Meilisearch needs to go next. Not incremental improvements, but fundamental shifts in what Meilisearch can do, who it can serve, and how it fits into the modern AI stack.

Today, we’re sharing our updated roadmap. It’s organized around four themes:

- Meilisearch for any workload
- Hybrid by default
- From API to platform
- RAG at scale

Here’s the full picture:

## **Meilisearch for any workload**

Meilisearch works great for most use cases today. But there’s a growing segment we want to serve better: large SaaS platforms with millions of tenants that need per-tenant search isolation. The economics simply don’t work when every tenant, active or not, requires dedicated compute.

We’re solving this with two complementary efforts:

### 1. Distributed search

We're well into this one already. [Sharding](https://github.com/meilisearch/meilisearch/releases/tag/v1.37.0) lets Meilisearch distribute indexes across multiple nodes, breaking through the single-machine ceiling that currently limits dataset size. Replication ensures [high availability:](https://www.meilisearch.com/blog/horizontal-scaling-with-sharding) if a node goes down, search continues to work.

For our [customers](https://www.meilisearch.com/customers), this means Meilisearch can handle datasets that were previously too large. For [SaaS companies](https://www.meilisearch.com/solutions/SaaS), it means they can finally trust Meilisearch as the backbone for search at real scale.

### 2. Serverless indexes

This is the bigger bet. The idea: indexes that aren’t being queried should cost almost nothing.

Today, whether an index serves 10,000 queries per second or zero, it allocates the same compute and memory. That’s wasteful. With serverless, inactive indexes get moved to cheap object storage (like S3) and only spin back up when a query arrives. Active indexes behave exactly as they do today, no performance compromise.

We're targeting Q3 2026 for the first release of serverless indexes, starting with Cloud.

What this unlocks is significant. A SaaS with 1 million tenants, of which only 5% are active at any given time, would pay for 50,000 warm indexes instead of 1 million. The other 950,000 sit on S3 for fractions of a cent. Per-tenant search isolation becomes affordable at any scale.

It also unlocks something we’ve long wanted: **a real free tier**. Today, offering free plans means provisioning compute for users who may never come back. With serverless, a dormant free user’s index costs us almost nothing in storage. When they return, it wakes up and works. We can finally offer a generous free plan without burning money on idle infrastructure.

Serverless also compounds beautifully with geo-replication. Without it, replicating across five regions would cost 5x as much. With serverless, replicas in low-traffic regions scale to zero and only wake when needed. Global distribution becomes affordable instead of multiplicatively expensive.

---

## Hybrid by default

Meilisearch already supports [hybrid search](https://www.meilisearch.com/solutions/hybrid-search), combining keyword matching with semantic understanding powered by AI embeddings. But setting it up requires too many decisions, too much configuration, and too much reliance on third-party providers. We want to change that.

This initiative is about making hybrid search the default, effortless experience, and building a genuine quality moat around it.

### Simpler setup for embeddings and reranking

Today, [getting semantic search](https://www.meilisearch.com/docs/learn/ai_powered_search/getting_started_with_ai_search) working requires choosing a provider, picking a model, understanding dimensions, and writing a document template. Most developers just want to “make my search smarter.”

We’re building a guided setup in the Cloud dashboard that walks you through enabling semantic search in under two minutes. Sensible defaults at every step. We’re also adding AI-generated document templates: Meilisearch samples your documents, analyzes them, and automatically generates the optimal template. No more guessing what text to embed.

On the reranking side, we’re opening support for more providers beyond Cohere (Jina, Voyage, and a generic REST option), and making the composite embedder pattern (a fast local model for queries, a high-quality remote model for indexing) a simple toggle rather than a manual configuration.

### A unified AI Gateway

Every embedding and reranking call currently goes directly to a third-party provider. Meilisearch has zero visibility: no cost tracking, no caching, no fallback when a provider goes down.

We’re building an AI gateway that sits between Meilisearch and all AI providers. It handles provider translation, retry with fallback, authentication, metering, and most importantly, aggressive caching. Embedding the same text always produces the same vector, so the gateway caches duplicate requests instead of paying for them again. This is especially valuable during re-indexing.

For Cloud users, this becomes the default. You enable semantic search, pick "meilisearch" as a source, and everything just works. No provider API keys needed. For self-hosted users, the gateway is available as a paid API with the same benefits. Switching providers becomes a configuration change; no re-indexing is required.

### Our own models

This is the long-term play. We’re training and hosting our own embedding and reranking models, purpose-built for search.

General-purpose models from OpenAI or Cohere are trained for broad similarity tasks. They often duplicate what keyword search already does well, instead of filling in what keywords miss. Our models will be trained specifically to complement Meilisearch’s keyword engine, using co-optimization techniques that teach the model to capture semantic meaning where keyword search falls short.

We’re planning multiple tiers (small, large, multilingual) and a local reranking model that runs inside the Meilisearch binary itself. No external API calls, no network latency.

The quality bar: match established providers on standard benchmarks within 5%, and outperform all general-purpose models on Meilisearch’s hybrid search quality. We’ll publish these benchmarks openly. This creates a real moat: models co-optimized with Meilisearch’s ranking pipeline deliver results that competitors can’t replicate by just calling the same API.

---

## **From API to Platform**

Meilisearch has grown far beyond a simple search engine. It supports sharding, index swaps, webhooks, document functions, detailed performance profiling, and more. But many of these capabilities are invisible in the Cloud dashboard. A feature that can’t be found might as well not exist.

This initiative is about closing the gap between what the engine can do and what customers can actually access, configure, and debug from the UI.

### Full engine access from the dashboard

We’re surfacing the full power of the Meilisearch engine in the Cloud dashboard. Many features exist in the engine today but have no UI: shard management, index swaps for zero-downtime reindexing, webhook configuration, data transfer between projects, document functions (in-place transforms via code), search performance breakdowns, and proper index management with disk usage and compaction.

All of these will get first-class dashboard interfaces. The goal is that everything you can do via the API can also be done from the Cloud UI, with visual feedback and safety checks built in.

### Built-in debugging and observability

Today, when something goes wrong, most customers file a support ticket. We want to change that by giving them the same tools our internal team uses.

The biggest one: we’re opening up our internal “top 50 slowest requests” dashboard to customers. It’s the same tool our team reaches for first when something feels slow. It will come with pattern detection and optimization suggestions for each slow query.

This builds on the per-request performance tracing we already shipped in [v1.35.0](https://github.com/meilisearch/meilisearch/releases/tag/v1.35.0), where adding `showPerformanceDetails: true` to any search query returns a full breakdown of where time is spent: tokenization, keyword search, semantic search, formatting, and more. If you've used the search preview in the Cloud dashboard, you've already seen this in action.

Beyond that, we’re building a better batch view (structured debugging instead of raw JSON), full engine telemetry with configurable alerts (latency percentiles, throughput, resource usage), an AI diagnostic helper that can analyze a slow request and explain what’s going on in plain language, an auto-optimize tool that suggests settings changes based on your data and usage patterns, and snapshot management for backups and point-in-time restore.

Together, these tools transform the [Cloud](https://www.meilisearch.com/cloud) dashboard from a basic management interface into a full observability and operations platform.

---

## **RAG at scale**

Since launching [Meilisearch Chat](https://www.meilisearch.com/solutions/rag) to general availability [last year](https://www.meilisearch.com/launch-week-q3-2025), we've seen strong adoption and learned a lot about what teams actually need when putting conversational search in front of real users. This next phase is squarely focused on experience: making it faster to configure Chat directly from the Cloud UI, reducing the setup friction that still requires touching code, and giving teams more control over how their *chat interface looks and behaves* directly from the dashboard.

### A smarter chat engine

The engine-level changes make the chat feature fundamentally more capable.

The biggest win is **parallel multi-search**. Today, the LLM must make sequential tool calls, one round-trip at a time. For a question like “compare Nike and Adidas running shoes under $150,” that means search, wait, search again, wait again. We’re replacing this with a single tool call that fires multiple searches in parallel. All results come back at once.

We’re also simplifying configuration dramatically. Today, setting up a chat agent requires N+1 API calls across two separate levels. We’re merging everything into a single agent configuration: provider, system prompt, guardrails, few-shot examples, and per-index search parameters in one call.

Other key improvements include dynamic facet discovery (the LLM explores filterable attributes on demand instead of relying on static data baked into the prompt), external tool calling (pass your own tools alongside Meilisearch search for complex agent workflows), conversational memory (the LLM stores and retrieves user preferences across sessions, scoped by tenant token), and better source attribution with real-time citation events streamed alongside the answer.

### From API to product

The engine improvements above mean nothing if developers can’t discover and use them.

We’re building a chat playground in the Cloud dashboard where you can test your agent, see which documents were retrieved, and watch the LLM’s internal tool calls in real-time. No code required to start experimenting.

The second big piece is auto-generation. Meilisearch already knows your data. We’ll use AI to analyze sample documents and automatically generate optimal configurations: system prompts, index descriptions, document templates, search parameters, guardrails, and few-shot examples. A developer should go from “I have indexed data” to “I have a working chat agent” in under five minutes.

We’re also building a Chat UI starter kit: a code generator that produces a complete, working chat component for your framework of choice (React, Vue, Next.js, Vanilla JS, Svelte), pre-wired to your Meilisearch endpoint. Copy, paste, ship. And comprehensive [documentation](https://www.meilisearch.com/docs/home) covering the full RAG pipeline, prompt design, and integration patterns.

### Multi-provider LLM infrastructure

Chat V1 relies on a single hardcoded LLM provider. Any outage means chat is down, any price increase hits margins directly, and customers with specific compliance requirements can’t be served.

We’re building an AI gateway, an internal Rust service deployed as a sidecar on each Cloud region, that sits between Meilisearch and all LLM providers. One unified OpenAI-compatible interface with support for OpenAI, Anthropic, Mistral, Cohere, Google Vertex, AWS Bedrock, and self-hosted models via Ollama.

The gateway provides fallback chains (if one provider fails, the next picks up transparently), per-tenant billing via metering, Bring Your Own Key for enterprise customers, per-tenant cost controls, and prompt optimization to reduce token usage. It’s invisible infrastructure: users interact with Meilisearch’s [Chat API](https://www.meilisearch.com/docs/reference/api/settings/get-chat), and the gateway handles everything behind the scenes.

---

## The bigger picture

These four themes aren’t isolated. They reinforce each other.

Serverless indexes make the AI gateway economically viable at scale. Millions of tenants can each have their own search and chat agents without burning compute on idle infrastructure. The AI gateway makes our own models reachable as the distribution channel for embeddings, reranking, and LLM access. Our models make hybrid search a genuine differentiator: purpose-built models that outperform generic alternatives. The platform improvements make everything accessible: debugging tools, observability, and a dashboard that surfaces the full power of the engine.

Together, they transform Meilisearch from a search engine into a complete information retrieval platform: search, hybrid AI, RAG-powered chat, and the infrastructure to run it all at any scale.

We’re incredibly excited about what’s ahead. As always, Meilisearch remains [open source](https://github.com/meilisearch/meilisearch) at its core, and we’ll share progress as we build. If any of this particularly interests you, or if you’re facing a problem that it would solve, we’d love to hear from you.

Stay tuned.

Want to stay up to date on our priorities and direction? Check out our [public roadmap](https://www.notion.so/Replication-system-2cb4b06b651f811d9457e961f80676ee?pvs=21) or sign up for the [Meilisearch newsletter](https://www.meilisearch.com/).

Roadmap roundup: where Meilisearch is heading

Retrieval-augmented generation (RAG) in AI is an architecture that combines information retrieval with text generation.

Instead of relying solely on a model’s training data, a RAG AI system retrieves relevant information from external data sources and uses it to generate accurate, grounded responses.

This guide is your go-to on how RAG works and how to build it properly. We’ll look at:

- A clear explanation of retrieval-augmented generation, how it connects large language models to external knowledge, and how it aids retrieval.
- How big a role RAG has in grounding generative AI and reducing AI hallucinations.
- A practical view of how a RAG pipeline forms through embeddings, vector search, APIs, and workflows.
- How semantic search, chunking, and hybrid retrieval improve relevance.
- Simple methods for evaluating RAG quality using accuracy and user trust signals.
- The most common RAG implementation failures that break AI systems at scale.
- Crucial security considerations for customer data, repositories, and internal knowledge sources.
- The importance of keeping RAG systems aligned with new data and up-to-date information.
- Where platforms like Meilisearch fit into modern RAG architectures.
- A clear direction for moving from concept to production in RAG implementation.

With this guide, you’ll learn a structured path to building reliable, scalable RAG systems.

Let’s start with what RAG is in AI.

## What is RAG in AI?

[RAG](https://www.meilisearch.com/blog/what-is-rag) in AI combines retrieval and generation in a single workflow.

Rather than relying solely on training data, RAG AI systems use external sources to retrieve critical information. Using the retrieved information, LLMs generate the most relevant response to a user query.

RAG is crucial today because it reduces AI hallucinations in results. Artificial intelligence systems can access up-to-date information rather than relying solely on static datasets used for training and retraining.

The most common example you see every day is that of [adaptive RAG](https://www.meilisearch.com/blog/adaptive-rag) in a chatbot. It retrieves answers from a live knowledge base and generates a relevant response for the user.

## What is retrieval-augmented generation in generative AI?

In generative AI (genAI), the RAG system provides LLMs with access to external knowledge sources before any AI-powered generation occurs.

With context derived from these sources, genAI produces a more suitable and relevant answer for the user because it isn’t relying on model memory alone (that’s [CAG](https://www.meilisearch.com/blog/rag-vs-cag)).

This is why RAG works well for customer support chatbots, AI assistants, and domain-specific AI applications, especially in healthcare. The AI model operates within a system, not in isolation, which reduces hallucinations and improves reliability.

![An illustration flow of how a RAG system works.](https://unable-actionable-car.media.strapiapp.com/How_RAG_Works_c0bd9d1be0.png)

A simple RAG AI workflow looks like this:

1.  A user asks the AI model a question
2.  The RAG system retrieves relevant documents from external sources
3.  The AI model generates an answer based on that context
4.  The user receives a more relevant and more accurate result

## How do you build a RAG pipeline?

A [RAG pipeline](https://www.meilisearch.com/blog/how-to-build-a-rag-pipepline) has three components in a single system: data, retrieval, and generation.

The goal of the RAG workflow is to retrieve the right information first (from real data sources) and then generate an answer based on the proper context.

![A graphic depicting the architecture of a RAG pipeline.](https://unable-actionable-car.media.strapiapp.com/Architecture_of_a_RAG_Pipeline_a5f737d3e0.png)

Here’s what a practical build sequence looks like:

1.  **Data ingestion:** First, you connect the external data sources to the AI model.
2.  **Chunking:** Next, the content is chunked into usable units and embedded for vector- and hybrid-search.
3.  **Indexing:** The embedded chunks are stored in a vector database or [search index](https://www.meilisearch.com/blog/rag-indexing), often with metadata to support filtering and ranking.
4.  **Retrieval:** When a user submits a query, it triggers the retrieval of relevant documents.
5.  **Prompting and generation:** The retrieved documents and context enrich the original user prompt, and the LLMs generate a relevant response.
6.  **Evaluation:** Finally, the system is evaluated against specific metrics, including relevance, latency, hallucinations, and quality.

Depending on your needs, you can add additional elements to the system, such as [hybrid search](https://www.meilisearch.com/blog/hybrid-search-rag), metadata filtering, ranking logic, monitoring pipelines, and continuous evaluation.

## How do you improve RAG retrieval relevance?

No matter how strong your model is, if the retrieved information doesn’t match user intent, the answers will fall below expectations. As such, retrieval should have the same priority as generation, especially in high-end [RAG pipelines](https://www.meilisearch.com/blog/rag-types).

Here are a few actionable ways to improve RAG retrieval relevance:

- **Use hybrid search:** Combine keyword and vector search to improve retrieval, as both exact matches and semantic similarity come into play.
- **Apply metadata filters:** These filters let you remove irrelevant data based on type, source, domain, time range, or access level metrics.
- **Add reranking layers:** A not-so-secret [technique](https://www.meilisearch.com/blog/rag-techniques) is to reorder retrieved documents using relevance-scoring models so that the most useful context appears first.
- **Use query rewriting:** Lets you expand or clarify user queries before retrieval.
- **Tune chunking strategy:** Adjusting chunk size and overlap is critical to ensure the retrieved content is sufficient without adding noise.
- **Optimize top-k selection:** Make sure that you only retrieve the most relevant documents instead of flooding the prompt with low-value data.

Now, let’s evaluate RAG model quality.

## How do you evaluate RAG quality?

How well a RAG system performs depends on its three layers: retrieval, generation, and system performance.

The best combination to evaluate such a system is offline testing and live production monitoring. Let’s see below:

- **Offline evaluation methods:** First up, you’ll need to use benchmark queries and labeled datasets to test retrieval precision. Are the relevant documents appearing in the top-k results? Similarly, evaluate generation quality using metrics such as groundedness checks, faithfulness scoring, and answer-to-source alignment.
- **Online evaluation methods:** You can also assess RAG quality by tracking real user interactions and feedback. Additionally, AI hallucination rates and answer acceptance will give you a solid idea of the workflow’s efficiency.
- **Retrieval metrics:** Metrics such as Precision@k, Recall@k, and retrieval coverage are your best friends.
- **Generation metrics:** Faithfulness, groundedness, hallucination rate, citation accuracy, and answer consistency.
- **System metrics:** Key metrics include latency, cost per query, API response time, embedding cost, and pipeline throughput.

This whole evaluation structure reveals where failures occur and what to optimize next.

## What are common RAG mistakes?

Most RAG failures stem from system design issues, not from model quality. These mistakes increase the likelihood of hallucinations and reduce trust in AI software.

Fixing them early can save you significant trouble in the future.

![A graph listing the most common Retrieval Augmented Generation mistakes.](https://unable-actionable-car.media.strapiapp.com/Common_RAG_mistakes_c0be4646fa.png)

### 1\. Subpar chunking strategies

Bad chunking equals fragmented answers and partial context in the results. To resolve this, adjust the chunk size and overlap. Instead of fixed splits, try semantic or recursive chunking. Each chunk should contain complete, meaningful context.

### 2\. Missing or weak metadata

If you want to keep surfacing relevant documents, ensure the metadata is strong. Otherwise, the retrieval can pull documents from differing domains and hurt retrieval quality. Structured metadata, including source, domain, timestamps, access level, and document type, is useful here.

### 3\. Stale indexes and outdated data

Outdated data leads to outdated search results. The simple fix is to automate reindexing, incremental updates, and refresh pipelines.

### 4\. No evaluation loop

Without continuous monitoring, systems will degrade and answer quality will decline. To address this, focus on tracking retrieval precision alongside latency and user feedback. Try to combine offline testing with live production monitoring.

### 5\. Treating the LLM as the entire system

The LLM cannot be treated as the entire system, nor can it be ignored. It needs to be considered an equal part of the workflow. Overreliance on the LLM will increase hallucinations, and undervaluing it will fragment responses.

## How do you secure data in RAG?

To secure data in RAG, you cannot enforce controls only at the generation layer. Control at the retrieval layer is equally important, if not more so.

After all, if sensitive data is never retrieved, it can never be generated.

Security in a RAG system is improved with access control, isolation, filtering, and visibility across the entire retrieval and generation workflow.

Here's the checklist:

- **Retrieval-time access control:** Restrict which documents can be retrieved based on user identity, role, or permissions. Enforce filtering before retrieval, not after.
- **Multi-tenant isolation:** The practice of maintaining separate indexes, namespaces, or datasets for different users, teams, or organizations.
- **PII handling and data classification:** You must tag sensitive fields and documents using metadata. Another healthy security practice is to exclude personal data and regulated content from retrieval pipelines unless explicitly authorized.
- **Secure API usage:** Never expose master keys in client-side environments. Use scoped keys, role-based access, and token-based authentication for all retrieval operations.
- **Audit logs and traceability:** Logging helps maintain traceability between user queries, retrieval, and generation in RAG systems.
- **Controlled data ingestion:** Strictly control the data being ingested. You must validate sources before indexing to prevent untrusted or unverified data from entering the knowledge base.

Security in RAG is architectural. When retrieval is controlled, generation becomes safe by design.

## How do you keep RAG results up to date?

RAG results can be kept up to date by controlling how data is indexed, refreshed, and re-embedded throughout the retrieval pipeline.

For large, stable datasets, such as policy archives or company documentation repositories used by AI agents, batch indexing is the preferred approach.

For older records, versioning is your best option, as it ensures they can be replaced safely. Combine this with deletion handling, and all outdated or invalid content will be removed from the indexes, reducing hallucinations from pre-trained models.

Re-embedding is also vital. It should be triggered when either the content meaning changes or when files are updated, especially when retrieval algorithms depend on semantic accuracy. Changes like structural edits or major rewrites require new embedding, while minor formatting updates don’t.

The optimal update cadence varies by data type. For instance, product data needs to be updated daily. On the other hand, static knowledge bases can be refreshed weekly or monthly. Support content and logs must be updated in near-real time, including content sourced from open-source systems.

## How does Meilisearch fit into a RAG system?

Meilisearch is a great retrieval layer for a RAG system.

Here’s what a RAG flow with Meilisearch looks like:

1.  A user submits a query.
2.  [Meilisearch](https://www.meilisearch.com/blog/rag-tools) retrieves the most relevant context.
3.  That context is passed to the LLM.
4.  The model generates a grounded response.

What makes Meilisearch powerful in RAG architectures is its handling of context quality. The [hybrid search from Meilisearch](https://www.meilisearch.com/solutions/hybrid-search) combines keyword matching with vector search to unlock exact precision and semantic understanding.

This automatically prevents common failure modes in which systems retrieve unnecessary information without any actual context.

Plus, Meilisearch’s fast filtering lets teams apply metadata constraints, such as tenant isolation, document type, access level, time range, or domain boundaries, before retrieval begins. Only valid, relevant context may be included in the prompt.

From an engineering perspective, Meilisearch simplifies the entire pipeline. There is no need to stitch together separate keyword engines, vector databases, ranking layers, and filtering services. Meilisearch integrates retrieval, ranking, hybrid search, and filtering into a single system. As a result, you reduce infrastructure complexity and eliminate glue code.

It’s important to emphasize that Meilisearch does not replace the LLM. It controls the information surface on which the LLM operates. That separation of responsibility is what makes RAG systems reliable and production-ready.

## What to do next with RAG in AI

RAG is not a feature but a system design approach.

The next step for RAG in AI isn’t to add more generative models like GPT, but to strengthen the retrieval quality, grounding logic, and architecture.

User trust is easily earned by teams that treat their RAG as infrastructure instead of experimentation.

How to build and optimize RAG in AI for reliable answers

# How Samtec modernizes product search with Meilisearch

[Samtec](https://www.samtec.com/) is a global leader in electronic interconnect solutions - designing, manufacturing, and selling a vast range of high-performance connectors and cables. Engineers and procurement specialists worldwide rely on Samtec to browse and purchase highly specific technical components from an extensive product catalog. With millions of monthly searches and an extremely complex catalog, search is one of the most critical experiences on the site.

When their previous solution no longer met expectations both in performance and cost, Samtec made the decision to modernize their search stack with Meilisearch.

## The challenge: search that couldn’t keep up

For years, Samtec relied on its previous search vendor to power product discovery across multiple regions. While it worked, it came with growing limitations.

- **Search relevancy wasn’t good enough** for engineers looking for highly specific components.
- **Normalizing data** across various product types was hard due to inconsistencies in product descriptions.
- **Latency issues** made the experience feel slower than expected.
- **Limited control over ranking and tuning** made it hard to prioritize results intelligently.
- **High costs** didn’t match the value delivered.
- Managing search across **multiple regional instances** added complexity.

At the same time, search behavior was evolving. Users no longer wanted to navigate through menus - they expected to type what they needed and find it instantly.

As Cody Wheeler, Software Development Manager at Samtec, explains:

> *“Every analytic we have shows the search bar is the hottest part of the site. People don’t want to navigate - they want to search and find answers quickly.”*
> 

With trillions of possible part combinations and complex attributes like *“plating type”* (e.g., whether the product is plated with gold or silver), “*pitch*” (the distance between pins), and “*stack height”* (how tall the pins are), Samtec needed a search experience that was both fast and precise. Engineers often search by technical specifications, not the product names, so the search system must support parametric filtering that lets users quickly narrow results based on these detailed criteria.

## The solution: a focused, developer-first search engine

Derek Winfield, Samtec.com’s Solution Architect, evaluated several search providers before landing on Meilisearch. The decision came down to a few key criteria: ease of use, clear documentation, cost efficiency, and a clean UI alongside robust API access.

Meilisearch stood out as a dedicated solution built specifically for search:

- **Developer-first experience:** Clear documentation and intuitive tooling meant the team could build independently, without needing constant support.
- **Instant performance:** Search results returned almost instantly, a dramatic improvement over the previous solution’s latency. Indexes were also fast to build and update.
- **Built-in relevancy:** Typo tolerance, customizable ranking rules, and granular control over search behavior out of the box.
- **Direct, responsive support:** A dedicated Slack channel gave the team instant access to the Meilisearch engineering team—a stark contrast to the ticket-based support of other vendors.

## Implementation: simple, fast, and low-friction

The team started with open source to validate the approach, then moved to Meilisearch Cloud in production. The rollout was phased:

- **Phase 1:** Content search (web pages, PDFs, videos)
- **Phase 2:** Product search, including deeper filtering and faceting capabilities

From proof of concept to production, the migration took a few weeks of work alongside other projects.

>  *“It was really simple to use. We didn’t get stuck. We were able to do it all on our own.”* - said Cody Wheeler.
> 

Performance improvements were immediate. Search results now return almost instantly, even across global instances. The team at Samtec also appreciated the direct Slack integration for support, highlighting it as a significant advantage during the implementation process:

> *"We were given a direct Slack integration to ask questions - you don’t see that from a lot of companies. That direct access means the world to people trying to integrate a product quickly.”*
> 

## The results: performance without disruption

For Cody, the biggest success indicator was simple: users didn’t even notice. In Cody’s words:

>  *"When you roll out something new and no one complains-that’s usually a good sign. People didn’t even notice we made a major switch of the backend. That to me was a large success.”*

Key outcomes since the migration:

- **1.5 million monthly searches & over 1 million indexed documents** processed across four regional instances with near-instant response times.
- **Improved relevancy and speed:** developers described results as “very fast” and “very accurate,” with indexes that are “extremely fast to update.”
- **Better cost efficiency** compared to the previous solution, with plans to consolidate additional vendor costs as product search comes online.
- **Self-sufficient team:** documentation and ease of use meant the team rarely needed to ask for outside help during the build.

## Conclusion: a search foundation built for scale

Samtec’s journey with Meilisearch is just getting started. The team is preparing to roll out full product search with faceted filtering, giving engineers the Amazon-like experience of typing a keyword and narrowing down through technical attributes. They’re also exploring how internally trained LLMs could integrate with Meilisearch to power semantic search-helping users move from vague queries to precise product matches.

With refreshed [analytics](https://www.meilisearch.com/docs/learn/analytics/events_endpoint) capabilities implementation next in the pipeline, Samtec plans to gain deeper visibility into how users search-enabling the team to continuously refine relevancy and ensure every visitor finds exactly what they need.

By choosing a dedicated search partner that prioritizes speed, simplicity, and direct support, Samtec has built a foundation that serves today’s 1 million indexed documents, and is ready for what comes next.

Meilisearch latest news and company updates