What is latent semantic indexing (LSI) and how does it work?

You know what's amazing about search? It's at the heart of how people interact with your application. But let's be honest: building and managing search infrastructure can feel like you're spending more time wrestling with configuration than creating value for your users. That's why we're excited to share something that changes this completely: the Meilisearch MCP server.

## Why MCP? Building bridges between search and AI

Before we dive in, let's talk about what MCP (Model Context Protocol) is and why we're excited about it. [MCP](https://www.anthropic.com/news/model-context-protocol) is an emerging open protocol that standardizes how applications provide context to Large Language Models (LLMs). Think of it as a universal translator between your tools and AI models.

Even though MCP isn't yet fully standardized, we believe it represents the future of how developers will interact with their tools. Here's why:
- It creates a consistent way for AI models to interact with different tools and services
- It puts security and user control at the forefront
- It enables natural language interactions while maintaining precise control
- It's being built in the open, with input from the developer community

## From complex commands to natural conversations

Remember the last time you needed to update your search settings? It probably involved looking up API documentation, crafting the perfect JSON payload, and triple-checking your syntax. We've all been there. But what if you could just tell your search engine what you want, in plain English?

That's exactly what we've built. By implementing MCP, we've enabled natural conversations with Meilisearch through AI assistants like Claude. Here's what this means in practice using [our movies database](https://where2watch.meilisearch.com/) as an example:

### Before:
```bash
curl \
  -X POST 'http://localhost:7700/indexes/movies/settings' \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer MASTER_KEY' \
  --data-binary '{
    "rankingRules": [
      "words",
      "typo",
      "proximity",
      "attribute",
      "sort",
      "exactness"
    ]
  }'
```

### After:
> "Update the ranking rules for the movies index to prioritize word matches and handle typos."

That's it. When paired with advanced LLMs like Claude Sonnet, the AI understands your intent and handles all the technical details behind the scenes. No more context-switching between documentation tabs.

## What can you do with it today?

Thanks to the capabilities of modern LLMs, our MCP integration already enables natural language control over your Meilisearch instance. Here's what you can do right now:

1. **Index management made simple**
```"Create a new index for our product catalog and optimize it for e-commerce search"```
The AI will guide you through setting up your index with appropriate configurations for e-commerce.

2. **Search optimization**
```"Our users are having trouble finding products when they make typos. How can we fix this?"```
Get instant guidance on configuring typo tolerance and seeing real-time results.

3. **Performance insights**
```"How is our search performance looking? Any areas we should optimize?"```
Understand your search analytics and get actionable recommendations.

## How does it work?

Our current MCP implementation exposes Meilisearch's capabilities through a set of carefully designed tools that AI assistants can use to manage your search infrastructure. When paired with advanced LLMs like [Claude Sonnet](https://www.anthropic.com/news/claude-3-5-sonnet), these tools enable natural conversations about complex search operations.

The best part? It works seamlessly with Claude Desktop and other MCP-compatible clients, giving you:
- Natural language control over your search infrastructure
- Instant access to best practices and optimization tips
- Real-time insights and analytics

## Getting started in 2 minutes

1. Install [Claude Desktop](https://claude.ai/download) (it's free!)
2. Add our MCP server to your configuration:

```json
{
  "mcpServers": {
    "meilisearch": {
      "command": "uvx",
      "args": ["-n", "meilisearch-mcp"]
    }
  }
}
```
3. Start managing your search like you're chatting with a colleague!

## Building the future of search

This is just the beginning of our mission to make search infrastructure more accessible. Here's what we're working on next:

### Smart prompts
We're developing pre-built templates for common scenarios like:
- Optimizing index settings for specific use cases (e-commerce, documentation, etc.)
- Debugging search behavior and performance issues
- Understanding and improving search analytics
- Implementing industry-specific best practices

### Enhanced capabilities
- More sophisticated analytics and monitoring tools
- Expanded AI assistant integrations
- Advanced collaborative features for teams
- Deeper integration with Meilisearch Cloud

## Open source and open for contributions

True to our values, the Meilisearch MCP server is completely open source. We believe in building in public and learning from our community. Check out our implementation, contribute, or adapt it for your needs:

[GitHub repository](https://github.com/meilisearch/meilisearch-mcp)

## Join us in simplifying search

- ⭐ Star [our repository](https://github.com/meilisearch/meilisearch)
- 💬 Join our [Discord community](https://discord.meilisearch.com/) & share your use cases (or do it on social media and tag us)
- 🛠️ Contribute to development on [GitHub](https://github.com/meilisearch/meilisearch-mcp)

We're excited to see how you'll use this to build better search experiences. Try it out and let us know what you think!

How we made Meilisearch talk to AI: introducing our MCP server

Building a RAG system with Meilisearch: a comprehensive guide


Have you ever noticed how Google suggests exactly what you're looking for before you finish typing? That's predictive search in action. It revolutionizes user interaction with software applications. In this guide, we'll explore implementing predictive search in your applications.

## Introduction to predictive search

### Definition and importance of predictive search

**Predictive search is an intelligent system that anticipates user intent and suggests relevant results in real-time as users type**. Unlike traditional search methods that wait for a complete query, predictive search processes each keystroke to provide immediate, relevant suggestions.

The importance of predictive search in software engineering is significant. It bridges user intent and content discovery, reducing the time and effort users spend searching. Modern applications leverage predictive search to enhance productivity and streamline workflows.

Software engineering teams implementing predictive search can expect improved user engagement metrics. For instance, e-commerce platforms often see a 25-30% increase in conversion rates with predictive search features. Developer tools report significant reductions in time spent searching for code snippets or documentation.

### Predictive search enhancing autocomplete

Traditional autocomplete completes words based on character matching. Predictive search, however, analyzes patterns in user behavior, search history, and contextual data to offer intelligent suggestions.

The personalization aspect of predictive search sets it apart from basic autocomplete. **Predictive search creates a more tailored experience by considering factors like previous searches, user location, and trending queries**.

For example, when a developer searches for "deploy" in a cloud platform's console, the system prioritizes suggestions based on their commonly used services or recent deployments.

<Callout intent="note">
	
	For example, in an e-commerce platform, **autocomplete might suggest "laptop bag"** when a user types "lap", predictive search might also suggest "computer accessories," "mobile workstation," or specific laptop models based on browsing history and market trends.
	
	
</Callout>

## Core concepts and components

### Predictive models in software engineering

Predictive models form the backbone of modern search capabilities. **They utilize complex algorithms to analyze user input patterns and generate relevant suggestions**.

Businesses can leverage large datasets, with the specific type of data varying depending on the industry and business objectives. It is possible to use various data sources like browsing history, user interactions, and market trends to improve predictive search strategies.

![Examples Of Data Predictive Search](https://unable-actionable-car.media.strapiapp.com/predictive_search_b4778b9493.avif)

These models process queries through multiple layers of analysis, including string matching, semantic understanding, and historical pattern recognition.

Predictive models often rely on prefix trees (or radix trees) for efficient string matching and retrieval. This structure enables rapid processing of partial queries, allowing the system to generate suggestions with minimal latency. The models continuously learn from user interactions, refining their prediction accuracy over time.

### Search engines and search queries

Search engines in predictive systems employ sophisticated indexing mechanisms to aid rapid query processing. Key components include query processors, suggestion generators, and ranking algorithms.

These elements work together to produce and prioritize results based on various relevance factors.

![Layers Of Logic Search Engine](https://unable-actionable-car.media.strapiapp.com/predictive_search_2c856b34f6.avif)

Modern search queries go through steps including tokenization, spell checking, and context analysis. **This ensures that even imperfect queries can yield useful suggestions**. Balancing immediate results and accuracy often involves using caching mechanisms to optimize frequently requested queries.

### Machine learning and AI power

Artificial intelligence significantly enhances predictive search through advanced natural language processing and machine learning algorithms.

These technologies help systems understand user intent beyond literal string matching, incorporating semantic understanding and contextual relevance.

AI improves search accuracy by:

- Capturing user intent with natural language understanding
- Adapting ranking algorithms to changing user behavior patterns
- Identifying relationships between different terms through semantic analysis
- Tailoring suggestions based on user profiles and historical interactions

Combining machine learning with traditional search algorithms creates a hybrid approach that delivers both speed and accuracy.

## Capabilities required for predictive search

A robust predictive search implementation requires several key capabilities to deliver a seamless user experience. Fast indexing and efficient query processing are foundational, but there's more to consider.

### Meilisearch's approach to enabling predictive search

Meilisearch combines traditional full-text search with AI-powered capabilities. Its architecture enables fast, relevant results and maintains simplicity in implementation.

**The integration process focuses on three key areas**: data indexing, search configuration, and result optimization. Here's an example of configuring search settings:

```typescript
// config/search.ts
const searchConfig = {
  settings: {
    rankingRules: [
      'words',
      'typo',
      'proximity',
      'attribute',
      'sort',
      'exactness'
    ],
    searchableAttributes: [
      'title',
      'description',
      'category'
    ],
    displayedAttributes: [
      'title',
      'description',
      'category',
      'thumbnail'
    ],
    // Enable AI-powered search with OpenAI
    embedder: {
      type: 'openai',
      documentTemplate: "Product: {{title}}nDescription: {{description}}"
    }
  }
};

await searchClient.index('products').updateSettings(searchConfig);
```

**This configuration enables both traditional search capabilities and AI-powered features**, creating a hybrid search experience that delivers both speed and relevance. The system handles typos, synonyms, and contextual understanding while maintaining sub-50ms response times.

<Callout intent="tip">
	
	Ready to supercharge your search experience? [Explore Meilisearch Cloud](https://meilisearch.com/cloud) and deliver lightning-fast search results that will keep your users engaged and boost your conversion rates.
	
	
</Callout>

## Applications and use cases

Modern software applications leverage predictive search in numerous ways, from e-commerce platforms to content management systems. Understanding these practical applications helps developers implement the right search strategies for their specific use cases.

### App search

**App search is a common implementation of predictive search in software applications**. Mobile apps benefit particularly due to limited screen space and typing constraints.

For instance, food delivery apps use predictive search to help users find restaurants or dishes quickly, considering factors like previous orders and current location.

### Site and media search

**Site and media search capabilities have evolved to handle complex content structures**. Content management systems and documentation platforms use predictive search to help users navigate large information hierarchies.

For example, technical documentation sites can suggest relevant API endpoints or code examples as developers type their queries.

<Callout intent="note">
	
	Check out how Meilisearch powers the discovery of 300,000+ AI models on [Hugging Face](https://www.meilisearch.com/blog/hugging-face-democratizes-ai), demonstrating the power of predictive search in organizing and accessing vast AI resources.
	
	
</Callout>

### Ecommerce search

E-commerce platforms showcase the potential of predictive search through sophisticated product discovery. Online stores suggest not only product names but also categories, brands, and specific features. **This capability extends to handling complex queries like "red summer dress under $50" accurately**.

<Callout intent="note">
	
	For another compelling example of predictive search in action, explore [Meilisearch's case study with Bookshop.org](https://www.meilisearch.com/blog/bookshop-increases-search-based-purchases), where they achieved a remarkable 43% increase in search-based purchases by enhancing their book discovery platform.
	
	
</Callout>

### Enterprise search

[Enterprise search](meilisearch.com/blog/enterprise-search) presents unique challenges because it handles diverse data sources while maintaining security constraints. Predictive search in enterprise environments must consider user permissions, department-specific terminology, and internal document hierarchies.

For instance, a company's internal tools might suggest different results for the same query based on the user's role and access level.

## Techniques and methodologies

### Genetic programming and test models

Genetic programming enhances predictive search capabilities by **evolving and refining search algorithms**. These techniques mimic natural selection to optimize search patterns and improve relevancy.

The system dynamically generates search algorithm variations. It then evaluates these algorithms by analyzing user interactions. Based on performance metrics, the system selects the most effective algorithm combinations to enhance future search experiences.

Test models developed through genetic programming adapt to changing search patterns. For instance, e-commerce search systems can automatically adjust prediction weights based on seasonal trends, user behavior shifts, and new product introductions. **The evolutionary nature of these algorithms ensures relevant search suggestions** as user preferences and patterns evolve.

### Base software and search base

Establishing a robust search base requires careful consideration of data structure and indexing strategies.

<Callout intent="note">
	
	For deeper insights into optimizing search indexing, check our comprehensive guide on [best practices for faster indexing](https://meilisearch.com/blog/best-practices-for-faster-indexing).
	
	
</Callout>

Typically, it consists of an inverted index system enabling quick lookup of terms while maintaining relationships between search elements. **This base structure must support both exact matches and fuzzy searching capabilities**.

Integration of predictive search within software frameworks demands:

- Efficient data storage and retrieval mechanisms to handle large volumes of search queries
- Flexible indexing strategies accommodating both structured and unstructured data
- Scalable architecture maintaining performance under increasing load
- Real-time update capabilities to reflect changes in underlying data

The search base must also handle edge cases and special scenarios. For example, with multilingual search capabilities, the base system must account for different character sets, writing systems, and language-specific patterns. **This ensures reliable and effective predictive search functionality across diverse use cases** and user requirements.

## Capabilities

Low latency is crucial for predictive search systems because users expect instant feedback while typing. Modern search implementations must maintain response times under 100ms to feel instantaneous. This speed requirement becomes even more critical when handling complex queries across large datasets.

Several optimization techniques help achieve these performance goals. **Intelligent caching strategies store frequently accessed results and partial queries**, significantly reducing database load and response times for common searches. Here's how response times typically break down:

| Operation | Target Latency | Optimization Method |
| --- | --- | --- |
| Query Processing | &lt; 20ms | In-memory processing |
| Index Lookup | &lt; 30ms | Cached indices |
| Result Ranking | &lt; 30ms | Pre-computed scores |
| Network Transfer | &lt; 20ms | Edge caching |

Ensuring reliability involves implementing robust error handling and failover mechanisms. **Search systems must gracefully handle scenarios like network issues, invalid queries, or backend service disruptions**. Implementing circuit breakers and fallback options helps maintain service availability.

Performance monitoring is vital for maintaining optimal search functionality. Modern search implementations use real-time metrics tracking to identify bottlenecks and optimization opportunities.

Key performance benchmarks for predictive search systems include:

- **Query response time:** 95th percentile under 200ms
- **Suggestion relevancy:** Minimum 80% accuracy rate for top 5 suggestions
- **System throughput:** Ability to handle 1000+ queries per second per node
- **Cache hit ratio:** At least 85% cache effectiveness

<Callout intent="tip">
	
	Meilisearch Cloud includes [built-in monitoring tools](https://www.meilisearch.com/docs/learn/analytics/monitoring) that enable real-time tracking of search performance metrics.
	
	
</Callout>

The architecture should scale horizontally to handle increasing load while maintaining performance. This typically involves:

1. Distributed index sharding for better resource utilization
2. Load balancing across multiple search nodes
3. Asynchronous index updates to prevent blocking operations
4. Smart query routing based on user location and server load

Reliability also extends to data consistency and accuracy. Search systems must handle index updates and real-time data changes without compromising search quality or availability. **This often requires sophisticated versioning and synchronization mechanisms** to ensure users always see the most relevant and up-to-date results.

## Future directions and innovations

As predictive search technology continues to evolve, several emerging trends and innovations are reshaping the future of search systems.

### Integration with advanced technologies

The future of predictive search is being shaped by **advancements in artificial intelligence and machine learning technologies**. Large language models (LLMs) are transforming how search systems understand and process user queries. These models can interpret natural language with high accuracy, enabling more conversational and context-aware search experiences.

Integration with advanced technologies is already showing promising results. Some systems are experimenting with multimodal search capabilities, processing text, images, and voice inputs simultaneously. This enables more intuitive search interfaces that match how humans naturally communicate.

### Trends in predictive search

LLMs are revolutionizing predictive search capabilities by fundamentally transforming how we interact with search engines. These sophisticated systems can now comprehend complex queries and generate natural, contextually appropriate suggestions that feel remarkably human-like.

Modern search systems now interpret both technical and informal language, moving beyond pattern matching to understand the true meaning behind user queries.

This advancement has enabled several crucial improvements in search functionality:

- More precise interpretation of user queries
- Sophisticated handling of ambiguous searches
- Enhanced recognition of specialized terminology
- Intelligent, context-aware result ranking

**These innovations are already being implemented in various forms**. Some search engines use hybrid approaches that combine traditional search algorithms with [AI-powered semantic understanding](meilisearch.com/blog/ai-powered-search). This allows them to maintain the speed and reliability of conventional search while leveraging advanced AI capabilities.

## Beyond horizons: the transformative potential of predictive search

Predictive search has evolved from a convenient feature to an essential component of modern software applications.

Successful implementation requires careful consideration of architecture, performance optimization, and user experience design.

The integration of advanced AI technologies and mobile-first approaches will continue to push the boundaries of what's possible in predictive search. This makes it an exciting area for continued innovation and development.

<Callout intent="tip">
	
	Unlock advanced search potential with customizable relevancy, typo tolerance, and more. [Enhance Your Search Strategy](https://meilisearch.com/features) with Meilisearch Cloud's powerful search capabilities.
	
	
</Callout>

Software engineering predictive search: a complete guide

In today's interconnected world, many applications need to handle content in multiple languages. Meilisearch offers powerful capabilities for multilingual search, combining intelligent language handling with specialized models for optimal relevance. This guide will walk you through setting up multilingual search with Meilisearch, covering both full-text and semantic search approaches.

Why multilingual search matters
===============================

Before we dive into the technical details, let's consider why multilingual search is crucial:

1.  Global reach: Expand your audience by catering to users in their native languages.
2.  Improved user experience: Provide relevant results regardless of the language used in queries or content.
3.  Increased engagement: Users are more likely to interact with content they can easily find and understand.

Now, let's explore how Meilisearch makes multilingual search a breeze.

Full-text search: intelligent language handling
-----------------------------------------------

At its core, Meilisearch excels at handling diverse languages out of the box. Our [advanced tokenizer](https://www.meilisearch.com/docs/learn/indexing/tokenization?utm_campaign=vector-search&utm_source=blog&utm_medium=multilingual_search) is designed to work with:

*   Languages that use whitespace to separate words (like English, Spanish, and French)
*   Languages with different writing systems (such as Chinese, Japanese, and Thai)

### Automatic language detection

When you add documents to your Meilisearch index, the engine automatically detects the language of each field. This means you don't need to specify languages manually in most cases - Meilisearch will handle it for you.

For example, if your documents contain a mix of English and Spanish content:

```json
[
  {
    "id": 1,
    "title": "The quick brown fox",
    "description": "El zorro marrón rápido salta sobre el perro perezoso"
  },
  {
    "id": 2,
    "title": "La luna y las estrellas",
    "description": "A beautiful night sky filled with twinkling stars"
  }
]
```

Meilisearch will automatically detect and properly handle the different languages in each field, ensuring accurate search results across your multilingual content.

### Specifying languages for specific fields

While automatic detection works well in most cases, you might want more control over language handling for certain fields. Meilisearch allows you to [specify languages](https://www.meilisearch.com/docs/reference/api/settings#localized-attributes?utm_campaign=vector-search&utm_source=blog&utm_medium=multilingual_search) for particular attributes using the `localizedAttributes` setting.

Here's how you can set it up:

```json
{
  "localizedAttributes": [
    {
      "locales": ["eng"],
      "attributePatterns": ["title_en", "*_english"]
    },
    {
      "locales": ["spa"],
      "attributePatterns": ["description_es", "*_spanish"]
    }
  ]
}
```

This configuration tells Meilisearch to treat fields ending with `_en` or `_english` as English, and fields ending with `_es` or `_spanish` as Spanish. This approach is particularly useful when you have a consistent naming convention for your multilingual fields or when you want to ensure specific treatment for certain attributes.

Semantic search: Enhancing multilingual relevance
=================================================

While full-text search works well for many multilingual scenarios, semantic search can provide even better results, especially for complex queries or when dealing with nuanced meanings across languages.

Meilisearch supports various embedding models for semantic search, offering powerful auto-embedding capabilities. This means you can easily set up semantic search without having to manage the embedding process yourself. Meilisearch handles the generation and storage of embeddings automatically, seamlessly integrating them into your search experience.

For multilingual applications, we recommend using specialized multilingual models rather than relying on general-purpose models. These models are designed to understand and represent meaning across multiple languages, leading to more relevant search results in multilingual contexts.

One excellent option for multilingual semantic search is the [Cohere Multilingual embedding model](https://docs.cohere.com/docs/multilingual-language-models). It offers a great balance of speed and relevance across a wide range of languages. Cohere's model is particularly adept at understanding context and nuance in different languages, making it an ideal choice for diverse, multilingual content. Follow [the guide](https://www.meilisearch.com/docs/guides/ai/cohere?utm_campaign=vector-search&utm_source=blog&utm_medium=multilingual_search) if you want to do it now.

To explore different models and their performance, you can use the [Meilisearch playground](https://playground.meilisearch.com). This interactive tool allows you to test various embedding models and see how they perform with your specific multilingual queries and content.

Conclusion
----------

We encourage you to experiment with different settings and models to find the optimal configuration for your specific use case. And remember, the Meilisearch community is always here to help - join us on [Discord](https://discord.meilisearch.com/) or check out our documentation for more detailed information and support.

Ready to get started with multilingual search? Try Meilisearch today [with a 14 day trial](https://cloud.meilisearch.com/?utm_campaign=vector-search&utm_source=blog&utm_medium=multilingual_search)!

Searching across multiple languages

How to add AI-powered search to a React app

_This is part 4 of a series of blog posts_ [_originally published on Clément Renault's blog_](https://blog.kerollmops.com/how-meilisearch-updates-a-millions-vector-embeddings-database-in-under-a-minute)_. Begin the journey with_ [_part 1_](/blog/spotify-inspired-hybrid-search-and-rust/)_,_ [_part 2_](/blog/refining-ann-performance-with-rust/)_ and_ [_part 3_](/blog/arroy-filtered-disk-ann/)_ This blog post considers that you’ve already read parts 1 and 2._

In this blog post, we’ll talk about how we implemented incremental indexing in [Arroy](https://github.com/meilisearch/arroy). By incremental, we mean that when inserting or deleting items in a tree, we can update only the required parts instead of rebuilding everything.

At Meilisearch, that’s essential because our users usually send updates regularly, and their content constantly changes. Rebuilding everything from scratch is very costly and won’t give a good experience; that’s why we already spent a lot of time on our latest release to make all the [data structures support incremental indexing](https://github.com/meilisearch/meiliSearch/releases/tag/v1.6.0) and arroy being one of them, we _HAD TO_ implement it as well!

The theory
----------

The following schema shows a high-level view of what should happen when adding and deleting documents into an existing tree.

As a reminder, arroy is a vector store based on [LMDB](http://www.lmdb.tech/doc/), an embedded transactional memory-mapped key-value store. It stores your vectors in trees composed of three kinds of nodes:

*   The split nodes - Store information on which item IDs are spread out between its left and right.
*   The descendants' nodes - Store multiple item IDs.
*   The items - A single vector.

For the following examples, we’ll consider the descendants node cannot contain **more** than two items and that the closer the IDs are, the closer the items are in space. For example, the item `1` is close to the item `2` but is farther from the item `10`.

![First step before inserting elements in the tree](https://unable-actionable-car.media.strapiapp.com/tamo_1_9ea29125b9.png)

Here, we’re trying to insert items `2` and `7` and delete items `3` and `11`.

Following what we do on the search, we split the items to insert between the left and right split nodes. But the deleted items are no longer present in the database! We don't have their associated vector, and therefore we'll need to iterate over all the leaves to find and delete them.

Fortunately, since we use roaring bitmaps, this list doesn't consume much memory, and since we never update it, we can share it with every thread without ever copying it 🎉

![Second step inserting element in the tree](https://unable-actionable-car.media.strapiapp.com/tamo_2_e4cd099c77.png)

Notice how the items to insert were balanced between the left and right split nodes. In the next step, we’ll follow the same process and split both lists of items to insert on the split node.

![Third step inserting into elements in the tree](https://unable-actionable-car.media.strapiapp.com/tamo_3_ab539d2465.png)

It’s on this step that all the good stuff happens. From the left to the right:

*   The first descendant node gets its item `3` removed, and item `2` added.
*   The second descendant node gets too big and must be replaced by a new split node.
*   The item `8` becomes a descendant containing both the items `7` and `8`.
*   After deleting item `10` in the last descendant node, we replace it with a direct link to an item to reduce the number of nodes in the tree (and improve search time by reducing the number of lookups).

![Last step inserting elements in the tree](https://unable-actionable-car.media.strapiapp.com/tamo_4_b07ebb4b8f.png)

Now that you’ve seen all the steps of our incremental indexing process, we’re still going to take a few notes on what happened:

*   In the first three steps, we had to read all of the tree nodes, which, by default, is equal to the total number of items in the database.
*   In the last step, we had to write four new tree nodes. Previously we would have rewritten the whole tree. The number of writes, which is the slowest operation in databases, is reduced to the bare minimum.
*   The process works on a single tree which means we can still multi-thread the computation of every tree.
*   The IDs aren’t sequential anymore, which doesn't play well with the ID generation strategy described in [Writing the tree in parallel](https://blog.kerollmops.com/multithreading-and-memory-mapping-refining-ann-performance-with-arroy).

How did we fix the ID generation?
---------------------------------

At the start of the indexing process, we need to gather all the existing tree IDs.  
The information we need to generate new nodes are:

*   The total number of tree nodes used to know when to stop building trees.
*   The « holes » in the used IDs that we want to re-use. This fragmentation is produced when we edit the trees.
*   The highest ID existing in the trees to generate the next new IDs.

![Selecting the next ID in the Roaring Bitmap](https://unable-actionable-car.media.strapiapp.com/tamo_5_1f7bb6a273.png)

The « simple » idea is to have a counter for the total number of tree nodes. A counter that we can increment to generate new fresh IDs. The most challenging part is to pick an available ID shared between threads and without big mutexes.

It took me quite some time to find a solution for this last point; I implemented a whole complex solution (150loc) that I ended up [rewriting entirely from scratch](https://github.com/meilisearch/arroy/pull/56/commits/0bf68e21cfd97ff00c7ef5a0ec116ba34c43c6a5) in one hour for an easy and more efficient solution. We're going to see all the ideas we went through before finding our final solution.

### Sharing an iterator over the available IDs

The first and most straightforward idea that comes to mind is to create and share an iterator over all the available IDs.

```rust
let mut available_ids = available_ids.into_iter();

roots.into_par_iter().for_each(|root| {
    // stuff
    let next_id = available_ids.next();
    // stuff
});
```

If you’re familiar with Rust, you’ll notice immediately that the `for_each` _cannot_ capture a mutable reference over the `available_ids` because we use the `into_par_iter` method from Rayon that execute the next methods on the iterator in a multi-threaded way. Which means we can't call `.next()`.

By using a mutex (short for [mutual exclusion](https://en.wikipedia.org/wiki/Mutual_exclusion)), we can make it compile:

```rust
let mut available_ids = Mutex::new(available_ids.into_iter());

roots.into_par_iter().for_each(|root| {
    // stuff
    let next_id = available_ids.lock().unwrap().next();
    // stuff
});
```

That is safe and will yield the right results, but won’t scale well as all the threads have to wait for each other on the lock.

### Let each thread own its list of IDs

When you need to remove a lock, a common solution is to split the work beforehand so each thread can work to its full capacity without ever having to know what’s happening on the other threads. That’s the solution [I implemented initially](https://github.com/meilisearch/arroy/pull/56/files/dc782ea61ef84f0ad8ad20e092476494d6eacc0c). The idea is to explode the roaring bitmap into as many smaller roaring bitmaps as there are trees to update.

The following function can split a roaring bitmap into `n` sub-bitmaps of equal size:

```rust
pub fn split_ids_in(available: RoaringBitmap, n: usize) -> Vec<RoaringBitmap> {
    // Define the number of elements per sub-bitmap
    let chunk_size = available.len() / n as u64;
    let mut ret = Vec::new();

    let mut iter = available.into_iter();
    for _ in 0..(n - 1) {
        // Create a roaring bitmap of `chunk_size` elements from the iterator without consuming it
        let available: RoaringBitmap = iter.by_ref().take(chunk_size as usize).collect();
        ret.push(available);
    }
    // the last element is going to contain everything remaining
    ret.extend(iter);

    ret
}
```

With this function available, it's then trivial to use one bitmap per tree to update with the [`zip`](https://docs.rs/rayon/latest/rayon/iter/trait.IndexedParallelIterator.html#method.zip) method on the parallel iterators of Rayon:

```rust
let available_ids = split_ids_in(available_ids, roots.len());

roots.into_par_iter().zip(available_ids).for_each(|(root, available_ids)| {
    let mut available_ids = available_ids.into_iter();
    // stuff
    let next_id = available_ids.next();
    // Here, we can use the available_ids without any locks or inter-thread synchronization
    // stuff
});
```

This works well while we're updating the already existing root trees. But later on, we may want to create new trees from scratch, and we can't know in advance how many new trees we’ll need to create. That's an issue because, without this information, we don't know how many sub-bitmaps we need to create.

At this point, I couldn’t see an easy fix to this problem, but I made the assumption that we rarely create a lot of new trees and that all new trees would use a lot of IDs. This means, giving all the IDs to the **first** new tree would _probably work out_ (it’s hard to be 100% sure without monitoring it in prod).

### The final solution

The previous solution was complex and did not even work perfectly. While I was scrolling through the documentation of the Roaring Bitmap, I saw [the `select` method](https://docs.rs/roaring/latest/roaring/bitmap/struct.RoaringBitmap.html#method.select) and immediately understood how I could make the initial approach works.

This method lets you get the values in your bitmap by index in an efficient way.  
For example, with the following values in a bitmap: `13, 14, 15, 98, 99, 100`:

*   `select(0)` returns `13`.
*   `select(3)` returns `98`.
*   `select(5)` returns `100`.
*   `select(6)` returns `None`.

With that in mind, if we share the bitmap with all threads in **read-only** and share the index in the bitmap as an atomic number, we can have multiple threads get available IDs at the same time with lock-free synchronization.

Once the `select` method returns `None`, it means we can stop picking IDs from the bitmap and instead simply generate new IDs from scratch with the old method.  
Even if it's super fast, calling the `select` method still takes some time, so once a thread gets a `None` returned from the method, we're going to update an atomic boolean telling us if there are still values to look at in the bitmap. If not, we can skip calling `select` and directly generate a new ID.

With that in mind, the cool `ConcurrentNodeIds` structure we showed you before in part 2 got a little bit more complex but it still let us generate IDs fairly without any locks!

```rust
/// A concurrent ID generator that will never return the same ID twice.
pub struct ConcurrentNodeIds {
    /// The current tree node ID we should use if no other IDs are available.
    current: AtomicU32,
    /// The total number of tree node IDs used.
    used: AtomicU64,

    /// A list of IDs to exhaust before picking IDs from `current`.
    available: RoaringBitmap,
    /// The current Nth ID to select in the bitmap.
    select_in_bitmap: AtomicU32,
    /// Tells if you should look in the roaring bitmap or if all the IDs are already exhausted.
    look_into_bitmap: AtomicBool,
}

impl ConcurrentNodeIds {
    /// Creates an ID generator returning unique IDs, avoiding the specified used IDs.
    pub fn new(used: RoaringBitmap) -> ConcurrentNodeIds {
        let last_id = used.max().map_or(0, |id| id + 1);
        let used_ids = used.len();
        let available = RoaringBitmap::from_sorted_iter(0..last_id).unwrap() - used;

        ConcurrentNodeIds {
            current: AtomicU32::new(last_id),
            used: AtomicU64::new(used_ids),
            select_in_bitmap: AtomicU32::new(0),
            look_into_bitmap: AtomicBool::new(!available.is_empty()),
            available,
        }
    }

    /// Returns a new unique ID and increases the count of IDs used.
    pub fn next(&self) -> Result<u32> {
        if self.look_into_bitmap.load(Ordering::Relaxed) {
            let current = self.select_in_bitmap.fetch_add(1, Ordering::Relaxed);
            match self.available.select(current) {
                Some(id) => Ok(id),
                None => {
                    self.look_into_bitmap.store(false, Ordering::Relaxed);
                    Ok(self.current.fetch_add(1, Ordering::Relaxed))
                }
            }
        } else {
            Ok(self.current.fetch_add(1, Ordering::Relaxed))
        }
    }
}
```

Performances in practice
------------------------

I haven’t run a lot of benchmarks yet (I would like to, though, and may update this post later), but on my benchmarks with a few hundred thousand items, here are the results I get:

![Performance improvement showed on a graph](https://unable-actionable-car.media.strapiapp.com/tamo_6_8c737d7e95.png)

*   On average, we get more than a 10x speedup after three batches of items indexed.
*   The little bumps we observe afterward that make us go from ~700ms to ~3s are due to the creation of new trees as the number of items increases.
*   I expect this benchmark to represent a worst-case scenario:
    *   All my item IDs are randomly generated, which means there are very few duplicates/updates but always new insertions and more writing.
    *   My vectors are also randomly generated with a uniform distribution, which means there shouldn’t be clusters as we observe in a real-life dataset.

This feature is now being used on production data with millions of documents, and we've seen numbers in the range of 9000 items added on a 2M items database in under a minute.

* * *

[Meilisearch](https://meilisearch.com/?ref=blog.meilisearch.com) is an open-source search engine enabling developers to build state-of-the-art experiences while enjoying simple, intuitive DX. 

For more things Meilisearch, you can join the community on [Discord](https://discord.gg/meilisearch/?ref=blog.meilisearch.com) or subscribe to the [newsletter](https://dub.sh/meili-newsletter). You can learn more about the product by checking out the [roadmap](https://roadmap.meilisearch.com/?ref=blog.meilisearch.com) and participating in [product discussions](https://github.com/orgs/meilisearch/discussions?ref=blog.meilisearch.com).

How Meilisearch updates a database with millions of vector embeddings in under a minute

On March 7th, we hosted a live showcase session introducing hybrid search and other Meilisearch product updates. Watch a recording and see the hybrid search demo [here](https://www.youtube.com/watch?v=Lgv8iTdWDQc&t=881s).

* * *

AI's influence on search
------------------------

The rapid advancement of AI technology is undeniably reshaping human-machine interactions. Historically, communication with machines necessitated intermediaries like programming languages and structured scripts.

Now, with the arrival of GPTs and AI-assisted bots, machine-learning models are approaching a deeper comprehension of natural human language. This evolution brings humans and machines closer, changing the way we seek and find information.

From keyword-based searches to discovery-driven inquiries, people’s approach to search evolves: we are moving from searching for known items to exploring unknown territories, searching and finding the concepts instead of focusing solely on words.

Take a streaming media company: the searches across their platform might include the inquiry for a specific movie title, such as "Back to the Future," as well as conceptually-driven searches like "Give me recommendations for a feel-good sci-fi movie from the 80s".

Consider everyday scenarios: spotting an item on the street and using an image to search for it, or conducting voice searches instead of typing in words on keyboards. These are manifestations of our changing search behaviors.

The pinnacle: hybrid search
---------------------------

While traditional search methods excel in keyword-based queries, they falter with complex, nuanced requests. This is where Meilisearch’s new hybrid search approach comes into play, enriching the efficiency and precision of our robust full-text search engine with a depth of semantic understanding, catering to a wide array of use cases driven by customer-generated data.

Integrating models from AI giants like [OpenAI](https://www.meilisearch.com/docs/learn/experimental/vector_search#generate-auto-embeddings-with-openai) (open-source embedders generation feature is set for the next release cycle), Meilisearch enables users to create and fine-tune vector embeddings, tailoring their search engine to specific business needs. Moreover, evolving AI models learn from user interactions, continuously improving search precision.

What‘s included in the new release package?
-------------------------------------------

The AI-enhanced hybrid search from Meilisearch leverages our existing robust search infrastructure, including advanced sorting, filtering, faceting, and geo-sorting capabilities, among others.

New release features
--------------------

*   **Enhanced search relevancy**: The AI-enhanced layers bring a deeper semantic understanding, significantly improving result quality.
*   **AI embedders autogeneration:** Choose between using a third-party solution like OpenAI or your own locally generated embeddings. An open-source embedders generation feature is set for the next release cycle.

![](https://unable-actionable-car.media.strapiapp.com/Group_143725866_7f8309846d.png)

Add a new AI embedder in Meilisearch Cloud.

*   **Improved prompting:** Enhanced to incorporate key information templates, aiding the system in prompt ingestion and relevant metadata interpretation.
*   **Semantic ratio control:** Customize your search to span anywhere from purely keyword-based to entirely semantic.  
    

![](https://unable-actionable-car.media.strapiapp.com/Group_143725865_1_f3910ea1ce.png)

Choose where you want your results to fall on the spectrum from pure keyword-based to fully semantic search.

*   **Multimodal search capabilities:** Expanding search functionality to include image, sound, and video content.

**Democratizing search**
------------------------

At Meilisearch, our journey has always been anchored in simplicity, performance, and out-of-the-box relevancy. Our core mission remains steadfast: make it easy to find anything anywhere by providing a go-to-search solution for every team.

> “Meilisearch's hybrid search will democratize advanced search technology, focusing on simplicity and open-source customizability”  
> \- **Quentin de Quelen, Meilisearch’s CEO and Co-founder.**

As we evolve, our methods become more sophisticated, adapting to the ever-changing needs of businesses and developers alike. In a world brimming with data, effective content discovery is essential, not just a luxury. Our vision combines innovation with accessibility, aiming to make cutting-edge search technology inclusive for all.

The launch of our hybrid search product marks a significant milestone for Meilisearch and the search technology landscape. We invite you to try the new solution (now generally available) and join us in shaping the future of search.

* * *

Stay in the loop by subscribing to our [newsletter](https://dub.sh/meili-newsletter?&utm_source=blog&utm_campaign=HybridSearchRelease). To learn more about Meilisearch's future and help shape it, take a look at our [roadmap](https://roadmap.meilisearch.com/?utm_source=Blog&utm_campaign=HybridSearchRelease) and participate in our [Product Discussions](https://github.com/orgs/meilisearch/discussions?utm_source=Blog&utm_campaign=HybridSearchRelease).

For anything else, join our developer community on [Discord](https://discord.com/invite/hTD5hU2yWV?utm_source=Blog&utm_campaign=HybridSearchRelease).

Introducing hybrid search: combining full-text and semantic search for optimal balance

_This is part 3 of a series of blog posts_ [_originally published on my personal blog_](https://blog.kerollmops.com/meilisearch-expands-search-power-with-arroy-s-filtered-disk-ann)_._ _Begin the journey with_ [_part 1_](/blog/spotify-inspired-hybrid-search-and-rust/) _and_ [_part 2_](/blog/refining-ann-performance-with-rust/)_._

We can store vectors in [Arroy](https://github.com/meilisearch/arroy) and efficiently compute the search tree nodes, but we still need some features to make it usable in Meilisearch. Our full-text search engine has great support for filtering; selecting subsets of documents is an essential feature to support. For instance, one of our biggest clients requires the ability to filter through over 100 million YouTube video metadata and their associated image embeddings to effectively select videos released within specific time frames, such as a single day or week. This represents the scalability and responsiveness challenges we aim to address with our filtering system, making it a perfect use case to tailor our developments.

Meilisearch supports the following operators on the filterable attributes of your documents: `<`, `<=`, `=`, `!=` `>=`, and `>`. Internally, we extensively [use `RoaringBitmap`s](https://github.com/RoaringBitmap/roaring-rs), which are well-optimized sets of integers that support fast binary operations like unions and intersections. When the engine receives a user request with filters, it first computes the subset of documents from the filter that will be inputted into the search algorithms, the one that ranks documents by quality. This subset is represented by a `RoaringBitmap`.

How was it working before Arroy?
--------------------------------

Ranking only the subset of filtered documents has been working well since 2018, but now that we have a new data structure to search on, we need to look at how to implement it. The engine had already supported the vector store feature for months but was inefficient. We were using an in-memory HNSW and were deserializing the whole data structure in memory, searching for the nearest neighbors of the target vector, which was returning an iterator.

```Rust
fn vector_store(db: Database, subset: &RoaringBitmap, target_vec: &[f32]) -> Vec<DocumentID> {
    // This takes a lot of time and memory.
    let hnsw = db.deserialize_hnsw();

    let mut output = Vec::new();
    for (vec_id, vec, dist) in hnsw.nearest_neighbors(target_vec) {
        let doc_id = db.document_associated_to_vec(vec_id).unwrap();
        if !output.contains(&doc_id) {
          output.push(doc_id);
          if output.len() == 20 { break }
        }
    }

    output
}
```

You may wonder why we are retrieving the document ID associated with the vector ID. Meilisearch supports multiple vectors by document since the beginning of the vector store feature. This is unfortunate because we must look up every vector we are iterating. We need to maintain this lookup table. The iterator can iterate on the whole vector dataset if the subset is small enough, e.g., `document.user_id = 32`. We want the document operations to be atomic and consistent, so we have to store the HNSW on disk and avoid having to maintain synchronization between the LMDB transactions and this vector store data structure. Ho! And the library doesn't support incremental insertion. We must reconstruct the HNSW in memory from scratch every time we insert a single vector.

Integrating Arroy in Meilisearch
--------------------------------

As we worked on updating Meilisearch to include the new vector store, arroy, we tried out mob programming for the first time. This is where we all code together at the same time. It might sound like it would slow us down, but it actually made us super productive! By tackling problems together right as they came up, we got arroy fitted into Meilisearch much faster than if we'd worked alone.

Arroy is different because it doesn’t return an iterator to give back search results. Now, our search engine is smarter and can figure out the exact number of results it needs to give back, even when there are filters to consider. This teamwork improved our search tool and showed us that working together is key when facing big tech challenges.

#### Filtering while searching in Arroy

You can find a description of the arroy internal data structure in [part 1 of this series](https://blog.kerollmops.com/spotify-inspired-elevating-meilisearch-with-hybrid-search-and-rust). Here is a list of the different kinds of nodes you can find:

*   The item nodes. The original vectors that the user provides. The small points on the left.
*   The normal nodes, also called split planes. They represent hyperplanes that split subsets of item nodes in half.
*   The descendant nodes. They are the tree leaves composed of the item IDs you'll find if you follow a certain normal node path.

![split-plane-combined-schema](https://unable-actionable-car.media.strapiapp.com/spli_plane_b53ab895e8.png)

A typical search will load all the normal nodes in a binary heap associated with an infinite distance. Remember that there are a lot of randomly generated trees, and there are, therefore, a lot of entry points. Ascending distances order the binary heap; the shortest nodes found will be popped first.

In the search algorithm, we pop the nearest item from this heap. If it's a normal node, we fetch the left and right sides of the hyperplane:

*   If we find normal nodes again, we associate the distance from the hyperplane with positive and negative distances from the targeted/queried vector.
*   If we find descendant nodes, we do not push them into the queue but rather directly add them to the potential output list as they represent the nearest vectors we have found.

You probably noticed where I want to go, but this is where magic happens. We modified arroy to store the list of descendants in `RoaringBitmap`s instead of a raw list of integers. This is another improvement compared to the original Spotify library, as those lists weigh less. Doing an intersection with the filtered subset is way easier now.

However, there is always an issue: the vector IDs are not the document IDs, and Meilisearch, after executing the filters, only knows about the documents. Iterating on the lookup table, I talked about before, constructing the final bitmap with all the vector IDs corresponding to the filtered documents would not be efficient when many documents are part of this subset, e.g., `document.user_id != 32`. I do not recommend using an `O(n)` algorithm in the search functions.

#### Using multiple indexes for efficient filtering

Fortunately, we developed a fun feature that wasn't meant to be used that way in arroy: the support for multiple indexes in a single LMDB database. We originally developed the multiple indexes feature to be able only to open a single LMDB database to store different vector types. Yes! In Meilisearch v1.6, you can describe the different embedders that live in a single index. You can identify the different vectors with different numbers of dimensions and distance functions you can store in a single document. Defining multiple vectors for a single document associated with the same embedder is also possible.

The indexes are identified by an `u16`. This feature can trick the algorithm into being even more efficient than the previous HNSW solution. Using one store by embedder and vector in a document is interesting because we can now identify the vectors using the document ID. No more lookup database and vector IDs. The vector IDs are reduced to the document IDs. We can use the output of the filters to filter the arroy indexes.

The search algorithm is different on the Meilisearch side. We request the nearest neighbors in every arroy index, sort the results by document ID to be able to deduplicate them and not return the same document multiple times, sort them again by distance, and then return only the top twenty documents. It may seem complex, but we are talking about twenty documents by the number of vectors for a single document. Usually, users will have a single vector.

```Rust
fn vector_search(
    rtxn: &RoTxn,
    database: Database,
    embedder_index: u8,
    limit: usize,
    candidates: &RoaringBitmap,
    target_vector: &[f32],
) -> Vec<(DocumentId, f32)> {
    // The index represents the embedder index shifted and
    // is later combined with the arroy index. There is an arroy
    // index by vector for a single embedded in a document.
    let index = (embedder_index as u16) << 8;
    let readers: Vec<_> = (0..=u8::MAX)
        .map(|k| index | (k as u16))
        .map_while(|index| arroy::Reader::open(rtxn, index, database).unwrap())
        .collect();

    let mut results = Vec::new();
    for reader in &readers {
        let nns = reader.nns_by_vector(rtxn, target_vector, limit, None, Some(candidates)).unwrap();
        results.extend(nns_by_vector);
    }

    // Documents can have multiple vectors. We store the different vectors
    // into different arroy indexes, we must make sure we don't find the nearest neighbors
    // vectors that correspond to the same document.
    results.sort_unstable_by_key(|(doc_id, _)| doc_id);
    results.dedup_by_key(|(doc_id, _)| doc_id);

    // Sort back the documents by distance
    results.sort_unstable_by_key(|(_, distance)| distance);
    results
}
```

The design philosophy behind these improvements
-----------------------------------------------

We're trying to make Meilisearch flexible enough for everyone's needs. Whether your documents are powered by a single embedding, like the kind OpenAI's nifty tools generate, or mixing it up with text and image embeddings, as many e-commerce sites do, we want your search experience to be seamless. We haven't forgotten you for the power users working with multiple embeddings from multi-modal embedders. Our latest improvements ensure that everyone gets to update and fine-tune their search indexes incrementally, making the whole process as smooth as butter.

Big thanks to everyone: Wrapping this up, a huge shout-out to the squad – [@dureuill](https://github.com/dureuill), [@irevoire](https://github.com/irevoire), and [@ManyTheFish](https://github.com/ManyTheFish) – for their genius and grunt work that brought our ideas to life. And watch out for [@irevoire](https://github.com/irevoire)'s upcoming article, where he'll explain how we've implemented incremental indexing in Arroy—meaning you can add new vectors without rebuilding everything from scratch. More on that soon!

You can comment about this article on [Lobste.rs](https://lobste.rs/s/hdokyh/meilisearch_expands_search_power_with), [Hacker News](https://news.ycombinator.com/), [the Rust Subreddit](https://www.reddit.com/r/rust/comments/18pqxdz/meilisearch_expands_search_power_with_arroys), or [X (formerly Twitter)](https://x.com/kerollmops/status/1738837058628551148?s=46&t=0s6OFYJM-Oyl_WD-1jijQA).

* * *

[Meilisearch](https://meilisearch.com/?ref=blog.meilisearch.com) is an open-source search engine that not only provides state-of-the-art experiences for end users but also a simple and intuitive developer experience.

For more things Meilisearch, you can join the community on [Discord](https://discord.gg/meilisearch/?ref=blog.meilisearch.com) or subscribe to the [newsletter](https://meilisearch.us2.list-manage.com/subscribe?u=27870f7b71c908a8b359599fb&id=79582d828e&ref=blog.meilisearch.com). You can learn more about the product by checking out the [roadmap](https://roadmap.meilisearch.com/?ref=blog.meilisearch.com) and participating in [product discussions](https://github.com/orgs/meilisearch/discussions?ref=blog.meilisearch.com).

Meilisearch latest news and company updates