We're excited to unveil Meilisearch v1.7. In this article, we’ll review the most impactful changes. For an exhaustive listing, check out the [changelog](https://github.com/meilisearch/meilisearch/releases/tag/v1.7.0) on Github.

<Callout>
  Meilisearch 1.7 is available on [Meilisearch Cloud](https://cloud.meilisearch.com/?utm_campaign=release-v1-7&utm_source=blog) too—upgrade now!
</Callout>

Improved hybrid & vector search
-------------------------------

<Callout>
  AI search is an experimental feature, read the docs to [enable AI search](https://www.meilisearch.com/docs/learn/experimental/vector_search?utm_campaign=release-v1-7&utm_source=blog#activate-vector-search).
</Callout>

### New OpenAI embedding models

Starting with Meilisearch 1.7, you can use OpenAI’s third-generation embedding models:

*   `text-embedding-3-small`
*   `text-embedding-3-large`

Compared to the previous `text-embedding-ada-002` model, these new models offer improvements allowing to balance search results relevancy and cost efficiency.

The `small` model is more cost-effective, generating smaller vector embeddings that require less storage. The `large` focuses on delivering higher precision in search results, making it ideal to optimize relevancy.

### Configuring OpenAI model dimensions

When using the new `text-embedding-3-small` and `text-embedding-3-large` OpenAI models, you can now configure **dimensions** for your model. When not provided, the model's defaults dimensions are used.

You can do so when updating your settings:

```json
{
  // other settings...

  "embedders": {
    "small": {
      "source": "openAi",
      "model": "text-embedding-3-large",
      "dimensions": 512 // must be >0, must be <= 3072 for "text-embedding-3-large"
    },
  }
}

```

Higher dimensions typically offer more complex embeddings which can lead to more accurate search results. However, increased dimensions are more costly to compute and require more storage.

<Callout>
  You cannot customize dimensions when using OpenAI `text-embedding-ada-002`.
</Callout>

### Use AI Search on Meilisearch Cloud

Meilisearch hybrid search and vector search are coming to Meilisearch Cloud. We're rolling out invites every day. Join the beta. 👇

[Get early access](https://share-eu1.hsforms.com/1RN-rGYhHQPu5BNivfknlwAfg3aq?utm_campaign=release-v1-7&utm_source=blog)

GPU support for computing Hugging Face embeddings
-------------------------------------------------

Meilisearch 1.7 comes with the ability to enable GPU support for [self-hosted Meilisearch](https://www.meilisearch.com/docs/learn/getting_started/installation?utm_campaign=release-v1-7&utm_source=blog) for computing Hugging Face embeddings. This can significantly speed up the embedding generation process.

Find detailed instructions in our dedicated guide on [Using GPU to compute Hugging Face embeddings](https://www.meilisearch.com/docs/learn/cookbooks/computing_hugging_face_embeddings_gpu?utm_campaign=release-v1-7&utm_source=blog).

Display ranking scores details at search
----------------------------------------

Meilisearch 1.7 stabilizes the `showRankingScoreDetails` search parameter that was introduced in [Meilisearch 1.3](/blog/v1-3-release/?utm_campaign=release-v1-7&utm_source=blog This parameter adds a `_rankingScoreDetails` field to each document in search results, enabling further inspection of the search results.

You can include the parameter when making searches:

```sh
curl \
  -X POST 'https://edge.meilisearch.com/indexes/movies/search' \
  -H 'Content-Type: application/json' \
  --data-binary '{
    "q": "batman",
    "showRankingScoreDetails": true
  }'

```

Learn more about the [`_rankingScoreDetails` object](https://www.meilisearch.com/docs/reference/api/search#ranking-score-details?utm_campaign=release-v1-7&utm_source=blog) and how to use it with your preferred SDK in the API reference.

Contributors shout-out
----------------------

We want to give a massive thank you to all contributors that made this release possible: [@capJavert](https://github.com/capJavert), [@codesmith-emmy](https://github.com/codesmith-emmy), [@Elliot67](https://github.com/Elliot67) and [@Gosti](https://github.com/Gosti) for their PRs in Meilisearch, and [@agourlay](https://github.com/agourlay), [@choznerol](https://github.com/choznerol), [@ngdbao](https://github.com/ngdbao), [@timvisee](https://github.com/timvisee), and [@xshadowlegendx](https://github.com/xshadowlegendx) for their work on [charabia](https://github.com/meilisearch/charabia).

Special thanks to our SDKs maintainers, thanks to whom Meilisearch is available across so many languages. 🫶

* * *

And that’s a wrap for v1.7! This release post highlights the most significant updates. For an exhaustive listing, read the [changelog on Github](https://github.com/meilisearch/meilisearch/releases/tag/v1.7.0).

Stay in the loop of everything Meilisearch by subscribing to our [newsletter](https://dub.sh/meili-newsletter). To learn more about Meilisearch's future and help shape it, take a look at our [roadmap](https://roadmap.meilisearch.com/?utm_campaign=release-v1-7&utm_source=blog) and come participate in our [Product Discussions](https://github.com/meilisearch/product/discussions).

For anything else, join our developers community on [Discord](https://discord.meilisearch.com/?utm_campaign=release-v1-7&utm_source=blog).

Meilisearch 1.7

On March 7th, we hosted a live showcase session introducing hybrid search and other Meilisearch product updates. Watch a recording and see the hybrid search demo [here](https://www.youtube.com/watch?v=Lgv8iTdWDQc&t=881s).

* * *

AI's influence on search
------------------------

The rapid advancement of AI technology is undeniably reshaping human-machine interactions. Historically, communication with machines necessitated intermediaries like programming languages and structured scripts.

Now, with the arrival of GPTs and AI-assisted bots, machine-learning models are approaching a deeper comprehension of natural human language. This evolution brings humans and machines closer, changing the way we seek and find information.

From keyword-based searches to discovery-driven inquiries, people’s approach to search evolves: we are moving from searching for known items to exploring unknown territories, searching and finding the concepts instead of focusing solely on words.

Take a streaming media company: the searches across their platform might include the inquiry for a specific movie title, such as "Back to the Future," as well as conceptually-driven searches like "Give me recommendations for a feel-good sci-fi movie from the 80s".

Consider everyday scenarios: spotting an item on the street and using an image to search for it, or conducting voice searches instead of typing in words on keyboards. These are manifestations of our changing search behaviors.

The pinnacle: hybrid search
---------------------------

While traditional search methods excel in keyword-based queries, they falter with complex, nuanced requests. This is where Meilisearch’s new hybrid search approach comes into play, enriching the efficiency and precision of our robust full-text search engine with a depth of semantic understanding, catering to a wide array of use cases driven by customer-generated data.

Integrating models from AI giants like [OpenAI](https://www.meilisearch.com/docs/learn/experimental/vector_search#generate-auto-embeddings-with-openai) (open-source embedders generation feature is set for the next release cycle), Meilisearch enables users to create and fine-tune vector embeddings, tailoring their search engine to specific business needs. Moreover, evolving AI models learn from user interactions, continuously improving search precision.

What‘s included in the new release package?
-------------------------------------------

The AI-enhanced hybrid search from Meilisearch leverages our existing robust search infrastructure, including advanced sorting, filtering, faceting, and geo-sorting capabilities, among others.

New release features
--------------------

*   **Enhanced search relevancy**: The AI-enhanced layers bring a deeper semantic understanding, significantly improving result quality.
*   **AI embedders autogeneration:** Choose between using a third-party solution like OpenAI or your own locally generated embeddings. An open-source embedders generation feature is set for the next release cycle.

![](https://unable-actionable-car.media.strapiapp.com/Group_143725866_7f8309846d.png)

Add a new AI embedder in Meilisearch Cloud.

*   **Improved prompting:** Enhanced to incorporate key information templates, aiding the system in prompt ingestion and relevant metadata interpretation.
*   **Semantic ratio control:** Customize your search to span anywhere from purely keyword-based to entirely semantic.  
    

![](https://unable-actionable-car.media.strapiapp.com/Group_143725865_1_f3910ea1ce.png)

Choose where you want your results to fall on the spectrum from pure keyword-based to fully semantic search.

*   **Multimodal search capabilities:** Expanding search functionality to include image, sound, and video content.

**Democratizing search**
------------------------

At Meilisearch, our journey has always been anchored in simplicity, performance, and out-of-the-box relevancy. Our core mission remains steadfast: make it easy to find anything anywhere by providing a go-to-search solution for every team.

> “Meilisearch's hybrid search will democratize advanced search technology, focusing on simplicity and open-source customizability”  
> \- **Quentin de Quelen, Meilisearch’s CEO and Co-founder.**

As we evolve, our methods become more sophisticated, adapting to the ever-changing needs of businesses and developers alike. In a world brimming with data, effective content discovery is essential, not just a luxury. Our vision combines innovation with accessibility, aiming to make cutting-edge search technology inclusive for all.

The launch of our hybrid search product marks a significant milestone for Meilisearch and the search technology landscape. We invite you to try the new solution (now generally available) and join us in shaping the future of search.

* * *

Stay in the loop by subscribing to our [newsletter](https://dub.sh/meili-newsletter?&utm_source=blog&utm_campaign=HybridSearchRelease). To learn more about Meilisearch's future and help shape it, take a look at our [roadmap](https://roadmap.meilisearch.com/?utm_source=Blog&utm_campaign=HybridSearchRelease) and participate in our [Product Discussions](https://github.com/orgs/meilisearch/discussions?utm_source=Blog&utm_campaign=HybridSearchRelease).

For anything else, join our developer community on [Discord](https://discord.com/invite/hTD5hU2yWV?utm_source=Blog&utm_campaign=HybridSearchRelease).

Introducing hybrid search: combining full-text and semantic search for optimal balance

🥳 Join our launch event
------------------------

[![Meilisearch product update: AI-powered hybrid search, monitoring, and more. Thursday, March 7th 4PM CET | 3PM GMT | 10AM EST](https://unable-actionable-car.media.strapiapp.com/product_launch_march_df203f4787.png)](https://us06web.zoom.us/meeting/register/tZMsdu2gqTooE9AcjcQEzj9uST040tSjWDKn#/registration)

Join us on **March 7th** to discuss our recent releases and get a sneak peak of what’s coming on Meilisearch Cloud. Spoilers—hybrid search, new embedding models, GPU support, and more. 🤫

[Register](https://us06web.zoom.us/meeting/register/tZMsdu2gqTooE9AcjcQEzj9uST040tSjWDKn#/registration)

🤔 Vector embeddings, explained
-------------------------------

[![Vector embeddings](https://unable-actionable-car.media.strapiapp.com/Vector_embeddings_1_3c8b6c87d6.png)](/blog/what-are-vector-embeddings/?utm_campaign=newsletter&utm_source=blog&utm_medium=february-updates-24)

Wondering what are vector embeddings? What is the semantic space? How is it relevant for search? Our very own [@CarolainFG](https://twitter.com/CarolainFG) dug deep into the matter to create this high-level educational piece.

[Read blog post](/blog/what-are-vector-embeddings/?utm_campaign=newsletter&utm_source=blog&utm_medium=february-updates-24)

🚀 Upgrade to 1.6
-----------------

![Meilisearch 1.6 release notes](https://unable-actionable-car.media.strapiapp.com/1_6_release_0a6958f6c2.jpeg)

We released Meilisearch 1.6 in January. Upgrade to benefit from **hybrid search**, **indexing speed improvements**, and further **customization of proximity** ranking rule. It’s time to upgrade. 👇

[Read release notes](/blog/meilisearch-1-6/utm_campaign=newsletter&utm_source=blog&utm_medium=february-updates-24

* * *

🖇️ Links, links, links
-----------------------

### 💡 [Indexing best practices](https://www.meilisearch.com/docs/learn/cookbooks/indexing_best_practices?utm_campaign=newsletter&utm_source=blog&utm_medium=february-updates-24)

All our tips to index data efficiently and speed up the indexing process.

### 🔄 [Syncing with your Postgres database](https://www.meilisearch.com/docs/learn/cookbooks/meilisync_postgresql?utm_campaign=newsletter&utm_source=blog&utm_medium=february-updates-24)

Learn how to keep your Meilisearch in sync with a Postgres database.

### 🐬 [Add search to your Appwrite application](https://appwrite.io/blog/post/add-a-search-function-to-your-app)

Learn how to add a search function to your Appwrite application.

### 🐇 [Meilisearch v1.7 release candidate](https://github.com/meilisearch/meilisearch/releases/tag/v1.7.0-rc.0)

Get a preview of what’s coming next (and give your feedback!)

### 🟧 [Laravel Scout guide](https://www.meilisearch.com/docs/learn/cookbooks/laravel_scout?utm_campaign=newsletter&utm_source=blog&utm_medium=february-updates-24)

Learn how to use Meilisearch in your Laravel 10 application.

* * *

🥰 Made with Meilisearch
------------------------

**Rerun.io** added a search bar to allow users to search through all their documentation and SDKs API references. The site is built with **React** and uses [Analytics](https://www.meilisearch.com/docs/learn/cloud/analytics?utm_campaign=newsletter&utm_source=blog&utm_medium=february-updates-24) to allow them to improve the docs content.

[![rerun.io search interface](https://unable-actionable-car.media.strapiapp.com/Capture_2024_02_27_155651_1_62b82f8316.png)](https://rerun.io/)

[rerun.io](https://rerun.io)

_We’re always looking to promote new projects from the community. Post yours in the_ [_#built-with-meilisearch_](https://discord.meilisearch.com/) _Discord channel!_

🤓 Life at Meili
----------------

Our engineer [@dureuill](https://github.com/dureuill) wrote about how dynamic management of virtual memory enabled us to remove limitations in Meilisearch indexing policy. Attention, this one is technical.

[![Squeezing millions of documents in only 128 TB of virtual memory](https://unable-actionable-car.media.strapiapp.com/squeezing_d635cb5972.png)](/blog/dynamic-virtual-address-management/?utm_campaign=newsletter&utm_source=blog&utm_medium=february-updates-24)

[Read blog post](/blog/dynamic-virtual-address-management/?utm_campaign=newsletter&utm_source=blog&utm_medium=february-updates-24)

🦸 We're hiring!
----------------

See our open positions:

*   [Head of Developer Marketing](https://www.meilisearch.com/careers?utm_campaign=newsletter&utm_source=blog&utm_medium=february-updates-24)

Meilisearch February updates

The new dashboard incorporates a set of metrics providing insights into your project’s performance. We want to help engineers move faster by making it easier to spot issues during implementation and having better visibility on your Meilisearch database health.

What metrics can you expect?
----------------------------

This first iteration comes with:

*   30 days of data retention
*   Real-time mode

**Search performance metrics**

*   Search latency
*   Maximum search queries per second

**Operations metrics**

*   Bandwidth (in/out)
*   API Calls — Succeeded/Failed

The default view combines the data from the indexes per project, but it’s also possible to filter out the insights for a specific index of your choice. The dashboard also allows you to select a custom range using a date picker or get real-time data insights:

![](https://unable-actionable-car.media.strapiapp.com/monitoring_metrics_custom_gif_gif_382681a817.jpg)

Why use monitoring**?**
-----------------------

With our brand-new dashboard, you can:

*   Ensure your users are getting lightning-fast search responses for every query
*   Monitor how your search functionality holds up under varying loads, not only during peak traffic conditions but also while indexing
*   Identify and rectify any performance bottlenecks or errors
*   Proactively maintain and optimize your project

**Availability and plans**
--------------------------

The new Cloud monitoring metrics are available for all Cloud users across all pricing tiers. Each tier has access to the same metrics and chart functionalities of the monitoring metrics dashboard.

[Try it in Meilisearch Cloud](https://cloud.meilisearch.com/?utm_campaign=monitoring-v1&utm_source=blog)

Learn how to [get started with Meilisearch Cloud](https://www.meilisearch.com/docs/learn/getting_started/cloud_quick_start?utm_campaign=monitoring-v1&utm_source=blog) and read the [Monitoring documentation](https://www.meilisearch.com/docs/learn/cloud/monitoring#requirements?utm_campaign=monitoring-v1&utm_source=blog).

* * *

Stay in the loop by subscribing to our [newsletter](https://meilisearch.us2.list-manage.com/subscribe?u=27870f7b71c908a8b359599fb&id=79582d828e&utm_campaign=monitoring-v1&utm_source=blog). To learn more about Meilisearch's future and help shape it, take a look at our [roadmap](https://roadmap.meilisearch.com/?utm_campaign=monitoring-v1&utm_source=blog) and participate in our [Product Discussions](https://github.com/meilisearch/product/discussions/?utm_campaign=monitoring-v1&utm_source=blog).

For anything else, join our developer community on [Discord](https://discord.meilisearch.com/?utm_campaign=teams-release&utm_source=blog).

Cloud monitoring metrics have arrived!

**🚀 Announcing Meilisearch 1.6**
---------------------------------

[![Meilisearch 1.6 release notes](https://unable-actionable-car.media.strapiapp.com/meilisearch_1_6_1_25b4cb0c21.jpg)](/blog/meilisearch-1-6/?utm_campaign=newsletter&utm_source=january-news-24&utm_medium=blog)

Meilisearch 1.6 brings **hybrid search**, significant **improvements in indexing speed**, and a new API to **customize the proximity** ranking rule. Read the release notes to get the full picture.

[Read release notes](/blog/meilisearch-1-6/?utm_campaign=newsletter&utm_source=january-news-24&utm_medium=blog)

🔍 Typesense comparison
-----------------------

[![Meilisearch vs Typesense](https://unable-actionable-car.media.strapiapp.com/meilisearch_vs_typesense_cover_1_3b17eec4c2.jpg)](/blog/meilisearch-vs-typesense/?utm_campaign=newsletter&utm_source=january-news-24&utm_medium=blog)

We sat down to compare the key features between Meilisearch and open-source search engine **Typesense**. You can read the comparison on our blog.

[Read comparison](/blog/meilisearch-vs-typesense/?utm_campaign=newsletter&utm_source=january-news-24&utm_medium=blog)

🤗 New docs home
----------------

![Meilisearch documentation](https://unable-actionable-car.media.strapiapp.com/Screenshot_2024_01_23_at_17_16_39_e046f3c318.png)

We spent some time reworking our **docs home page** to make it more welcoming for developers. We also made **example apps** easier to find. Let us know what you think!

[View documentation](https://www.meilisearch.com/docs?utm_campaign=newsletter&utm_source=january-news-24&utm_medium=blog)

🖇️ Links of the month
----------------------

🔒 [Master key and API keys](https://www.meilisearch.com/docs/learn/security/master_api_keys?utm_campaign=newsletter&utm_source=january-news-24&utm_medium=blog)  
Learn to secure your Meilisearch instance and manage public API keys.

⚡ [Meilisearch indexes documents 2x faster](https://twitter.com/StriftCodes/status/1747566185791553639)  
Take a look at the performance boosts in Meilisearch 1.6.

🐬 [MySQL Full-Text Search Limits and New Alternatives Solutions](https://data-sleek.com/blog/the-limits-of-mysql-full-text-search-and-the-quest-for-alternative-solutions/)  
Comparing MySQL full-text search features with other search engines.

🎓 [Dropdown Autocomplete Search Anything in Laravel](https://codecourse.com/watch/dropdown-autocomplete-search-anything-in-laravel)  
Learn to build instant dropdown search with Laravel and Meilisearch.

⚛️ [The Two Reacts](https://overreacted.io/the-two-reacts/)  
Dan Abramov writes on components state, server data, and React.

🥰 Made with Meilisearch
------------------------

**YoungCapital NEXT** introduced their new website with a fresh branding and design. Built with **Next.js** and **Vercel**, their [job portal search](https://next.youngcapital.nl/vacatures) is powered by Meilisearch.

[![YoungCapital NEXT](https://unable-actionable-car.media.strapiapp.com/Screenshot_2024_01_23_at_17_34_37_1_552b1d480f.png)](https://next.youngcapital.nl/vacatures)

_We’re always looking to promote new projects from the community. Post yours in the_ [_#built-with-meilisearch_](https://discord.meilisearch.com/) _Discord channel!_

🏡 Life at Meili
----------------

[![Discord](https://unable-actionable-car.media.strapiapp.com/discord_2yf3soo8o238y21k_2d54bc8412.jpg)](/blog/from-slack-to-discord-our-migration/?utm_campaign=newsletter&utm_source=january-news-24&utm_medium=blog)

Our team wrote a blog post explaining our reasons for migrating our community to a Discord server, how we went about it, and why we shut down our public Slack.

[Read blog post](/blog/from-slack-to-discord-our-migration/?utm_campaign=newsletter&utm_source=january-news-24&utm_medium=blog)

🦸 We're hiring!
----------------

See our open positions:

*   [Head of Developer Marketing](https://www.meilisearch.com/careers?utm_campaign=newsletter&utm_source=january-news-24&utm_medium=blog)

* * *

Get these monthly updates straight to your inbox by subscribing to our [newsletter](https://meilisearch.us2.list-manage.com/subscribe?u=27870f7b71c908a8b359599fb&id=79582d828e&ref=blog.meilisearch.com).

For more things Meilisearch, join our developers community on [Discord](https://discord.meilisearch.com/). You can learn more about the product by checking out the [roadmap](https://roadmap.meilisearch.com/?ref=blog.meilisearch.com) and participating in [product discussions](https://github.com/meilisearch/product/discussions?ref=blog.meilisearch.com).

Until next time.

Meilisearch January Updates

Meilisearch vs Typesense

We’re announcing the release of Meilisearch 1.6. Let’s dive into some of the most important changes. You can also view the [full changelog](https://github.com/meilisearch/meilisearch/releases/tag/v1.6.0) on GitHub.

<Callout>
  Meilisearch 1.6 is also available on [Meilisearch Cloud](https://cloud.meilisearch.com/login?utm_campaign=release-v1-6&utm_source=blog), including all experimental features.
</Callout>

Experimental feature: hybrid search
-----------------------------------

Meilisearch introduces hybrid search. It combines full-text and semantic search to enhance the accuracy and comprehensiveness of search results. Picture a movie app like [where2watch](https://where2watch.meilisearch.com/?utm_campaign=release-v1-6&utm_source=blog). Now, your users will be able to find those movies they can't quite name but remember the story.

Furthermore, Meilisearch now streamlines the creation of vector embeddings. Choose your preferred embedders and Meilisearch will handle all interactions with external tools for you.

### Configuring embedders

You can [configure the embedders](https://www.meilisearch.com/docs/learn/experimental/vector_search?utm_campaign=release-v1-6&utm_source=blog#generate-vector-embeddings) in your index settings. Select from three types of embedders for your needs:

`openAI`:

*   Uses the [OpenAI](https://openai.com/) API for computing embeddings
*   Requires an OpenAI API key for operation

`huggingFace`:

*   Enables local computation of embeddings by downloading models from the [HuggingFace Hub](https://huggingface.co/models)
*   Operates on your CPU–not your GPU–which may impact indexing performance

`userProvided`:

*   Functions similarly to Meilisearch v1.3, with a key difference: you must define a specific embedder
*   Allows you to add pre-computed embeddings into your documents. You perform searches using vectors instead of text.

To use hybrid search, define at least one embedder in the index settings:

```JSON
{
  "embedders": {
    "default": {
      "source":  "openAi",
      "apiKey": "<your-OpenAI-API-key>",
      "model": "text-embedding-ada-002",
      "documentTemplate": "A movie titled '{{doc.title}}' whose description starts with {{doc.overview|truncatewords: 20}}"
    },
    "image": {
      "source": "userProvided",
      "dimensions": 512
    },
    "translation": {
      "source": "huggingFace",
      "model": "sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2",
      "documentTemplate": "A movie titled '{{doc.title}}' whose description starts with {{doc.overview|truncatewords: 20}}"
    }
  }
}

```

The `documentTemplate` field acts as a blueprint for creating your document's embedding. It uses the [Liquid template language](https://shopify.dev/docs/api/liquid). While its inclusion is optional, it is highly recommended, especially since embedding models are optimized for concise texts. It keeps only the necessary content, excluding non-essential data like `id`, and helps in adding context to increase relevancy.

### **Hybrid search**

To [perform a hybrid search](https://www.meilisearch.com/docs/learn/experimental/vector_search?utm_campaign=release-v1-6&utm_source=blog#vector-search-with-auto-embeddings), use the hybrid field within the `POST /index/:index_uid/search` route.

```JSON
{
    "q": "Plumbers and dinosaurs",
    "hybrid": {
        "semanticRatio": 0.9,
        "embedder": "default"
    }
}

```

`embedder`: an embedder from the options configured in your index settings.

`semanticRatio`: a floating value ranging from `0` to `1`; `1` is a  fully semantic search; `0` is an exact, match-focused full-text search; the default is `0.5` which mixes both methods.

Your control over the semantic ratio directly influences how search results are ranked. A higher semantic ratio shifts the focus towards the context and meaning behind your query, ranking results that are more semantically relevant higher. 

On the other hand, a lower semantic ratio increases the weight given to keyword accuracy in the ranking process, bringing results that closely match your specific search terms to the forefront.

### **Breaking changes in the experimental vector search API**

Meilisearch v1.6 introduces some breaking changes in the vector search API.

Previously, you could send vectors without specifying a model. Now, you must define a model in the settings:

```JSON
"embedders": {
    "default": {
      "source": "userProvided",
      "dimensions": 512
    }
}

```

Because Meilisearch now supports multiple embedders, it has updated the vector submission format from arrays to JSON objects.

*   Previews format: `“_vectors”: [[0.0, 0.1]]`
*   New format: `“_vectors”: {“image2text”: [0.0, 0.1, …]}`

For detailed information on these updates, refer to the [documentation](https://www.meilisearch.com/docs/learn/experimental/vector_search?utm_campaign=release-v1-6&utm_source=blog).

For in-depth technical information, explore the [series of articles](/blog/spotify-inspired-hybrid-search-and-rust/?utm_campaign=release-v1-6&utm_source=blog on [Arroy](https://github.com/meilisearch/arroy), an open-source repository based on [Spotify’s Annoy](https://github.com/spotify/annoy) and developed in Rust. This library, created and maintained by the Meilisearch engine team, specializes in searching for vectors in a space that are near a specified query vector.

**Performance optimization**
----------------------------

### Improved indexing speed

We're thrilled to share a major enhancement in Meilisearch's indexing performance. Our recent tests, including scenarios with frequent and partial document updates, have shown impressive results: **a reduction in indexing time by up to 50%**, and in some cases, **even as much as 75%**. 

Thanks to our latest optimizations, Meilisearch now **stores and pre-computes less data**. Additionally, during document updates, it **re-indexes or deletes only the necessary data**. For instance, in an e-commerce dataset, updating the stock level of a product results in re-indexing just the 'stock' field, rather than the entire product document.

### Disk space usage reduction

Meilisearch reduces internal data storage, leading to a more **compact database size** on your disk. With a dataset of approximately 15Mb, we observed a **40% to 50% reduction in database size**.

This enhancement not only reduces the database size, but also **improves its stability**, making the space savings more evident as the number of documents increases.

**New feature: customize proximity precision**
----------------------------------------------

To further reduce indexing speed, Meilisearch now allows you to tailor the accuracy of the [proximity ranking rule](https://www.meilisearch.com/docs/learn/core_concepts/relevancy?utm_campaign=release-v1-6&utm_source=blog#3-proximity) to your specific needs.

The proximity ranking rule is computationally demanding and may lead to longer indexing times. Reducing its accuracy can greatly enhance performance, and in most scenarios, it will not substantially affect the relevancy of the results.

To adjust its impact [configure the `proximityPrecision` setting](https://www.meilisearch.com/docs/reference/api/settings?utm_campaign=release-v1-6&utm_source=blog#proximity-precision):

```JSON
curl \
  -X PATCH 'http://localhost:7700/indexes/books/settings/proximity-precision' \
  -H 'Content-Type: application/json'  \
  --data-binary '{
    "proximityPrecision": "byAttribute"
  }'
```

The default `proximityPrecision` setting is `byWord`, which calculates proximity based on exact word distances. 

The `byAttribute` setting considers words in the same attribute as proximate, regardless of their exact distance.

Using `byAttribute` can boost the indexing speed, but it might slightly change how relevant the results are. This becomes more noticeable in searches where it's important for words to be close to each other.

For example, when you're looking through song lyrics or long articles, like trying to find 'world war' in a bunch of Wikipedia pages, you might end up with results that contain these words but not necessarily close together or in the desired order. This is also true for [phrase searches](https://www.meilisearch.com/docs/reference/api/search?utm_campaign=release-v1-6&utm_source=blog#phrase-search) and for searches involving [multi-word synonyms](https://www.meilisearch.com/docs/learn/configuration/synonyms?utm_campaign=release-v1-6&utm_source=blog#multi-word-synonyms), where the specific combination of words is crucial.

**New feature: task queue webhook**
-----------------------------------

Meilisearch now offers a webhook feature to notify a custom URL when an asynchronous task finishes (either succeeds, fails, or gets canceled).

This feature is particularly useful for streamlining workflows, saving you from polling the tasks route.

Set up your webhook at launch using these environment variables:

```bash
MEILI_TASK_WEBHOOK_URL=https://mywebsite.com/my-super-webhook?user=1234&number=8

MEILI_TASK_WEBHOOK_AUTHORIZATION_HEADER='Bearer 12340987546wowowlolol'
```

You can also use the respective command-line options.

Once set up, the webhook sends out a payload in [JSON Lines (ndjson) format](https://www.meilisearch.com/docs/learn/core_concepts/documents?utm_campaign=release-v1-6&utm_source=blog#ndjson) to your specified URL, containing the list of finished tasks:

```json
//POST HTTP request to https://myproject.com/mywebhook?common=people

{"uid":4,"indexUid":"movie","status":"failed","type":"indexDeletion","canceledBy":null,"details.deletedDocuments":0,"error.message":"Index `movie` not found.","error.code":"index_not_found","error.type":"invalid_request","error.link":"https://docs.meilisearch.com/errors#index_not_found","duration":"PT0.001192S","enqueuedAt":"2022-08-04T12:28:15.159167Z","startedAt":"2022-08-04T12:28:15.161996Z","finishedAt":"2022-08-04T12:28:15.163188Z"}
{"uid":5,"indexUid":"movie","status":"failed","type":"indexDeletion","canceledBy":null,"details.deletedDocuments":0,"error.message":"Index `movie` not found.","error.code":"index_not_found","error.type":"invalid_request","error.link":"https://docs.meilisearch.com/errors#index_not_found","duration":"PT0.001192S","enqueuedAt":"2022-08-04T12:28:15.159167Z","startedAt":"2022-08-04T12:28:15.161996Z","finishedAt":"2022-08-04T12:28:15.163188Z"}
```

**Experimental feature: limit the number of batched tasks**
-----------------------------------------------------------

To speed up the indexing process, Meilisearch processes similar [tasks](https://www.meilisearch.com/docs/learn/async/asynchronous_operations?utm_campaign=release-v1-6&utm_source=blog#tasks-and-asynchronous-operations) in large batches. However, excessive [queued tasks](https://www.meilisearch.com/docs/learn/async/asynchronous_operations?utm_campaign=release-v1-6&utm_source=blog#task-queue) can occasionally cause crashes or stalls.

To control the number of batched tasks, set the limit at launch using either the command-line argument `--experimental-max-number-of-batched-tasks`, the `MEILI_EXPERIMENTAL_MAX_NUMBER_OF_BATCHED_TASKS` environment variable, or the [configuration file](https://www.meilisearch.com/docs/learn/configuration/instance_options?utm_campaign=release-v1-6&utm_source=blog#configuration-file).

**Contributors shout-out**
--------------------------

We are really grateful for all the community members who participated in this release.We would like to thank [@Karribalu](https://github.com/Karribalu) and [@vivek-26](https://github.com/vivek-26) for their help with [Meilisearch](https://github.com/meilisearch/meilisearch). We also want to send a special shout-out to our SDKs maintainers 🦸

* * *

And that’s it for v1.6! This release post highlights the most significant updates. For an exhaustive listing, read the [changelog on Github](https://github.com/meilisearch/meilisearch/releases/tag/v1.6.0).

Stay in the loop of everything Meilisearch by subscribing to [the newsletter](https://meilisearch.us2.list-manage.com/subscribe?u=27870f7b71c908a8b359599fb&id=79582d828e&ref=blog.meilisearch.com). To learn more about Meilisearch's future and help shape it, take a look at our [roadmap](https://roadmap.meilisearch.com/?utm_campaign=release-v1-6&utm_source=blog) and come participate in our [Product Discussions](https://github.com/meilisearch/product/discussions?ref=blog.meilisearch.com).

For anything else, join our developers community on [Discord](https://discord.meilisearch.com/?utm_campaign=release-v1-6&utm_source=blog).

Meilisearch v1.6

⏳ Meilisearch v1.6 release candidate is now available
-----------------------------------------------------

[![](https://unable-actionable-car.media.strapiapp.com/1_6_RC_74d5e70575.jpg)](https://github.com/meilisearch/meilisearch/releases/tag/v1.6.0-rc.0?utm_campaign=newsletter&utm_source=december-news-23&utm_medium=blog)

Get a sneak peek at **Meilisearch v1.6**, launching **mid-January**! This update promises **major indexing performance improvements** and introduces **hybrid search** for more comprehensive results.

⚠️ The current **release candidate** is for **testing only** and not recommended for production use. [**Feedback**](https://github.com/orgs/meilisearch/discussions) and [**bug reports**](https://github.com/meilisearch/meilisearch/issues/new/choose) are welcome to refine the final release.

[Read pre-release notes](https://github.com/meilisearch/meilisearch/releases/tag/v1.6.0-rc.0?utm_campaign=newsletter&utm_source=december-news-23&utm_medium=blog)

🦀 Spotify-inspired: Elevating Meilisearch with Hybrid Search and Rust
----------------------------------------------------------------------

[![](https://unable-actionable-car.media.strapiapp.com/spotify_inspired_51f4d0331b.jpeg)](/blog/spotify-inspired-hybrid-search-and-rust?utm_campaign=newsletter&utm_source=december-news-23&utm_medium=blog)

How we created [**Arroy**](https://github.com/meilisearch/arroy), a **Rust library** building upon the foundations of **Spotify**'s Annoy.

[Read article](/blog/spotify-inspired-hybrid-search-and-rust?utm_campaign=newsletter&utm_source=december-news-23&utm_medium=blog)

🌟 We reached 40,000 GitHub stars!
----------------------------------

[![](https://unable-actionable-car.media.strapiapp.com/40_K_a03904ec32.jpg)](https://github.com/meilisearch/meilisearch/)

A huge **thank you** to our **incredible community** for helping us reach this milestone! Your **contributions**, **feedback**, and **enthusiasm** have been pivotal in our journey. Thank you for being a part of our growing family.

**Let's continue to reach new heights together!**

**📰 In the news**
------------------

### **🔍** [**Using vector search**](https://www.meilisearch.com/docs/learn/experimental/vector_search?utm_campaign=newsletter&utm_source=december-news-23&utm_medium=blog)

Everything you need to know to get started with this feature.

### 📈 [Introducing Cloud search analytics](/blog/search-analytics-release/?utm_campaign=newsletter&utm_source=december-news-23&utm_medium=blog)

Get immediate actionable insight to optimize your search.

### 👥 [Teams feature is available on Meilisearch Cloud](/blog/introducing-teams/?utm_campaign=newsletter&utm_source=december-news-23&utm_medium=blog)

 Host and manage your projects in collaboration with your teammates.

### ✈️ [Migrating to Meilisearch Cloud](https://www.meilisearch.com/docs/learn/update_and_migration/migrating_cloud?utm_campaign=newsletter&utm_source=december-news-23&utm_medium=blog)

Migrating from self-hosted? Here’s how to handle the migration.

### 🎓 [Bucket sort guide](/blog/bucket-sort-guide/?utm_campaign=newsletter&utm_source=december-news-23&utm_medium=blog)

Understand the algorithm behind Meilisearch relevancy.

🥰 Made with Meilisearch
------------------------

[**Bookshop.org**](https://bookshop.org/?utm_campaign=newsletter&utm_source=december-news-23&utm_medium=blog) is an online bookstore based in Brooklyn, New York, with a mission to support local, independent bookstores financially. They have integrated Meilisearch to enhance the search functionality within their extensive catalog.

[![Search results page from Bookshop.org, displaying over 1000 results for "m", and refining as the user keeps typing in the search bar](https://unable-actionable-car.media.strapiapp.com/bookshop_meilisearch_9498082583.gif)](https://bookshop.org/?utm_campaign=newsletter&utm_source=december-news-23&utm_medium=blog)

_We’re always looking to promote new projects from the community. Post yours in the_ [_#built-with-meilisearch_](https://discord.meilisearch.com/) _Discord channel!_

🏡 Life at Meili
----------------

For us, what you value in work is as important as your technical skills, if not more so. Here’s a recruiter's advice on how to answer interview questions about values.

[![ A spinning top on a wooden surface.](https://unable-actionable-car.media.strapiapp.com/recruiters_advice_0b884ab386.jpeg)](/blog/advice-on-answering-interview-questions-about-values/?utm_campaign=newsletter&utm_source=december-news-23&utm_medium=blog)

[Read blog post](/blog/advice-on-answering-interview-questions-about-values/?utm_campaign=newsletter&utm_source=december-news-23&utm_medium=blog)

🦸 We're hiring!
----------------

See our open positions:

*   [Head of Developer Marketing](https://jobs.lever.co/meili/6c9637b0-399a-4300-9f03-1e6f4c450efa)

* * *

Get these monthly updates straight to your inbox by subscribing to our [newsletter](https://meilisearch.us2.list-manage.com/subscribe?u=27870f7b71c908a8b359599fb&id=79582d828e&ref=blog.meilisearch.com).

For more things Meilisearch, join our developers community on [Discord](https://discord.gg/meilisearch/?ref=blog.meilisearch.com). You can learn more about the product by checking out the [roadmap](https://roadmap.meilisearch.com/?ref=blog.meilisearch.com) and participating in [product discussions](https://github.com/meilisearch/product/discussions?ref=blog.meilisearch.com).

Until next time.

Meilisearch December Updates

_This is part 3 of a series of blog posts_ [_originally published on my personal blog_](https://blog.kerollmops.com/meilisearch-expands-search-power-with-arroy-s-filtered-disk-ann)_._ _Begin the journey with_ [_part 1_](/blog/spotify-inspired-hybrid-search-and-rust/) _and_ [_part 2_](/blog/refining-ann-performance-with-rust/)_._

We can store vectors in [Arroy](https://github.com/meilisearch/arroy) and efficiently compute the search tree nodes, but we still need some features to make it usable in Meilisearch. Our full-text search engine has great support for filtering; selecting subsets of documents is an essential feature to support. For instance, one of our biggest clients requires the ability to filter through over 100 million YouTube video metadata and their associated image embeddings to effectively select videos released within specific time frames, such as a single day or week. This represents the scalability and responsiveness challenges we aim to address with our filtering system, making it a perfect use case to tailor our developments.

Meilisearch supports the following operators on the filterable attributes of your documents: `<`, `<=`, `=`, `!=` `>=`, and `>`. Internally, we extensively [use `RoaringBitmap`s](https://github.com/RoaringBitmap/roaring-rs), which are well-optimized sets of integers that support fast binary operations like unions and intersections. When the engine receives a user request with filters, it first computes the subset of documents from the filter that will be inputted into the search algorithms, the one that ranks documents by quality. This subset is represented by a `RoaringBitmap`.

How was it working before Arroy?
--------------------------------

Ranking only the subset of filtered documents has been working well since 2018, but now that we have a new data structure to search on, we need to look at how to implement it. The engine had already supported the vector store feature for months but was inefficient. We were using an in-memory HNSW and were deserializing the whole data structure in memory, searching for the nearest neighbors of the target vector, which was returning an iterator.

```Rust
fn vector_store(db: Database, subset: &RoaringBitmap, target_vec: &[f32]) -> Vec<DocumentID> {
    // This takes a lot of time and memory.
    let hnsw = db.deserialize_hnsw();

    let mut output = Vec::new();
    for (vec_id, vec, dist) in hnsw.nearest_neighbors(target_vec) {
        let doc_id = db.document_associated_to_vec(vec_id).unwrap();
        if !output.contains(&doc_id) {
          output.push(doc_id);
          if output.len() == 20 { break }
        }
    }

    output
}
```

You may wonder why we are retrieving the document ID associated with the vector ID. Meilisearch supports multiple vectors by document since the beginning of the vector store feature. This is unfortunate because we must look up every vector we are iterating. We need to maintain this lookup table. The iterator can iterate on the whole vector dataset if the subset is small enough, e.g., `document.user_id = 32`. We want the document operations to be atomic and consistent, so we have to store the HNSW on disk and avoid having to maintain synchronization between the LMDB transactions and this vector store data structure. Ho! And the library doesn't support incremental insertion. We must reconstruct the HNSW in memory from scratch every time we insert a single vector.

Integrating Arroy in Meilisearch
--------------------------------

As we worked on updating Meilisearch to include the new vector store, arroy, we tried out mob programming for the first time. This is where we all code together at the same time. It might sound like it would slow us down, but it actually made us super productive! By tackling problems together right as they came up, we got arroy fitted into Meilisearch much faster than if we'd worked alone.

Arroy is different because it doesn’t return an iterator to give back search results. Now, our search engine is smarter and can figure out the exact number of results it needs to give back, even when there are filters to consider. This teamwork improved our search tool and showed us that working together is key when facing big tech challenges.

#### Filtering while searching in Arroy

You can find a description of the arroy internal data structure in [part 1 of this series](https://blog.kerollmops.com/spotify-inspired-elevating-meilisearch-with-hybrid-search-and-rust). Here is a list of the different kinds of nodes you can find:

*   The item nodes. The original vectors that the user provides. The small points on the left.
*   The normal nodes, also called split planes. They represent hyperplanes that split subsets of item nodes in half.
*   The descendant nodes. They are the tree leaves composed of the item IDs you'll find if you follow a certain normal node path.

![split-plane-combined-schema](https://unable-actionable-car.media.strapiapp.com/spli_plane_b53ab895e8.png)

A typical search will load all the normal nodes in a binary heap associated with an infinite distance. Remember that there are a lot of randomly generated trees, and there are, therefore, a lot of entry points. Ascending distances order the binary heap; the shortest nodes found will be popped first.

In the search algorithm, we pop the nearest item from this heap. If it's a normal node, we fetch the left and right sides of the hyperplane:

*   If we find normal nodes again, we associate the distance from the hyperplane with positive and negative distances from the targeted/queried vector.
*   If we find descendant nodes, we do not push them into the queue but rather directly add them to the potential output list as they represent the nearest vectors we have found.

You probably noticed where I want to go, but this is where magic happens. We modified arroy to store the list of descendants in `RoaringBitmap`s instead of a raw list of integers. This is another improvement compared to the original Spotify library, as those lists weigh less. Doing an intersection with the filtered subset is way easier now.

However, there is always an issue: the vector IDs are not the document IDs, and Meilisearch, after executing the filters, only knows about the documents. Iterating on the lookup table, I talked about before, constructing the final bitmap with all the vector IDs corresponding to the filtered documents would not be efficient when many documents are part of this subset, e.g., `document.user_id != 32`. I do not recommend using an `O(n)` algorithm in the search functions.

#### Using multiple indexes for efficient filtering

Fortunately, we developed a fun feature that wasn't meant to be used that way in arroy: the support for multiple indexes in a single LMDB database. We originally developed the multiple indexes feature to be able only to open a single LMDB database to store different vector types. Yes! In Meilisearch v1.6, you can describe the different embedders that live in a single index. You can identify the different vectors with different numbers of dimensions and distance functions you can store in a single document. Defining multiple vectors for a single document associated with the same embedder is also possible.

The indexes are identified by an `u16`. This feature can trick the algorithm into being even more efficient than the previous HNSW solution. Using one store by embedder and vector in a document is interesting because we can now identify the vectors using the document ID. No more lookup database and vector IDs. The vector IDs are reduced to the document IDs. We can use the output of the filters to filter the arroy indexes.

The search algorithm is different on the Meilisearch side. We request the nearest neighbors in every arroy index, sort the results by document ID to be able to deduplicate them and not return the same document multiple times, sort them again by distance, and then return only the top twenty documents. It may seem complex, but we are talking about twenty documents by the number of vectors for a single document. Usually, users will have a single vector.

```Rust
fn vector_search(
    rtxn: &RoTxn,
    database: Database,
    embedder_index: u8,
    limit: usize,
    candidates: &RoaringBitmap,
    target_vector: &[f32],
) -> Vec<(DocumentId, f32)> {
    // The index represents the embedder index shifted and
    // is later combined with the arroy index. There is an arroy
    // index by vector for a single embedded in a document.
    let index = (embedder_index as u16) << 8;
    let readers: Vec<_> = (0..=u8::MAX)
        .map(|k| index | (k as u16))
        .map_while(|index| arroy::Reader::open(rtxn, index, database).unwrap())
        .collect();

    let mut results = Vec::new();
    for reader in &readers {
        let nns = reader.nns_by_vector(rtxn, target_vector, limit, None, Some(candidates)).unwrap();
        results.extend(nns_by_vector);
    }

    // Documents can have multiple vectors. We store the different vectors
    // into different arroy indexes, we must make sure we don't find the nearest neighbors
    // vectors that correspond to the same document.
    results.sort_unstable_by_key(|(doc_id, _)| doc_id);
    results.dedup_by_key(|(doc_id, _)| doc_id);

    // Sort back the documents by distance
    results.sort_unstable_by_key(|(_, distance)| distance);
    results
}
```

The design philosophy behind these improvements
-----------------------------------------------

We're trying to make Meilisearch flexible enough for everyone's needs. Whether your documents are powered by a single embedding, like the kind OpenAI's nifty tools generate, or mixing it up with text and image embeddings, as many e-commerce sites do, we want your search experience to be seamless. We haven't forgotten you for the power users working with multiple embeddings from multi-modal embedders. Our latest improvements ensure that everyone gets to update and fine-tune their search indexes incrementally, making the whole process as smooth as butter.

Big thanks to everyone: Wrapping this up, a huge shout-out to the squad – [@dureuill](https://github.com/dureuill), [@irevoire](https://github.com/irevoire), and [@ManyTheFish](https://github.com/ManyTheFish) – for their genius and grunt work that brought our ideas to life. And watch out for [@irevoire](https://github.com/irevoire)'s upcoming article, where he'll explain how we've implemented incremental indexing in Arroy—meaning you can add new vectors without rebuilding everything from scratch. More on that soon!

You can comment about this article on [Lobste.rs](https://lobste.rs/s/hdokyh/meilisearch_expands_search_power_with), [Hacker News](https://news.ycombinator.com/), [the Rust Subreddit](https://www.reddit.com/r/rust/comments/18pqxdz/meilisearch_expands_search_power_with_arroys), or [X (formerly Twitter)](https://x.com/kerollmops/status/1738837058628551148?s=46&t=0s6OFYJM-Oyl_WD-1jijQA).

* * *

[Meilisearch](https://meilisearch.com/?ref=blog.meilisearch.com) is an open-source search engine that not only provides state-of-the-art experiences for end users but also a simple and intuitive developer experience.

For more things Meilisearch, you can join the community on [Discord](https://discord.gg/meilisearch/?ref=blog.meilisearch.com) or subscribe to the [newsletter](https://meilisearch.us2.list-manage.com/subscribe?u=27870f7b71c908a8b359599fb&id=79582d828e&ref=blog.meilisearch.com). You can learn more about the product by checking out the [roadmap](https://roadmap.meilisearch.com/?ref=blog.meilisearch.com) and participating in [product discussions](https://github.com/orgs/meilisearch/discussions?ref=blog.meilisearch.com).

Meilisearch latest news and company updates