Meilisearch 1.16

We’re thrilled to announce the release of Meilisearch 1.16. In this article, we’ll highlight the key new features: multi-modal embeddings, exporting to other instances, and improvements to the documents API.

For an exhaustive listing of the changes, please consult the changelog on GitHub.

New: Multi-modal search and embeddings

Meilisearch 1.16 introduces the new fragments API. This allows you to deconstruct documents and search queries into distinct semantic parts, or “fragments.“

Leveraging fragments, you can now create embeddings based on semantically different parts of the document and build multimodal search. For example, this allows users on ecommerce sites and marketplaces to find products by describing them.

Let’s take the example of a movie document with the following structure:

{
  "title": "Star Wars",
  "description": "A long time ago, in a galaxy...",
  "image_url": "https://images.com/star-wars.jpg"
}

We want to use Voyage AI to generate embeddings at indexing time as follows:

• At indexing time — Create a text embedding based on title and description as text, and create an image embedding based on the image URL
• At search time — Allow matching text based on a text query and image based on an inlined image

The payload below allows updating embedder settings accordingly:

{
	"voyage": {
		"source": "rest",
		"url": "https://api.voyageai.com/v1/multimodalembeddings",
		"apiKey": "$VOYAGE_API_KEY",
		// describes how we index documents
		"indexingFragments": {
			"text": {
				"value": {
					"content": [
						{
							"type": "text",
							"text": "A movie titled {{doc.title}} whose description starts with {{doc.overview|truncateWords:20}}."
						}
					]
				}
			},
			"image": {
				"value": {
					"content": [
						{
							"type": "image_url",
							"image_url": "{{doc.poster}}"
						}
					]
				}
			}
		},
		// describes how to search in the index
		"searchFragments": {
			"image": {
				"value": {
					"content": [
						{
							"type": "image_base64",
							"image_base64": "data:{{media.image.mime}};base64,{{media.image.data}}"
						}
					]
				}
			},
			"text": {
				"value": {
					"content": [
						{
							"type": "text",
							// uses the `q` field from search queries
							"text": "{{q}}"
						}
					]
				}
			}
		},
		"request": {
			"inputs": [
				"{{fragment}}",
				"{{..}}"
			],
			"model": "voyage-multimodal-3"
		},
		"response": {
			"data": [
				{
					"embedding": "{{embedding}}"
				},
				"{{..}}"
			]
		}
	}
}

In the indexing fragments, the value object depends on the embedding API. In this example, it adheres to the Voyage AI API.

With these settings in effect, you can make a search request using image data as follows:

{
  "media": {
    "image": {
      "mime": "image/jpeg",
      "data": "/9j/4AAQSkZJRgABAQAAAQABAAD/2w... images bytes encoded as base64"
    }
  "hybrid": {
    "embedder": "voyage"
  }
}

You can also perform a hybrid search using a text query. Note that since the Voyage model we’re using here is multi-modal, Meilisearch will also compare the generated embeddings with the movie image. This means that describing the image could allow finding the movie.

{
  "q": "A movie with lightsabers in space",
  "hybrid": {
    "embedder": "voyage",
    "semanticRatio": 0.5
  }
}

New: Transferring documents between Meilisearch instances

Meilisearch 1.16 introduces a new exports API to transfer documents from one instance to another. This new API does not require creating a dump or a snapshot. The goal is to ease the migration from a local environment to a production Meilisearch Cloud project and minimize deployment overhead.

Consult the documentation for more information about using the exports API.

New: Sorting on the documents API

Meilisearch 1.16 brings sorting to the documents API. The sorting behaves exactly like the sorting with the search API. This improves the administrative toolkit to manage your Meilisearch instance. Consult the documentation for more information about using the documents API.

Contributors shout-out

We want to give a massive thank you to the contributors who made this release possible. Special thanks @martin-g, @lblack00, @mcmah309, @nnethercott, @arthurgousset, @Mubelotix, @diksipav, @Nymuxyzo, @kametsun for their contributions to Meilisearch and to @nnethercott for their contribution to Arroy.

And that’s a wrap for v1.16! These release notes only highlight the most significant updates. For an exhaustive listing, refer to the changelog on GitHub.

For more information, subscribe to our monthly newsletter, or join our Product Discussions.

For anything else, join our developers community on Discord.