> ## Documentation Index
> Fetch the complete documentation index at: https://www.meilisearch.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Handling multilingual datasets

> This guide covers indexing strategies, language-specific tokenizers, and best practices for aligning document and query tokenization.

When working with datasets that include content in multiple languages, it’s important to ensure that both documents and queries are processed correctly. This guide explains how to index and search multilingual datasets in Meilisearch, highlighting best practices, useful features, and what to avoid.

## Recommended indexing strategy

### Create a separate index for each language (recommended)

If you have a multilingual dataset, the best practice is to create one index per language.

#### Benefits

* Provides natural sharding of your data by language, making it easier to maintain and scale.

* Lets you apply language-specific settings, such as [stop words](/reference/api/settings/get-stopwords), and [separators](/reference/api/settings/get-separatortokens).

* Simplifies the handling of complex languages like Chinese or Japanese, which require specialized tokenizers.

#### Searching across languages

If you want to allow users to search in more than one language at once, you can:

* Run a [multi-search](/reference/api/multi-search/perform-a-multi-search), querying several indexes in parallel.

* Use [federated search](/reference/api/multi-search/perform-a-multi-search), aggregating results from multiple language indexes into a single response.

### Create a single index for multiple languages

In some cases, you may prefer to keep multiple languages in a **single index**. This approach is generally acceptable for proof of concepts or datasets with fewer than \~1M documents.

#### When it works well

* Suitable for languages that use spaces to separate words and share similar tokenization behavior (e.g., English, French, Italian, Spanish, Portuguese).

* Useful when you want a simple setup without maintaining multiple indexes.

#### Limitations

* Languages with compound words (like German) or diacritics that change meaning (like Swedish), as well as non-space-separated writing systems (like Chinese, or Japanese), work better in their own index since they require specialized [tokenizers](/capabilities/indexing/advanced/tokenization).

* Chinese and Japanese documents should not be mixed in the same field, since distinguishing between them automatically is very difficult. Each of these languages works best in its own dedicated index. However, if fields are strictly separated by language (e.g., title\_zh always Chinese, title\_ja always Japanese), it is possible to store them in the same index.

* As the number of documents and languages grows, performance and relevancy can decrease, since queries must run across a larger, mixed dataset.

#### Best practices for the single index approach

* Use language-specific field names with a prefix or suffix (e.g., title\_fr, title\_en, or fr\_title).

* Declare these fields as [localized attributes](/reference/api/settings/get-localizedattributes) so Meilisearch can apply the correct tokenizer to each one.

* This allows you to filter and search by language, even when multiple languages are stored in the same index.

## Language detection and configuration

Accurate language detection is essential for applying the right tokenizer and normalization rules, which directly impact search quality.

By default, Meilisearch automatically detects the language of your documents and queries.

This automatic detection works well in most cases, especially with longer texts. However, results can vary depending on the type of input:

* **Documents**: detection is generally reliable for longer content, but short snippets may produce less accurate results.
* **Queries**: short or partial inputs (such as type-as-you-search) are harder to identify correctly, making explicit configuration more important.

When you explicitly set `localizedAttributes` for documents and `locales` for queries, you **restrict the detection to the languages you’ve declared**.

**Benefits**:

* Meilisearch only chooses between the specified languages (e.g., English vs German).

* Detection is more **reliable and consistent**, reducing mismatches.

For search to work effectively, **queries must be tokenized and normalized in the same way as documents**. If strategies are not aligned, queries may fail to match even when the correct terms exist in the index.

### Aligning document and query tokenization

To keep queries and documents consistent, Meilisearch provides configuration options for both sides. Meilisearch uses the same `locales` configuration concept for both documents and queries:

* In **documents**, `locales` are declared through `localizedAttributes`.
* In **queries**, `locales` are passed as a [search parameter](/reference/api/search/search-with-post).

#### Declaring locales for documents

The [`localizedAttributes` setting](/reference/api/settings/get-localizedattributes) allows you to explicitly define which languages are present in your dataset, and in which fields.

For example, if your dataset contains multilingual titles, you can declare which attribute belongs to which language:

<CodeGroup>
  ```json theme={null}
  {
    "id": 1,
    "title_en": "Danube Steamship Company",
    "title_de": "Donaudampfschifffahrtsgesellschaft",
    "title_fr": "Compagnie de navigation à vapeur du Danube"
  }
  ```
</CodeGroup>

<CodeGroup>
  ```bash cURL theme={null}
  curl \
    -X PUT 'MEILISEARCH_URL/indexes/INDEX_NAME/settings/localized-attributes' \
    -H 'Content-Type: application/json' \
    --data-binary '[
      { "attributePatterns": ["*_en"], "locales": ["eng"] },
      { "attributePatterns": ["*_de"], "locales": ["deu"] },
      { "attributePatterns": ["*_fr"], "locales": ["fra"] }
    ]'
  ```
</CodeGroup>

#### Specifying locales for queries

When performing searches, you can specify [query locales](/reference/api/search/search-with-post#body-locales) to ensure queries are tokenized with the correct rules.

<CodeGroup>
  ```javascript theme={null}
  client.index('INDEX_NAME').search('schiff', { locales: ['deu'] })
  ```
</CodeGroup>

This ensures queries are interpreted with the correct tokenizer and normalization rules, avoiding false mismatches.

<Note>
  If the `locales` search parameter and the `localizedAttributes` index setting disagree, `locales` wins. The value passed on the query takes precedence over the index-level configuration for that request. This is useful when a user explicitly picks a language in your UI (for example, a locale switcher), but it also means a mis-specified `locales` value can override your carefully tuned index settings for that one search.
</Note>

## Monitoring search quality across languages

When serving multiple languages, search quality can vary between them. [Meilisearch Cloud analytics](/capabilities/analytics/overview) can help you identify issues such as high no-result rates or low click-through rates for specific languages, so you can fine-tune settings per language or adjust your indexing strategy.

## Conclusion

Handling multilingual datasets in Meilisearch requires careful planning of both indexing and querying.
By choosing the right indexing strategy, and explicitly configuring languages with `localizedAttributes` and `locales`, you ensure that documents and queries are processed consistently.

## Next steps

<CardGroup cols={2}>
  <Card title="Tokenization" href="/capabilities/indexing/advanced/tokenization">
    Learn how Meilisearch breaks text into tokens for different languages.
  </Card>

  <Card title="Indexing best practices" href="/capabilities/indexing/advanced/indexing_best_practices">
    Tips to speed up the indexing process and optimize performance.
  </Card>

  <Card title="Getting started with indexing" href="/capabilities/indexing/getting_started">
    Add your first documents and configure your index settings.
  </Card>
</CardGroup>