If you’ve just played around with Meilisearch using our quick start guide’s [movies dataset](https://www.meilisearch.com/movies.json), indexing your data may have taken Meilisearch only a few seconds. But if you worked with a larger dataset, it has probably taken a lot longer than that. In this article, we will review the best practices to help you index your data efficiently and speed up the indexing process.

Define your needs
-----------------

Meilisearch stores data in the form of discrete records, called [documents](https://docs.meilisearch.com/learn/core_concepts/documents.html), and each document must have a unique identifier—the [primary key](https://docs.meilisearch.com/learn/core_concepts/primary_key.html). Documents are grouped into collections, called [indexes](https://docs.meilisearch.com/learn/core_concepts/indexes.html).

To offer a search-as-you-type experience, Meilisearch needs to store and organize data in multiple manners so it can retrieve it in the most efficient way possible. Because of this, documents must undergo thorough processing before they are ready to be searched.

There are around twenty data structures per index in Meilisearch and constructing them is the most time-consuming part of the indexing process. Changing index settings may invalidate many of these data structures and require reindexing your data. Because of that, it’s usually a good idea to define the index settings before adding your documents.

### Searchable attributes

By default, all document fields added to Meilisearch are [searchable](https://docs.meilisearch.com/learn/configuration/displayed_searchable_attributes.html#the-searchableattributes-list). However, the words present in the fields listed in the [searchable attributes](https://docs.meilisearch.com/learn/configuration/displayed_searchable_attributes.html#searchable-fields) are the ones that require the most data structures–eleven, to be exact. A good way to speed up indexing is to make sure all the attributes in the searchable attributes list are the ones you really want to be checked for matching query words.

Let's take, for example, a document containing a field with an image URL. You probably want to display the image to the user, but I doubt that users are interested in being able to search for query terms in the URL. Don’t forget that not all [displayed fields](https://docs.meilisearch.com/learn/configuration/displayed_searchable_attributes.html#displayed-fields) need to be searchable!

It is not only important to specify in the list of searchable attributes the fields that are actually required, but it’s also essential to avoid having meaningless, random, or unique values in the searchable fields. Imagine all those databases out there full of ‘https’, ‘www’, ‘com’, or values like ‘I77lHE’—not exactly useful when trying to find a specific product or movie 😱

Speaking of which: 🤔unique values…does that ring a bell? 💡The primary key! That’s another field you can safely remove from the searchable attributes list.

#### Under the hood

To better understand the importance of customizing the searchable attributes index setting, let’s take a look at the largest data structures needed for the searchable attributes. Out of the eleven we mentioned, there are three that take the longest time to build: `WORD_DOCIDS`, `WORD_POSITION_DOCID`, and `WORD_PAIR_PROXIMITY_DOCIDS`.

To better understand how each data structure works, we’ll use the following set of documents as examples:

```JSON
{
    "id": 1,
    "description": "New York City is the most populous city in the USA"
},
{
    "id": 2,
    "description": "New York was named in honor of the Duke of York"
},
{
    "id": 3,
    "description": "Tel Aviv is the new most expensive city in the world"
}

```

In `WORD_DOCIDS`, each word is associated with the [primary key](https://docs.meilisearch.com/learn/core_concepts/primary_key.html) of the documents that contain it:

*   `“new” => [1, 2, 3]`
*   `“york” => [1, 2]`

In `WORD_POSITION_DOCID`, the keys are words together with their position in the document. The values associated with the keys are the documents where this word occupies the same position. In the documents above, `id` would be the attribute 0, and `description` the attribute 1:

*   `new(1,0) => [1, 2]` : in documents 1 and 2, the word “new” is located in attribute 1, position 0–it’s the first word of the attribute `description`
*   `new(1, 4) => [3]`: in document 3, “new” is located in attribute 1, position 4

Finally, in `WORD_PAIR_PROXIMITY_DOCIDS` Meilisearch keeps track of the distance between pairs of words across all documents in an index. Terms must be within 8 words of each other to be stored, as words further away are not considered part of the same context and, therefore, not relevant.

In the example documents above, Meilisearch would store the following pairs:

*   `newyork1 => [1, 2]`
*   `newcity2 => [1]`
*   `newcity3 => [3]`

The numbers attached to the word pairs represent the distance between them:

*   1 means the words are next to each other
*   2 means they are separated by one word
*   3 means they are separated by two words

As you can see, each new word represents extra rows in Meilisearch’s internal data structures. Values like unique ids or URL strings can make the databases grow tremendously–and most likely unnecessarily.

Refining which fields Meilisearch should search is critical to reduce indexing time. It can also lead to greater relevance and search speed, as the results won’t be contaminated with irrelevant data.

### Filterable and sortable attributes

Certain fields don’t contain any words, but might still be crucial to help your users find the results they need. This type of data might be a perfect match for filtering and sorting than regular text-based search.

[Filterable attributes](https://docs.meilisearch.com/learn/configuration/settings.html#filterable-attributes) are attributes that can be used as filters to refine search results. You can use them to restrict search results for a particular user or to create faceted search interfaces that allow the user to narrow down the list of results according to the criteria of their choice. Boolean-type values make great filters.

[Sortable attributes](https://docs.meilisearch.com/learn/configuration/settings.html#sortable-attributes) are attributes that can be used for sorting at search time, which allows users to decide which documents they want to see first. Numeric values are perfect for sorting.

As a rule of thumb, if your dataset contains documents with numeric and boolean field values, take the time to assess if they can be part of the filterable or sortable attributes lists.

### Ranking rules

Ranking rules are responsible for the relevancy of your search results. Meilisearch contains six built-in ranking rules:

```json
[
  "words",
  "typo",
  "proximity",
  "attribute",
  "sort",
  "exactness"
]
```

You can also add custom rules for ascending and descending sort to this list:

```json
[
  "words",
  "typo",
  "proximity",
  "attribute",
  "sort",
  "exactness",
  "price:asc"
]
```

Unlike sortable attributes, which are used to sort at search time, custom ranking rules are used to set a default order.

If you need this kind of default sorting, it is better to configure it beforehand. Adding a new rule after documents have already been indexed will trigger a reindexing and potentially your nerves!

Size down and batch up
----------------------

Index only what you really need. Are all the columns in your database necessary? The fewer columns in your dataset, the smaller the documents. The smaller the documents, the less they will weigh, and the faster they will reach Meilisearch, which will then be able to process them sooner.

Meilisearch combines consecutive document addition requests into a single batch and processes them together. This significantly speeds up the indexing process. Since the final size of each batch will depend on how much data Meilisearch received while it was processing the latest document addition request, it is therefore encouraged to send documents in groups rather than one by one.

For the same reason, consider compressing your data. Meilisearch supports the `br`, `deflate`, and `gzip` compression methods. You can find more information in our [documentation](https://docs.meilisearch.com/reference/api/overview.html#content-encoding).

Stay up to date and keep us updated
-----------------------------------

Use the latest [stable version of Meilisearch](https://github.com/meilisearch/meilisearch/releases). New releases often include performance improvements that can significantly increase indexing speed.

If you experience any indexing issues, please [report them](https://discord.gg/meilisearch)! It’s key to improving the product!

Don’t trust me, trust the data
------------------------------

While creating some benchmarks, [Many](https://github.com/ManyTheFish), software engineer of the Engine team, created the following graph. Please note that the indexing time is highly dependent on the size of the machine (CPUs, RAM) and the dataset: for a similar-sized dataset, you may obtain different results.

![Line graph that compares the indexing time with two different configurations (default and custom) in relation to the dataset size. In all cases, the speed is higher with a custom configuration.](https://lh4.googleusercontent.com/l8KRD6C64sT2wPL39Zx6EVzzqk1UOIbYUItL9uWSTT2hd1vfPMEp-j_yMCAXciebMOZQBm9IyVp5XnJHk7XWt3hVl4QUiqXHkRReBq3ERs-jZwVw6QQgU-_eircOWWxu0-VmCRGrQy3F8OTqvQOjU-Z-iCVi1ECgyqesCnTQhG4s-OWM-mz0zJQZ6oNfSw)

The horizontal axis represents the dataset's size, and the vertical axis represents the indexing time

The red line represents indexing time using default settings, and the blue line represents indexing time using a custom configuration. With the same machine and dataset, we observed a 36% gain in indexing speed when using custom settings.

We observed similar improvements in database size, which was considerably smaller when using custom settings:

![Bar graph of the database size where the database with custom settings is significantly smaller than the one with default settings.](https://lh3.googleusercontent.com/5whDEpcglxFIPRfTzHqmJudSWis4ig2BMfvSwJT5LrLVjp43PnsHXRBvJrsH9RTBkKYDjHuOPAcfu4ZWsUIuZnVNJGepYzRG_WD0zvkMoEMZyBGZcSuJcdFELNi96tzagnfe_H9Q8v7SG-kE7H8ysXfRxMTQyi-_Wmw5JPak5kwsQWdLo6QriHLyYN-n8Q)

Database size with custom settings in blue and default settings in red

* * *

Learn more about what Meilisearch can bring to your business

[Talk to a search expert](https://dub.sh/meili-sales)

* * *

And that’s all for now! We covered the best practices to speed up the indexing process. Did you know any of these tips? Have you noticed a difference in indexing speed after following them? Share your experience on our [Discord](https://discord.meilisearch.com/)!

Your feedback is key to helping us improve Meilisearch! I can't stress this enough and I never tire of repeating it, because it is the truth. Our community has built Meilisearch with us and continues to help us shape it.

[Issues](https://github.com/meilisearch/meilisearch/issues/new/choose) or [product discussions](https://github.com/meilisearch/product/discussions) on GitHub, our [roadmap](https://roadmap.meilisearch.com), our [Discord server](https://discord.meilisearch.com/)… No matter where, no matter how, we want to hear from you!

Are you indexing the smart way?

Hi, I’m [Quentin de Quelen](/blog/author/quentin/)he CEO of Meilisearch, an open-source search engine API built in Rust. At Meilisearch, we are committed to the open-source community and believe in the power of collaboration to drive innovation. This year, we made significant strides in delivering the best possible search experience to our users and customers, including raising funds, launching new products in beta, and continuing to optimize our engine and improve our API.

This article will highlight some of the major improvements that we made in 2022.

New product: Meilisearch Cloud
------------------------------

Back in March of 2022, we communicated for the first time about [Meilisearch Cloud](https://cloud.meilisearch.com/), our fully-managed search engine solution for organizations of all sizes. This new offering makes it easy and fast to set up a search engine without technical expertise or infrastructure management.

After almost a full year of closed beta, [we opened Meilisearch Cloud](/blog/the-history-of-meilisearch-cloud/)o the world at the end of 2022. While there are still a few improvements coming before we officially release the product—such as usage-based price which aims to give users the flexibility to scale their search engine as needed—we’re tremendously proud of the progress we made in 2022.

We believe Meilisearch Cloud will revolutionize the search industry and are dedicated to continuously improving and evolving the product. We would love to hear your feedback on this new offering and are excited to see how it can benefit our users and customers.

Strengthening the search engine
-------------------------------

In 2022, we released six major versions of the Meilisearch engine: [v0.25](/blog/whats-new-in-v0-25/)[v0.26](/blog/whats-new-in-v0-26/) [v0.27](/blog/whats-new-in-v0-27/) [v0.28](/blog/whats-new-in-v0-28/) [v0.29](/blog/whats-new-in-v0-29/) and [v0.30](/blog/whats-new-in-v0-30/)ach release brought significant new features, performance improvements, and quality-of-life changes. This resulted in a **fivefold increase** in the number of Meilisearch users in only one year.

![](https://unable-actionable-car.media.strapiapp.com/monthly_active_databases_9dfee93d83.png)

Usage of Meilisearch over 2022

### New features

********************************Nested documents********************************  
This year, Meilisearch gained full support for [nested document fields](https://docs.meilisearch.com/learn/advanced/datatypes.html#objects). This feature allows users to access and use nested fields in any API parameter that accepts document attributes, simply by using dot notation. We want users to be able to take advantage of Meilisearch's capabilities without ever having to format their data.

******************API keys and tenant tokens******************  
We introduced API key management in Meilisearch in 2022. With [tenant tokens](https://docs.meilisearch.com/learn/security/tenant_tokens.html#multitenancy-and-tenant-tokens), developers can now create JWTs (JSON Web Tokens) on the fly that restrict access to specific documents based on the end user making the search request. This allows for multi-tenant indexes and gives developers more control over the search experience for their users.

**************New filter operations**************  
We've added two new operators to the [existing filtering capabilities](https://www.meilisearch.com/docs/learn/advanced/filtering): `EXISTS` and `IN`. When combined with the `NOT` (`!`) operator, they provide greater flexibility and precision in search results. This makes it easier for users to find what they're looking for and get the most out of Meilisearch's powerful search capabilities.

************************************Configuration file************************************  
Meilisearch now accepts a [configuration file](https://docs.meilisearch.com/learn/configuration/instance_options.html#configuration-file) in the `.toml` format as an alternative to command-line options and environment variables. Configuration files can be easily shared, versioned, and allow for the definition of multiple options.

************************************Full pagination support************************************  
Pagination is critical for some applications, but for a long time Meilisearch was not well equipped for [pagination-based navigation](/blog/pagination-vs-infinite-scroll-vs-load-more/)o address this, we introduced two new search parameters: `page` and `hitsPerPage`. Queries with these parameters return the exhaustive number of `totalHits` and `totalPages`, which can be used to create UI elements such as numbered page selectors.

### Performance improvements

Coming into 2022, we knew that indexation performance was one of the biggest issues with Meilisearch. While we were incredibly fast on search, this brought with it a tradeoff on indexing.

A few months ago, we recommended to people with more than 10M documents not to use Meilisearch, as the time to index their documents would be absurd. I'm proud to say that this is no longer the case!

********************************************Optimized task manager********************************************  
This year, we introduced [auto-batching](https://docs.meilisearch.com/learn/core_concepts/documents.html#auto-batching) of tasks, which has greatly improved performance. This was accompanied by a completely rewritten task manager that adds a lot of intelligence to task queue management. It can recognize different tasks, reorganize them, and even delete unnecessary ones. This new task manager also enabled us to add features like task [cancellation](https://docs.meilisearch.com/reference/api/tasks.html#cancel-tasks) and deletion, and [index swapping](https://docs.meilisearch.com/reference/api/indexes.html#swap-indexes) for zero downtime deployments.

****************Optimized indexation****************  
Our team has also worked hard to improve indexation speed. In brief, we added a soft deletion mechanism that improves both deletion and document update, optimized all aspects of filtering and sorting, and optimized RAM and CPU consumption during indexation.

This work is not yet complete; we have many more ideas to improve indexation performance. Nonetheless, we believe that the progress we made this year was significant! If you are still having issues with indexation speed, please reach out to us on [Discord](https://discord.meilisearch.com).

### Better developer experience

Our engine is built on three pillars: relevancy, speed, and developer experience, or DX. We have always prioritized DX as we made changes to the engine, but this came with the trade-off of introducing many breaking changes as we pursued an intuitive HTTP API.

Many API endpoints changed in 2022. The first is the search endpoint; we have completely reimplemented the [cropping](https://docs.meilisearch.com/reference/api/search.html#attributes-to-crop) of fields, with something smarter that looks at the field context to crop in the best place. We also, as mentioned before, changed the task management system entirely with a new API that allows canceling and deleting tasks.

The good news is that the breaking changes are almost over: **stability is coming to Meilisearch’s API**. You won't have to wait long for [v1](https://github.com/meilisearch/meilisearch/milestone/37), as it's planned for February 2023!

### Overall improvements

In 2022, our team implemented several enhancements to make Meilisearch more customizable and user-friendly.

****************************Typo tolerance****************************  
In v0.27, we added the ability to [customize the typo tolerance](https://docs.meilisearch.com/learn/configuration/typo_tolerance.html#configuring-typo-tolerance) applied to search queries. This allows users to set the level of tolerance they desire for misspellings, making it easier to find relevant results even if the query is not perfectly accurate.

**********************************************Improved cropping and highlighting**********************************************  
In addition, Meilisearch now allows users to customize how search results are displayed, including the level of [cropping](https://docs.meilisearch.com/reference/api/search.html#crop-marker) and [highlighting](https://docs.meilisearch.com/reference/api/search.html#highlight-tags) applied to the results. You can now choose your own highlight tags and put the HTML ones you need, or [select the number of words](https://docs.meilisearch.com/reference/api/search.html#crop-length) you want to keep while cropping.

******************************************Search result limits******************************************  
Meilisearch has been improved to allow for [customizable limits](https://docs.meilisearch.com/reference/api/settings.html#pagination) on the number of search results returned. This can help optimize search performance and limit the amount of data returned, as well as improve security practices.

**********************Matching strategy**********************  
Finally, Meilisearch has added the ability to customize the [matching strategy](https://docs.meilisearch.com/reference/api/search.html#sort) used to match search queries to documents. You can now choose between two different strategies: one that requires all words present in the query to be present in matches, and one that accepts results that contain only some of those words.

**************************New language optimizations**************************  
Meilisearch has recently added Japanese, Hebrew, and Thai to its already extensive [list of supported languages](https://docs.meilisearch.com/learn/what_is_meilisearch/language.html#language). Contributors can now more easily propose new languages to support, helping the Meilisearch team to continue expanding its language offerings.

A growing toolbox
-----------------

I could not be more proud to showcase the accomplishments of our integration team, the team at Meilisearch responsible for creating new integrations and diligently updating existing ones.

In 2022, our integration team:

*   Developed three major integrations: [Firebase](/blog/firebase-meilisearch/)Gatsby](/blog/add-a-search-bar-to-your-gatsby-project-using-meilisearch/ and [Strapi](/blog/strapi-v4-plugin-meilisearch/)   Kept our 10 officially supported SDKs up to date: [JavaScript,](https://github.com/meilisearch/meilisearch-js) [PHP,](https://github.com/meilisearch/meilisearch-php) [Python,](https://github.com/meilisearch/meilisearch-python) [Ruby,](https://github.com/meilisearch/meilisearch-ruby) [.Net,](https://github.com/meilisearch/meilisearch-dotnet) [Dart,](https://github.com/meilisearch/meilisearch-dart) [Golang,](https://github.com/meilisearch/meilisearch-go) [Java,](https://github.com/meilisearch/meilisearch-java) [Rust,](https://github.com/meilisearch/meilisearch-rust) [Swift](https://github.com/meilisearch/meilisearch-swift)
*   Updated integrations for three frameworks: [Laravel Scout](https://github.com/laravel/scout), [Symfony](https://github.com/meilisearch/meilisearch-symfony), [Ruby on Rails](https://github.com/meilisearch/meilisearch-rails)
*   Updated five frontend integrations: [Angular](https://github.com/meilisearch/meilisearch-angular), [React](https://github.com/meilisearch/meilisearch-react), [Vue](https://github.com/meilisearch/meilisearch-vue), [instant-meilisearch](https://github.com/meilisearch/meilisearch-js-plugins/tree/main/packages/instant-meilisearch), [doc-searchbar.js](https://github.com/meilisearch/docs-searchbar.js)

![](https://unable-actionable-car.media.strapiapp.com/integrations_meilisearch_2e28ef98ef.png)

Meilisearch integrations

These achievements demonstrate our dedication to providing our users and customers with smooth integration options for their search needs. With the upcoming release of Meilisearch v1, our integration team will be able to focus even more on new integrations and projects. So, don't forget to give us [feedback](https://roadmap.meilisearch.com/tabs/1-under-consideration) on which integrations you'd like to see next!

A constant focus on the community
---------------------------------

2022 has been an exciting year for the Meilisearch community. We saw tremendous growth in engagement and contributions, resulting in significant improvements to our search engine.

This year, the number of stars on our main repository reached an impressive 31.5k. We had approximately 500 external contributors, who together merged over 1000 pull requests. This level of participation and collaboration has been instrumental in driving innovation and enhancing the Meilisearch experience for our users and customers.

### Special events

In addition to our regular contributions, we had a successful [Hacktoberfest](/blog/hacktoberfest-2022-recap/)his year. This global event brings together developers to work on open-source projects, and we were delighted to be one of them. The influx of talent and energy from Hacktoberfest resulted in improvements to our SDKs, integrations, and engine.

We also participated in [Advent of Code](https://aoc.meilisearch.com) for the first time. This programming challenge is a great way to engage with the community and encourage problem-solving and collaboration. We are always looking for ways to foster a strong and engaged community, and Advent of Code was a great opportunity to do so.

Finally, we recently migrated our community from [Slack to Discord](/blog/from-slack-to-discord-our-migration/)his move has been highly anticipated, and we are excited about the potential for increased communication and collaboration. We believe Discord will be a great place for our community to share ideas, get help, and work on projects.

### Documentation

In addition to our work in the community, we also made significant strides forward in our [documentation](https://docs.meilisearch.com/) in 2022.

Besides keeping the Meilisearch documentation up to date, this year our documentation team published many new guides and tutorials, including the [Meilisearch 101](https://docs.meilisearch.com/learn/getting_started/filtering_and_sorting.html), a brief overview of everything you need to know to create with Meilisearch. We also released a guide on [migrating from Algolia](https://www.meilisearch.com/docs/learn/update_and_migration/algolia_migration), a [benchmark](https://docs.meilisearch.com/learn/what_is_meilisearch/comparison_to_alternatives.html#comparison-table) of how Meilisearch stacks up against the competition, and tons of guides related to new features such as [API keys](https://docs.meilisearch.com/learn/security/master_api_keys.html), [tenant tokens](https://docs.meilisearch.com/learn/security/tenant_tokens.html), and [pagination](https://docs.meilisearch.com/learn/advanced/pagination.html). All of these resources are designed to help our users and customers get the most out of Meilisearch.

Our documentation team also undertook a major review and reorganization of our documentation, with the goal of making it even more user-friendly and comprehensive. This is just the start of our efforts in this area, and we are committed to continuously improving the quality and accessibility of our documentation.

### Blog

Throughout 2022, we continued to release high-quality articles on our blog. It’s impossible to choose just one post, but some highlights include [a comparison of different types of pagination](/blog/pagination-vs-infinite-scroll-vs-load-more/)the announcement of our $15M Series A](/blog/meilisearch-series-a/ an explanation of [how we implemented a salary grid at Meilisearch](/blog/our-approach-to-salaries/)nd [a lovely series of interviews](/blog/tag/meet-the-team/)ith Meilisearch employees. We also published numerous [demos, guides, and tutorials](/blog/tag/using-meilisearch/)!

We love creating content for our blog and sharing stories from within the company. If you enjoyed one of our blog posts this year, [let us know](https://twitter.com/meilisearch) which one!

Being a better company
----------------------

This year, we focused on our employees' well-being. First of all, we implemented a transparent salary grid to ensure fair pay for all team members. Additionally, we established a comprehensive mental health support program with resources and assistance for those who need it.

Inclusivity is a core value at Meilisearch. We've placed great emphasis on creating an inclusive work environment and culture. This is evident in our recruitment process, which includes scorecards to reduce bias and aims to select candidates with diverse backgrounds, experiences, and skills.

At Meilisearch, we believe that by investing in the growth and development of our employees, we can create a stronger and more successful company. To support this growth, this year we implemented an improved feedback loop that allows for ongoing communication and coaching between employees and their managers. By understanding our employees' aspirations better, we can provide them with the support and resources they need to succeed, both at Meilisearch and in the rest of their careers.

Conclusion
----------

To wrap this up, it has been an outstanding year for Meilisearch. I am grateful to every single member of the team for the work they put in to make our search engine a success. I also want to thank the community for their invaluable support and contributions. I think I speak for everyone at Meilisearch when I say that we look forward to bringing this forward momentum with us into 2023 and continuing to provide our users and customers with the best possible search experience.

Looking ahead to the coming year, we are excited to continue delivering new and improved features to our users and customers. If you have any feedback or suggestions for our [roadmap](https://roadmap.meilisearch.com), we encourage you to reach out and let us know what you need. We are always looking for ways to improve and evolve, and your input is an important part of that process. Thank you again for your support, and we look forward to continuing to work with you in 2023!

Meilisearch in 2022

Today we’re sitting down with [Clémentine](https://www.notion.so/Cl-mentine-Urquizar-90214cbb5f1e4fa583546c7ed2be1362), Manager of the Engine Team at Meilisearch, to talk about how the open-source community shaped our company. Clémentine was among the first developers hired by Meilisearch, back in the distant times of 2019. She’s been with us since the very beginning and saw the company go from a pure SaaS model to fully embracing open source.

From SaaS to Open Source
========================

To many familiar with Meilisearch, it might be surprising to hear we initially launched our product as a SaaS. Clémentine explains the change:

_”Back in the day, the engine and the SaaS built around it were buggy, user experience was poor, documentation non-existent, and users required many advanced features from the get-go—they were expected to pay for our product, so of course they wanted a full fledged search engine. In addition to all this, being a paid service meant we didn’t always receive the relevant feedback needed to improve and grow as a company: what worked well, what didn’t, or what could work much better with a tiny bit of adjustment.”_

The founders of Meilisearch quickly realized their business model wasn’t sustainable and decided to do a 180: make Meilisearch open-source.

![](https://unable-actionable-car.media.strapiapp.com/Screenshot_2023_01_04_at_11_11_32_92c18bbda3.png)

GitHub chart of contributions to Meilisearch

Clémentine tells us that such a drastic change of direction was equally exciting and frightening. Exciting because the modus operandi was very different from what they’d been doing up until that point. Frightening because they weren’t sure the project was going to survive.

_“I decided to join the company because I had faith in the founders and their ideas. I believed in what they were trying to accomplish and that the people working in Meilisearch were competent enough to make it happen. Seeing things fail at that stage would have been seriously heartbreaking.”_

Working on a now-open-source project, Meilisearch put its focus on growing the project by making it more reliable and performant, and adding various integrations. This meant we wouldn’t generate revenue for a while, and thus that our survival depended on a successful fundraiser. To everyone’s relief, it was a fundraiser the founders did manage to secure.

In the day-to-day reality of development, the change from SaaS to open source wasn’t as difficult, but still very much radical. Clémentine shares the surprise she’d felt when she first saw people actually participating in the project through PRs such as [this](https://github.com/meilisearch/meilisearch-php/pull/13) or [this](https://github.com/meilisearch/meilisearch-php/pull/43): experienced and humble programmers who enjoyed seeing projects evolve and grow began to contribute. PHP and Rust developers were the first to bring their enthusiasm and support, she recalls.

_“I would never have guessed how much the open-source community would shape Meilisearch, but it was the best possible decision the founders could have taken.”_

A new structure in Meilisearch
==============================

These new and exciting times required the team to rethink itself to ensure a successful shift from SaaS to open source: we had to foster our community, we had to offer good documentation, and the engine itself had to become easier to use. 

Transparency became our focus. There’s nothing better for a company than live feedback: we get to know what is important for our users while still being able to decide how to implement new features in a way that moves Meilisearch in the right direction—the last one is a skill we’ve only learned to master with time. It was paramount to ensure our community was happy with our decision as our growth was (and still is) closely linked to their involvement. 

Since transparency is not the result of a single action but of continued effort, we used the money from our first fundraiser to help us nurture our open-source community. We hired technical writers to create great documentation, developer advocates to improve communication between company and users, and additional programmers to maintain repositories for our growing number of integrations. All of this was necessary to cultivate an ecosystem that made sure we were going in a direction that made sense to our contributors.

![](https://unable-actionable-car.media.strapiapp.com/star_history_202314_34fe7a3a6d.png)

Evolution of the number of Meilisearch's GitHub stars

Clémentine’s scope moved to integration and in November 2020 she became manager of the Integrations Team—the first official manager in Meilisearch! Shortly after, she also took on the responsibility of managing the Engine Team. From that point on, Meilisearch went from a setup where “everyone does everything” to real teams with synchronization and dedicated scopes.

She confesses she did not expect to become a manager, even if in hindsight her natural predisposition toward organizing people was clear to the founders:

_“I was a bit surprised, but the more I worked at Meilisearch the more I realized I liked pushing projects and people forward, seeing things growing and becoming the best versions of themselves. I don’t have much time to code anymore—only occasional bug fixes or CI work—but I really have no regrets. In retrospect I can see I wasn’t fully satisfied as a full-time developer. But it took some time to make peace with that idea, and to close that chapter of my life.”_

The future of Meilisearch
=========================

The next step is the long-awaited Meilisearch v.1.0:

_“Our users deserve to be serene when using our search engine. Releasing v1.0, our first major version, is a big and necessary step to show Meilisearch is stable and ready for production.”_

This goes together with the development of a SaaS setup:

_“Meilisearch has grown so much these past few years. Thinking back at what we had in 2019, when we first tried developing a SaaS, and comparing it with the product we have now is amazing—and it’s all thanks to the open-source community. One of our main goals moving forward will be to keep giving back to all our users, even as we begin to dedicate resources to the SaaS solution.”_

In conclusion
=============

Reflecting back on all that was accomplished, Clémentine shares:

_“Meilisearch grew, grows, and will keep on growing thanks to the open-source community. No matter what we build in the future, the open-source community is our biggest strength. We will always need our contributors, and we intend to do our best to keep them in the loop and show our appreciation for their support.”_

Open source became part of our DNA. Its values—asynchronicity, pragmatism, transparency—are also reflected in our company policies and the way we operate internally, from our programmers to our managers, writers, and developer advocates.

Clémentine herself never stopped growing alongside the organization: after starting as an intern, she became a full fledged developer, then moved on to lead the Engine Team. Finally, she’s just recently been promoted to the position of Head of Engineering, overseeing the work of programmers across the company !

We wish to extend a warm thank you to Clémentine for her candor and to the contributors that allowed us to grow and evolve as we did.

How Meilisearch became open-source: a retrospective

The following guide will walk you through the steps required to update your Meilisearch instance on DigitalOcean, AWS, or GCP when using the Meilisearch official images.

Before you begin, please note the following:

*   This guide does not work for versions below v0.15. Since dumps were introduced in v0.15, you’ll need to reindex all your documents and settings manually if you are on v0.14 or below
*   If you are updating to v0.28 or above, keys imported from the old version will have their `key` and `uid` fields regenerated
*   You’ll need to connect via SSH to your cloud instance, and depending on the user you are connecting with (root, admin, etc.), you may need to prefix some commands with `sudo`

Also, if you are using v0.22 or above, you can use our [migration script](https://github.com/meilisearch/meilisearch-migration) and let it do all the work for you!

Step 1: Check your Meilisearch version
--------------------------------------

You need to know your Meilisearch version. If you don’t know or are not sure, you can use the get version endpoint:

```bash
curl \
  -X GET 'http://<your-domain-name>/version' \
  -H 'Authorization: Bearer API_KEY'

```

If you get a `missing_authorization_header` error code, you might be on v0.24 or below. Change the authorization header to `X-MEILI-API-KEY` as follows:

```bash
curl \
  -X GET 'http://<your-domain-name>/version' \
  -H 'X-Meili-API-Key: API_KEY'  

```

Check your `pkgVersion` if it's “0.21.0” or higher, you can go straight to [step 3](#step-3-create-the-dump). If not, please proceed to the next step.

Step 2: Set all fields as displayed attributes
----------------------------------------------

<Callout>
  This step is only mandatory if you are on v0.20 or below.
</Callout>

‌‌

When creating dumps using Meilisearch versions below v0.21, all fields must be [displayed](https://docs.meilisearch.com/learn/configuration/displayed_searchable_attributes.html#displayed-fields) in order to be saved in the dump.

First, you need to verify the displayed attributes list.

```bash
# whenever you see {index_uid}, replace it with your index's unique id
curl \
  -X GET 'http://<your-domain-name>/indexes/{index_uid}/settings/displayed-attributes' \
  -H 'X-Meili-API-Key: API_KEY'  

```

If the response includes `{'displayedAttributes': '["*"]'}`, you can move on to [step 3](#step-3-create-the-dump).

‌‌If it's something else, save the current list to restore it after the migration is complete, and proceed to reset the list of displayed attributes to its default value (`[“*”]`):

```bash
curl \
  -X DELETE 'http://<your-domain-name>/indexes/{index_uid}/settings/displayed-attributes' \
  -H 'X-Meili-API-Key: API_KEY'  

```

This command returns an `updateId`. You can use this to track the status of the operation.

```bash
# replace {updateId} with the updateId returned by the previous request
curl \
  -X GET 'http://<your-domain-name>/indexes/{index_uid}/updates/{updateId}' \
  -H 'X-Meili-API-Key: API_KEY'  

```

Once the status is `processed`, you're good to go.

Now that all fields are displayed proceed to the next step.

Step 3: Create the dump
-----------------------

Before creating your dump, make sure that your [dump directory](https://docs.meilisearch.com/learn/configuration/instance_options.html#dumps-destination) is accessible. By default, dumps are created in a folder called dumps in the configuration file directory: `/var/opt/meilisearch/dumps`.

‌‌You can then create a dump of your Meilisearch database:

```bash
# replace the header with -H 'X-Meili-API-Key: API_KEY' if you are on v0.24 or below
curl \
  -X POST 'http://<your-domain-name>/dumps' \
  -H 'Authorization: Bearer API_KEY' 


```

‌‌If you are on v0.27 or below, dumps use a separate queue from the task queue. The response should return a dump `uid`. You can use it to track the status of the dump with the get dumps status endpoint:

```bash
# replace the header with -H 'X-Meili-API-Key: API_KEY' if you are on v0.24 or below
curl -X GET 'http://<your-domain-name>/dumps/{uid}/status' \
  -H 'Authorization: Bearer API_KEY' 


```

Once the dump status is `done`, you can move on to the next step.

```JSON
{
    "uid":"20221130-110940190",
    "status":"done",
    "startedAt":"2022-11-30T11:09:40.190772Z",
    "finishedAt":"2022-11-30T11:09:45.94484Z"
}

```

‌‌‌‌If you are on v0.28 or above, dumps are considered tasks and share the task queue. The response should then return a `taskUid` you can use to track its status with the get task endpoint:

```bash
curl \
  -X GET 'http://<your-domain-name>/tasks/{taskUid}' \
  -H 'Authorization: Bearer API_KEY’ 

```

‌‌Once the `dumpCreation` task shows `"status": "succeeded"`, you're ready to move on to the next step.

```json
{
  "uid": 1,
  "indexUid": null,
  "status": "succeeded",
  "type": "dumpCreation",
  "canceledBy": null,
  "details": {
	"dumpUid": "20220621-161029217"
  },
  "error": null,
  "duration": "PT0.025872S",
  "enqueuedAt": "2022-06-21T16:10:29.217688Z",
  "startedAt": "2022-06-21T16:10:29.218297Z",
  "finishedAt": "2022-06-21T16:10:29.244169Z"
}

```

‌Step 4: Stop the Meilisearch instance
--------------------------------------

If you haven’t done it yet, connect via SSH to your cloud instance and execute the following command to stop Meilisearch. Remember you may need to prefix it with `sudo` if you are not connected as root.

```bash
systemctl stop meilisearch
```

‌‌Step 5: Save the current binary and database for backup
---------------------------------------------------------

Move the binary of the current Meilisearch version and the database to the tmp/ folder:

```bash
mv /usr/bin/meilisearch /tmp
mv /var/lib/meilisearch/data.ms /tmp/

```

‌Step 6: Install the desired version of Meilisearch
---------------------------------------------------

‌‌‌‌You can use this command to download the Meilisearch binary. Replace `{meilisearch_version}` with the version of your choice formatted like this: `vX.X.X`.

```json
curl "https://github.com/meilisearch/meilisearch/releases/download/{meilisearch_version}/meilisearch-linux-amd64" --output meilisearch --location --show-error
```

Make the meilisearch binary executable:

```bash
chmod +x meilisearch
```

Move the new Meilisearch binary to the directory containing the executable files:

```bash
mv meilisearch /usr/bin/meilisearch
```

Step 7: Launch Meilisearch and import the dump
----------------------------------------------

‌Now that you’ve got the desired Meilisearch version, execute the command below to import the dump at launch.

Don’t forget to replace the `dump_uid.dump` with the actual dump file name.

```json
meilisearch --db-path /var/lib/meilisearch/data.ms --import-dump "/var/opt/meilisearch/dumps/dump_uid.dump"
```

Importing a dump requires indexing all the documents it contains. Depending on the size of your dataset, this process can take a long time and cause a spike in memory usage.

Step 8: Restart Meilisearch as a service
----------------------------------------

Once your dump has been correctly imported, press `Ctrl`+`C`  to stop Meilisearch. Next, execute the command below to run the script to configure Meilisearch and restart it as a service:

```json
meilisearch-setup
```

Don’t forget to set `displayedAttributes` back to its previous value if necessary. You can do this using the [update displayed attributes endpoint](https://docs.meilisearch.com/reference/api/settings.html#update-displayed-attributes).

🎉 Congratulations! You have successfully migrated your Meilisearch database to the latest version!

Step 9: Clean files or rollback
-------------------------------

### Clean files

If you’ve gone through all the previous steps and Meilisearch is up and running, it’s time to do some cleanup. Remember those files we saved for backup? You can delete them with the following commands:

```bash
rm /tmp/meilisearch
rm -r /tmp/data.ms
```

‌Since you no longer need it, you can also delete the dump file:

```bash
rm /var/opt/meilisearch/dumps/{dump_uid.dump}
```

### Ro‌‌llback

If for some reason, something went wrong, you can always roll back to the previous version. Move the files back to their previous location using:

```bash

mv /tmp/meilisearch /usr/bin/meilisearch
mv /tmp/data.ms /var/lib/meilisearch/data.ms
```

‌And run the configuration script:

```bash
meilisearch-setup
```

‌‌Everything should be as it was. Phew!

I hope this guide has helped you to enjoy all the advantages of the latest version of Meilisearch. Don’t forget our support team is always ready to help! We have a dedicated support channel on our brand new [Discord](https://discord.gg/meilisearch) server!

‌‌‌‌‌‌‌‌‌‌

How to update your app to the latest Meilisearch version

v0.30 of Meilisearch was released earlier this month. With it, our primary goal was to make the Meilisearch workflow as smooth as possible. After all, we are on the road to v1, and our destination is closer than ever. For that, having index deployment with zero downtime was a crucial step. We’re happy to say that v0.30 solves this problem, making it easier than ever to update an index in production. Read on to learn more about how the feature works!

  
The challenge
----------------

Suppose you have changes in your database that you need to synchronize with your Meilisearch index in production. What do you do? Well, if you were working on Meilisearch v0.29, you might:

### Update your index

🙅‍♀️  Simple, right? But if you’ve ever tried it, then you should already know…

![It's a trap!](https://media.tenor.com/FtNhFA1nSo0AAAAC/star-wars-its-a-trap.gif)

Updating your production index while it receives search queries can result in inconsistencies with results or even missing information. You don’t want to offer that search experience, do you? Otherwise, you wouldn't be using Meilisearch 😁

### Delete the index, create a new index with the same name, and re-index the data

⌛ Each step will take some time. Even if it’s just a few seconds, it’s a few seconds of downtime… And that’s at least three requests! A few seconds here plus a few seconds there can result in minutes of downtime! No, thank you very much.

### Create a new index with another name, stage the changes there, then modify all the clients to point to the new index

🤔 Well, it’s not the worst option. But again, updating the clients leads to downtime 😞 In some cases, it isn’t even up to the developer when the client update will be pushed to production. Take an iOS app for example: Apple has to review the new version before you can release it, which can be a major drag on your release plan.

### Create an intermediate redirection layer to avoid downtime

🧞 Very clever! While this is a good approach, it's not the most straightforward. It requires architectural knowledge, time, and extra tools.

Aren't you glad you never have to do any of these things again?

The silver bullet
-----------------

Meilisearch has always been committed to offering the best developer experience.

Our team has been working hard for the past months, and after several iterations, they’ve come up with a solution that integrates seamlessly into the developer’s stack: **index swapping**.

Let me show you how it works.

Let’s say you have an index in production—`indexA`—in which your clients search. You want to sync the changes in your primary database with Meilisearch. To do so, you would follow the steps below.

### Step 1: Create a new index with up-to-date data

First, you need to create an index—let’s call it `indexA_new`—representing the new version of `indexA` that you want to deploy to your search clients. Add up-to-date documents from your database, and update the index settings if necessary.

Don’t forget to check that all the tasks related to the index creation have successfully completed, including `indexCreation`, `settingsUpdate`, and `documentAdditionOrUpdate` task types. You can use the `/tasks` [route](https://docs.meilisearch.com/reference/api/tasks.html#tasks) to get information about their progress.

### Step 2: Test the new index

Before sending your index to production, you want to be sure everything is working as expected.

![Meme: image of a city on fire in the middle of a thunderstorm, the text reading: "text in production" they said, "what could go wrong" they said ](https://unable-actionable-car.media.strapiapp.com/test_in_production_meme_1_63b857925c.jpg)

Make sure to update the settings for any new fields you may have introduced with the updated data. For example, you may want to add new fields to `searchableAttributes`, `filterableAttributes`, and/or `sortableAttributes`. Don't forget to double-check the relevancy of search results any time you add new data!

👉 Keep in mind that the `searchableAttributes` list not only designates the fields that are searchable, but also dictates the [attribute ranking order](https://docs.meilisearch.com/learn/core_concepts/relevancy.html#attribute-ranking-order). Make sure any new fields you may have introduced are added to the list in the right order.

### Step 3: Swap indexes

Once your `indexA_new` has been successfully created, filled with data, and tested, it’s ready to be deployed with an **index swap**. To do so, send a `POST` request to the `/swap-indexes` endpoint. Specify the indexes you want to swap in the payload. Since it’s a swap, the order doesn’t matter.

```bash
curl \
-X POST 'http://localhost:7700/swap-indexes' \
-H 'Content-Type: application/json' \
--data-binary '[
	{ "indexes": ["indexA", "indexA_new"] }
]'
```

👉 In a [protected](https://docs.meilisearch.com/learn/security/master_api_keys.html#protecting-a-meilisearch-instance) Meilisearch instance, the API key used to swap indexes must have access to the `indexes.swap` action as well as the indexes you want to swap. If not, Meilisearch will throw an `invalid_api_key` [error](https://docs.meilisearch.com/reference/errors/error_codes.html#invalid-api-key). For more information about creating API keys with specific permissions, see the [documentation](https://docs.meilisearch.com/learn/security/master_api_keys.html#creating-an-api-key).

You can use the response's `taskUid` to track the status of your request with the `GET /tasks/{task_uid}` [endpoint](https://docs.meilisearch.com/reference/api/tasks.html#get-tasks). A successful index swap should look like this:

```JSON
{
  "uid": 23,
  "indexUid": null,
  "status":"succeeded",
  "type":"indexSwap",
  "details":{
	"swaps": [
  	     {"indexes": ["indexA", "indexA_new"]},
	]
  },
  "duration": "PT1S",
  "enqueuedAt": "2021-08-10T14:29:17.000000Z",
  "startedAt": "2021-08-10T14:29:18.000000Z",
  "finishedAt": "2021-08-10T14:29:19.000000Z"
}

```

Your indexes have been swapped without any downtime! The documents, settings, and task history of `indexA`—except for any enqueued tasks—have been swapped with those of `indexA_new`. Every mention of `indexA` in the task history has been replaced by `indexA_new` and vice-versa (`enqueued` tasks are left unmodified).

![Animation illustrating index swapping: an index receives search queries while the another is being created in the background. When the second index is ready, they swap, then the unnecessary index is deleted.](https://unable-actionable-car.media.strapiapp.com/ezgif_4_520c610091_8f72080b3b.gif)

After the swap, `indexA_new` holds the outdated content. You can delete it or keep it as a backup, should something go wrong and you need to swap back. Better safe than sorry!

And that’s it! Only three steps, two if you like to live dangerously. Could this get any better?

The cherries—yes, plural—on the cake 🍰
---------------------------------------

What if I tell you that you can swap several indexes with just one request?

I am not kidding. A single request can swap as many index pairs as you wish. Meilisearch can deploy all changes at the same time. Clients will access the new version of all indexes at once without any downtime.

```bash
curl \
  -X POST 'http://localhost:7700/swap-indexes' \
  -H 'Content-Type: application/json' \
  --data-binary '[
    {
        "indexes": ["indexA", "indexA_new"]
    },
    {
        "indexes": ["indexB", "indexB_new"]
    },
    {
        "indexes": ["indexC", "indexC_new"]
    }
]'

```

In the example above, three swap operations will occur simultaneously and atomically.

Wait, what?

Yes, you can read that again. It’s atomic! Either all indexes are successfully swapped, or none are. Either all the content is swapped, or none is.

Why is that important? It prevents partial changes in the database, ensuring consistency and, thus, a top-notch search experience.

Conclusion
----------

As I mentioned earlier, this feature has been in the works for several months, and it all started with user feedback. The [“swap indexes” card](https://roadmap.meilisearch.com/c/65-swap-indexes) on our roadmap got 38 votes and almost as many comments explaining use cases where this feature is a must-have. Here are some of our favorites:

*   “It would be very useful when trying to change rules on production database to tweak for the best results”
*   “Would help if index was accidentally created with wrong name or need to rename/change it for various reasons.”
*   “We need it to clean out all deleted items.”

These are just a few examples. This kind of insight is extremely helpful for our team, as it allows us to shape the product to fit the users' needs.

While very convenient—you can have an overall view of all the feature ideas submitted, in the making, and released—the [roadmap](https://roadmap.meilisearch.com/) is just one of the options to provide feedback.

The [product discussions](https://github.com/meilisearch/product/discussions) on GitHub are, in my opinion, one of the best places to explain your needs. You can have direct contact with the Meilisearch product team, and, being public, it allows anyone to participate and enrich the process.

The last option is our brand-new [Discord server](https://discord.gg/meilisearch). Don’t hesitate to join us and talk about what you’ve built with Meilisearch, your use case, and your specific needs.

No matter which option you choose, we look forward to hearing from you!

Zero downtime index deployment

Meilisearch v0.30 is out and brings with it the ability to create numbered page selectors. You can read more about the new features and updates from v0.30 in our [release blog post](/blog/whats-new-in-v0-30/)his article will go over the different ways of navigating search results and the pros and cons of each method.

### What is pagination, and what are some common alternatives?

Let’s say you go to an ecommerce website to buy a new pair of shoes. It would be quite overwhelming if you got 700 pairs of shoes to choose from as soon as the page loads—not to mention that fetching every pair of shoes would certainly slow down the site.  

To avoid this, most websites will break search results up into chunks, such as pages, and load a certain number at a time based on input from the user.  

There are three main ways to display large datasets: pagination, infinite scroll, and load more. Each method has benefits and downsides that make it suited for certain use cases and content types. For example, Google’s use of numbered pagination makes it easier to find a specific result again, while Twitter, Facebook, and Instagram use infinite scroll to keep users glued to their screens. Google uses load more on mobile devices to display search results.  

Changing from one type of pagination to another can completely alter user experience.  Imagine waking up to a paginated version of Twitter, where you have to continuously click a “Next” button to view more posts in your feed (thanks, Elon). This would completely change the way you interact with the site!  

In making the choice of how to display search results, you decide how much content to show users at one time, what action they have to take to view more, and how convenient it is to find a specific result again.

### Pagination

Pagination works by breaking search results down into separate pages. There are two main types of pagination interfaces: numbered ones, where you find a list of numbered pages at the bottom of each page, and unnumbered ones, where you navigate from page to page using the “Previous” and “Next” buttons.  

Pagination is one of the most popular ways to display search results and is particularly common on ecommerce websites. Pagination breaks down content, making it easier to process. Since each page is distinct, users can easily return to items they liked or share them with others. This helps users to search for something specific instead of browsing through a never-ending list of items.  

The biggest downside to pagination is that most users don’t look beyond the first few pages and hence never get to look at more results. When was the last time you went to the second page of Google’s search results?

![](https://media.tenor.com/zmDrt3416pcAAAAC/think-winniethepooh.gif)

Since the results are replaced with every new page, users need to switch between pages to compare results. In addition, pagination isn’t very mobile-friendly. Mobile users prefer load more or infinite scroll as it’s more intuitive. It also requires additional actions from the user: clicking the “Next” button and waiting for the page to load. This may also give the impression that pagination is slow.

#### Numbered page selection

Page numbers add clarity to the search process and guide users to what they’re searching for. Numbered page selection offers numerous advantages, such as:  

*   Users can be shown the total number of pages of results, allowing them to estimate how long they will need to find a particular item. (“I have 25 pages worth of shoes to sift through!”)
*   With numbered pages, it’s clear that some pages are more relevant than others. (“I didn’t find those shoes on the first five pages, so this website probably doesn’t sell them.”)
*   Users can remember the positioning of an item, and numbered pages allow them to return to it faster. (“I liked the pink shoes on page 3.”)
*   Users can generally stop in the middle of a session and continue from where they left off. (“I stopped at page 5 yesterday.”)

![](https://unable-actionable-car.media.strapiapp.com/Screen_Shot_2022_11_29_at_16_12_39_22ff2f4296.png)

#### “Previous” and “Next” buttons

In numberless pagination, users navigate via “Next” and “Previous” buttons instead of a list of numbered pages. This alternative to numbered pagination is often used for mobile displays where there isn’t enough space to display page numbers. Since there are only two buttons, it offers less precision, as users can’t jump to specific pages. However, it offers better performance as it isn’t necessary to calculate the total number of search results.  

![](https://lh4.googleusercontent.com/7g66Jnt6E-GR1esTMZrmCv6MDvHMOn-FprG5cHxYQhmHGQLS8vYFvlmOlpaBdjmf_pc285dIACe3SeUNSj2UJ9C2JMfk4cxoxbqFuS_eD4q9qZp1h2yhIaz-9ng7xB3ly39_PvvtUNqm6L0a5euKFYvZSn9ttg5NzKPQMmz4X0k37XiGqD7RnkuVwT9XKg)

### Infinite scroll

Infinite scroll has become increasingly popular over the past decade due to the rise in smartphones. It loads more content seamlessly as the user scrolls, reducing friction. It’s also more intuitive than pagination; you just keep scrolling to view more content! Additionally, since all the content is loaded to a single page, it may seem faster than traditional pagination.

![](/blog/content/images/2022/11/IMG_3391.GIF

The biggest downside to infinite scroll is poor usability.  The page length increases infinitely, making it difficult to find previously viewed results again, nor can users bookmark or share a particular set of items on the list.  Page performance slows down as more content is loaded, making the page “heavier”. Meanwhile, the site footer may be inaccessible since infinite scroll pushes it endlessly down as more content is loaded. Since Googlebot cannot emulate scrolling, infinite scroll also has a negative impact on SEO.  

However, the greatest impact by far is on user behavior. At some point, users of infinite scroll will stop paying attention to individual results and begin to focus more on scrolling, leading to decreased engagement. You can read more about this in [Baymard Institute's study](https://www.smashingmagazine.com/2016/03/pagination-infinite-scrolling-load-more-buttons/) on design patterns for loading products on desktops and mobile devices.  

Based on the above, infinite scroll may not be the best solution for ecommerce websites. How do you find the pink shoes you liked 15-ish scrolls ago?  

There is one use case where infinite scroll cannot be beat. For entertainment websites whose main purpose is to keep the users with no particular aim on the page for as long as possible, infinite scroll all but guarantees a longer time on site. This is why infinite scroll is closely linked with social media and other business models that rely on advertising or data harvesting.

### Load more

Like infinite scroll, load more is a mobile-friendly option that works by extending one page infinitely. However, unlike infinite scroll, load more allows users to choose whether they want to view more results by clicking a “Load more” button at the bottom of the page. This critical difference gives users more control, making it easier, for example, to access the footer or keep track of a specific result. Studies show that it also increases the amount of attention users give to each individual item.  

Load more shares many downsides with infinite scroll, such as poor page performance. Like infinite scroll, it’s not ideal for SEO as Googlebot cannot click the “Load more” button. This means it cannot crawl or index all of your content.  

![](https://lh5.googleusercontent.com/-xmPwy0C6ZdgZoHNMV4fmRdxR_g9scVY5J0ktGY_k0oh24Un80WooYE9zSRo_TxNhPJE5--hAQJ-UJIRPejnFZt4pcNca6l4z6JedvwrjdKucMBAurVeBgMgIGYWS9unr2cOEA-1CZYBwlQ0ySexv2HqYSWqCSemBCVK5_jb-VweLwczjNcXQjl3ghxdmA)

### So, what do I use?

The short answer: it depends on the kind of product experience you want.  

**Pagination**

*   Your users are looking for highly specific results
*   SEO is very important to you

**Infinite scroll**

*   You’re building a social media site
*   Your users want to explore a large amount of content

**Load more**

*   You want to hide your content behind a paywall
*   You want to build an ethical mobile experience
*   You want to gain some of the benefits of both pagination and infinite scroll  
    

Or you could go hybrid! Consider Ikea: they use a load more button but also display the total number of search results and a progress bar, enabling users to see how many results they’ve viewed so far.  

![](https://unable-actionable-car.media.strapiapp.com/Screen_Shot_2022_11_29_at_18_45_56_68c9b2fc30.png)

While we have covered the three main ways to display search results today, they are far from set in stone. At the end of the day, your goal is to help your users find the content they need. If you understand which key goals you want to accomplish with your search navigation interface and which pitfalls you want to avoid, you may be able to design something even better than what’s been done before. Good luck, and let us know what you create with Meilisearch!

Navigating search results: pagination vs infinite scroll vs load more

These past few years, our team has observed the growth of our community across multiple online spaces such as Twitter, GitHub, LinkedIn, and especially our public Slack workspace. We were and are thrilled to see more and more people interested in our work and our company, and applied ourselves to making everyone feel welcome. Eventually, however, we began to consider a new option: the possibility of migrating our community from Slack to [our very own brand new Discord server.](https://discord.meilisearch.com) It was only a matter of time, since Discord has established itself as _the_ reference in terms of community platforms.

Such a change had to be carefully considered since it meant migrating our quite large Slack community onto a brand new platform. We didn't want to go through with it without a good reason and even better plan of action.

**First of all, why did we decide to move to Discord?**

Discord is difficult to ignore. It has become an extremely popular and highly democratized platform, with an average of 150 million active monthly users\*. Discord’s large user base was a very good argument in its favor, but it wasn't the only reason for our decision. Discord is already the happy home of various online communities from within the tech space, including our friends at [AppWrite](https://appwrite.io/) and [Orbit](https://orbit.love/). As such, the platform is very well designed for maintainers. Some of its benefits include:

*   A large variety of moderation tools, many of which can be automated thanks to bots
*   A variety of roles that can be assigned with different permissions, to both valorize community members and identify moderators
*   The option to make the Code of Conduct mandatory before joining the servers to help protect the space from bad actors
*   Unlimited and expiration-free invites
*   Many inclusive options, such as allowing people to assign themselves their preferred pronouns

When considering all of this, the general consensus was that a move to Discord was the best option for both our community members and the moderators.

Luckily, we aren't the first ones to consider this move and go through with it! After taking the decision, our next step was to diligently do our homework: research how other companies had handled the change and apply their best practices.

![](https://unable-actionable-car.media.strapiapp.com/Screenshot_2022_11_28_at_16_26_14_a0a03dd987.png)

**Secondly, how to manage the migration?**

As we embarked on the actual migration, the very first step taken well in advance was to prepare our future Discord server. We asked around internally to figure out which features would be expected and whether each team wanted its own space. Based on the initial feedback, we created the first proposal and invited the team to have a look around and share their impressions. At this time some team members raised issues around inclusivity, especially concerning the text-to-speech feature, and we did our best to address these in order to ensure a good experience for as many people as possible. Based on this second wave of feedback, the server was adjusted accordingly and the design was locked in.

There was also the question of timing. Our original plan was to have a post-new year 2023 migration. However, we had to consider the impact of other upcoming announcements. We recently announced and are soon to release our Cloud offering and v.0.30 of our search engine, which we expect to translate into an increase in awareness and an influx of newcomers to our community. We were concerned about having users join us only to be told a month later they would have to migrate to a completely different platform. Alongside this, we were preparing for our first sponsorship of the [Advent of Code](https://adventofcode.com/), a month long event offering challenging programming puzzles, for which we also wanted to create an [original coding challenge](https://aoc.meilisearch.com/). It is common for tech-oriented servers to host their very own Advent of Code channel; without a Discord, it wasn't something we could easily offer.

A December migration seemed very interesting from a strategic standpoint, the only issue being the short turnaround to communicate the upcoming changes to our Slack users.

**Finally, what happens now with our Slack?**

As we mentioned earlier, this change was greatly motivated by our desire to offer a more friendly space to our community members. Our aim isn't to take people by surprise and rush them around. Our Slack workspace will remain open for the next few months so that members who missed the initial announcement won’t simply be cut out and left to wonder what exactly happened. We will keep publishing regular announcements throughout these next few months to explain the situation and instruct people on how they can join us on this new platform. These efforts will also be supported on our various social media to get the word around as much as possible. We strongly encourage our community members to ask any questions they might have so that we can answer these queries transparently and reassure anyone feeling uncertain about this decision.

This migration was very carefully considered by Meilisearch, and we look forward to hearing your opinion about our new server. We intend to take full advantage of Discord's new options, and are excited to have you [joining us](https://discord.meilisearch.com) in this new adventure. Thank you for your continued support!

\*[https://www.cloudwards.net/discord-statistics/](https://www.cloudwards.net/discord-statistics/)

From Slack to Discord: our migration

In this article, let's take a look at some of the most significant changes in Meilisearch's latest update. You can read [the full changelog on GitHub](https://github.com/meilisearch/meilisearch/releases/tag/v0.30.0), but we’ll go over the main ones in this article.

Zero downtime index deployment
------------------------------

Syncing your database changes with your Meilisearch index results in downtime. You cannot afford a “broken” search: user experience is paramount. You’ve come up with some workaround, but it takes time and resources. There must be a simpler way to handle this, right?

v0.30 introduces index swapping. You can now deploy a new index version with zero downtime!

To synchronize the changes between your primary database and your Meilisearch index  —`indexA`— in production, you just need to take the following steps:

*   Step 1: Create a new version of the index —`indexA_new`— containing all those changes
*   Step 2: Swap indexes, which means you replace the documents, settings, and task history of `indexA` with those of `indexA_new`. This is the request you need to send for the magic to happen:

```bash
curl \
  -X POST 'http://localhost:7700/swap-indexes'\
  -H 'Content-Type: application/json' \
  --data-binary '[
      { "indexes": ["indexA", "indexA_new"] }
  ]'
```

⏲️ Wait for the task’ status to update to `succeeded`...

🎉 Tada! Your changes are now in production! Your users are still searching in `indexA`, but it holds the data of `indexA_new` now, and `indexA_new` holds the data of `indexA`.

New features: task cancelation and deletion
-------------------------------------------

v0.30 gives you more control over asynchronous operations: you can now cancel tasks and delete your task history.

### Task cancelation

The inability to cancel tasks has been one of the recurring problems users have raised over time. Not being able to interrupt tasks enqueued by mistake or taking too long to process can be very frustrating.

With v0.30, this problem is now a thing of the past. With the new API route `/tasks/cancel`, you can cancel enqueued or processing tasks based on their:

*   `uid`
*   `status`
*   `type`
*   `indexUid`
*   the date they were enqueued or started (`beforeEnqueuedAt`, `afterEnqueuedAt`, `beforeStartedAt`, `afterStartedAt`)

You can learn more about tasks in our [documentation](https://docs.meilisearch.com/learn/advanced/asynchronous_operations.html#understanding-tasks).

To cancel all enqueued tasks that add or update documents to an index, you should send the following request:

```bash
curl \
  -X POST 'http://localhost:7700/tasks/cancel?statuses=enqueued&types=documentAdditionOrUpdate’

```

### Task deletion

Succeeded or failed —and now canceled— task information can take up much-needed disk space in instances with a large number of asynchronous operations. Clean up finished tasks with the new `DELETE /tasks` endpoint.

Just as for task cancelation, task deletion can be done based on the tasks’:

*   `uid`
*   `status`
*   `type`
*   `indexUid`
*   the date they were enqueued or started (`beforeEnqueuedAt`, `afterEnqueuedAt`, `beforeStartedAt`, `afterStartedAt`)

But you can also delete them based on the date they finished processing and what task canceled them:

*   `beforeFinishedAt`
*   `afterFinishedAt`
*   `canceledBy`

The following request deletes all tasks processed before October 11, 2020, at 11:49:53 am.

```bash
curl \
  -X DELETE 'http://localhost:7700/tasks?beforeFinishedAt=2020-10-11T11:49:53.000Z'

```

New feature: configuration file support
---------------------------------------

Thanks to our dear contributor [mlemesle](https://github.com/mlemesle/), a new way of configuring your Meilisearch instance has entered the chat. Along with environment variables and CLI options, you can use a configuration file 🥳

Configuration files can be easily shared and versioned. They allow you to manage all configuration options in one place and give an overall view at a glance.

To do so, set the desired instance options in a single  `.toml`  file:

```toml
env = "production"
master_key = "YOUR_MASTER_KEY_VALUE"

```

You can use the `--config-file-path` CLI option or the `MEILI_CONFIG_FILE_PATH` environment variable to specify a configuration file. Otherwise, Meilisearch will use the file `./config.toml` in the working directory as the default configuration file. You can download a default configuration file with the following command:

```json
curl https://raw.githubusercontent.com/meilisearch/meilisearch/main/config.toml > config.toml
```
<Callout>
  Please note that specifying the `config-file-path` key in the configuration file will throw an error.
</Callout>

If no configuration file is provided, the `Config file path` is set to `none`.

Improvement: search result pagination
-------------------------------------

v0.30 introduces a numbered pagination mode for users who need pagination interfaces.

Up to v0.29, when using the search endpoint, Meilisearch only returned `estimatedTotalHits`, an estimate of how many documents matched the query. For this reason, creating interfaces with a page selector was not recommended.

You can now use the new search parameters `page` or `hitsPerPage` to get the exhaustive number of hits and the total number of pages for a query.

```bash
curl \
  -X POST 'http://localhost:7700/indexes/movies/search' \
  -H 'Content-Type: application/json' \
  --data-binary '{ "q": "american ninja", "page": 1 }'

```

The above request will fetch the first page of results for the given query and will return the following response:

```JSON
{
  "hits": [
        // … 20 hits
    ],
  "query": "american ninja",
  "processingTimeMs": 2,
  "hitsPerPage": 20,
  "page": 1,
  "totalPages": 49,
  "totalHits": 966
}

```

As you can see, it returns the exhaustive number of search results (`totalHits`) and the number of search results pages (`totalPages`), which allows you to create numbered pagination interfaces easily.

  
👉 Note that `totalHits` and `totalPages` are calculated based on the [pagination.maxTotalHits index setting](https://www.meilisearch.com/docs/reference/api/settings#pagination).

Breaking changes
----------------

### Field rename

When [deleting documents by batch](https://docs.meilisearch.com/reference/api/documents.html#delete-documents-by-batch), the corresponding task object contains a field with the number of document ids received. This field, previously called `receivedDocumentIds`, has been renamed to `providedIds`.

### New error

In cases where users try accessing the `/keys` route without setting a master key, Meilisearch now throws a more helpful error  `missing_master_key`:

```JSON
{
    "message": "Meilisearch is running without a master key. To access this API endpoint, you must have set a master key at launch.",
    "code": "missing_master_key",
    "type": "auth",
    "link": "https://docs.meilisearch.com/errors#missing_master_key"
}
```

### Task filter and error naming

To keep naming consistent with the value they can have, we have pluralized the query parameters used to filter tasks:

*   `indexUid` becomes `indexUids`
*   `type` becomes  `types`
*   `status` becomes  `statuses`

We have also renamed the following error codes:

*   `invalid_task_type` to `invalid_task_types_filter`
*   `invalid_task_status` to `invalid_task_statuses_filter`

Hacktoberfest contributions
---------------------------

We are infinitely grateful for all the contributions we received during Hacktoberfest. Most of the changes mentioned above were from these contributions, and we would like to highlight two more. We would not have released them in this version if it were not for our contributors.

*   Support for compressed API requests (Gzip, Brotli, Zlib) by [@mou](https://github.com/mou)
*   Apple Silicon binaries are available thanks to [@jeertmans](https://github.com/jeertmans)

Other changes
-------------

*   Meilisearch has been binding by default to `127.0.0.1:7700` , which is an IPv4 address. In order to make Meilisearch reachable through both [IPv4](https://en.wikipedia.org/wiki/IPv4) (`127.0.0.1:7700`) and [IPv6](https://en.wikipedia.org/wiki/IPv6) (`[::1]:7700`) addresses, we have changed the default address to `localhost:7700`.
*   All task objects now contain the `canceledBy` and `error` fields
*   New [task filters](https://docs.meilisearch.com/learn/advanced/asynchronous_operations.html#filtering-tasks) are available for the `GET /tasks` endpoint:
    *   `uids`
    *   `beforeEnqueuedAt` / `afterEnqueuedAt`
    *   `beforeStartedAt` / `afterStartedAt`
    *   `beforeFinishedAt` / `afterFinishedAt`
    *   `canceledBy`
*   We have introduced a new task type: `snapshotCreation`. Snapshots tasks are now visible in [the task list](https://docs.meilisearch.com/reference/api/tasks.html#get-tasks)
*   The indexing speed has been significantly increased
*   We have reduced both the size of the Meilisearch binary and the size the Meilisearch instance takes up in disk space
*   All instance options now have their corresponding environment variable. Some related to the snapshot and dump features were missing

A big thank you to all our contributors! This project wouldn't have gotten very far without your help 🙏

And that’s it for v0.30! Remember to check the [changelog](https://github.com/meilisearch/meilisearch/releases/tag/v0.30.0) for the full release notes.

What's new in v0.30

Time to close the door behind Hacktoberfest 2022! What an event it was, the time has come to sum up our learnings for this year to better prepare for the future.

For the third time in a row, everyone at Meilisearch was very excited to be participating in Hacktoberferst, probably one of the most popular celebrations of the open-source community online.

We were very grateful for everyone support and contributions throughout the month, please find below the recap we’ve shared internally.

Our teams really enjoyed taking part into the event, leaving us no choice but to go for one more round next year: Hacktoberfest 2023, here we come!

Here, you can find the slides of the presentation with a few comments for context:

![](https://unable-actionable-car.media.strapiapp.com/External_Hacktoberfest_2022_A_recap_636027ab45.jpg)

![](https://unable-actionable-car.media.strapiapp.com/External_Hacktoberfest_2022_A_recap_3_082f4c1b7d.jpg)

![](https://unable-actionable-car.media.strapiapp.com/External_Hacktoberfest_2022_A_recap_4_54734eb823.jpg)

![](https://unable-actionable-car.media.strapiapp.com/External_Hacktoberfest_2022_A_recap_5_f0c74dca0b.jpg)

It’s important to keep in mind the way we count our PRs might be slightly different from the official counting. In the following graph, we counted any PR merged during October, and we included the PRs that were validated but not merged in time on October 31st. Some PRs were open before October 1st but still benefited from the “Hacktoberfest approved” tag since they were completed during October.

![](https://unable-actionable-car.media.strapiapp.com/External_Hacktoberfest_2022_A_recap_6_cbd60a838b.jpg)

![](https://unable-actionable-car.media.strapiapp.com/External_Hacktoberfest_2022_A_recap_7_7b0c2029be.jpg)

![](https://unable-actionable-car.media.strapiapp.com/External_Hacktoberfest_2022_A_recap_8_d35b08263e.jpg)

![](https://unable-actionable-car.media.strapiapp.com/External_Hacktoberfest_2022_A_recap_9_4c4e155893.jpg)

![](https://unable-actionable-car.media.strapiapp.com/External_Hacktoberfest_2022_A_recap_10_1_78e13c206c.jpg)

We were truly blown away by the number of contributions we received this year! We didn’t quite reach our fourth goal, but we set it as such, knowing it was definitely going to be a challenge. At the same time, we were thrilled to see we weren’t that far away from it!  
  
These objectives being hit isn’t just an ego boost for us. It also translates directly into monetary donations for various organizations we’ve selected before the start of the event. The following organizations were selected on the basis of causes that are important to us. We also tried to find some groups that were geographically close to some of the employees of Meilisearch. This second criterion is because we wish to establish a stronger link with the organizations we donate to. We’d love to remain in touch with them and maybe see how our donation- by extension yours!- has potentially improved things for them.  
  
As we mentioned briefly, these donations wouldn’t have been achievable without your support and contributions throughout the whole month of October! We wish to extend a big thank you to everyone for that.

![](https://unable-actionable-car.media.strapiapp.com/External_Hacktoberfest_2022_A_recap_11_1_8804545d22.jpg)

![](https://unable-actionable-car.media.strapiapp.com/External_Hacktoberfest_2022_A_recap_12_a474cd83ac.jpg)

![](https://unable-actionable-car.media.strapiapp.com/External_Hacktoberfest_2022_A_recap_13_625ef09d3b.jpg)

![](https://unable-actionable-car.media.strapiapp.com/External_Hacktoberfest_2022_A_recap_14_00a1b66b56.jpg)

![](https://unable-actionable-car.media.strapiapp.com/External_Hacktoberfest_2022_A_recap_15_1_ba4bae4f61.jpg)

![](https://unable-actionable-car.media.strapiapp.com/External_Hacktoberfest_2022_A_recap_18_2_c65de291a9.jpg)

![](https://unable-actionable-car.media.strapiapp.com/External_Hacktoberfest_2022_A_recap_19_70512c60c4.jpg)

![](https://unable-actionable-car.media.strapiapp.com/External_Hacktoberfest_2022_A_recap_20_ad1a2df74e.jpg)

Meilisearch latest news and company updates