Today we are releasing the first release candidate (RC) of Meilisearch v0.21. This is the initial step towards the next version of Meilisearch, and a pretty important one given that it will be our first major release since March.

A big part of v0.21 will be relatively invisible to most end-users: our core team has been hard at work refactoring Meilisearch’s engine, something that is as time-consuming as it is crucial for the long-term health of our codebase. In this post, I want to go over why we decided to refactor Meilisearch, how that impacted our product, and where we’ll go from here.

We won’t be discussing everything that’s changed between v0.20 and v0.21—that’s for another blog post—but we will touch on several of the new features we are most excited about.

Why was refactoring needed?
===========================

We first realized that something was wrong when it became increasingly difficult for our core team to add new features without impacting performance. We could see our code was becoming more difficult to maintain, which is something particularly concerning for an open-source project: our community is our greatest ally and keeping code clean, readable, and pleasant-to-use is crucial if we want to keep our allies happy. We were clearly reaching the limits of what the previous codebase could offer us in terms of performance and ease-of-use, so we decided it was time for a change.

How did we do?
==============

We have a lot of good news—but, as with any major software release, there were some goals we weren’t able to attain.

### Search relevancy

The v0.21.0 will provide new ways to make your search more relevant:

*   You can now use numeric attributes as facets—think things like the price of a product or the release date of a book
*   This was actually requested by our users: the equivalent of Algolia’s `words` criterion [is now available](https://www.meilisearch.com/docs/learn/core_concepts/relevancy#1-words) in Meilisearch!
*   Phrase search: users can now force the engine to only return exact matches by enclosing query terms in double quotes (`"`)
*   Highlighting is now available for array and object values

### Search performance

Searching through huge datasets with Meilisearch has never been so fast! With a 1GB dataset containing around 115 million documents:

*   Queries that return many results are more than ten times faster: searching for a single letter takes less than 50ms in v0.21, compared with 2 seconds in v0.20
*   Similar speed gains when filtering on numbers: most searches take less than 50ms in v0.21

### Indexation

TLDR; indexation is now faster in most cases, but there are exceptions. Lots of room for improvement!

First, the good bits: indexation is globally faster with the refactored search engine.

Unfortunately, we noticed a loss of performance when parsing datasets with lengthy description values (think a list of blog posts or scientific articles with excerpts and abstracts) in v0.21. This is not a complete surprise: the new search engine requires more calculations, which means indexation might take more time. This is not ideal, but we would rather release something that’s not completely optimized than delay v0.21 and all the new features it will introduce. Rest assured, we are doing our best to improve indexation performance.

Another bit of bad news: while we have made significant steps in the right direction and reduced memory consumption across the board, the issue of Meilisearch sometimes crashing due to excessive memory usage has still not been fixed. This is another priority for us going forward.

### **Are there any breaking changes**?

There are indeed! We did our best to avoid breaking changes, but that’s not always possible when we want to keep improving things and making our API easier to use. The two biggest changes related to v.021’s refactoring are:

*   The parameters `filters` and `facetFilter` have been replaced by `filter`. Having two parameters doing almost the same thing added unnecessary complexity when filtering query results
*   Several naming changes that we believe make using Meilisearch more intuitive. None of these have any impact on the end-user experience. A full list will be available when we officially release v0.21

What’s next?
============

Once v0.21 is officially released some time within the next three months, we will return to our monthly release cycle. We are particularly looking forward to working on `sortBy` and geographical search—two features that are highly requested by our users. We hear you loud and clear!

If you are curious about Meilisearch’s upcoming version, feel free to test it and give us as much feedback as possible—we depend on you to catch as many remaining bugs as possible. You can [download the release candidate on GitHub](https://github.com/meilisearch/MeiliSearch/releases/tag/v0.21.0rc0).

Also, a word of advice: we do not recommend using this RC in production. We are confident v0.21 RC1 is looking pretty good, but there are still issues that we need to work out before it is ready for the spotlight.

Thank you again, and see you soon for the official v0.21.0 release!

Refactoring Meilisearch: v0.21 has entered the chat

Meilisearch currently maintains more than 50 open-source projects on GitHub. In the last six months alone we merged over 1000 Pull Requests (PRs), 160 of which came from external contributors.

Ensuring the codebase remains stable and that code quality is always top-notch requires us to be very careful when it comes to merging a new PR. As the number of contributions grows, the automation of this process becomes a necessity in order to keep everything simple for the contributors and safe for Meilisearch's codebase.

In this post I will describe why we decided to use [Bors](https://github.com/bors-ng/bors-ng?ref=blog.meilisearch.com), a merge bot for Github pull requests, to create a better environment for both maintainers and contributors.

## Our merging process
During an open-source contribution, any changes must be done on their own separate branch. Afterwards, when the work on this branch is finished, one of our main requirements before merging is that the new branch is up-to-date with the primary branch—called `main` in most of our repositories.

If the PR is not brought up to date with `main`, you could say that most of the work is done, but *most of the work* is not enough for us. Even if all the automated tests pass on the PR itself, they are not guaranteed to pass when merged if the PR branch and `main` are not aligned.

Before using Bors, this meant we often had to ask PR authors to **rebase their branch** if, for instance, a different PR had been merged meanwhile. This made contribution more complex than we wanted, since not everyone is comfortable with Git’s `rebase` command.

### Why rebasing?
At Meilisearch, we believe keeping a clear Git history is really important for two main reasons:
1 It allows everyone to follow all changes made to the project
2 When debugging, a clear history helps identify which commit introduced the issue

The goal of a rebase is to move the commits from a PR branch to right after the last commit on `main`. Therefore, **rebasing keeps your Git history clear and consistent with the order in which you have merged your PRs**.

Rebasing is not the only way to bring a branch up-to-date with the `main` branch: you can also pull the changes from `main` to the PR branch, for instance. The issue with this approach is that the pull would add a merge commit to the PR branch, which would not keep the repository history clear.

Keeping in mind how important it is to maintain a clear commit tree, **rebasing is the only way to guarantee that the work that is going to be merged fits the current work in production**. If the tests pass on an up-to-date branch, they will also pass once merged into `main`, ensuring your production branch will not break.

## Finding the perfect tool
With our rebase requirements in mind, I began my search for the perfect tool. I wanted to be efficient, so I narrowed my scope and only evaluated tools that were:
* **Free and open source:** the tool had to fit the values of Meilisearch and its community
* **Highly rated:** the tool had to be stable and reliable, since any mistakes when handling PRs are bound to have impactful consequences
* **Well documented and easy to set up:** good documentation saves time that we can use improving Meilisearch

Unfortunately, I did not find as many solutions as I expected within this scope.

The first tool I tried was [Shipits](https://github.com/shipitjs/shipit?ref=blog.meilisearch.com), which I discovered thanks to [this article](https://engineering.shopify.com/blogs/engineering/successfully-merging-work-1000-developers?ref=blog.meilisearch.com) on Shopify’s engineering blog. Though Shipits seemed really solid and promising, I felt it was a better fit for complex integration processes instead of open-source projects just trying to keep their Git history clean.

Another tool I tested was [Kodiak](https://github.com/chdsbd/kodiak?ref=blog.meilisearch.com). The set up was really easy, in large part due to their great documentation. It was a good candidate for our use-case, but the automatic merge process is based on labels that we would have to manage, which does not really fit our workflow. For example, our minimum requirement when merging a PR is obtaining the approval of one reviewer; sometimes, though, I want two people reviewing one PR and it is easy to forget to communicate this to Kodiak. The moment it receives the first approval, Kodiak automatically merges the PR because the one-reviewer-requirement has been respected.

Another thing we noticed was that Kodiak does not update the branch originating the PR by rebasing it and instead creates a Git commit by pulling the main branch into the PR branch, which, as we saw, muddles the repository’s history. In the end, Kodiak did not fit our needs, even if its setup and configuration provided a seamless and hassle-free experience!

Contrary to all the other tools I looked into, **[Bors](https://github.com/bors-ng/bors-ng?ref=blog.meilisearch.com)** was built following a unique and powerful principle described by the original Bors creator in [this article](https://graydon2.dreamwidth.org/1597.html?ref=blog.meilisearch.com):

> *The Not Rocket Science Rule Of Software Engineering: automatically maintain a repository of code that always passes all the tests.*

This "rule" perfectly fit our requirements and made Bors our final choice.

## How does Bors work?
Bors is a merge bot—meaning, it is an application that **automates aspects of the PR merging process**. Its setup is straightforward and, like Kodiak, very well documented. Indeed, configuration can be a single `bors.toml` file containing the names of tests we want to execute before merging.

All Bors commands have to be given through comments in a PR. To merge a PR, all we need to do is use `bors merge`.
Bors first applies a rebase before running the tests and, if they succeed, merges them back into `main`. To be more accurate, Bors does not actually rebase the branch of the PR: it merges the commits of the PR into another branch, called `staging`, that is already up-to-date with `main`. If the tests do not pass on `staging`, Bors returns a failure and does not merge the PR.

The icing on the cake: **Bors is also able to manage merging multiple PRs at the same time**, which means we don't need to merge them one by one. Each PR is added to a queue and will be checked with the last version on `main` at regular intervals.

If Bors finds conflicts between the PR and the main branch, it cannot make the decision about which final changes to apply. In this situation, the contributors have to rebase the branch manually—if that happens to you, I wrote this [quick tutorial](https://gist.github.com/curquiza/5f7ce615f85331f083cd467fc4e19398?ref=blog.meilisearch.com) on how to rebase from a forked repository.

## Conclusion: which problems have we solved?
We at Meilisearch have used Bors for a few months now and are really satisfied. We know that keeping the Git history clear while providing a stable codebase can be very complex, but so far Bors has helped us greatly optimize our workflow. With Bors integrated in our workflow there is:
* No need to ask and wait for PR authors to rebase
* No need to test and merge PRs one by one
* No frustration for our contributors: they all do amazing work, but some of them are not really comfortable with Git’s rebase feature, which is indeed not always easy to use—especially when working from a forked repository!

**Bors makes both maintainers and contributors happier**; we can, therefore, recommend it for projects with the same use-case as ours.

Automate pull requests merging with Bors

<Callout>
  This guide is for Strapi v3. If you are using Strapi v4, check [this post](/blog/strapi-v4-plugin-meilisearch/)
</Callout>

As the web evolves, more and more tools are available to take care of complex jobs for us. Why create a CRUD API from scratch when you can generate one in a few clicks? Why struggle with difficult search operations and queries, when simple and efficient solutions exist?

Maybe we're intimidated by new technologies, or concerned about investing time in tools that don't pan out. Or, more likely, _we may not even be aware that a simple solution exists._

![](https://unable-actionable-car.media.strapiapp.com/truemen_d2b99b9b0a.gif)

Meilisearch in action

This tutorial will show you, in the fewest steps possible, how to use Strapi and Meilisearch to create a search-based web-app. First, let's get acquainted with our tools!

Introducing our Tools
---------------------

### Strapi

[Strapi](https://strapi.io/) is a headless CMS. It provides a quick way to create a back-end where we can add and store data. The data is then available through a REST API without having to code anything or configure our database.

### Meilisearch

[Meilisearch](https://www.meilisearch.com/docs/learn/what_is_meilisearch/overview) is a fast, open-source search engine that's easy to use and deploy. The aim of Meilisearch is to create a relevant search experience out of the box in very few steps and with no configuration needed.

Requirements
------------

To be able to follow this tutorial, you'll need the following:

*   An open [terminal or command prompt](https://www.ionos.com/help/email/troubleshooting-mail-basicmail-business/access-the-command-prompt-or-terminal)
*   [Node.js](https://nodejs.org/en/download/) {">= 10.16 and <= 14"}: this makes it possible to run JS outside a browser
    *   You can check the active version with the command `node --version`
    *   If your version of Node is outside this range, we recommend that you [install nvm](https://github.com/nvm-sh/nvm/blob/master/README.md#about) and use it to access Node 14
*   NPM and NPX >= 6.x (installed with Node.js): package managers that help you access and use JS libraries

Steps:
------

1.  [Create a working directory](#create-a-directory-for-our-app)
2.  [Create Strapi Back End](#create-a-back-end-using-strapi)
3.  [Start Meilisearch](#launch-and-start-meilisearch)
4.  [Connect Strapi and Meilisearch](#connect-strapi-and-meilisearch)
5.  [Conclusion](#conclusion)

Create a directory for our app
------------------------------

We don't want our app to be all over the place! Let's create a directory where we will add every part of our application.

```bash
mkdir my-app
cd my-app
```

If you are not using a [UNIX-based operating system](https://www.eukhost.com/blog/webhosting/linux-flavors/) please follow this guide to [create a directory](https://www.computerhope.com/issues/ch000742.htm).

Create a back end using Strapi
------------------------------

In this section we are going to create and run a back-end using Strapi.

Our first step is to generate a Strapi Application. Go to your open terminal window and run the following command:

```bash
npx create-strapi-app@3.5.4 back --quickstart

```
<Callout>
  Using [npx](https://www.npmjs.com/package/npx) we run a specific version of the package without installing it globally. Note that this guide may not work with a different version.
</Callout>

This command creates a Strapi app in a new directory called `back` and opens the admin dashboard. Create an account or log in so that you have access.

![](https://unable-actionable-car.media.strapiapp.com/Screenshot_2021_04_20_at_14_28_54_1_453a624ffa.png)

If you lose this page at any time, just go to `http://localhost:1337/admin` in your browser

Once you have created your account, you should be redirected to Strapi's admin dashboard. This is where we will configure our back-end API.

Our next step is to create a new collection type—in this case, it'll be a collection of restaurants. Complete [step 3 of Strapi's quick start guide](https://strapi.io/documentation/developer-docs/latest/getting-started/quick-start.html#_3-create-a-restaurant-content-type) to continue.

![](https://unable-actionable-car.media.strapiapp.com/Screenshot_2021_04_20_at_15_31_41_2effed8eb3.png)

Admin dashboard landing page

After finishing [step 3 of Strapi's quick start guide](https://strapi.io/documentation/developer-docs/latest/getting-started/quick-start.html#_3-create-a-restaurant-content-type), a new collection named `Restaurants` should have appeared under `Collection Types`. If we click on it, we can see that there are still no restaurants. Let's add some!

![](https://unable-actionable-car.media.strapiapp.com/Screenshot_2021_04_20_at_15_07_36_63c48765e5.png)

`Restaurants` collection landing page

Click on `+ Add New Restaurants` in the upper-right corner of the dashboard. Now let's add three restaurants, one by one:

*   Name: `The Butter Biscotte`. Description:  `All about butter, nothing about health.`
*   Name: `The Slimy Snail` Description: `Gastronomy is made of garlic and butter.`
*   Name: `The Smell of Blue` Description: `Blue Cheese is not expired, it is how you eat it. With a bit of butter and a lot of happiness.`

![](https://unable-actionable-car.media.strapiapp.com/Screenshot_2021_04_20_at_15_15_38_2c431c5150.png)

Create an entry in the Restaurant Collection.

For each restaurant, you need to press `save` and then `publish`. Once the three restaurants have been added, you should end up with the following page:

![](https://unable-actionable-car.media.strapiapp.com/Screenshot_2021_04_20_at_15_22_55_e0137d6c9d.png)

Main page of `Restaurants` showcasing the restaurants we have added

Our Strapi back end is now up and running! Strapi automatically creates a REST API for our `Restaurants` collection. Check out Strapi's documentation for all available [API endpoints](https://strapi.io/documentation/developer-docs/latest/developer-resources/content-api/content-api.html#api-endpoints).

Next, it's time to bring Meilisearch into the mix.

Launch and Start Meilisearch
----------------------------

**tl;dr:** [download, install and start Meilisearch](https://docs.meilisearch.com/learn/getting_started/installation.html#download-and-launch).

Want to avoid a local installation? Use our [Meilisearch cloud](https://cloud.meilisearch.com/) free 14-day trial or our [one-click install on DigitalOcean](https://marketplace.digitalocean.com/apps/meilisearch) and [continue to the next section](#connect-strapi-and-meilisearch).

If you made it this far, you should already have an open terminal window running Strapi. If not or if you closed it by accident, please [follow this guide](https://www.ionos.com/help/email/troubleshooting-mail-basicmail-business/access-the-command-prompt-or-terminal), navigate to the directory `.../my-app/back` that we created, and run the command `npm run develop` to restart Strapi.

Let's take a moment to **open a second terminal window** and move to the `my-app` directory. This window will be used to run Meilisearch.

Inside `my-app`, create a new directory called `meilisearch-binary` .

```bash
mkdir meilisearch-binary
cd meilisearch-binary
```

Create and move to directory where we will run Meilisearch

Execute the following command to download Meilisearch in the current directory. If cURL doesn't work for you, consider trying one of the [many other methods to download and install](https://docs.meilisearch.com/learn/getting_started/installation.html#download-and-launch) Meilisearch.

```bash
# Install Meilisearch
curl -L https://install.meilisearch.com | sh
```

Download and install Meilisearch

We now have the `meilisearch` binary in our directory. Let's run it!

```bash
# Launch Meilisearch
./meilisearch
```

Launch Meilisearch

You should see a big chunk of text (including some ASCII art) confirming that Meilisearch is up and running.

![](https://unable-actionable-car.media.strapiapp.com/MS_adress_75a76c4b2b.png)

Running Meilisearch

Find the line titled `Server listening on`. This lets us know where Meilisearch is being served. Once you've found it, open this adress in your browser.

You should arrive on a web page without much to look at.

![](https://unable-actionable-car.media.strapiapp.com/Screenshot_2021_04_21_at_13_21_44_d9162084ee.png)

The Meilisearch web interface w/ empty index

This is the Meilisearch [web interface](https://www.meilisearch.com/docs/learn/what_is_meilisearch/search_preview). It will remain empty until we add some data to our Meilisearch instance.

Data, you say? It just so happens we already have some in our Strapi app! Time to connect the two and start searching through our data with Meilisearch.

Connect Strapi and Meilisearch
------------------------------

In order to add the Meilisearch plugin to Strapi, we need to quit our Strapi app. Go to the terminal window running Strapi (_make sure it's not the one running Meilisearch!_) and push `control + c` to kill the process.

Once you've done that, move inside the `back` directory and install the plugin.

```json
cd back
npm install strapi-plugin-meilisearch
```

After the installation, we have to rebuild our Strapi app before starting it again in development mode, since it makes configuration easier.

```bash
npm run build
npm run develop
```

At this point, our Strapi app should be running once again on the default address: [`http://localhost:1337/admin/`](http://localhost:1337/admin/) . Visiting it in your browser, you should see an admin sign-in page; enter the same credentials as before.

Once connected, you should see a new `Plugin` on the left side of the screen: Meilisearch! Let's click on it.

![](https://unable-actionable-car.media.strapiapp.com/Screenshot_2021_04_21_at_14_32_44_c00e3d34f8.png)

Arriving on the Meilisearch plugin page, we can see that the information is split into two sections:

*   Meilisearch credentials
*   Strapi collections

![](https://unable-actionable-car.media.strapiapp.com/Screenshot_2021_04_21_at_14_44_12_6b821d2d2e.png)

Meilisearch plugin page of Strapi admin dashboard

First, we need to add the credentials of our Meilisearch instance so that Strapi knows where to send our data.

You may remember from the previous step that Meilisearch runs on `http://localhost:7700` by default. Add this in the `Meilisearch Host` field, then click `Add` to connect your Meilisearch instance to Strapi.  

![](https://unable-actionable-car.media.strapiapp.com/credentials_6fe7804ffa.gif)

Add credentials to Meilisearch.

> Note that we leave `Meilisearch Api Key` blank because we started Meilisearch without a [master key](https://www.meilisearch.com/docs/learn/configuration/instance_options#master-key).

Great! Now it's time to add your Strapi collection to Meilisearch.

Try clicking on the check box next to `restaurant` and watch as Meilisearch instantly begins processing your data. Because we only have three documents, the indexing time is practically non-existent.

![](https://unable-actionable-car.media.strapiapp.com/Large_GIF_956x212_5f52b4fd0f.gif)

Add restaurants to Meilisearch

We can now go back to our Meilisearch host address: `http://localhost:7700`. After refreshing, you should observe that the page is not as empty as before.

![](https://unable-actionable-car.media.strapiapp.com/Screenshot_2021_04_22_at_10_07_08_6dbd424e4d.png)

Demo page of Meilisearch

We did it! Try inputting "butter" and check out the results.

#### Front-end Interface

Using [instant-meilisearch](https://github.com/meilisearch/meilisearch-js-plugins/tree/main/packages/instant-meilisearch), you can implement a front-end search interface in [React](https://github.com/meilisearch/meilisearch-react), [Vue](https://github.com/meilisearch/meilisearch-vue), or [Angular](https://github.com/meilisearch/meilisearch-angular) in no time at all.  
  
Here's an interactive [code sandbox](https://codesandbox.io/s/react-movies-demo-e1dqh) (in React) where you can get a feel for the different components. You can't break anything, so feel free to play around!

### Conclusion

I hope this article was a pleasant introduction to the new [Meilisearch plugin for Strapi](https://github.com/meilisearch/strapi-plugin-meilisearch). However, this is just the beginning!

If you have any questions, please join us [on Discord](https://discord.gg/meilisearch). If you like Meilisearch, [a star on GitHub means a lot](https://github.com/meilisearch/meilisearch). For more information, see our:

*   [Documentation](https://www.meilisearch.com/docs/)
*   [GitHub](https://github.com/meilisearch/meilisearch)
*   [Strapi Plugin](https://github.com/meilisearch/strapi-plugin-meilisearch)
*   [Discord](https://discord.gg/meilisearch)

How to add relevant search to your Strapi app using Meilisearch

_[Originally published](https://docs.airbyte.io/tutorials/slack-history) on Airbyte site_

**Overview**
------------

The [Slack free tier](https://slack.com/pricing/paid-vs-free) saves only the last 10K messages. For social Slack instances, it may be impractical to upgrade to a paid plan to retain these messages. Similarly, for an open-source project like [Airbyte](https://airbyte.io) where we interact with our community through a public Slack instance, the cost of paying for a seat for every Slack member is prohibitive.

However, searching through old messages can be really helpful. Losing that history feels like some advanced form of memory loss. What was that joke about Java 8 Streams? This contributor question sounds familiar—haven't we seen it before? But you just can't remember!

This tutorial will show you how you can, for _free_, use Airbyte to save these messages (even after Slack removes access to them). It will also provide you a convenient way to search through them.

Specifically, we will export messages from your Slack instance into an open-source search engine called [Meilisearch](https://github.com/meilisearch/meilisearch). We will be focusing on getting this setup running from your local workstation. We will mention at the end how you can set up a more productionized version of this pipeline.

We want to make this process easy, so while we will link to some external documentation for further exploration, we will provide all the instructions you need here to get this up and running.

**Set up Meilisearch**
----------------------

First, let's get Meilisearch running on our workstation. Meilisearch has extensive docs for [getting started](https://docs.meilisearch.com/reference/features/installation.html#download-and-launch). For this tutorial, however, we will give you all the instructions you need to set up Meilisearch using Docker.

```Shell
docker run -it --rm \
  -p 7700:7700 \
  -v $(pwd)/data.ms:/data.ms \
  getmeili/meilisearch
```

That's it!

> Meilisearch stores data in `$(pwd)/data.ms`, so if you prefer to store it somewhere else, just adjust this path.

**How to replicate your Slack messages to Meilisearch**
-------------------------------------------------------

### Set up Airbyte

Make sure you have Docker and Docker Compose installed. If you haven’t set Docker up, follow the [instructions here](https://docs.docker.com/desktop/) to set it up on your machine. Then, run the following commands:

```Shell
git clone https://github.com/airbytehq/airbyte.git
cd airbyte
docker-compose up
```

If you run into any problems, feel free to check out our more extensive [getting started](https://docs.airbyte.io/getting-started) for more help.

Once you see an Airbyte banner, the UI is ready to go at `http://localhost:8000/`. Once you have set your user preferences,  you will be brought to a page that asks you to set up a source. In the next step, we'll go over how to do that.

### Set up Airbyte’s Slack source connector

In the Airbyte UI, select Slack from the dropdown. We provide step-by-step instructions for setting up the Slack source in Airbyte [here](https://docs.airbyte.io/integrations/sources/slack#setup-guide). These will walk you through how to complete the form on this page.

![](https://lh3.googleusercontent.com/ZCxW4_nPPsLB4_fiszkcCIWFIDS8nSifxcwLweC2tH__sUdhEF-34YXQmQXJCSZGg4upzpv1NbICIKNKS0ayap4X41Unxhf7qLmfmDCzEbkl8M2R9ggUnMi-czMU-v_QSAD2mL07)

By the end of these instructions, you should have created a Slack source in the Airbyte UI. For now, just add your Slack app to a single public channel (you can add it to more channels later). Only messages from that channel will be replicated.

The Airbyte app will now prompt you to set up a destination. Next, we will walk through how to set up Meilisearch.

### Set up Airbyte’s Meilisearch destination connector

Head back to the Airbyte UI. It should still be prompting you to set up a destination. Select "Meilisearch" from the dropdown. For the host field, set: `http://localhost:7700`. The api\_key can be left blank.

### Set up the replication

On the next page, you will be asked to select which streams of data you'd like to replicate. We recommend unchecking "files" and "remote files" since you won't really be able to search them easily in this search engine.

![](https://lh6.googleusercontent.com/O-bUkd7k_8Eo-K9PNZA1kCd-bbg0DekSwyNfaXdDHIb3Pcyd9gi8nJXljU0NJL3ELaY6sDq8HoUuaMklFi5fQBpTplfTDklOuYdoQF_lpe_vFAZ9wSOLtPp76iWM7VFnwuC9rkbh)

For frequency, we recommend every 24 hours.

**Search Meilisearch**
----------------------

After the connection has been saved, Airbyte should start replicating the data immediately. When it completes you should see the following:

![](https://lh5.googleusercontent.com/ZErxFD14kuzXkBx2Fv6OtjPM80AogKllbLPlMQJZ8rfb1OTVhKRdt0rr1rS6e7bcnngfs0qJ7pLzNL4CsXYAByzJSspjSXPgbqkyemyiaRP7aPeMT8Wqs7gqtkuTUmKSmZSwdN_3)

When the sync is done, you can sanity check that this is all working by making a search request to Meilisearch. Replication can take several minutes depending on the size of your Slack instance.

```Shell
curl 'http://localhost:7700/indexes/messages/search' --data '{ "q": "<search-term>" }'
```

For example, I have the following message in one of the messages that I replicated: welcome to airbyte..

```Shell
curl 'http://localhost:7700/indexes/messages/search' --data '{ "q": "welcome to" }'
# => {"hits":[{"_ab_pk":"7ff9a858_6959_45e7_ad6b_16f9e0e91098","channel_id":"C01M2UUP87P","client_msg_id":"77022f01-3846-4b9d-a6d3-120a26b2c2ac","type":"message","text":"welcome to airbyte.","user":"U01AS8LGX41","ts":"2021-02-05T17:26:01.000000Z","team":"T01AB4DDR2N","blocks":[{"type":"rich_text"}],"file_ids":[],"thread_ts":"1612545961.000800"}],"offset":0,"limit":20,"nbHits":2,"exhaustiveNbHits":false,"processingTimeMs":21,"query":"test-72"}
```

**Search via a UI**
-------------------

Making curl requests to search your Slack History is a little clunky, so we have modified the example UI that Meilisearch provides in [their docs](https://docs.meilisearch.com/learn/tutorials/getting_started.html#integrate-with-your-project) to search through the Slack results.

Download (or copy and paste) this [html file](https://github.com/airbytehq/airbyte/blob/master/docs/tutorials/slack-history/index.html) to your workstation. Then, open it using a browser. You should now be able to write search terms in the search bar and get results instantly!

![](https://lh6.googleusercontent.com/JfzrnvxFz-08gIL6hMyWeW5TB8_XGgAUibx26o0gXhxJVc97IgW_YnqvWEV2ta8OvJbTQwQUUpYpsHggvbvRSJp8YVTw9w6Y3Ocl8E33Ab-L0E5visu-Beh_kqJwolt-9m_ZEk8N)

**"Productionizing" saving Slack history**
------------------------------------------

You can find instructions for how to host Airbyte on various cloud platforms [here](https://docs.airbyte.io/deploying-airbyte).

Documentation on how to host Meilisearch on cloud platforms can be found [here](https://www.meilisearch.com/docs/learn/cookbooks/running_production).

If you want to use the UI mentioned in the section above, we recommend statically hosting it on S3, GCS, or equivalent.

Save and search through your Slack history on a free Slack plan

Hey everybody! As some of you may know, I [recently joined Meilisearch](/blog/caro/)s a Developer Advocate. Part of my job is to help the community by answering questions, so my priority over the last few months has been to get to know Meilisearch. I believe the best way to learn new software is to use it just as any developer would: to build something. So, I set out to make a demo.

<Callout>
  This demo was created when the latest version of Meilisearch was v0.19, some of the workarounds described in this blogpost are no longer necessary due to the addition of new features in more recent versions of Meilisearch.
</Callout>

Meilisearch x MoMA
------------------

The first step was to find a nice dataset that I enjoyed working with. After several unsuccessful attempts to tame some unstructured datasets, I stumbled upon the [repository of the Museum of Modern Art (MoMA)](https://github.com/MuseumofModernArt/collection). What a lucky find! They have two datasets: artists and artworks. I decided to go for the artworks dataset since it has all the essential information about each art piece: title, artist, dimensions, medium, even the artist’s biography. But the best part is that it’s available in JSON format! No conversion needed, no MongoDB dump to restore, no scraping: Meilisearch ready 😀

I used [Meilisearch JavaScript](https://github.com/meilisearch/meilisearch-js) to create the indexes, add the documents, and customize the settings. For the front-end, I decided to go for [Instant Meilisearch](https://github.com/meilisearch/meilisearch-js-plugins/tree/main/packages/instant-meilisearch) combined with [Vue InstantSearch](https://github.com/algolia/vue-instantsearch/) as described in the [meilisearch-vue repository](https://github.com/meilisearch/meilisearch-vue/). I’m a little lazy, so if I find a tool that provides nice results effortlessly, I will use it! Fortunately, InstantSearch provides all the front-end tools to customize the search environment at will, so you can experience the power of Meilisearch and its search-as-you-type feature with just a few lines of code. Finally, I tried to add some decent styling, but it’s not my forte 😅 I apologize in advance for the visual design!

**You can see the result** [**here**](https://moma.meilisearch.com/)**.**

![Web interface with searchbar and results for the query "Picasso"](https://unable-actionable-car.media.strapiapp.com/Capture_d_e_cran_2021_02_18_a_16_40_57_41c7a62d0f.png)

Solving Problems
----------------

So, I had found a dataset, indexed it in a Meilisearch instance, and set up a front-end for the search. All done, right? Not exactly; now came the time to iterate and make the search results feel good.

I was lucky to find a structured JSON dataset that worked effortlessly with Meilisearch. Nevertheless, I had to make some small changes to better display the results. Since Meilisearch’s [automatic highlighting](https://www.meilisearch.com/docs/learn/what_is_meilisearch/features#highlighting) doesn’t work on query results that are stored in an array, I had to **convert array values into strings** before adding them.

I also wanted to allow users to **sort the results by date**. Since the date field didn’t have a standard format, I had to retrieve the year and store it in a new \`sortByDate\` field. Then there was another problem: Meilisearch doesn’t support sorting at query time (**at least, not yet**)! To circumvent that and implement this feature, I created two more indexes with custom ranking rules desc(DateToSortBy) and asc(DateToSortBy). Every time you change the sorting option in the drop-down menu, Meilisearch is actually searching in a completely different index, but it’s so fast that the change is not visible to the human eye 🤯

Here you can see the sorting in action:

![Web interface with the query "Picasso" sorting by ascending and then descending date gif](https://unable-actionable-car.media.strapiapp.com/sorting2_f59a0e2432.gif)

🧐 You might have noticed that I said Meilisearch doesn’t support query-time sorting **yet**—that’s because it’s one of the most requested features on [our public roadmap](https://roadmap.meilisearch.com/tabs/1-under-consideration), and it’s now under consideration 🥳 So if you haven’t done it yet, take a look at Meilisearch’s roadmap and vote for your favorite feature or submit your own ideas! **👉 Update note:** query-time sorting was released with Meilisearch v0.22.

The most challenging part of creating this demo was working with an unknown dataset of 138,151 documents, because the large number of documents makes it difficult to know the possible values and length of the different fields. I first wanted the user to be able to filter results by \`medium\` (e.g.  “photograph” or “paint”). That is, until I realized some of the artworks had \`medium\` fields that were many lines long! Take this one for example:

> "H (vol. IX): Cover with lithographed manuscript text by Kliun on front; 1 lithographed illustration by Terent'ev; lithographed manuscript text. L (vol. XIII): Cover with lithographed illustration and manuscript design by Kirill Zdanevich on front; lithographed manuscript and typed text. M (vol. XIV): Cover with lithographed manuscript design and illustration on front by Kirill Zdanevich; lithographed manuscript and typed text. P (vol. XVII): Cover with lithographed manuscript design on front by Goncharova; and lithographed manuscript text by Mikhail Pustynin and Olga Olesha-Suok"

Can you imagine having this as a [facet filter](https://www.meilisearch.com/docs/learn/advanced/faceted_search)? Not very nice for the UI…

Close to Perfect
----------------

The easiest part of making this demo was setting up Meilisearch. Once I decided which attributes I wanted to be [displayed](https://www.meilisearch.com/docs/learn/configuration/displayed_searchable_attributes#displayed-fields) and [searchable](https://www.meilisearch.com/docs/learn/configuration/displayed_searchable_attributes#searchable-fields), the rest was a piece of cake. Aside from the highlighting and sorting issues mentioned above, the last thing I changed was [the attribute ranking order](https://www.meilisearch.com/docs/learn/core_concepts/relevancy#attribute-ranking-order). By default, the attribute ranking order is generated automatically based on the attributes' order of appearance in the first document indexed. In my case, the first attribute was \`title\`, which meant that artwork with matching query terms in the \`title\` field topped the results list. This delivered decent results, but I was pretty sure I could do better.

I not only had to consider which attributes are most important for this particular dataset; I also had to think about what types of queries are more likely to be made. If a user queries for a specific art piece using its title, he will get relevant results because the words used are unlikely to be found in lots of other fields. However, if a user queries the name of a famous artist, it is very possible to find that name in an artwork title, for example as a tribute. So I decided to place \`artist\` at the top of [the searchable attributes list](https://www.meilisearch.com/docs/learn/configuration/displayed_searchable_attributes#the-searchableattributes-list). Before I made the change, this is what you would find among the first results when searching for “Van Gogh” :

![Information and image of a litograph called "Do You Leave without seeing the 150 Van Gogh's" by Willem Jacob Henri Berend Sandberg](https://unable-actionable-car.media.strapiapp.com/Capture_d_e_cran_2021_02_22_a_11_34_08_073b869794.png)

As you can see, it’s an art piece created by an artist named Willem Jacob Henri Berend Sandberg where the name “Van Gogh” appears in the title, but it’s not an actual Van Gogh.

Now, after changing the order of the searchable attributes in [the settings](https://www.meilisearch.com/docs/reference/api/settings#searchable-attributes), the first result is well and truly a piece of art created by Vincent Van Gogh:

![Information and image of the litograph "The Potato Eaters" by Vincent van Gogh](https://unable-actionable-car.media.strapiapp.com/Capture_d_e_cran_2021_02_22_a_11_41_41_7bdabf659b.png)

  
Pretty amazing how a simple change like this can impact the search results. Meilisearch is ultra relevant out-of-the-box, but it lets you tune the relevancy of the results according to your needs, and that is essential to provide an awesome experience for the end user.

An Open-Source Museum Experience
--------------------------------

The demo source code is available on [GitHub](https://github.com/meilisearch/demo-MoMA/). Feel free to play with it and show me what you can do! 😁

How cool is it that we’re able to access this extraordinary catalog? Thanks to MoMA’s generosity, you can explore the museum from anywhere in the world. And now, thanks to Meilisearch, it’s only 50 milliseconds away 🚀

* * *

_Photo by [Robert Bye](https://unsplash.com/@robertbye?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText) on [Unsplash](https://unsplash.com/?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText)_

Meilisearch: a blank canvas

Happy New Year, Everyone! 2021 brings [a new version](https://github.com/meilisearch/MeiliSearch/releases/tag/v0.18.0?ref=blog.meilisearch.com) of Meilisearch. How good is that?

In this release, we have fixed some bugs and raised the default payload size when you add documents to Meilisearch. The star of the show, however, is the **new tokenizer**.

Our main goal is to offer the **best search experience**, and a good tokenizer is critical to having quality search results. We are very proud of our core team and our contributors—they worked hard, and they delivered! Let’s take a closer look at it.

## New Tokenizer
The **tokenizer** is sort of like the brain of Meilisearch: it understands how languages work and adapts the way documents are stored in Meilisearch accordingly. Without a tokenizer, we wouldn't be able to know where a word starts and where it ends. We wouldn’t be able to understand what a user asks for when performing a search.

### A Closer Look
The tokenizer’s role is to **find and retrieve all the words in a string based on the language’s particularities**. Every language requires a unique process. For example:

- In **Latin-based languages**, words are generally separated by spaces.
- In **Chinese**, splitting words is more complicated.

Meilisearch’s new tokenizer goes **field by field**, determining the most likely language for the field and running a different pipeline for each language. Since the tokenizer is **modular**, adding new languages is much easier than before.

The new tokenizer might not feel different for **Latin-based alphabets** such as English, but it should dramatically improve the experience for our **Chinese users**.

## The Evolution of Meilisearch in Chinese

Before, the tokenizer considered **one Hanzi character (Chinese character) as one word**. The new tokenizer:

- Identifies **Chinese words** made of one or more characters.
- Assesses the **distance between matched query terms** more accurately (see: [proximity rule](https://docs.meilisearch.com/learn/core_concepts/relevancy.html#proximity-rule)).
- **Positively improves search relevancy**.
- A **single search query** now gives results in both **traditional and simplified Chinese**.

## Increased Max Payload Size

In previous versions of Meilisearch, the **default limit** of a document payload was **10MB**. We have increased this limit to **100MB**. Of course, you still have the possibility to change this limit.

```bash
$ ./meilisearch --http-payload-size-limit=209715200
# Change limit to 200MB
```


## Bug Fixes

- Meilisearch now **correctly adds attributes** to `searchableAttributes`, even when they have been added to `attributesForFaceting` first.
- Attributes are now added to `searchableAttributes` **in order of appearance**, as intended.
- When no attribute is set to be displayed, Meilisearch **returns an empty object **`` instead of an error.


## Other Changes

- We’ve made some modifications to our **testing suite**.
- **Dump tests can now run in parallel**.
- Most of the **dependencies have been updated**.

For a **specific list of changes and commits**, please check the [release notes](https://github.com/meilisearch/meilisearch/releases).


## Contributions

We are fortunate to have this **amazingly supportive community**.

Special thanks to:

- [**@piaoger**](https://github.com/piaoger) for beta-testing the new tokenizer.
- [**@woshilapin**](https://github.com/woshilapin) for improving the CI.
- [**@sanders41**](https://github.com/sanders41) for increasing the default payload size.

We would also like to thank **all the contributors** who help maintain the toolkits that make Meilisearch easier to use.

Your involvement **means a lot to us**! 💜

---

## 💬 Get Involved!

We are always eager to hear our users’ and contributors’ **suggestions**! Come and talk with us using the method you prefer:

- [GitHub Discussions](https://github.com/meilisearch/meilisearch/discussions)
- [Discord](https://discord.meilisearch.com)
- [Twitter](https://twitter.com/meilisearch)

🚀 Happy coding!


What's new in v0.18.0

**tl;dr:** We recently raised 1,5M euros from LocalGlobe, Seedcamp, Kima & Tiny.vc. This funding will allow us to focus on building a community of early adopters & contributors around our open search engine [Meilisearch](https://github.com/meilisearch/MeiliSearch).

We are still at the very beginning of our open source adventure and we are thrilled to see that more and more people believe in this project. This is our first fundraise and it means the world for the Meili team 🎉

### Meili got funded!

Meili SAS (the company) was created in November of 2018 with the idea of building an alternative to existing search engine APIs like Algolia. After some successes and failures as a team and company, we decided to raise some money to keep focus on building Meilisearch and its ecosystem.

We believe that an exceptionalsearch experience on websites and applications should be the norm, not an exclusive feature. Thus we are building the most straightforward technology for developers to offer state-of-the-art search to their users, scaling from the tiniest of operations all the way up to enterprise.

![](https://unable-actionable-car.media.strapiapp.com/david_armstrong_srp_E_a5_W418_unsplash_beb73be50e.jpg)

How we got there
----------------

Meilisearch has been open source since the [earliest days](/blog/open-source-or-not/)f its development. As developers, we made our project accessible on GitHub from day one but without the idea of being an "open source company". First and foremost we set out to make a simpler alternative to Algolia for smaller businesses who needed something more developer-friendly and affordable.

After describing our project to many developers, we quickly realize that one of the main differences from Algolia was that Meilisearch is open source. Still we weren't using open source as a force.

In late 2019, after some substantial discussion within the team, we decided to [fully commit to orienting the business around open source](/blog/to-open-source-and-beyond/)e could not quietly bootstrap anymore since we decided to build a community around our project to get feedback and contributions from external developers.

We wrote our [Beta release blog post](/blog/meilisearch-beta-release/)or Meilisearch in November of 2019. Since then we have been working hard at making Meilisearch the new obvious choice for developers when it comes to building a search box.

Charlotte, who joined us in August of 2019, did an incredible job of directing the writing and making of our [documentation](https://docs.meilisearch.com/) while Clémentine, who joined us one month later, took charge of developing the [SDKs and integrations](https://github.com/meilisearch/integration-guides) to ease the use of Meilisearch.

Fundraise
---------

In late 2019 we began to meet with investors because we knew we wouldn't be able to develop the business without help from industry veterans.

The first key meeting that we had was with [Alexis from Kima](https://twitter.com/ale6_) who helped us make our pitch understandable to VCs. Alexis was presented to us as `THE investor who understand tech` in Paris. Not only did he understood what we wanted to build, he also did an amazing job at introing us to relevant funds regarding our stage & business 😁  
  
Then from meeting to meeting we met the different persons we are now working with: [Mish](https://twitter.com/mishmashkautsan?lang=en) & [Louis](https://www.linkedin.com/in/knightwebb/?originalSubdomain=uk) from [LocalGlobe](https://localglobe.vc/) as lead investor, [Sia](https://twitter.com/siahouchangnia?lang=en) from [SeedCamp](https://seedcamp.com/), [Alexis](https://twitter.com/ale6_) from [Kima](https://www.kimaventures.com/) & [Philipp](https://twitter.com/pmoe?lang=en) from [Tiny.vc](https://www.tiny.vc/).

The deal we made with the VCs is that we develop the community and the usage around Meilisearch while we better understand the use-cases of our search engine. From there we will be able to generate revenue. [Our business plan is publicly available](/blog/meilisearch-open-business-plan-v0-1/)s a iteratively developed document.

What's next?
------------

### Core features

We made an [open roadmap](http://roadmap.meilisearch.com/) and we will publish more roadmap communications in the coming weeks to get better feedback on what developers need in a search engine API.

There are some features that we know we need to build in order to be competitive as a search engine backend (basic analytics and authorization system 🚧) that are already planned. We are also discussing with developers & companies to gain more insight on features that should be game-changers in the enterprise search industry!

### Integrations & Documentation

All of this work around Meilisearch would not make any sense if Meilisearch is not used. To ease the integration of Meilisearch we are building different [SDKs](https://github.com/meilisearch/integration-guides) in different languages and improving our [documentation](https://docs.meilisearch.com/) every week.

We want anyone to be able to integrate Meilisearch in any common programming environment in less than 5 minutes. Today, with our most mature front- and back-end libraries you should be able to setup a proof-of-concept in less than an hour. We are hard at work shaving off those remaining 55 minutes to get where we want to be.

### Open source & values

As we learn to build a commercial open source software company, we are thriving for transparency on everything we do. To raise awareness around open source sustainability, Erlend is publishing a podcast called [Value in Open](/blog/value-in-open-1-posthog/)e already have our first episode released, and the second one should follow shortly 😁

![](https://unable-actionable-car.media.strapiapp.com/kelly_sikkema_XX_2_W_Tb_Lr3r8_unsplash_1_877c2e9d79.jpg)

  
We are delighted and excited about the future of Meilisearch 😁 if you want to reach us for anything, [please feel free to do so](https://docs.meilisearch.com/learn/what_is_meilisearch/contact.html)!

Meili raises 1.5M€ for open source search in Rust

Announcing Meilisearch v0.13.0  

=================================

Meilisearch v0.13.0 introduces a bunch of new features and changes. Bugs have  
been fixed and some behavior have changed for more consistency, here what's new in meilisearch:

Placeholder search
------------------

  
We've been asked for a placeholder search feature a lot, and it's landed in  
v0.13.0. It allows you to perform a search without specifying query strings,  
and get "placeholder" search results. The results are sorted according to the  
ranking rules you specified, and it supports faceting and filtering.

Post Route search
-----------------

  
A new route has been added to allow you to perform search with a `POST` route instead of `GET`. This will improve performance for searches that require token  
authentication, by allowing [preflight request](https://developer.mozilla.org/en-US/docs/Glossary/Preflight_request) caching.  
  
This is now the recommended way to perform searches with Meilisearch.

New settings behavior
---------------------

  
We have made a couple of changes to the way settings work in order to make them easier to use and intuitive. `searchableAttributes` and `displayedAttributes` now default to all attributes (`*` ). This means that unless you specify otherwise, all attributes will be searchable by default. This removes the need for the `acceptNewField`, that no longer exists.

Sys-info route removal
----------------------

  
We decided not to support the sys-info route anymore, and remove it. This was motivated by the fact that there already are better tools to do this job, and it was not worth the time maintaining this part of the code.

Check that Meilisearch is launched with the correct database version
--------------------------------------------------------------------

  
We added a check to ensure that Meilisearch is launched with a database that's compatible with the engine version. Meilisearch will refuse to start unless versions match.

Bug fixes
---------

*   `facetDistribution` returning the values in lowercase, instead of the correct case.
*   Bug that caused Meilisearch to add documents with the same document id twice, if they were added in the same batch.

Special thanks
--------------

  
We'd like to give a special thank to our external contributors on this release.  
Their involvement means a lot to us!

*   [@balajisivaraman](https://github.com/balajisivaraman) for his help on making our tests faster and cleaner
*   [@Rio](https://github.com/Rio) for his work on authentication and analytics
*   [@aeriksson](https://github.com/aeriksson) for finding and fixing typos in our code

If you feel like contributing, and don't know where to start, ping us on [slack](https://slack.meilisearch.com/).

Meilisearch latest news and company updates