Meilisearch: v0.30.0 🦫

This release introduces index deployment with zero downtime and adds task management features. It also brings the capability to navigate search results by page selection. Thanks to Hacktoberfest, tons of updates and bug fixes have been done by the fantastic Meilisearch community. You were more present than ever, and words cannot do justice to just how grateful we are for your support. A big thank you from the whole Meilisearch team! ❤️ 🧰 Most integrations (SDKs, clients, libraries) will be compatible with this version within four hours of the release. Sometimes this can take up to 48 hours, depending on the issues we encounter during the release. Here is the exhaustive list of integrations **not immediately compatible** with v0.30.0: - [meilisearch-dart](https://github.com/meilisearch/meilisearch-dart) and [meilisearch-swift](https://github.com/meilisearch/meilisearch-swift): though you can still use both libraries with v0.30.0, the new available features are not yet available. Let us know if you really need it—and contributions are always welcome! - [meilisearch-java](https://github.com/meilisearch/meilisearch-java): Our team is still working on refactoring the code; this is reaching the end, but the SDK cannot be used with v0.30.0 yet. Follow the repository to know when it will be compatible. # New features and improvements 🔥 ## Improve search result navigation: exhaustive number of search results When paginating search results with `offset` and `limit`, Meilisearch only returns an estimate of the total number of results. Since `estimatedTotalHits` can change, creating reliable pagination interfaces with numbered page selectors is challenging. v0.30 introduces two new search parameters, `page` and `hitsPerPage`. Queries with these parameters return an exhaustive number of `totalHits` and `totalPages`, which you can then use to create UI elements such as numbered page selectors. The following example fetches the second page of results for a given query: ```bash curl \ -X POST 'http://localhost:7700/indexes/movies/search' \ -H 'Content-Type: application/json' \ --data-binary '{ "q": "shifu", "page": "2", "hitsPerPage": "10" }' ``` ```json { "hits": [ // … 10 hits ], // … "page": 2, "hitsPerPage": 10, "totalHits": 2100, "totalPages": 210 } ``` Done in #2601 by @ManyTheFish. ## Cancel processing and enqueued tasks Meilisearch now allows you to cancel `enqueued` or `processing` tasks through a new API route, `/tasks/cancel`. This can be useful when you need to interrupt a task that is taking too much time to be completed or enqueued by mistake. To cancel tasks, filter them by specifying one of the following query parameters: `uids`, `statuses`, `types`, or `indexUids`. Tasks can be filtered by date fields: - `beforeEnqueuedAt` / `afterEnqueuedAt` - `beforeStartedAt` / `afterStartedAt` For example, to cancel tasks by `uid`: ```bash curl \ -X POST 'http://localhost:7700/tasks/cancel?uids=1,2' ``` ```json { "taskUid": 1, "indexUid": "null", "status": "enqueued", "type": "taskCancelation", "enqueuedAt": "2021-08-12T10:00:00.000000Z" } ``` To cancel all enqueued tasks across all indexes in an instance: ```bash curl \ -X POST 'http://localhost:7700/tasks/cancel?statuses=enqueued' ``` Read more about [cancelling tasks](https://docs.meilisearch.com/reference/api/tasks.html#cancel-tasks). Done in #2763 by @irevoire, @Kerollmops and @loiclec. ## Delete the task history This release introduces a new `DELETE` endpoint to the `/tasks` route. You can use it to remove finished tasks (`succeeded`, `failed` or `canceled`) from Meilisearch's task history. This can be helpful in reducing the amount of occupied disk space. To delete finished tasks, filter them by specifying one of the following query parameters: `uids`, `statuses`, `types`, `indexUids`, or `canceledBy`. Tasks can be filtered by date fields: - `beforeEnqueuedAt` / `afterEnqueuedAt` - `beforeStartedAt` / `afterStartedAt` - `beforeFinishedAt` / `afterFinishedAt` For example, to delete tasks by `uid`: ```bash curl \ -X DELETE 'http://localhost:7700/tasks?uids=1,2' ``` ```json { "taskUid": 3, "indexUid": null, "status": "enqueued", "type": "taskDeletion", "enqueuedAt": "2021-08-12T10:00:00.000000Z" } ``` To delete all finished tasks in an instance: ```bash curl \ -X DELETE 'http://localhost:7700/tasks?statuses=failed,succeeded,canceled' ``` Read more about [deleting finished tasks](https://docs.meilisearch.com/reference/api/tasks.html#delete-tasks). Done in #2763 by @irevoire, @Kerollmops, and @loiclec. ## Deploy indexes with zero downtime Deploy new indexes version with zero downtime to the search clients. Meilisearch now provides a seamless way to deploy multiple indexes atomically with the new swap indexes API. To swap indexes, use the new `/swap-indexes` route: ```bash curl \ -X POST 'http://localhost:7700/swap-indexes'\ -H 'Content-Type: application/json' \ --data-binary '[ { "indexes": ["indexA", "indexB"] } ]' ``` ```json { "taskUid": 3, "indexUid": null, "status": "enqueued", "type": "indexSwap", "enqueuedAt": "2021-08-12T10:00:00.000000Z" } ``` Read more about [swapping indexes](https://docs.meilisearch.com/learn/core_concepts/indexes.html#swapping-indexes). Done in #2763 by @irevoire, @Kerollmops, and @loiclec. ## Configuration file support You can now use a configuration file to set [instance options](https://docs.meilisearch.com/learn/configuration/instance_options.html#all-instance-options). By default, Meilisearch will search the working directory for a file named `./config.toml`. You can use the `--config-file-path` option if you want to keep your configuration file in another directory: ```bash meilisearch --config-file-path="./my-config.toml" ``` Example of configuration file: ```toml env = "production" master_key = "MY_MASTER_KEY" schedule_snapshot = true ``` You can download a default configuration file from our repository: ```bash curl https://raw.githubusercontent.com/meilisearch/meilisearch/main/config.toml > config.toml ``` Read more about [customizing Meilisearch with a configuration file](https://docs.meilisearch.com/learn/configuration/instance_options.html#configuration-file). Done in #2745, #2928, #2804, #2841, and #2961 by @mlemesle, @choznerol, @arriven, @LunarMarathon and @curquiza, ## Other improvements * The fresh new filters are also availabe for the `GET /tasks`: `uids`, `canceledBy`, `beforeEnqueuedAt`, `afterEnqueuedAt`, `beforeStartedAt`, `afterStartedAt`, `beforeFinishedAt`, `afterFinishedAt`. * Full support for compressed API requests (Gzip, Brotli, Zlib) (#2876) @mou * Significant indexing speed improvements (#2763, meilisearch/milli#639, meilisearch/milli#619) @loiclec * Change default bind address from `127.0.0.1` to `localhost`, making the default address available in IPv4 and IPv6 (#2861) @Fall1ngStar * Provide Apple Silicon binaries (#2837) @jeertmans * Add missing environment variables for dump and snapshot features (#2738) @gmourier * Increase max concurrent readers from 126 to 1024 (#2830) @arriven * Reduce the size of the Meilisearch binary: allow excluding specialized tokenizations when building binaries (#2773) @jirutka * Introduce the `snapshotCreation` task type: snapshot tasks are added to the task list, for more transparency (#2763) @Kerollmops * Reduce the size taken by Meilisearch on the disk space (#2763, meilisearch/milli#639) @loiclec * `canceledBy` and `error` fields are now always part of the `task` object (#2763) @irevoire, @Kerollmops, and @loiclec # Behavior changes ⚠️ * Rename `receivedDocumentIds` into `providedIds` in the task object when performing a [`delete-batch` action](https://docs.meilisearch.com/reference/api/documents.html#delete-documents-by-batch) (#2826) @Ugzuzg * Return a new error when using `/keys` without any master key set: `missing_master_key` instead of `invalid_api_key` (#2922) @vishalsodani * Some names of filter parameters for `/tasks` and some error `code`s have been renamed (#3023) @Kerollmops @loiclec - `indexUid` query parameter is renamed `indexUids` - `type` query parameter is renamed `types` - `status` query parameter is renamed `statuses` - `invalid_task_type` error is renamed `invalid_task_types_filter` - `invalid_task_status` error is renamed `invalid_task_statuses_filter` # Fixes 🐞 * Improve error message when adding documents: replace a meaningless serde message (#2819) @onyxcherry * Fix `dumpCreation` tasks bug (#2890) @washbin * Don't panic when the error length is slightly over 100 (#2727) @onyxcherry * Fix phrase search bug (#2763, meilisearch/milli#639 and meilisearch/milli#647) @loiclec * Fix filtering bug (meilisearch/milli#619) @loiclec * Fix `facetDistribution` when setting `maxFacetPerValue` if only a few documents are returned (meilisearch/milli#619) @loiclec * Correct variant returned for `invalid_api_key_indexes` error when creating an API key (#3032) @ManyTheFish * Prevent phrase search containing stop words never retrieves any documents (#3036 and meilisearch/milli#664) @Samyak2 @Kerollmops * Add missing `IN`/`NOT IN` operators to the `invalid_filter` error message (#3036 and meilisearch/milli#676) @Pranav-yadav @Kerollmops * Filtering by "inf", "infinity", or "NaN" as numbers will yield an internal error (#3036 and meilisearch/milli#689) @dureuill @Kerollmops * Prevent duplicate documents in the search route (#3036 and meilisearch/milli#690) @Kerollmops * Re-introduce missing key in the documents database (#3036 and meilisearch/milli#690) @Kerollmops * Don't remove DB if unreadable (#3077) @dureuill * Fix `MDB_BAD_VALSIZE` error (#3084 meilisearch/milli#696) @Kerollmops @loiclec * Prevent OS error 22 when creating indexes (#3084 meilisearch/milli#699) @Kerollmops @dureuill * Display the `dumpUid` as `null` until we create it (#3122) @Kerollmops # Misc * GitHub CIs and tests * Add dry run for publishing binaries: check if the compilation works (#2726 and #2733) @curquiza * Add CI workflow to update the Meilisearch version in Cargo.toml files (#2741 and #2744) @curquiza * Add CI manifest to automate some steps when closing/creating a Milestone (#2739, #2852, #2853) @curquiza * Improve Docker CI for cloud team (#2790, #2896) @curquiza * Update checkout v2 to v3 in CI manifests and use a unique GitHub PAT (#2740) @curquiza * Uncomment cache steps in Github CI (#2868) @AM1TB * Skip dashboard test if the mini-dashboard feature is disabled (#2814) @jirutka * Use pre-compiled binaries for faster CI (meilisearch/milli#685) @azzamsa * Refactorize the whole test suite (#3085) @irevoire * Dependencies * Upgrade dependencies (#2847) @loiclec * Upgrade clap to 4.0 (#2851) @choznerol * Upgrade to alpine 3.16 in Dockerfile (#2827) @nwnt * Documentation * Update internal CLI documentation (#2839) @jeertmans * Improve issue template to avoid support questions in Meilisearch issues (#2772) @curquiza * Update Hacktoberfest section in CONTRIBUTING.md (#2817, #2794, #2793) @meili-bot @curquiza @Luna-meili * Fix typos (#2789) @kianmeng * Fix broken link in CONTRIBUTING.md (#2845) @AnirudhDaya * Delete v1.rs since it is not included in project (#2834) @Himanshu664 * Download-latest script: refactoring (#2913) @nfsec * Make Clippy happy (#2831) @Kerollmops * Improve debuggability by naming spawned threads (#3061) @dureuill ❤️ Thanks again to our external contributors: - [Meilisearch](https://github.com/meilisearch/meilisearch): @AM1TB, @AnirudhDaya, @arriven, @choznerol, @Fall1ngStar, @Himanshu664, @jeertmans, @jirutka, @kianmeng, @LunarMarathon, @mlemesle, @mou, @nfsec, @nwnt, @onyxcherry, @Ugzuzg, @vishalsodani and @washbin. - [Milli](https://github.com/meilisearch/milli): @anirudhRowjee, @azzamsa, @ehiggs, @jeertmans, @msvaljek, @Pranav-yadav, @Samyak2, @vincent-herlemont, and @vishalsodani.