Meilisearch: v1.14.0 🦫

Meilisearch v1.14 gives more granular control over which parts of filters you can disable for indexing performance optimization. This release also includes composite embedders, which can improve embedding generation during search and indexing, and a new route to retrieve multiple documents by their IDs. 🧰 All official Meilisearch integrations (including SDKs, clients, and other tools) are compatible with this Meilisearch release. Integration deployment happens between 4 to 48 hours after a new version becomes available. Some SDKs might not include all new features. Consult the project repository for detailed information. Is a feature you need missing from your chosen SDK? Create an issue letting us know you need it, or, for open-source karma points, open a PR implementing it (we'll love you for that ❤️). # New features and updates 🔥 ## Granular filterable attribute settings v1.14 gives you more control over which types of filter you want to disable in your searches. This allows you to further optimize indexing speeds by letting you activate only the filter features you need. Use `PATCH /indexes/INDEX_NAME/settings` to specify which filters you want to enable for each attribute in your documents: ```json { "filterableAttributes": [ { "attributePatterns": ["genre", "artist"], "features": { "facetSearch": true, "filter": { "equality": true, "comparison": false } } }, { "attributePatterns": ["rank"], "features": { "facetSearch": false, "filter": { "equality": true, "comparison": true } } } ] } ``` For more details about this feature, please refer to [the dedicated documentation page](https://www.meilisearch.com/docs/reference/api/settings#filterable-attribute-object). Done by @ManyTheFish in [#5254](https://github.com/meilisearch/meilisearch/pull/5254). ## Composite embedders This feature allows using different embedders at search and indexing time. This can be useful when optimizing AI-powered search performance. For example, you may prefer to use: - A remote embedder during indexing, as remote embedders have higher bandwidth and can generate more embeddings per second - A local embedder when answering to search queries, as local embedders have lower latency and can respond more quickly to user input To use the feature, follow these steps: 1. Enable the `Composite embedders` feature with the Meilisearch Cloud interface, or with the `/experimental-features` route: ```sh curl MEILISEARCH_URL/experimental-features \ -H 'Content-Type: application/json' \ -d '{"compositeEmbedders": true}' ``` 2. Next, create an embedder, setting its `source` to `"composite"` and defining one `searchEmbedder` and one `indexingEmbedder`: ```jsonc { "embedders": { "text": { "source": "composite", "searchEmbedder": { "source": "huggingFace", "model": "baai/bge-base-en-v1.5", "revision": "a5beb1e3e68b9ab74eb54cfd186867f64f240e1a" }, "indexingEmbedder": { "source": "rest", "url": "https://URL.endpoints.huggingface.cloud", "apiKey": "hf_XXXXXXX", "documentTemplate": "Your {{doc.template}}", "request": { "inputs": [ "{{text}}", "{{..}}" ] }, "response": [ "{{embedding}}", "{{..}}" ] } } } } ``` 3. Once the composite embedder has been created, Meilisearch will use its `indexingEmbedder` during indexing and `searchEmbedder` when responding to user queries For more details about this feature, please refer to its [public usage page](https://www.notion.so/meilisearch/Composite-embedder-usage-14a4b06b651f81859dc3df21e8cd02a0?pvs=4#19f4b06b651f809ab3ecc0daae7d5856). Done by @dureuill in [#5371](https://github.com/meilisearch/meilisearch/pull/5371) and [#5401](https://github.com/meilisearch/meilisearch/pull/5401). ## Retrieve multiple documents by ID It is now possible to retrieve multiple documents by their IDs: ``` curl -H 'Content-Type: application/json' MEILISEARCH_URL/indexes/INDEX_UID/documents -d '{ "ids": ["cody", "finn", "brandy", "gambit"] }' ``` ```json { "results": [ { "id": "brandy", "info": 13765493 }, { "id": "finn", "info": 35863 }, { "id": "cody", "info": 122263 }, { "id": "gambit", "info": 22222 } ], "offset": 0, "limit": 20, "total": 4 } ``` > [!WARNING] > Documents are *not* returned in the queried order. Non-existent documents are ignored. Done by @dureuill in [#5384](https://github.com/meilisearch/meilisearch/pull/5384). ## Other improvements * Batch together `/documents` requests using either `PUT` or `POST` by @Kerollmops in [#5293](https://github.com/meilisearch/meilisearch/pull/5293) * Display timestamped internal indexing steps on the `/batches` route by @Kerollmops in [#5356](https://github.com/meilisearch/meilisearch/pull/5356) and [#5364](https://github.com/meilisearch/meilisearch/pull/5364) * Introduce `exhaustiveFacetCount` parameter to `/facet-search` route to retrieve an exhaustive facet count by @ManyTheFish in [#5369](https://github.com/meilisearch/meilisearch/pull/5369) * Reduce RAM consumption of arroy by @irevoire in https://github.com/meilisearch/arroy/pull/105 * Experimental feature: Cache embeddings during search (Read more in the [feature discussion](https://github.com/orgs/meilisearch/discussions/818)) by @dureuill in [#5418](https://github.com/meilisearch/meilisearch/pull/5418) * Extend batch progress view to include indexing of vectors by @irevoire in [#5420](https://github.com/meilisearch/meilisearch/pull/5420) * Armenian characters are no longer case-sensitive by @ManyTheFish in https://github.com/meilisearch/meilisearch/pull/5454 * Avoid reindexing searchables when the order changes by @ManyTheFish in https://github.com/meilisearch/meilisearch/pull/5402 * Accept cancellation tasks even when the disk is full by @irevoire in https://github.com/meilisearch/meilisearch/pull/5492 # Fixes * Geo update bug by @ManyTheFish in https://github.com/meilisearch/meilisearch/pull/5407 * Fix prefix search on attributes listed in `disabledOnAttributes` settings by @ManyTheFish in https://github.com/meilisearch/meilisearch/pull/5415 (fixes [#5347](https://github.com/meilisearch/meilisearch/issues/5347) and [#5452](https://github.com/meilisearch/meilisearch/issues/5452)) * Fix CI to work with merge queues by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5456 * Support fetching the pooling method from the model configuration by @dureuill in [#5355](https://github.com/meilisearch/meilisearch/pull/5355) * Delete unused prefixes by @Kerollmops in [#5413](https://github.com/meilisearch/meilisearch/pull/5413) * Fixes mini dashboard to prevent the panel from popping up every time by @curquiza in https://github.com/meilisearch/meilisearch/pull/5436 * Enabling `rankingScoreThreshold` no longer causes `_rankingScore` to be miscalculated by @barloes and @dureuill in https://github.com/meilisearch/meilisearch/pull/5313 * Validate dimensions of embedding when receiving documents with `_vectors` by @dureuill in https://github.com/meilisearch/meilisearch/pull/5478 # Misc * Dependencies updates * Bump Ubuntu in the CI from 20.04 to 22.04 by @Kerollmops in [#5338](https://github.com/meilisearch/meilisearch/pull/5338) * Bump heed to v0.22 by @irevoire and @Kerollmops in [#5406](https://github.com/meilisearch/meilisearch/pull/5406) * Bump ring to v0.17.14 to compile on old aarch64 by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5423 * Bump zip from 2.2.2 to 2.3.0 by @dependabot in https://github.com/meilisearch/meilisearch/pull/5426 * CIs and tests * Improve test performance of get_index.rs by @DerTimonius in [#5210](https://github.com/meilisearch/meilisearch/pull/5210) * Ollama Integration Tests by @Kerollmops in [#5308](https://github.com/meilisearch/meilisearch/pull/5308) * Ensure the settings routes are properly configured when a new field is added to the Settings struct by @MichaScant in [#5149](https://github.com/meilisearch/meilisearch/pull/5149) * Skip a snapshot test on Windows by @Kerollmops in [#5383](https://github.com/meilisearch/meilisearch/pull/5383) * Fix CI to work with merge queues by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5456 * Accept total batch size in human size by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5421 * Add more progress levels to measure merging and post-processing by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5422 and https://github.com/meilisearch/meilisearch/pull/5468 * Isolate word fst usage to dedicate it to typo-correction by @ManyTheFish in https://github.com/meilisearch/meilisearch/pull/5415 * Show database sizes batches by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5457 and https://github.com/meilisearch/meilisearch/pull/5464 > ⚠️ Please consider that the batches stats content can change anytime. * Improve performance of computing document stats by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5465 ❤️ Thanks again to our external contributors: - [Meilisearch](https://github.com/meilisearch/meilisearch): @MichaScant - [Heed](https://github.com/meilisearch/heed): @oXtxNt9U - [Arroy](https://github.com/meilisearch/arroy): @ptondereau - [Charabia](https://github.com/meilisearch/charabia): @NarHakobyan, @mosuka