Dify: v0.3.31

## New Features - Added support for multi-datasets retrieval, hybrid search, and result re-ranking. Learn more: [Dify.AI v0.3.31: Surpassing the Assistants API – Dify's RAG Demonstrates an Impressive 20% Improvement](https://dify.ai/blog/dify-ai-rag-technology-upgrade-performance-improvement-qa-accuracy) ## Important Upgrade Notice Current version adds `full-text search` and `hybrid search` functions for datasets, significantly improving search results. If you want to use this feature, you need to make the following configurations: - If the vector database you are using is `Weaviate` Please update your version to **v1.19.0** or later in **/dify/docker/docker-compose.yaml**. For more details, please refer to the [Weaviate release list](https://github.com/weaviate/weaviate/releases?page=5). ``` weaviate: image: semitechnologies/weaviate:1.19.0 restart: always volumes: # Mount the Weaviate data directory to the container. - ./volumes/weaviate:/var/lib/weaviate environment: # The Weaviate configurations # You can refer to the [Weaviate](https://weaviate.io/developers/weaviate/config-refs/env-vars) documentation for more information. QUERY_DEFAULTS_LIMIT: 25 AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'false' PERSISTENCE_DATA_PATH: '/var/lib/weaviate' DEFAULT_VECTORIZER_MODULE: 'none' CLUSTER_HOSTNAME: 'node1' AUTHENTICATION_APIKEY_ENABLED: 'true' AUTHENTICATION_APIKEY_ALLOWED_KEYS: 'WVF5YThaHlkYwhGUSmCRgsX3tD5ngdN8pkih' AUTHENTICATION_APIKEY_USERS: '[email protected]' AUTHORIZATION_ADMINLIST_ENABLED: 'true' AUTHORIZATION_ADMINLIST_USERS: '[email protected]' ``` - If the vector database you are using is `Qdrant` 1. Since Dify's full-text index relies on the multilingual tokenizer in Qdrant's full-text index, by default, Qdrant does not support Chinese, Japanese, and Korean language tokenization. 2. Therefore, we provide a full-language tokenizer mirror for Qdrant: **langgenius/qdrant:latest**, which can be deployed and used directly. You can also build your own Qdrant image using the following command: ``` docker buildx build . --build-arg FEATURES="multiling-chinese,multiling-japanese,multiling-korean" --tag=langgenius/qdrant ``` 3. Due to the lack of full-text index in previous Qdrant data, you need to execute the following commands in the **api** container to complete the process: 1. Start a bash shell inside the running Docker container: ``` docker exec -it docker-api-1 bash ``` 2. Execute the command to create the full-text-index index: ``` flask add-qdrant-full-text-index ``` - `Milvus/Zilliz` temporarily does not support Hybrid search ## What's Changed * Update README.md by @goocarlos in https://github.com/langgenius/dify/pull/1525 * fix: conversation rename always auto generate by @takatost in https://github.com/langgenius/dify/pull/1530 * new readme by @crazywoola in https://github.com/langgenius/dify/pull/1528 * doc: update readme by @crazywoola in https://github.com/langgenius/dify/pull/1536 * feat: optimize timezone of server by @takatost in https://github.com/langgenius/dify/pull/1537 * Doc/update readme patch 1 by @crazywoola in https://github.com/langgenius/dify/pull/1538 * feat: add code of conduct by @crazywoola in https://github.com/langgenius/dify/pull/1541 * fix: upload image by @zxhlyh in https://github.com/langgenius/dify/pull/1522 * feat: remove plugin page by @zxhlyh in https://github.com/langgenius/dify/pull/1544 * update images by @crazywoola in https://github.com/langgenius/dify/pull/1548 * update images by @crazywoola in https://github.com/langgenius/dify/pull/1549 * fix: upload file not clickable in firefox by @crazywoola in https://github.com/langgenius/dify/pull/1552 * feat: supports for new version of openllm by @takatost in https://github.com/langgenius/dify/pull/1554 * Add some interesting badges :) by @crazywoola in https://github.com/langgenius/dify/pull/1558 * Feat/add retriever rerank by @JohnJyong in https://github.com/langgenius/dify/pull/1560 * Feat/jp and es by @crazywoola in https://github.com/langgenius/dify/pull/1562 * feat: add retriever rank fe by @iamjoel in https://github.com/langgenius/dify/pull/1557 * fix hybrid search reranking check by @JohnJyong in https://github.com/langgenius/dify/pull/1563 * fix the error message by @JohnJyong in https://github.com/langgenius/dify/pull/1564 * chore: retrieval docs links and enchance help doc translation by @iamjoel in https://github.com/langgenius/dify/pull/1570 * feat: fetch stream compatibility enhance by @rickythink in https://github.com/langgenius/dify/pull/1551 * fix(web): Sidebar create new chat context by @JunIce in https://github.com/langgenius/dify/pull/1569 * Feat/sdk vision support by @GarfieldDai in https://github.com/langgenius/dify/pull/1531 * fix(api): patch Windows timezone set by @JunIce in https://github.com/langgenius/dify/pull/1575 * feat: configurable invite expiry time by @MatriQ in https://github.com/langgenius/dify/pull/1573 * Fix/check rerank model selected by @zxhlyh in https://github.com/langgenius/dify/pull/1577 * fix: some text by @zxhlyh in https://github.com/langgenius/dify/pull/1579 * fix multi retrieval with resource score issue by @JohnJyong in https://github.com/langgenius/dify/pull/1578 * feat: add `supported_model_types` field and filter in provider list by @takatost in https://github.com/langgenius/dify/pull/1581 * Compatible model saving error by @JohnJyong in https://github.com/langgenius/dify/pull/1582 * feat: bump version to 0.3.31 by @takatost in https://github.com/langgenius/dify/pull/1584 ## New Contributors * @rickythink made their first contribution in https://github.com/langgenius/dify/pull/1551 **Full Changelog**: https://github.com/langgenius/dify/compare/0.3.30...0.3.31