Dify: v0.6.11

## ✨ What’s New in v0.6.11? ✨ Hey everyone, we’ve got a fresh update for you! Version 0.6.11 is here, and it’s packed with new features, enhancements, and fixes. Let’s dive into what’s changed: ### 🚀 New Features #### 📜 Knowledge - **🔥 Firecrawl Support**: Use [Firecrawl](https://www.firecrawl.dev/) as a "Sync from website" solution in Knowledge by @iamjoel in #5226 and @JohnJyong @nickscamara @guchenhe in #5232. <img width="597" alt="image" src="https://github.com/langgenius/dify/assets/5485478/46000382-2e93-4279-bf58-b807e1bec2e2"> #### 🚀 Workflow - **Note Support**: Add notes directly into your workflows to enhance documentation and readability when editing or reviewing shared DSLs by @zxhlyh in #5164. <img width="743" alt="image" src="https://github.com/langgenius/dify/assets/5485478/3d5a7c35-e7d1-4414-8b6f-b25551269e5a"> #### 🤖 Model Support - **Jina**: Support for Jina-CLIP-v1 embedding model by @sinomoe in #5146. - **Zhipu**: Added GLM4 new models and Zhipu embedding-2 by @orangeclk in #5089. - **ERNIE-4.0-8K-Latest**: Support for ERNIE-4.0-8K-Latest by @iamsk in #5216. - **Doubao**: Support for Doubao LLM function calling by @sinomoe in #5100. - **Hunyuan**: Support for Hunyuan LLM models by @xielong in #5013. - **SiliconFlow**: Added support for SiliconFlow by @orangeclk in #5129. - **Novita.ai**: Added Novita.ai as a model provider by @jasonhp in #4961. #### 🛠️ Tools - **DuckDuckGo Tools**: Added DuckDuckGo image search, translate, and AI chat tools by @hjlarry in #5074. - **Feishu multi-dimensional table Tools**: Added Feishu multi-dimensional table operation tools by @hgnulb in #5213. #### 🗄️ Vector DB - **TiDB Vector**: Support for TiDB vector by @Weaxs in #4588. - **Chroma Vector Store**: Support for Chroma vector store by @bowenliang123 in #5015. - **Tencent Vector DB**: Support for Tencent vector DB by @quicksandznzn in #3568. #### ☁️ AWS - **EKS Pod Identity**: Upgraded boto3 library to support EKS Pod Identity by @nanasi880 in #5064. - **IAM Role for Bedrock**: Allow using IAM Role for Bedrock by @tmokmss in #5188. - **AWS S3 IAM Check**: Added AWS S3 IAM check by @JohnJyong in #5174. #### 🏗️ API Service Infrastructures - **Poetry Support**: Initial support for the Poetry build tool by @MatriQ in #4513. > ⚠️ In the next version, we will deprecate pip as the primary package management tool for Dify api service, currently Poetry and pip coexist. - **Poetry in CI**: Use Poetry as the default build system for dependency installation in CI jobs by @bowenliang123 in #5088. - **Pydantic Migration**: Modernized validation by migrating Pydantic from 1.x to 2.x by @bowenliang123 in #4592. ### Other Enhancements - **Editor User Permission**: New editor user permission profile by @chazzhou in #4435. - **Memory Messages Limit**: Set default memory messages limit to infinite by @takatost in #5002. - **DOCX Image Handling**: Improved handling of external images when extracting DOCX images by @JohnJyong in #5024. - **Dataset Delete Endpoint**: Added dataset delete endpoint by @perzeuss in #5048. - **Knowledge Retrieval**: Optimized knowledge retrieval performance by batching dataset queries by @wangiii in #4917. - **Jina Tools**: Added optional API key, proxy server, and bypass cache parameters to Jina tools by @XiaoLey in #5197. - **HTTP Request Fix**: Fixed HTTP request delete method not working by @th3n00b13 in #4975. For additional enhancements and updates, refer to the What's Changed section below. ## Upgrade Guide ### Docker compose deployments 1. Back up your customized docker-compose YAML file (optional) ```bash cd docker cp docker-compose.yaml docker-compose.yaml.$(date +%s).bak ``` 2. Get the latest code from the main branch ```bash git checkout main git pull origin main ``` 3. Stop the service，Command, please execute in the docker directory ```bash docker compose down ``` 4. Back up data ```bash tar -cvf volumes-$(date +%s).tgz volumes ``` 5. Upgrade services ```bash docker compose up -d ``` ### Source Code deployments 1. Stop API server, Worker and Web frontend Server. 2. Get the latest code from the main branch: ```bash git checkout main git pull origin main ``` 3. Update Python dependencies: - If using pip ```bash cd api pip install -r requirements.txt ``` - If using poetry ```bash cd api poetry install ``` 7. Then, let's run the migration script: - If using pip ```bash flask db upgrade ``` - If using poetry ```bash cd api poetry run python -m flask db upgrade ``` 8. Finally, run API server, Worker and Web frontend Server again. ## What's Changed * change "Import from text file" to "Import from file" by @harrywang in https://github.com/langgenius/dify/pull/4935 * feat: pricing page add llm load balancing info by @iamjoel in https://github.com/langgenius/dify/pull/4942 * Optimize knowledge retrieval performance by batching dataset quries. by @wangiii in https://github.com/langgenius/dify/pull/4917 * fix: end node limit in next step by @zxhlyh in https://github.com/langgenius/dify/pull/4945 * fix: import error in web/app/components/header/account-setting/model-provider-page/declarations.ts by @crazywoola in https://github.com/langgenius/dify/pull/4944 * chore: fix indention violations by applying E111 to E117 ruff rules by @bowenliang123 in https://github.com/langgenius/dify/pull/4925 * fix: missing dataset patch parameters in settings modal by @chazzhou in https://github.com/langgenius/dify/pull/4901 * fix: missing iterator in task pipeline by @Yeuoly in https://github.com/langgenius/dify/pull/4948 * fixing a bug of handling header row when parsing xls file, and tune xls/xlsx parsing result to be more structured by @yc-huang in https://github.com/langgenius/dify/pull/3600 * fix: duckduckgo search does not work by @hjlarry in https://github.com/langgenius/dify/pull/4949 * feat: support tidb vector by @Weaxs in https://github.com/langgenius/dify/pull/4588 * Typo in Knowledge settings by @WeepsDanky in https://github.com/langgenius/dify/pull/4958 * fix some tidb bugs by @JohnJyong in https://github.com/langgenius/dify/pull/4960 * add meta.doc_id index for tidb by @JohnJyong in https://github.com/langgenius/dify/pull/4963 * feat: add create tenant command by @takatost in https://github.com/langgenius/dify/pull/4974 * chore: rename vdb tests for PGVector and PGvectoRS by @bowenliang123 in https://github.com/langgenius/dify/pull/4973 * fix: gemini timeout error by @ZhouhaoJiang in https://github.com/langgenius/dify/pull/4955 * fix: llm selector position is incorrect in not workflow app by @iamjoel in https://github.com/langgenius/dify/pull/4982 * feat: support login by given mail by @iamjoel in https://github.com/langgenius/dify/pull/4991 * fix: add handling for non-string type in variable template parser by @Yeuoly in https://github.com/langgenius/dify/pull/4996 * fix: typo in sd3 by @Yeuoly in https://github.com/langgenius/dify/pull/5000 * feat: set default memory messages limit to infinite by @takatost in https://github.com/langgenius/dify/pull/5002 * fix: some base models cannot be selected in Azure OpenAI Service setting page by @doufa in https://github.com/langgenius/dify/pull/4985 * fix: modal z-index and cleanup by @chazzhou in https://github.com/langgenius/dify/pull/4978 * Fix/azure blob new version by @JohnJyong in https://github.com/langgenius/dify/pull/5004 * Fix: z-index in header by @JzoNgKVO in https://github.com/langgenius/dify/pull/5017 * deal the external image when extract docx image by @JohnJyong in https://github.com/langgenius/dify/pull/5024 * Fix language setting not success by @doufa in https://github.com/langgenius/dify/pull/5023 * fix(style): some style issues by @HiChen404 in https://github.com/langgenius/dify/pull/5029 * feat: feature modal style by @zxhlyh in https://github.com/langgenius/dify/pull/5032 * improve: generalize vector factory classes and vector type by @bowenliang123 in https://github.com/langgenius/dify/pull/5033 * 🔧 fix docker-compose ssrf_proxy service WARNING: You should probably remove '::/0' from the ACL named 'all' by @takuya-o in https://github.com/langgenius/dify/pull/5005 * fix: update presence_penalty configuration for wenxin AI ernie-4.0-8k and ernie-3.5-8k models by @XiaoLey in https://github.com/langgenius/dify/pull/5039 * fix: issue where an error occurs when invoking TTS without selecting a voice by @takatost in https://github.com/langgenius/dify/pull/5046 * fix: wrong link to web app repo in chatflow mode by @perzeuss in https://github.com/langgenius/dify/pull/5062 * feat: add dataset delete endpoint by @perzeuss in https://github.com/langgenius/dify/pull/5048 * chore: update maas model provider description by @sinomoe in https://github.com/langgenius/dify/pull/5056 * 🔧 Fix(docker/volumes/ssrf_proxy/squid.conf): The squid process on ssrf_proxy docker service crashes at startup by @takuya-o in https://github.com/langgenius/dify/pull/5050 * build: initial support for poetry build tool by @MatriQ in https://github.com/langgenius/dify/pull/4513 * Arabic README.md by @ra2230 in https://github.com/langgenius/dify/pull/5078 * fix: dependency package versions are not synchronized to requirements.txt by @takatost in https://github.com/langgenius/dify/pull/5084 * feat: add glm4 new models and zhipu embedding-2 by @orangeclk in https://github.com/langgenius/dify/pull/5089 * Fix: infinite loading not work when message is too short by @JzoNgKVO in https://github.com/langgenius/dify/pull/5075 * fix: autoHeightTextarea dimensions in Firefox by @ZuzooVn in https://github.com/langgenius/dify/pull/4891 * fix: Google HL Parameter for SearchApi by @ugyuji in https://github.com/langgenius/dify/pull/5071 * feat: add duckduckgo img search, translate, ai chat by @hjlarry in https://github.com/langgenius/dify/pull/5074 * fix: #4970 by @iamjoel in https://github.com/langgenius/dify/pull/5093 * dalle3 add style consistency parameter by @charli117 in https://github.com/langgenius/dify/pull/5067 * fix: remote_url doesn't work for gemini by @rerorero in https://github.com/langgenius/dify/pull/5090 * build: use Poetry as default build system for dependency installation in CI jobs by @bowenliang123 in https://github.com/langgenius/dify/pull/5088 * feat: support doubao llm function calling by @sinomoe in https://github.com/langgenius/dify/pull/5100 * chore: workflow syncing modal by @zxhlyh in https://github.com/langgenius/dify/pull/5108 * fix(typo): misspelling by @HiChen404 in https://github.com/langgenius/dify/pull/5094 * feat: support hunyuan llm models by @xielong in https://github.com/langgenius/dify/pull/5013 * fix: select field not work when it is not required by @hjlarry in https://github.com/langgenius/dify/pull/5101 * fixed a typo and grammar error in sampled app by @harrywang in https://github.com/langgenius/dify/pull/5061 * Fix: fix load_yaml logging, Avoid setting the log level to warning by @fishisnow in https://github.com/langgenius/dify/pull/5019 * chore: sync pyproject.toml from requirements.txt by @takatost in https://github.com/langgenius/dify/pull/5130 * remove dalle3 seed by @charli117 in https://github.com/langgenius/dify/pull/5136 * feat: support siliconflow by @orangeclk in https://github.com/langgenius/dify/pull/5129 * chore: update question classifier prompt by @0xff-dev in https://github.com/langgenius/dify/pull/5137 * fix localai,baichuan agent chat err #4943 by @yanghx-git in https://github.com/langgenius/dify/pull/5138 * chore: update qdrant_vector.py by @eltociear in https://github.com/langgenius/dify/pull/5128 * chore:add some new api version for azure openai by @leslie2046 in https://github.com/langgenius/dify/pull/5142 * feat: support jina-clip-v1 embedding model by @sinomoe in https://github.com/langgenius/dify/pull/5146 * fix: style misalignment and inconsistency by @hjlarry in https://github.com/langgenius/dify/pull/5149 * fix: front end error when same tool is called twice at once by @chazzhou in https://github.com/langgenius/dify/pull/5068 * chore(docs): fix minor small typos by @kurokobo in https://github.com/langgenius/dify/pull/5124 * Upgrade boto3 library to support EKS Pod Identity. by @nanasi880 in https://github.com/langgenius/dify/pull/5064 * Fix: google storage init with sa and download by @kazukitakamatsu in https://github.com/langgenius/dify/pull/5054 * feat: support Chroma vector store by @bowenliang123 in https://github.com/langgenius/dify/pull/5015 * feat: introduce APP_MAX_EXECUTION_TIME by @Yeuoly in https://github.com/langgenius/dify/pull/5167 * improve: CI experience by @Yeuoly in https://github.com/langgenius/dify/pull/5168 * improve: mordernizing validation by migrating pydantic from 1.x to 2.x by @bowenliang123 in https://github.com/langgenius/dify/pull/4592 * fix: Vertex AI Anthropic models authentication failed by @hellof20 in https://github.com/langgenius/dify/pull/4971 * fix: issues by pydantic2 upgrade by @takatost in https://github.com/langgenius/dify/pull/5171 * fix: JSON mode with an image doesn't work for Gemini by @rerorero in https://github.com/langgenius/dify/pull/5169 * fix: milvus_vector default dataset index_struct type from weaviate to milvus by @sagarey in https://github.com/langgenius/dify/pull/5098 * fix: pydantic2 error by @takatost in https://github.com/langgenius/dify/pull/5172 * chore(deps): bump azure-identity from 1.15.0 to 1.16.1 in /api by @dependabot in https://github.com/langgenius/dify/pull/5116 * refactor: Delete the dataset to verify whether it is in use by @gubinjie in https://github.com/langgenius/dify/pull/5112 * chore(deps): bump authlib from 1.2.0 to 1.3.1 in /api by @dependabot in https://github.com/langgenius/dify/pull/5115 * Chore: refactor embedded chatbot by @JzoNgKVO in https://github.com/langgenius/dify/pull/5125 * fix(core/tools): Fix the issue with iterating over None in `_transform_tool_parameters_type`. by @laipz8200 in https://github.com/langgenius/dify/pull/5190 * Fix: conversation id display & support copy by @JzoNgKVO in https://github.com/langgenius/dify/pull/5195 * Modify docs in JP by @HanqingZ in https://github.com/langgenius/dify/pull/5185 * fix: workflow as tool create error by type misuse by @takatost in https://github.com/langgenius/dify/pull/5205 * chore: remove bump-pydantic dependency by @bowenliang123 in https://github.com/langgenius/dify/pull/5177 * chore: make the Celery command more noticeable by @doufa in https://github.com/langgenius/dify/pull/5203 * feat: allow to use IAM Role for Bedrock by @tmokmss in https://github.com/langgenius/dify/pull/5188 * add aws s3 iam check by @JohnJyong in https://github.com/langgenius/dify/pull/5174 * feat: workflow add note node by @zxhlyh in https://github.com/langgenius/dify/pull/5164 * Add novita.ai as model provider by @jasonhp in https://github.com/langgenius/dify/pull/4961 * fix(model_providers/ollama): Fix OllamaLargeLanguageModel to correctly set the stop option by @laipz8200 in https://github.com/langgenius/dify/pull/5217 * support ERNIE-4.0-8K-Latest by @iamsk in https://github.com/langgenius/dify/pull/5216 * feat: Add Optional API Key, Proxy Server, and Bypass Cache Parameters to Jina Tools by @XiaoLey in https://github.com/langgenius/dify/pull/5197 * fix: add repo check for build-push.yml by @GOODBOY008 in https://github.com/langgenius/dify/pull/5141 * feat: support tencent vector db by @quicksandznzn in https://github.com/langgenius/dify/pull/3568 * Update README, deploy dify with YAML file on Kubernetes by @Winson-030 in https://github.com/langgenius/dify/pull/5131 * Fix: http_request delete method not working by @th3n00b13 in https://github.com/langgenius/dify/pull/4975 * feat: new editor user permission profile by @chazzhou in https://github.com/langgenius/dify/pull/4435 * fix: number variable cause type error in openai moderation by @rerorero in https://github.com/langgenius/dify/pull/5222 * fix: workspace member's last_active should be last_active_time, but not last_login_time by @DomKing in https://github.com/langgenius/dify/pull/4906 * chore: development script for syncing Poetry lockfile by @bowenliang123 in https://github.com/langgenius/dify/pull/5170 * feat(Tools): Add Feishu multi-dimensional table operation function by @hgnulb in https://github.com/langgenius/dify/pull/5213 * feat: support firecrawl frontend code by @iamjoel in https://github.com/langgenius/dify/pull/5226 * fix: initialize site with customized icon and icon_background by @kurokobo in https://github.com/langgenius/dify/pull/5227 * fix: allow the name and icon of the web app to be set independently of that of the bot itself by @kurokobo in https://github.com/langgenius/dify/pull/5225 * Update README.md by @takatost in https://github.com/langgenius/dify/pull/5228 * fix: z-index by @crazywoola in https://github.com/langgenius/dify/pull/5229 * fix: note editor italic by @zxhlyh in https://github.com/langgenius/dify/pull/5230 * update tooltip by @guchenhe in https://github.com/langgenius/dify/pull/5235 * Feat/firecrawl data source by @JohnJyong in https://github.com/langgenius/dify/pull/5232 * version to 0.6.11 by @takatost in https://github.com/langgenius/dify/pull/5224 * fix: ExtractSetting optional value missing None as default val by @takatost in https://github.com/langgenius/dify/pull/5238 ## New Contributors * @harrywang made their first contribution in https://github.com/langgenius/dify/pull/4935 * @wangiii made their first contribution in https://github.com/langgenius/dify/pull/4917 * @yc-huang made their first contribution in https://github.com/langgenius/dify/pull/3600 * @ZhouhaoJiang made their first contribution in https://github.com/langgenius/dify/pull/4955 * @takuya-o made their first contribution in https://github.com/langgenius/dify/pull/5005 * @XiaoLey made their first contribution in https://github.com/langgenius/dify/pull/5039 * @ra2230 made their first contribution in https://github.com/langgenius/dify/pull/5078 * @rerorero made their first contribution in https://github.com/langgenius/dify/pull/5090 * @fishisnow made their first contribution in https://github.com/langgenius/dify/pull/5019 * @0xff-dev made their first contribution in https://github.com/langgenius/dify/pull/5137 * @yanghx-git made their first contribution in https://github.com/langgenius/dify/pull/5138 * @kurokobo made their first contribution in https://github.com/langgenius/dify/pull/5124 * @nanasi880 made their first contribution in https://github.com/langgenius/dify/pull/5064 * @kazukitakamatsu made their first contribution in https://github.com/langgenius/dify/pull/5054 * @laipz8200 made their first contribution in https://github.com/langgenius/dify/pull/5190 * @HanqingZ made their first contribution in https://github.com/langgenius/dify/pull/5185 * @tmokmss made their first contribution in https://github.com/langgenius/dify/pull/5188 * @jasonhp made their first contribution in https://github.com/langgenius/dify/pull/4961 * @GOODBOY008 made their first contribution in https://github.com/langgenius/dify/pull/5141 * @quicksandznzn made their first contribution in https://github.com/langgenius/dify/pull/3568 * @Winson-030 made their first contribution in https://github.com/langgenius/dify/pull/5131 * @th3n00b13 made their first contribution in https://github.com/langgenius/dify/pull/4975 * @hgnulb made their first contribution in https://github.com/langgenius/dify/pull/5213 **Full Changelog**: https://github.com/langgenius/dify/compare/0.6.10...0.6.11