Commit c30f069
V4.9.11 feature (#4969)
* Feat: Images dataset collection (#4941)
* New pic (#4858)
* 更新数据集相关类型,添加图像文件ID和预览URL支持;优化数据集导入功能,新增图像数据集处理组件;修复部分国际化文本;更新文件上传逻辑以支持新功能。
* 与原先代码的差别
* 新增 V4.9.10 更新说明,支持 PG 设置`systemEnv.hnswMaxScanTuples`参数,优化 LLM stream 调用超时,修复全文检索多知识库排序问题。同时更新数据集索引,移除 datasetId 字段以简化查询。
* 更换成fileId_image逻辑,并增加训练队列匹配的逻辑
* 新增图片集合判断逻辑,优化预览URL生成流程,确保仅在数据集为图片集合时生成预览URL,并添加相关日志输出以便调试。
* Refactor Docker Compose configuration to comment out exposed ports for production environments, update image versions for pgvector, fastgpt, and mcp_server, and enhance Redis service with a health check. Additionally, standardize dataset collection labels in constants and improve internationalization strings across multiple languages.
* Enhance TrainingStates component by adding internationalization support for the imageParse training mode and update defaultCounts to include imageParse mode in trainingDetail API.
* Enhance dataset import context by adding additional steps for image dataset import process and improve internationalization strings for modal buttons in the useEditTitle hook.
* Update DatasetImportContext to conditionally render MyStep component based on data source type, improving the import process for non-image datasets.
* Refactor image dataset handling by improving internationalization strings, enhancing error messages, and streamlining the preview URL generation process.
* 图片上传到新建的 dataset_collection_images 表,逻辑跟随更改
* 修改了除了controller的其他部分问题
* 把图片数据集的逻辑整合到controller里面
* 补充i18n
* 补充i18n
* resolve评论:主要是上传逻辑的更改和组件复用
* 图片名称的图标显示
* 修改编译报错的命名问题
* 删除不需要的collectionid部分
* 多余文件的处理和改动一个删除按钮
* 除了loading和统一的imageId,其他都resolve掉的
* 处理图标报错
* 复用了MyPhotoView并采用全部替换的方式将imageFileId变成imageId
* 去除不必要文件修改
* 报错和字段修改
* 增加上传成功后删除临时文件的逻辑以及回退一些修改
* 删除path字段,将图片保存到gridfs内,并修改增删等操作的代码
* 修正编译错误
---------
Co-authored-by: archer <[email protected]>
* perf: image dataset
* feat: insert image
* perf: image icon
* fix: training state
---------
Co-authored-by: Zhuangzai fa <[email protected]>
* fix: ts (#4948)
* Thirddatasetmd (#4942)
* add thirddataset.md
* fix thirddataset.md
* fix
* delete wrong png
---------
Co-authored-by: dreamer6680 <[email protected]>
* perf: api dataset code
* perf: log
* add secondary.tsx (#4946)
* add secondary.tsx
* fix
---------
Co-authored-by: dreamer6680 <[email protected]>
* perf: multiple menu
* perf: i18n
* feat: parse queue (#4960)
* feat: parse queue
* feat: sync parse queue
* fix thirddataset.md (#4962)
* fix thirddataset-4.png (#4963)
* feat: Dataset template import (#4934)
* 模版导入部分除了文档还没写
* 修复模版导入的 build 错误
* Document production
* compress pictures
* Change some constants to variables
---------
Co-authored-by: Archer <[email protected]>
* perf: template import
* doc
* llm pargraph
* bocha tool
* fix: del collection
---------
Co-authored-by: Zhuangzai fa <[email protected]>
Co-authored-by: dreamer6680 <[email protected]>
Co-authored-by: dreamer6680 <[email protected]>1 parent bb810a4 commit c30f069
File tree
198 files changed
+4927
-2283
lines changed- docSite
- assets/imgs
- template
- content/zh-cn/docs
- development
- openapi
- upgrading
- guide/knowledge_base
- packages
- global
- common
- file
- frequenctLimit
- string
- core/dataset
- apiDataset
- image
- training
- plugins/src/bocha
- service
- common
- api
- buffer/rawText
- file
- gridfs
- system
- frequencyLimit
- timerLock
- core
- ai
- config/provider
- dataset
- apiDataset
- custom
- feishuDataset
- yuqueDataset
- collection
- data
- image
- search
- training
- workflow/dispatch
- agent/runTool
- chat
- support/permission/dataset
- worker
- web
- components/common
- Icon
- icons
- core
- app/simpleMode
- dataset
- workflow/template
- MyMenu
- MyPopover
- i18n
- en
- zh-CN
- zh-Hant
- projects/app/src
- components
- MyImage
- common/NextHead
- core
- app
- chat
- ChatContainer/ChatBox/components
- components
- dataset
- global/core/dataset
- pageComponents
- account/team/OperationLog
- processors
- chat/ChatQuoteList
- dataset
- detail
- CollectionCard
- Import
- components
- diffSource
- Info
- components
- components
- data
- list
- pages
- account/team
- api
- admin
- common/file
- core
- app
- chat
- quote
- dataset
- apiDataset
- collection
- create
- data
- v2
- file
- image
- training
- v1/audio
- dataset/list
- service
- common/system
- core
- chat
- dataset
- data
- queues
- training
- web
- common/file
- core/dataset
- components
- context
- image
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
198 files changed
+4927
-2283
lines changedLoading
Loading
Loading
Loading
Loading
Loading
Loading
Loading
Loading
0 commit comments