S3 Support apart from local upload.

[Arshil-Akkala]

Hi, was curious to know about s3 support to Ragflow. I was digging in the code and found some relative code already present in the backend for s3. From UI, are you going to give an object browser for an S3? so currently the local-upload is being taken care by a minio bucket. I want to know the plan, like with an s3 how it is going to be.

[dosubot[bot]]

RAGFlow supports AWS S3 as a backend storage option, configurable via the service_conf.yaml and the STORAGE_IMPL=AWS_S3 environment variable. You can switch from MinIO to S3 by providing your AWS credentials and bucket details, then rebuilding and restarting the service source.

On the UI side, there is currently a tab for S3 upload in the file upload modal, but it only displays a "Coming Soon" message. There is no implementation of an S3 object browser or any UI for browsing or managing S3 buckets or objects—only local upload and management features are available right now source. The presence of the S3 tab suggests upload support is planned, but there’s no evidence in the codebase or recent PRs of advanced S3 browsing or management features being developed source.

So, S3 integration is available for backend storage, but the UI does not yet offer an object browser or S3 file management—just a placeholder for future S3 upload functionality.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[Arshil-Akkala]

"S3 integration is available for backend storage", so how will the "/upload" API work in this case? As for local upload, we are actually giving the file to the API. So i am curious how will the flow be for S3 integration or any additional changes have to be done?

[dosubot[bot]]

Yes, that's correct. When you configure S3 as the backend, the /upload API still expects files as multipart form data. The backend then stores the uploaded files and metadata in S3 (not MinIO), using the storage abstraction layer. No changes are needed to the API or client-side upload flow—just update your backend configuration to use S3, and all uploads will go directly to your S3 bucket as intended. Source

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[Arshil-Akkala]

Actually, my intention was to replace the local file upload with an S3. So what is the plan when supporting S3, will we be able to select a file from the S3 object browser and that in turn will call the /upload API as is..in multipart form format?

[dosubot[bot]]

Currently, RAGFlow does not have an S3 object browser or any UI for selecting files directly from S3. The S3 tab in the file upload dialog just shows a "Coming Soon" message—there’s no implemented functionality for browsing or picking files from your S3 bucket in the UI, and no workflow for triggering the /upload API with files selected from S3. All uploads still use multipart form data from local files, regardless of whether your backend storage is MinIO or S3. There are no code changes or roadmap notes indicating this feature is planned or in development yet source, source.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[Arshil-Akkala]

So, is the planning for that in progress?

[dosubot[bot]]

There’s no evidence that planning or development for an S3 object browser or direct S3 file selection in the UI is currently in progress. The S3 tab in the upload dialog just shows a "Coming Soon" message, and there are no comments, TODOs, or roadmap notes in the codebase or recent PRs indicating this feature is being worked on or planned at this time source source.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[Arshil-Akkala]

I do have another question, currently the backend is tied to mysql, I have seen postgres related config in the service_config.yaml. So if I comment mysql config and uncomment the postgres related config and restart the service, will the backend automatically shift to postgres db?

[dosubot[bot]]

Yes, you can switch RAGFlow's backend from MySQL to PostgreSQL by updating your configuration (commenting out MySQL and enabling PostgreSQL in service_config.yaml, setting DB_TYPE=postgres in your environment, and providing the necessary PostgreSQL connection details), then restarting the service. The backend will automatically connect to PostgreSQL, and the ORM will handle schema creation and migrations for you—no manual migration steps are needed for schema or tables if you’re starting fresh. If you have existing data in MySQL that you want to keep, you’ll need to migrate that data to PostgreSQL yourself, as RAGFlow does not automate data transfer between databases. The backend expects the target PostgreSQL database to be created and accessible; tables and schema will be initialized automatically on service start. Reference

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[Arshil-Akkala]

Is it possible to plug in any other vector database?

[dosubot[bot]]

RAGFlow supports plugging in other vector databases, but there are strict requirements. Out of the box, only Elasticsearch, OpenSearch, Infinity, and Vespa are supported, because RAGFlow needs advanced features like BM25 scoring, phrase/sloppy queries, query term/field weights, minimum should match, dynamic query pruning (WAND), and rank feature queries. Any new vector database must implement all these features to be compatible—databases like Faiss or Milvus are not currently supported because they don't meet these requirements yet reference reference.

Technically, RAGFlow uses an abstraction layer (the DocStoreConnection interface) for vector database integration. To add a new backend, you’d need to implement this interface with all required methods (index management, CRUD, search, etc.) and ensure your database supports the necessary search features reference. There’s no plugin system—each new connector is a manual code addition. There is active development to add more engines (e.g., Baidu VDB) reference.

So, it’s possible to plug in another vector database, but only if it fully supports all the required search and ranking features and you implement the connector class.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[Arshil-Akkala]

What are the supported files for upload and parsing?

[dosubot[bot]]

RAGFlow supports uploading and parsing a wide range of file types. The most common formats include: PDF, DOCX, XLSX, XLS, PPT, PPTX, MD, MDX, TXT, CSV, JSON, EML, HTML, JPEG, JPG, PNG, TIF, and GIF. The exact set of supported formats depends on the chunking template you select for your dataset. For example, the "General" template supports all the above formats, while specialized templates like "Q&A" or "Table" focus on spreadsheet and text formats.

Some important details:

• For Q&A parsing, CSV and TXT files must use TAB (\t) as the separator and be UTF-8 encoded, with the question in the first column and the answer in the second. XLSX files must have two adjacent columns for Q&A, no header required.
• DOCX parsing extracts text and tables, but embedded images are ignored unless you extract and reference them as URLs in TXT or HTML.
• PDF parsing includes text, tables, figures, and layout info, with OCR for images inside PDFs.
• Image formats (JPEG, JPG, PNG, TIF, GIF) are supported for picture datasets.
• Markdown (.md) files are fully supported, including tables and code blocks.

You can find a detailed mapping of chunking templates to supported file formats in the official documentation.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[Arshil-Akkala]

can you tell me based on each chunking template, what all file types are supported?

[Arshil-Akkala]

Is laws chunking method implemented?

[ZhenhangTung]

We're working on it. Expected to be released in our next version.

[Arshil-Akkala]

Hi, thanks for the response. One more question, can you give me what all chunking templates are present currently in production?