Configure Data Sources

You can manage the data sources used in a knowledge base: upload new files or remove existing ones.

Data Source Formats

Supported file formats: .csv, .htm, .html, .md, .txt, .xml, .pdf.

Maximum file size: 50 MB per file.

• Only textual data should be used as sources.
• There are no content or formatting restrictions, except for tables: the first row must contain column headers.
• Media files (images, video, etc.) will not prevent indexing, but the agent will only work with text.
• Sources can include URLs (e.g., your own documentation or online catalogs) so the agent can provide relevant links. However, URLs are treated as text—the agent cannot read or analyze the contents of a webpage directly.

Adding a Data Source

1. Go to AI Agents → Knowledge Bases.
2. Click on the knowledge base and open the Data Sources tab.
3. Click Add source and choose the method: file upload or link connection.

Specialized parsers are used for each format. For example, when processing HTML pages, scripts (JS), CSS, meta tags, and other technical elements are ignored—only page content is indexed.

File Upload

• Select the Upload File tab and add 1–100 files.

Connecting by URL

• Select the Connect via link tab and paste the link into the Source link field.
• Click Connect. You can add multiple URLs at once.

Requirements for URL sources:

• The page must be accessible without authentication.
• The page must fully load via curl, without client-side generation (no SPA).

For URL sources, you can enable automatic reindexing. Turn on the Auto reindexing on schedule toggle. Reindexing occurs when the ETag header changes. If no ETag is provided, the index updates at each scheduled run.

Managing Data Sources

The Data Sources tab displays all added sources:

• File icon: manually uploaded source
• Globe icon: source added via URL

Hovering over a source shows the Reindex button, which reloads the content and updates the index:

• Files: reindexes the current version
• URLs: fetches the latest content from the link

Clicking the three-dot menu next to a source allows you to:

• Download the file (for manually uploaded sources)
• Edit the URL or schedule (for linked sources)
• Delete the source