Skip to main content

Document Loaders

A document loader is the bridge between where your content lives and the store that will search it. Flowera ships 38 of them.

What a loader does​

Three jobs, in order:

  1. Fetch the content — read a file, crawl a page, call an API.
  2. Extract text from it, handling the format's quirks.
  3. Split that text into chunks, using the splitter you choose.

The output is a set of chunks belonging to the store. One store can hold many loaders, and each keeps its own configuration — a PDF manual and a scraped FAQ page can live side by side and be searched together.

Choosing a loader type​

If your content is…Use
A PDF, Word, Excel, PowerPoint, EPUB, CSV, or plain text filePdf File, Microsoft Word, Microsoft Excel, Microsoft PowerPoint, Epub File, Csv File, Text File
A folder of mixed filesFile Loader, which takes several files at once
A public web pageCheerio Web Scraper (fast) or Playwright / Puppeteer (for JavaScript-heavy pages)
A whole siteApify Website Content Crawler, FireCrawl, or Spider
In a SaaS toolNotion, Confluence, Jira, Github, GitBook, Airtable, Figma
In Google WorkspaceGoogle Drive, Google Sheets
In cloud storageS3, S3 Directory
Behind your own APIAPI Loader, or Custom Document Loader for logic the others can't express

Web-scraping and SaaS loaders need a credential for the service. File loaders don't.

Configuring a loader​

Press Add Document Loader in the store, pick the type, and fill in its fields. Most loaders ask for three kinds of thing:

  • The source — a file to upload, a URL, a Notion database, an S3 bucket.
  • A text splitter — how to cut the extracted text into chunks. See Chunking. Leaving it empty stores each document as one chunk, which is only right for content that's already short.
  • Metadata — optional key/value pairs attached to every chunk from this loader. Useful for recording where a passage came from, so an answer can cite it.

Previewing chunks before saving​

Every loader has Preview Chunks. Use it. Every time.

The preview shows exactly what will be stored — the actual text of each chunk, in order, with the total count. Thirty seconds here catches the problems that are otherwise invisible until a customer gets a bad answer:

  • A PDF that extracted as gibberish — it's a scan, and needs OCR before Flowera can read it.
  • Chunks that cut mid-sentence — the splitter or the chunk size is wrong for this document.
  • A web page that loaded the cookie banner and nothing else — the page needs a JavaScript-capable scraper.
  • One enormous chunk — no splitter was selected.

Chunks that look wrong in the preview will retrieve badly, and no amount of prompt engineering downstream will fix them.

After saving​

A saved loader is processed in the background — the store shows Stale until that finishes, then Sync. Either way the content isn't searchable until you upsert it.