<!-- Source: https://docs.snoutdata.com/flows/sources -->

# Sources

There are four ways data gets into a flow.

## Files and PDFs

CSV, TSV, JSON, NDJSON, log files and PDFs. Drop them in any number and any mix.

SnoutData works out what each file is **from its content**, not from its file extension: a `.txt`
that is really TSV is read as TSV, and a `.csv` that is really JSON is read as JSON. When a file
is something we cannot get records out of, it is **declined by name** and the rest of the flow
carries on. Recognising a file and refusing it is a real answer, not an error.

## Web pages

Give a flow a URL and it fetches the page, renders it, and can crawl on from there.

The rendering matters. Many sites build their content in the browser, so fetching the HTML
directly returns an empty container. SnoutData opens the page the way a browser would and reads
it once it has built itself, which is why a client-side site produces the records a person can
see rather than one row of boilerplate. `robots.txt` is respected, and requests to the same host
are paced.

## Databases

Any connection you already have. Read a whole table or a query.

Reads use keyset paging, so a large table does not have to be pulled in one piece, and
re-running an [incremental flow](https://docs.snoutdata.com/flows/schedules-and-runs) moves only the rows that are new.

## A model, with no input file

Sometimes the data does not exist yet. Name the subject, the columns and how many rows you
want, and a model writes the records. This is useful for building a test corpus or a training
set, and it pairs with the [fine-tuning destination](https://docs.snoutdata.com/flows/destinations).

Generation runs on **your own** infrastructure: your Ollama server (local or remote, with an
optional bearer token), or your own vendor key. SnoutData's own AI gateway is never a
generation source. Every generated record names the seed it was written from, so you can trace
where a row came from.

You can seed generation from your own files or PDFs, in which case the records are written
against material you supplied rather than from the model's own knowledge.
