Sources

There are four ways data gets into a flow.

Files and PDFs

CSV, TSV, JSON, NDJSON, log files and PDFs. Drop them in any number and any mix.

SnoutData works out what each file is from its content, not from its file extension: a .txt that is really TSV is read as TSV, and a .csv that is really JSON is read as JSON. When a file is something we cannot get records out of, it is declined by name and the rest of the flow carries on. Recognising a file and refusing it is a real answer, not an error.

Web pages

Give a flow a URL and it fetches the page, renders it, and can crawl on from there.

The rendering matters. Many sites build their content in the browser, so fetching the HTML directly returns an empty container. SnoutData opens the page the way a browser would and reads it once it has built itself, which is why a client-side site produces the records a person can see rather than one row of boilerplate. robots.txt is respected, and requests to the same host are paced.

Databases

Any connection you already have. Read a whole table or a query.

Reads use keyset paging, so a large table does not have to be pulled in one piece, and re-running an incremental flow moves only the rows that are new.

A model, with no input file

Sometimes the data does not exist yet. Name the subject, the columns and how many rows you want, and a model writes the records. This is useful for building a test corpus or a training set, and it pairs with the fine-tuning destination.

Generation runs on your own infrastructure: your Ollama server (local or remote, with an optional bearer token), or your own vendor key. SnoutData's own AI gateway is never a generation source. Every generated record names the seed it was written from, so you can trace where a row came from.

You can seed generation from your own files or PDFs, in which case the records are written against material you supplied rather than from the model's own knowledge.