Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions step-types/llm/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,32 @@ See [Reliability](/step-types/llm/reliability).

See [Web Search](/step-types/llm/reasoning-web-search#web-search).

## Files

`with.files` gives the model files to read with the prompt: a path, a glob pattern, a list of them, or a list an earlier step published, such as the emails `mail.search` found with `save_attachments: true`.

```yaml
steps:
- id: read
action: chat.completion
with:
prompt: Read the invoice and give its number, date, and total.
files: ~/Downloads/invoice-*.pdf
output_schema:
type: object
properties:
number: {type: string}
date: {type: string}
total: {type: number}
required: [number]
```

- PDF, Word (`.docx`), Excel (`.xlsx`) and text files go to the model as text ahead of the prompt, each in a `<file name="…">` block, a PDF page by page. Text in Shift_JIS is decoded.
- A PDF page without text of its own, such as a scan, goes as an image, and so does an image file (PNG, JPEG, GIF, WebP), scaled down to a long edge of 1568 pixels when larger. The text names each image by its place: `[Image 1 is receipt.jpg.]`, or `--- page 2: image 1 ---`.
- A file may hold up to 50 MB; together the files may hold 1,000,000 characters of text and 50 images. A missing file, a pattern matching nothing, and a file of another kind fail the step before any request.
- The session keeps the text, so a later step of the session still reads it; the images go with the one request. A push-back resends the feedback without the files.
- `files` belongs on the step, under `with`; a DAG-level `llm` block refuses it.

## Structured output

Set `output_schema` on the step, next to `with`, when later steps need values rather than prose:
Expand Down
23 changes: 23 additions & 0 deletions writing-workflows/scheduling.md
Original file line number Diff line number Diff line change
Expand Up @@ -392,6 +392,29 @@ Live scheduled runs are also blocked while a catchup run is queued for the same

At most 1000 missed runs are buffered per DAG. If more than 1000 runs were missed, only the 1000 most recent are replayed.

## Watched Folders

A DAG that acts on files as they land in a folder watches the folder instead of polling it on a schedule:

```yaml
watch:
path: ~/Downloads/invoices
pattern: "*.pdf"
interval: 30s
steps:
- id: read
action: chat.completion
with:
prompt: Read the invoice and give its number, date, and total.
files: ${WATCH_FILE}
```

- `path` is the folder; `pattern` is a glob the file's name must match (empty, or `*`, matches every file); `interval` is how often the folder is read, 10s by default and at least 1s. A DAG with `watch` needs no `schedule`; both may be set.
- A file counts as arrived once its size and modification time are the same on two readings in a row, so a file still being copied waits. Each arrival starts one run, queued like a webhook's, with the file's absolute path as `WATCH_FILE` and its name as `WATCH_NAME`; the run's trigger type is `file`.
- The scheduler remembers each file by its size and time, so a file runs once and runs again only when it is replaced. The files a folder holds when the watch starts, or when `path` or `pattern` change, are left alone; a file that arrives while the scheduler is off is new when it returns.
- Hidden files, Office lock files (`~$`), folders, and files still being written (`.tmp`, `.part`, `.partial`, `.crdownload`, `.download`) never match. A folder that cannot be read is reported once and read again every interval. A suspended DAG's files wait and run once it resumes.
- `WATCH_FILE` and `WATCH_NAME` are chosen by whoever put the file in the folder: a `run:` step reads them with `printenv` or through `template.render` rather than interpolating them into its command.

## Pausing the Scheduler

Suspending a single DAG stops that DAG only. To freeze every scheduled run at once, for a maintenance window or an incident, pause the scheduler.
Expand Down
Loading