diff --git a/step-types/llm/index.md b/step-types/llm/index.md index 076ba34..ea872da 100644 --- a/step-types/llm/index.md +++ b/step-types/llm/index.md @@ -172,6 +172,32 @@ See [Reliability](/step-types/llm/reliability). See [Web Search](/step-types/llm/reasoning-web-search#web-search). +## Files + +`with.files` gives the model files to read with the prompt: a path, a glob pattern, a list of them, or a list an earlier step published, such as the emails `mail.search` found with `save_attachments: true`. + +```yaml +steps: + - id: read + action: chat.completion + with: + prompt: Read the invoice and give its number, date, and total. + files: ~/Downloads/invoice-*.pdf + output_schema: + type: object + properties: + number: {type: string} + date: {type: string} + total: {type: number} + required: [number] +``` + +- PDF, Word (`.docx`), Excel (`.xlsx`) and text files go to the model as text ahead of the prompt, each in a `` block, a PDF page by page. Text in Shift_JIS is decoded. +- A PDF page without text of its own, such as a scan, goes as an image, and so does an image file (PNG, JPEG, GIF, WebP), scaled down to a long edge of 1568 pixels when larger. The text names each image by its place: `[Image 1 is receipt.jpg.]`, or `--- page 2: image 1 ---`. +- A file may hold up to 50 MB; together the files may hold 1,000,000 characters of text and 50 images. A missing file, a pattern matching nothing, and a file of another kind fail the step before any request. +- The session keeps the text, so a later step of the session still reads it; the images go with the one request. A push-back resends the feedback without the files. +- `files` belongs on the step, under `with`; a DAG-level `llm` block refuses it. + ## Structured output Set `output_schema` on the step, next to `with`, when later steps need values rather than prose: diff --git a/writing-workflows/scheduling.md b/writing-workflows/scheduling.md index 7edf09c..a21f622 100644 --- a/writing-workflows/scheduling.md +++ b/writing-workflows/scheduling.md @@ -392,6 +392,29 @@ Live scheduled runs are also blocked while a catchup run is queued for the same At most 1000 missed runs are buffered per DAG. If more than 1000 runs were missed, only the 1000 most recent are replayed. +## Watched Folders + +A DAG that acts on files as they land in a folder watches the folder instead of polling it on a schedule: + +```yaml +watch: + path: ~/Downloads/invoices + pattern: "*.pdf" + interval: 30s +steps: + - id: read + action: chat.completion + with: + prompt: Read the invoice and give its number, date, and total. + files: ${WATCH_FILE} +``` + +- `path` is the folder; `pattern` is a glob the file's name must match (empty, or `*`, matches every file); `interval` is how often the folder is read, 10s by default and at least 1s. A DAG with `watch` needs no `schedule`; both may be set. +- A file counts as arrived once its size and modification time are the same on two readings in a row, so a file still being copied waits. Each arrival starts one run, queued like a webhook's, with the file's absolute path as `WATCH_FILE` and its name as `WATCH_NAME`; the run's trigger type is `file`. +- The scheduler remembers each file by its size and time, so a file runs once and runs again only when it is replaced. The files a folder holds when the watch starts, or when `path` or `pattern` change, are left alone; a file that arrives while the scheduler is off is new when it returns. +- Hidden files, Office lock files (`~$`), folders, and files still being written (`.tmp`, `.part`, `.partial`, `.crdownload`, `.download`) never match. A folder that cannot be read is reported once and read again every interval. A suspended DAG's files wait and run once it resumes. +- `WATCH_FILE` and `WATCH_NAME` are chosen by whoever put the file in the folder: a `run:` step reads them with `printenv` or through `template.render` rather than interpolating them into its command. + ## Pausing the Scheduler Suspending a single DAG stops that DAG only. To freeze every scheduled run at once, for a maintenance window or an incident, pause the scheduler.