AFM — Foundation Models HTTP Server

A SwiftUI app for macOS 27 / iOS 27 that serves Apple's FoundationModels through a subset of the OpenAI Responses API. Requires Xcode 27 / Swift 6.4. SwiftNIO owns HTTP parsing and framing; FoundationModels owns inference.

Run

Open AFM.xcodeproj in Xcode and run the app target’s AFM scheme (existing local checkouts may still have an App scheme). Choose My Mac for native macOS, My Mac (Mac Catalyst) for Catalyst, or an iOS device/simulator. The app target creates a signed .app bundle with identifier tools.ars.afm. Package.swift exposes the Server and ChatHistory libraries and their tests; it does not create a bare UI executable. UIKit/Catalyst requires a real app bundle at startup. The main window is a chat workspace with a searchable conversation sidebar. Open Server settings from the toolbar to configure the server. It starts automatically at launch and whenever the app becomes active again. On macOS, server settings have a separate window, also accessible through the menu bar item; closing the chat window does not stop the app or server. On iOS, settings open as a separate sheet; background serving remains subject to iOS execution limits.

The chat uses URLSession and SSE against the same endpoint used by external clients. Complete conversation turns are sent with every request; server sessions are isolated per request. Stop cancels the HTTP request and generation. Interrupted, failed and output-limited responses remain visible, with a Retry response action, but are excluded from later inference history along with their user turn.

Conversations and SwiftData storage

Conversation history now uses a versioned SwiftData schema in Application Support at AFM/History.store (inside the app container when sandboxed). SwiftData manages the SQLite store and its external binary-data files as one persistence unit.

• StoredConversation: stable ID, title, timestamps, sequence counter, message count, and a short sidebar preview. Deleting it cascades to its messages.
• StoredMessage: conversation, sequence, author, generation state, timestamp, actual model, error/context notice, and ordered content parts. Drafts are user messages with state draft and sequence -1; sending promotes the same record.
• ContentPart: common identity, owning message, position, and content type. TextPart stores searchable String content. ImagePart stores JPEG Data with @Attribute(.externalStorage) and image metadata. Message deletion cascades through the base relationship to both subclasses.
• Internal store metadata records selection and successful legacy migration. CloudKit synchronization is explicitly disabled.

The sidebar fetches 50 conversation summaries at a time. Search queries titles and TextPart contents in the store, without loading image bytes. The chat opens with the newest 60 messages; Load earlier messages fetches the preceding page. Inactive conversations are released from the UI cache, except for an ongoing response. Draft edits and stream deltas update only the affected model records; there is no periodic whole-archive JSON encoding. Saves remain batched at roughly 700 ms and occur at turn completion and scene deactivation.

Inference uses up to 80 recent stored messages, retaining complete user/assistant pairs and enforcing the four-image request budget. UI pagination does not change that inference window. A notice explains omitted context; the full saved history remains available. Failed, incomplete, and interrupted turns remain visible but are excluded from later inference. Pending responses reopen as interrupted.

On first use, existing AFM/conversations.json and AFM/Images files are imported in one transaction, preserving conversation/message identities, order, drafts, response states, and image contents. The import-completion marker is committed with the imported records, preventing reimport after restart or later deletion. The original JSON and image files are retained unchanged as a migration backup; they are no longer used or rewritten during normal chat operation. Missing legacy attachments or invalid archives stop migration, roll back imported records, and show an error with a retry action instead of creating a replacement empty history.

Assistant messages retain Markdown, fenced code, selection/copy, stop/retry, search, rename, and confirmed conversation deletion. Run swift test for the server, legacy compatibility, and actual on-disk SwiftData persistence tests.

SwiftData validation: all ordinary tests passed (24 passed, one optional inference smoke test skipped). Native macOS and iOS Simulator builds passed. Native macOS checks migrated the existing three conversations and their images, rendered a migrated image, generated and saved a new response, searched its response text, and restored selection, the new conversation, and a draft after restarting. The original JSON checksum was unchanged after migration and subsequent chatting.

Default address: http://127.0.0.1:11535/v1. Binding a LAN interface exposes an unauthenticated service to that network; loopback is the default. On iOS, serving is intended for foreground use. The app includes local-network usage text and macOS sandbox network permissions. For physical iOS devices, select your signing team in the AFM target; simulator and local macOS builds do not require a distribution profile.

Persistent server settings

Host, the last valid port, PCC preference, and the common prompt are stored as a small versioned UserDefaults entry (AFM.serverSettings.v1). Invalid in-progress port edits do not replace the last valid saved value. A saved LAN host is preserved when an interface is temporarily unavailable; bind failures remain visible. The server starts at launch and on foreground recovery (including macOS app activation when its windows remain open). A manual Stop lasts until the next activation or restart; running/stopped state is not persisted. iOS still controls execution while suspended, so this does not guarantee background serving.

Common prompt in Server settings is prepended to client instructions for all requests, including external API clients. It is saved automatically and can be edited while serving. Each request snapshots the prompt before inference setup; an edit affects subsequent requests, without altering active responses.

If a chat request exceeds the context window before emitting any answer text, the client retries with fewer complete older turns. Recovery is bounded by halving the older turns; the current user message is always preserved. A notice on the answer explains the omission, and the saved conversation is unchanged. An oversized current message fails explicitly instead of looping.

Validation before the SwiftData migration: all 21 package tests passed; native macOS and iOS Simulator builds passed from a source-only checkout with no shared schemes or user state. Native macOS checks confirmed auto-start after relaunch, common-prompt persistence and application to an external Responses request, and machine-readable context rejection.

Xcode user state and xcshareddata are ignored. Fresh checkouts use Xcode's automatically generated AFM app scheme; project.pbxproj, app assets, entitlements, Info.plist, and the UserDefaults privacy manifest remain versioned.

Image input in chat

Use Attach image → Camera, Attach image → Choose image file…, Photos, or the Paste button above the composer. On native macOS, an image copied to the clipboard can also be attached with ⌘V while the message field is focused. Image providers can be dropped onto the composer. Add a question, or send the image alone. Camera capture asks for permission only when opened and stops when its sheet closes; microphone access is not requested.

Draft attachments have previews and individual remove buttons. Sent images remain in their originating messages and are restored with the conversation after restart. Images are stored as SwiftData ImagePart records with external binary storage; original metadata is discarded, orientation is normalized, and the longest edge is limited to 1536 pixels. Source images are limited to 32 MiB and prepared images to 2 MiB each. Existing text-only archives remain readable.

The chat sends Responses input_image parts as base64 JPEG data URLs. AFM allows four images across an entire request, including history: the client keeps a contiguous suffix of recent complete turns that fits that limit, followed by the new message. Older turns remain in saved history. Unavailable image data fails the request explicitly. SwiftData manages the lifetime of binary content through its owning message; the application no longer performs filesystem garbage collection.

Validation: image normalization, invalid/oversized inputs, wire format, archive compatibility, reference preservation/cleanup, and history image budgeting are covered by tests. Native macOS UI checks confirmed file attachment, image preview, clipboard paste using both the Paste button and ⌘V, actual on-device inference identifying a red/blue test image, and image-message restoration after restarting. Camera capture, photo-library selection, and iOS physical-device behavior require device checks.

PCC configuration and UI

PCC code is compiled, but use is disabled by default. No PCC model is instantiated in this default build. The UI shows PCC is not configured for this app, and disables the preference toggle. This is an app configuration gate, distinct from Apple's model availability: framework availability may report available even for an unprovisioned executable.

After Apple grants the managed entitlement and the app's signing/provisioning is configured, add AFM_ENABLE_PCC to the Server target's Swift compilation conditions (-Xswiftc -DAFM_ENABLE_PCC for a CLI build). The flag does not grant entitlement. The UI then checks framework availability and daily quota before enabling the Prefer PCC when available toggle. Change this setting while the server is stopped.

Routing:

Requested model	Behavior
apple-auto	PCC only when the preference is on and PCC is available; otherwise on-device
apple-on-device	Always on-device
apple-pcc	Explicit PCC selection; returns 503 when unconfigured/unavailable

PCC generation errors are surfaced. There is no automatic replay after generation starts, so a stream never mixes outputs from two models. Responses identify the actual model. /status reports effective PCC availability, reason and preference. /v1/models advertises usable backends and the automatic alias.

Instructions use DynamicInstructions. System/developer instructions are kept separate from user/assistant history, which is passed through the new session initializer. Each request constructs its own instruction and history snapshot.

Official references:

• PCC API
• PCC eligibility
• Dynamic instructions

HTTP API

• GET /health: server liveness.
• GET /status: availability, capabilities and token budgets per usable model.
• GET /v1/models (/models alias): model IDs.
• POST /v1/responses: JSON or SSE. Chat Completions and Messages are not supported.

curl -N http://127.0.0.1:11535/v1/responses \
  -H 'Content-Type: application/json' \
  -d '{"model":"apple-auto","stream":true,"store":false,"input":"Hello!"}'

model and input are required. Input accepts a string or Responses input items. Supported features include instructions, message history, temperature, max_output_tokens, function tools and tool_choice (auto/none/required/named), and user input_image parts. Capability checks use the selected native model. This is a subset, not a complete OpenAI API emulator. Unsupported options return 400 rather than silently emulating a capability.

Tools and history

Tools use the Responses function format (type, name, description, parameters). Native schema-guided generation supplies arguments. AFM returns function_call items; the client executes the functions, then sends previous input and output items plus function_call_output with matching call_id. AFM does not execute client functions. With parallel_tool_calls=false, at most one call is returned per turn. Tool schemas support the subset representable by FoundationModels GenerationSchema; unsupported keywords and open-ended object properties are rejected. Structured text output formats are not implemented.

Requests are stateless: send the full history each time. store must be false (or omitted), and previous_response_id is unsupported. SSE sends Responses lifecycle, text delta and function argument events, ending in response.completed, response.incomplete or response.failed; it does not send Chat Completions [DONE]. Usage comes from FoundationModels token counts. Since the native stream does not expose a finish reason, reaching the requested output budget is conservatively reported as incomplete.

Vision and reasoning

Images must be base64 data URLs (PNG/JPEG/WebP/GIF), up to four images and 8 MiB per image. Images are downsampled to 2048 pixels; animated formats use the first frame. Remote URLs and file IDs are unsupported. The SDK's image token-count preflight currently fails, so image requests rely on native generation's context limit enforcement; text-only requests are checked before generation.

Reasoning is enabled only when the backend reports the capability (PCC). reasoning.effort low/medium/high maps to the native light/moderate/deep setting. The native API cannot disable reasoning on PCC, so effort=none is rejected there. On-device reasoning is unsupported. Raw thoughts and textual summaries are not exposed. Reasoning output items carry an empty summary and AES-GCM encrypted native continuation state in encrypted_content. Include these items in subsequent input. State is bound to the selected model and this server instance; restarting the app invalidates it. PCC generation still requires entitlement/provisioning and has not been exercised on this machine.

VS Code Copilot

The server panel copies the API base URI and port individually. For a Responses client, append /responses to the copied /v1 URI. Configure clients with apiType=responses and statefulResponses=false where applicable. For clients requiring a key, any nonempty placeholder works; AFM has no auth.

The token monitor includes all HTTP inference clients, with live input/output counts, reasoning tokens (a subset of output), the most recently updated request, and totals since the last successful server start. Failed or cancelled generations retain whatever usage the SDK reports; requests rejected before generation are not counted. Counts are SDK usage, not billing or PCC quota measurements. Stopping preserves the display; restarting resets it.

The current on-device model reports a 4096-token total context. AFM partitions that into maxInputTokens=3072 and maxOutputTokens=1024. These are AFM's conservative budgets, not two independent native context limits. Other models' values are calculated from their runtime context size, including PCC once usable. /status exposes these budgets.

VS Code 1.141.2 recognizes this endpoint and displays a 4K context. However, the local Copilot Ask smoke test failed in its prompt builder with No lowest priority node found (path: vie) before sending HTTP. The small context budget appears to be the constraint; registration alone does not establish that Copilot's agent prompts fit. Do not advertise a larger context than the model actually supports. Direct text, image and tool requests to AFM work independently.

Limits and validation

Requests are capped at 16 MiB. Reception has a 30-second deadline and processing a 300-second deadline. Connections close after one response. NIO handles HTTP parsing and chunked framing; streaming writes are awaited for backpressure. Disconnecting cancels generation. Stop closes both listener and accepted sockets.

Run swift test from the repository root. It covers validation, schema conversion, reasoning-state authentication, SSE lifecycle, token budgets and real loopback HTTP lifecycle. It requires permission to bind a loopback socket. AFM_INFERENCE_SMOKE=1 swift test additionally checks on-device text streaming, output limits, a real function-call/result round trip and image input. PCC inference requires a provisioned app and is not exercised by these tests.