Zoe

A SwiftUI sample for simple, low-cost web content subscriptions using cloud and on-device models.

A cloud model builds and tests workflows. Once saved, they run locally with WebKit and Apple Foundation Models, avoiding repeated cloud-model calls.

Supports iOS, iPadOS and macOS, with adaptive layouts for iPhone Duo.

Requirements

• Xcode 27 and iOS 27 / iPadOS 27 / macOS 27.
• An Apple Intelligence-capable device with Apple Intelligence enabled and the local model ready.
• A Gemini API key or Apple's Private Cloud Compute entitlement to build new workflows. Bundled samples can run without a cloud API key.

For iPhone Duo, build with Xcode 27.1 or later (iOS 27.1 SDK or later) to enable edge-to-edge layout and adaptive vertical toolbars. Running an older-SDK build on iOS 27.1 does not enable these layouts. See Apple's Prepare your app for iPhone Duo.

Run

1. Open Zoe.xcodeproj, select the Zoe scheme and choose My Mac. For iPhone or iPad, configure your signing team and bundle identifier.
2. Wait for the model status to show ready.
3. Choose a sample, review it, select Save Workflow, then Run on Device.

The bundled samples were generated and tested by the cloud builder:

• Hacker News: Apple-related stories and sentiment in their first three top-level comments.
• Swift Evolution: Active Review proposals filtered and classified by topic.
• arXiv: Recent cs.AI papers selected by topic and summarized from their abstracts.

Saved workflows can also run through Shortcuts. Background and locked-device execution are experimental.

To create a workflow, enter a start URL and a short goal, choose a builder, and select Build Workflow. For Gemini, paste your API key into the app or set GEMINI_API_KEY; the key stays in memory. Review the generated workflow before saving.

Screenshots

Workflow review and on-device results on the iPhone Duo simulator (iOS 27.1), with layouts for the unfolded inner display and folded outer display.

How it works

The cloud builder inspects pages, writes JavaScript and short local-model instructions, and tests the workflow. During replay, Swift coordinates page navigation and model calls; JavaScript handles page interaction and extraction.

flowchart LR
    subgraph Build["1. Build and verify"]
        LLM["Cloud model<br/>Gemini / PCC"]
        Tools["On-device build tools<br/>Page inspection, JavaScript and local-model trials"]
        LLM <--> Tools
    end

    Saved[("Saved workflow JSON<br/>Steps, JavaScript and local-model instructions")]
    Tools -->|Verify, then user review and save| Saved

    subgraph Replay["2. Reuse on device"]
        Runner["WorkflowRunner<br/>Swift coordinator"]
        WebKit["WebKit browser<br/>Page interaction, extraction and navigation"]
        AFM["Apple Foundation Models<br/>2K input / 4K total replay budget"]
        Results["Results and run log"]

        Saved --> Runner
        Runner <-->|act / read| WebKit
        Runner <-->|semantic| AFM
        Runner --> Results
    end

Loading

Build and verification

sequenceDiagram
    autonumber
    actor User
    participant Host as Builder host (on device)
    participant Builder as Cloud model (Gemini / PCC)
    participant Runner as WorkflowRunner
    participant Browser as WebKit browser
    participant LocalModel as On-device model (AFM)

    User->>Host: Goal and start URL
    Host->>Browser: Load start page
    Browser-->>Host: Page ready
    Host->>Builder: Goal and start URL
    loop Explore and test as needed
        Builder->>Host: inspect / read / explore
        Host->>Browser: Inspect or interact with the page
        Browser-->>Host: DOM outline / JSON
        Host-->>Builder: Observation
        opt Test a local-model task
            Builder->>Host: testSemantic(taskJSON, inputJSON)
            Host->>LocalModel: Evaluate task on sample input
            LocalModel-->>Host: Output and token usage
            Host-->>Builder: Output, usage and run log
        end
    end
    loop verifyWorkflow, then finish (repair and retry if needed)
        Builder->>Host: verifyWorkflow(workflowJSON) / finish(summary)
        Host->>Runner: Run candidate / verified workflow
        Runner->>Browser: Restart at the start URL
        Browser-->>Runner: Fresh page ready
        loop Execute workflow steps in order
            alt Browser step
                Runner->>Browser: act / read
                Browser-->>Runner: Completion / extracted data
            else Semantic step
                Runner->>LocalModel: Select / classify / summarize
                LocalModel-->>Runner: Task output
            end
        end
        Runner-->>Host: Status, output and run log
        Host-->>Builder: Replay feedback
    end
    Builder-->>Host: Closing response
    Host-->>User: Workflow ready to review and save

Loading

Both verification passes rerun the workflow on a fresh page. Partial results may be accepted when individual items fail; the run log retains the reasons. Verification does not guarantee model accuracy.

The local model selects, classifies or summarizes small inputs using independent sessions. Replay limits input to 2K tokens and the total context budget to 4K; selection uses batches of up to ten records. Results retain source links, and skipped items appear in the run log.

Workflows and the latest results are stored locally.

Code

• WorkflowBuilder.swift: cloud model and building tools.
• OnDeviceModel.swift: local profile, structured output and token budgets.
• WorkflowRunner.swift: browser and model coordination.

Tests

Unit and WebKit fixture tests cover execution and data contracts:

xcodebuild test -project Zoe.xcodeproj -scheme Zoe \
  -destination 'platform=macOS' -derivedDataPath /private/tmp/zoe-tests CODE_SIGNING_ALLOWED=NO

Real-model checks require an Apple Silicon Mac with the Core model ready. They use synthetic inputs and offline pages; inspect the reported scores rather than treating completion as an accuracy guarantee.

zsh Tools/smoke.sh model /private/tmp/zoe-core-check
zsh Tools/smoke.sh run-offline-hn Zoe/Resources/hn-ai-watch.json /private/tmp/zoe-hn-offline-check

Run one smoke command at a time. Results and logs are saved in the chosen directory.

Scope and privacy

Simple public list and detail pages are the intended scope. Logins, CAPTCHAs and complex interactions may fail; website changes may require rebuilding. Automatic scheduling and model accuracy need further validation.

Before running a workflow, check and respect the site's robots.txt and automation policy. Do not crawl disallowed pages or bypass access restrictions.

Building sends page observations to the chosen cloud model. Replay processes content locally, while websites receive normal browsing requests. Generated scripts can have side effects, and the host allowlist covers document navigation rather than all network activity. Review workflows and follow each site's automation policy. See PRIVACY.md.

Foundation Models reference

Apple Foundation Models dynamically determine the on-device model variant and context window based on hardware tier:

Variant	Architecture	Context window	Reasoning level	Hardware tier
.core3 (Core)	3B dense	4K tokens (4,096)	Standard / .light	iPhone 15 Pro, iPhone 16 series, M1 / M2 Macs
.coreAdvanced3 (Core Advanced)	20B sparse (1–4B active)	8K tokens (8,192)	Supports .deep	iPhone 17 Pro, iPhone Air, M4 iPads (≥12 GB), M3+ Macs (≥12 GB)
PCC (Private Cloud Compute)	Server-grade foundation model	32K tokens (32,768)	Full reasoning	Cloud-hosted; requires developer entitlement

Zoe targets the .core3 baseline (3B dense / 4K context) for its replay token budgeting and batch packing limits. This ensures workflows remain fully operable across the entire Apple Intelligence device fleet without assuming high-tier hardware.

References

Apple's Origami and Book Tracker samples.

License

Apache License 2.0. See LICENSE.