diff --git a/CHANGELOG.md b/CHANGELOG.md index 237f7d5..15d6144 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,31 @@ # Changelog +## Unreleased + +- Add reminder reading. `reminder()` returns one armed alarm as a + `ScheduledReminder`, and `reminders()` lists every key of one operation. Both + apply the intents staged so far in the turn, so a read agrees with what the + commit will write. Reading works on the SQL backends and on Durable Objects. +- Leave a one-shot reminder that already fired out of `reminder()` and + `reminders()`. Its row stays as `completed`, so a next-run lookup reported an + old time rather than nothing, and an existence check refused to re-arm an + alarm that could never fire again. +- Refuse an unknown operation in `unschedule()` and `unscheduleAll()`. + `schedule()` already threw `UnknownOperation` for one, so a typo cancelled + nothing quietly and left a recurring reminder running. +- Add reminder cancellation. `unschedule()` removes one alarm by operation, by + operation and key, or by the handle `schedule()` now returns. + `unscheduleAll()` removes every key of one operation. Both stage an intent + beside the schedules, so they apply in the order the turn called them, commit + with the state change that decided them, and cancel nothing when a turn + throws. Cancellation works on the SQL backends and on Durable Objects. A + cancellation that lands on an occurrence the scheduler claimed but has not yet + enqueued pre-empts it, and the scheduler continues rather than failing. +- `schedule()` now returns a `ReminderHandle` (`{ name: string }`) instead of + `void`. A handle is a plain object, so it survives in actor state and still + cancels after a deactivation. Code that assigned the result to `void` needs + updating, as `emit` required in 0.15.0. + ## 0.15.2 - 2026-09-21 - Stop the deadlock between concurrent callers that create the same actor from diff --git a/docs/api.md b/docs/api.md index 7de0fe7..204aefe 100644 --- a/docs/api.md +++ b/docs/api.md @@ -45,6 +45,9 @@ authorization, capability boundaries, and release validation. - `Actor`: base class providing `ref()`, `actorId`, `currentMessage`, `observables()`, `reject()`, `emit()`, `transmit()`, `commitAction()`, `schedule()`, `sendTo()`, and protected lifecycle hooks. + `ScheduledOperationsFor` types the map `schedule()` returns, whose calls return a + `ReminderHandle`. `TransmittedOperationsFor` types the map `transmit()` + returns, whose calls return nothing. - `broadcastValue(value)`: mark an observable so its changed value enters the durable invalidation envelope. - `broadcastInvalidation(value)`: compare the real observable value but put @@ -67,10 +70,16 @@ createdAtMs }` shape returned by `SolidObjectsRuntime.snapshotWithIncarnation`. `DestroyOptions`: the options for authorization, idempotency, time, and schedule that the reference methods use. +`ScheduledReminder` is one armed reminder as an actor reads it, and +`ReminderReader` is how a runtime supplies them. + `ActorIntents`, `EffectIntent`, `CommitActionIntent`, `ReminderIntent`, +`UnscheduleIntent`, `UnscheduleAllIntent`, `ReminderMutation`, `OutboundMessageIntent`, `ReminderOptions`, `OutboundMessageOptions`, `PayloadBroadcasts`, and `PayloadBroadcastValue` describe actor-declared -transactional work and typed personalized projections. +transactional work and typed personalized projections. `ReminderMutation` is the +union of one scheduled reminder and the two cancellations, held in one list so +they apply in the order the turn called them. `EffectFailurePayload`, `EffectSuccessPayload`, and `SerializedError` describe effect callback messages. They are also exported @@ -162,6 +171,130 @@ row per item. It also cannot strand an entry when the runtime coalesces an occurrence. Prefer it for a large queue of interchangeable items. Prefer `key` when one item needs an alarm that you can move on its own. +#### Cancelling a reminder + +`schedule()` returns a `ReminderHandle` (`{ name: string }`) naming the alarm it +armed. `unschedule()` cancels one alarm, by operation, by operation and key, or +by that handle. `unscheduleAll()` cancels every key of one operation. + +```typescript +const MONTH = 30 * 24 * 60 * 60 * 1000 + +class Subscription extends Actor { + static override readonly actorType = "subscriptions" + + status = "trialing" + renewal: ReminderHandle | null = null + + startTrial(): void { + this.schedule({ at: new Date(Date.now() + (14 * MONTH) / 30) }).trialExpired() + } + + convertToPaid(): void { + this.status = "active" + this.unschedule("trialExpired") + this.renewal = this.schedule({ + at: new Date(Date.now() + MONTH), + everyMilliseconds: MONTH, + }).chargeRenewal() + } + + cancelled(): void { + this.status = "cancelled" + if (this.renewal) this.unschedule(this.renewal) + } + + trialExpired(): void { + this.status = "expired" + } + + chargeRenewal(): void {} +} +``` + +`this.unschedule(this.renewal)` and `this.unschedule("chargeRenewal")` cancel the +same alarm. Prefer the handle when the actor already stored one, because it +cannot drift from the name that armed the reminder. + +A keyed alarm cancels by the key that armed it, and `unscheduleAll()` cancels +every key of one operation: + +```typescript +class Shipment extends Actor { + static override readonly actorType = "shipments" + + dispatch({ carrierIds }: { carrierIds: string[] }): void { + for (const carrierId of carrierIds) { + this.schedule({ at: new Date(Date.now() + MONTH / 30), key: carrierId }).chaseCarrier({ + carrierId, + }) + } + } + + shipped({ carrierId }: { carrierId: string }): void { + this.unschedule("chaseCarrier", { key: carrierId }) + } + + stopChasing(): void { + this.unscheduleAll("chaseCarrier") + } + + chaseCarrier(_options: { carrierId: string }): void {} +} +``` + +`unschedule()` and `unscheduleAll()` refuse an operation the actor does not +declare, with the `UnknownOperation` that `schedule()` already throws, so a typo +fails the turn rather than cancelling nothing. A handle skips that check, because +the `schedule()` call that produced it was already checked. + +#### Reading the schedule + +`reminder()` returns the armed alarm as a `ScheduledReminder`, or `undefined`. +`reminders()` returns every key of one operation. Both are async, because an +actor reads its own rows rather than holding them in memory: + +```typescript +async nextChargeAt(): Promise { + return (await this.reminder("chargeRenewal"))?.runAtMilliseconds ?? null +} + +async pendingCarriers(): Promise<(string | null)[]> { + return (await this.reminders("chaseCarrier")).map((reminder) => reminder.key) +} +``` + +A read applies the intents staged so far in the turn, so an actor that schedules +and then reads sees what the commit will write, and one that cancels and then +reads sees the alarm gone. + +`key` and `intervalMilliseconds` are `null` rather than `undefined` when absent, +so a `ScheduledReminder` returns from an operation without a serialization error. + +Reading is available during a turn and from a snapshot projection, so an +observable can report what is armed. `reminder()` and `reminders()` refuse an +operation the actor does not declare, as `schedule()` and `unschedule()` do. + +A one-shot that already fired is not reported. Its row stays as `completed`, and +an alarm that cannot fire again is not armed. + +`ScheduledReminder` carries no occurrence count. The SQL backends track one and +Durable Objects does not, so it is left out rather than reported for one backend +only. + +A cancellation is staged like a schedule, so it commits with the state change +that decided it and a turn that throws cancels nothing. Both apply in the order +the turn called them, so cancelling and then scheduling the same name leaves it +armed at the new time. + +Cancelling an alarm that does not exist is not an error. `unschedule()` returns +nothing, because it stages an intent rather than applying one, and an answer +given at call time could be stale by the time the turn commits. + +A handle is a plain object, so it survives in actor state and still cancels +after a deactivation. Passing a handle together with `key` is a `TypeError`, +because the handle already names the key. + ### Recovering abandoned effects `emit` returns an `EffectHandle` (`{ id: string }`) on every successful call. Save @@ -358,9 +491,11 @@ effect results and the handler's declared argument/result types in agreement. ### Typed operation references `schedule` and `transmit` infer this actor's operation names and arguments, including -inside actor methods and for inherited application operations. The returned -`ScheduledOperationsFor` values return `void` and preserve required, -optional, and zero-argument operation signatures. No non-null assertion is needed: +inside actor methods and for inherited application operations. A scheduled operation +call returns a `ReminderHandle`, through `ScheduledOperationsFor`, and a +transmitted one returns `void`, through `TransmittedOperationsFor`. Both +preserve required, optional, and zero-argument operation signatures. No non-null +assertion is needed: ```typescript class ChatRun extends Actor { diff --git a/docs/correctness.md b/docs/correctness.md index 17858a3..e34af95 100644 --- a/docs/correctness.md +++ b/docs/correctness.md @@ -5,6 +5,14 @@ - Delivery is ordered per actor identity and at least once. - Different identities may execute concurrently. - Sequence allocation and durable enqueue are one transaction. +- An actor reads its own schedule. A read applies the intents staged so far in + the turn, so it agrees with what the commit will write rather than with what + the turn began with. +- A reminder can be cancelled. A cancellation commits with the state change that + decided it, and applies in the order the turn called it. A cancellation cannot + recall an occurrence the scheduler already turned into a message. It does + pre-empt one the scheduler claimed but has not yet enqueued, and the scheduler + treats that as ordinary work rather than a failure. - Concurrent callers that create the same actor produce one instance row and distinct sequences. The mailbox locks that row by its primary key, so MySQL does not upgrade a shared lock and the enqueue does not deadlock. diff --git a/src/actor.ts b/src/actor.ts index 336c66d..7a6e718 100644 --- a/src/actor.ts +++ b/src/actor.ts @@ -9,7 +9,9 @@ import { createStagedOperations, type ActorReference, type ScheduledOperations, + type StagedOperationMap, type ScheduledOperationsFor, + type TransmittedOperationsFor, type StagedOperations, } from "./reference.js" import { jsonObject, normalizeJson } from "./serialization.js" @@ -17,6 +19,9 @@ import type { ActorIdentifier, EffectHandle, JsonObject, + ReminderHandle, + ReminderReader, + ScheduledReminder, JsonValue, MessageContext, } from "./types.js" @@ -99,6 +104,7 @@ export interface CommitActionIntent { } export interface ReminderIntent { + cancel?: undefined /** Without a key this is the operation. */ name: string operation: string @@ -108,6 +114,18 @@ export interface ReminderIntent { missedPolicy: "all" | "latest" } +export interface UnscheduleIntent { + cancel: "one" + name: string +} + +export interface UnscheduleAllIntent { + cancel: "all" + operation: string +} + +export type ReminderMutation = ReminderIntent | UnscheduleIntent | UnscheduleAllIntent + export interface OutboundMessageIntent { actorType: string actorId: string @@ -121,7 +139,7 @@ export interface ActorIntents { effects: EffectIntent[] effectRecoveries?: EffectRecoveryIntent[] commitActions: CommitActionIntent[] - reminders: ReminderIntent[] + reminders: ReminderMutation[] outboundMessages: OutboundMessageIntent[] } @@ -163,6 +181,41 @@ function validatedReminderKey(key: string | number | undefined): string | undefi * reverse, and it is refused here rather than at the insert, once the turn is * already doing work. */ +function handleName(handle: ReminderHandle, key: string | number | undefined): string { + if (key !== undefined) throw new TypeError("a reminder handle already names its key") + + const name = handle?.name + if (typeof name !== "string" || name.length === 0) { + throw new TypeError("a reminder handle returned by schedule is required") + } + + return name +} + +function applyReminderIntent(view: Map, intent: ReminderMutation): void { + if (intent.cancel === "all") { + for (const [name, status] of view) { + if (status.operation === intent.operation) view.delete(name) + } + return + } + if (intent.cancel === "one") { + view.delete(intent.name) + return + } + + view.set(intent.name, { + name: intent.name, + operation: intent.operation, + key: intent.name === intent.operation ? null : intent.name.slice(intent.operation.length + 1), + runAtMilliseconds: intent.atMilliseconds, + intervalMilliseconds: intent.intervalMilliseconds ?? null, + missedPolicy: intent.missedPolicy, + status: "scheduled", + handle: { name: intent.name }, + }) +} + function reminderName(operation: string, key: string | undefined): string { if (key === undefined) return operation @@ -195,6 +248,8 @@ export abstract class Actor { } readonly #actorId: string + #readReminders: ReminderReader | undefined + readonly #intents: ActorIntents = { effects: [], commitActions: [], @@ -303,8 +358,8 @@ export abstract class Actor { transmit( this: Actor & Pick & (Partial | NoInfer), - ): ScheduledOperationsFor> - transmit(): ScheduledOperations { + ): TransmittedOperationsFor> + transmit(): StagedOperationMap { return createStagedOperationMap(this.#operations, (operation, argumentsValue) => { this.#intents.effects.push({ name: TRANSMIT_EFFECT, @@ -331,8 +386,9 @@ export abstract class Actor { const key = validatedReminderKey(options.key) return createStagedOperationMap(this.#operations, (operation, argumentsValue) => { + const name = reminderName(operation, key) this.#intents.reminders.push({ - name: reminderName(operation, key), + name, operation, atMilliseconds, arguments: jsonObject(argumentsValue), @@ -341,9 +397,65 @@ export abstract class Actor { ? {} : { intervalMilliseconds: options.everyMilliseconds }), }) + return { name } }) } + unschedule(operationOrHandle: string | ReminderHandle, options: { key?: string | number } = {}) { + this.#intents.reminders.push({ + cancel: "one", + name: this.#reminderNameOf(operationOrHandle, options), + }) + } + + unscheduleAll(operation: string) { + this.#assertOperation(operation) + this.#intents.reminders.push({ cancel: "all", operation }) + } + + async reminder( + operationOrHandle: string | ReminderHandle, + options: { key?: string | number } = {}, + ): Promise { + return (await this.#reminderView()).get(this.#reminderNameOf(operationOrHandle, options)) + } + + async reminders(operation: string): Promise { + this.#assertOperation(operation) + const view = await this.#reminderView() + return [...view.values()].filter((status) => status.operation === operation) + } + + #reminderNameOf( + operationOrHandle: string | ReminderHandle, + options: { key?: string | number }, + ): string { + if (typeof operationOrHandle === "string") { + this.#assertOperation(operationOrHandle) + return reminderName(operationOrHandle, validatedReminderKey(options.key)) + } + + return handleName(operationOrHandle, options.key) + } + + async #reminderView(): Promise> { + if (!this.#readReminders) { + throw new TypeError("reading reminders is not available outside an actor turn") + } + const view = new Map() + for (const reminder of await this.#readReminders()) { + if (reminder.status !== "completed") view.set(reminder.name, reminder) + } + for (const intent of this.#intents.reminders) applyReminderIntent(view, intent) + return view + } + + #assertOperation(operation: string): void { + if (this.#operations.has(operation)) return + + throw new UnknownOperation(`unknown operation ${JSON.stringify(operation)}`) + } + sendTo( reference: ActorReference, options: OutboundMessageOptions = {}, @@ -367,8 +479,9 @@ export abstract class Actor { } /** @internal */ - prepare(operations: ReadonlySet): void { + prepare(operations: ReadonlySet, readReminders?: ReminderReader): void { this.#operations = operations + this.#readReminders = readReminders } /** @internal */ diff --git a/src/cloudflare/engine.ts b/src/cloudflare/engine.ts index 3cc562a..0fbd723 100644 --- a/src/cloudflare/engine.ts +++ b/src/cloudflare/engine.ts @@ -34,6 +34,7 @@ import type { JsonObject, JsonValue, SerializedError, + ScheduledReminder, } from "../types.js" import type { CloudflareSettings } from "./configuration.js" import { actorName, callHost, type ActorIdentity, type HostRequest } from "./protocol.js" @@ -357,9 +358,29 @@ export class ActorEngine { return { definition, instance, state } } + private readReminders = async (): Promise => + this.store.rows("SELECT record FROM reminders ORDER BY name").map((reminder) => ({ + name: reminder.name, + operation: reminder.operation, + key: + reminder.name === reminder.operation + ? null + : reminder.name.slice(reminder.operation.length + 1), + runAtMilliseconds: reminder.at, + intervalMilliseconds: reminder.interval, + missedPolicy: reminder.missed, + status: reminder.status, + handle: { name: reminder.name }, + })) + private async snapshot(identity: ActorIdentity): Promise { const { definition, instance, state } = this.committed(identity) - const actor = hydrateActor({ definition, actorId: identity.actorId, state }) + const actor = hydrateActor({ + definition, + actorId: identity.actorId, + state, + readReminders: this.readReminders, + }) const before = stableJson(actorState(actor, definition.stateKeys)) const snapshot: JsonObject = { ...state } await withActorProjection({ actor, runtime: this.runtime }, async () => { @@ -387,7 +408,12 @@ export class ActorEngine { }): Promise { const { input, payloadNames } = options const { definition, instance, state } = this.committed(input) - const actor = hydrateActor({ definition, actorId: input.actorId, state }) + const actor = hydrateActor({ + definition, + actorId: input.actorId, + state, + readReminders: this.readReminders, + }) const identity = { actorType: input.actorType, actorId: input.actorId, @@ -486,6 +512,7 @@ export class ActorEngine { storedVersion: instance.stateVersion, storedState: instance.state, }), + readReminders: this.readReminders, }) if (this.cached?.actor !== actor) { await withActorContext({ actor, runtime: this.runtime }, () => actor.activate()) @@ -673,6 +700,14 @@ export class ActorEngine { }) } for (const intent of intents.reminders) { + if (intent.cancel === "all") { + this.store.deleteRemindersFor(intent.operation) + continue + } + if (intent.cancel === "one") { + this.store.deleteReminder(intent.name) + continue + } this.store.saveReminder({ name: intent.name, generation: crypto.randomUUID(), diff --git a/src/cloudflare/storage.ts b/src/cloudflare/storage.ts index 31f23d7..6885d59 100644 --- a/src/cloudflare/storage.ts +++ b/src/cloudflare/storage.ts @@ -176,6 +176,17 @@ export class ActorStorage { ) } + deleteReminder(name: string): void { + this.storage.sql.exec("DELETE FROM reminders WHERE name = ?", name) + } + + deleteRemindersFor(operation: string): void { + const names = this.rows("SELECT record FROM reminders") + .filter((reminder) => reminder.operation === operation) + .map((reminder) => reminder.name) + for (const name of names) this.deleteReminder(name) + } + saveSubscription(subscription: Subscription): void { this.storage.sql.exec( "INSERT INTO subscriptions(id, expires_at, record) VALUES (?, ?, ?) ON CONFLICT(id) DO UPDATE SET expires_at = excluded.expires_at, record = excluded.record", diff --git a/src/definition.ts b/src/definition.ts index 7936282..37ffb25 100644 --- a/src/definition.ts +++ b/src/definition.ts @@ -2,6 +2,7 @@ import { Actor, type ActorClass } from "./actor.js" import { withApplicationWritesForbidden } from "./context.js" import { ApplicationWriteForbidden, InvalidActor, StateMigrationError } from "./errors.js" import { deepCopy, jsonObject, normalizeJson } from "./serialization.js" +import type { ReminderReader } from "./types.js" import type { JsonObject } from "./types.js" export type PayloadBroadcastHandler = ( @@ -146,10 +147,11 @@ export function hydrateActor(options: { definition: ValidatedActorDefinition actorId: string state: JsonObject + readReminders?: ReminderReader }): ActorType { const { definition, actorId, state } = options const actor = new definition.actorClass(actorId) - actor.prepare(new Set(definition.operations)) + actor.prepare(new Set(definition.operations), options.readReminders) const target = actor as unknown as Record for (const key of definition.stateKeys) target[key] = deepCopy(state[key]) return actor diff --git a/src/index.ts b/src/index.ts index 792dc8b..769b8b6 100644 --- a/src/index.ts +++ b/src/index.ts @@ -15,6 +15,9 @@ export { type PayloadBroadcasts, type PayloadBroadcastValue, type ReminderIntent, + type ReminderMutation, + type UnscheduleAllIntent, + type UnscheduleIntent, type ReminderOptions, } from "./actor.js" export { @@ -106,6 +109,7 @@ export { type ActorSnapshot, type ScheduledOperations, type ScheduledOperationsFor, + type TransmittedOperationsFor, type StagedOperations, } from "./reference.js" export type { @@ -127,6 +131,9 @@ export type { EffectContext, EffectFailurePayload, EffectHandle, + ReminderHandle, + ReminderReader, + ScheduledReminder, EffectSuccessPayload, InvocationOptions, JsonObject, diff --git a/src/reference.ts b/src/reference.ts index 6dced92..bd2de94 100644 --- a/src/reference.ts +++ b/src/reference.ts @@ -10,6 +10,7 @@ import type { JsonValue, MessageStatus, SnapshotOptions, + ReminderHandle, } from "./types.js" type FunctionKeys = { @@ -70,6 +71,12 @@ type StagedMethod = Method extends (...argumentsValue: any[]) => any : (...argumentsValue: OperationArguments) => void : never +type ScheduledMethod = Method extends (...argumentsValue: any[]) => any + ? [OperationArguments] extends [never] + ? never + : (...argumentsValue: OperationArguments) => ReminderHandle + : never + type DirectMessages = { [Key in ActorOperationNames]: InvokedMethod } @@ -93,13 +100,21 @@ export type StagedOperations = { } export interface ScheduledOperations { - [operation: string]: (argumentsValue?: Record) => void + [operation: string]: (argumentsValue?: Record) => ReminderHandle } export type ScheduledOperationsFor = { + [Key in ActorOperationNames]: ScheduledMethod +} + +export type TransmittedOperationsFor = { [Key in ActorOperationNames]: StagedMethod } +export interface StagedOperationMap { + [operation: string]: (argumentsValue?: Record) => void +} + export type ActorReference = ActorReferenceCore & DirectMessages & DirectQueries @@ -260,10 +275,10 @@ export function createStagedOperations( return createStagedOperationMap(operations, dispatch) as unknown as StagedOperations } -export function createStagedOperationMap( +export function createStagedOperationMap( operations: ReadonlySet, - dispatch: (operation: string, argumentsValue: JsonObject) => void, -): ScheduledOperations { + dispatch: (operation: string, argumentsValue: JsonObject) => Result, +): { [operation: string]: (argumentsValue?: Record) => Result } { return new Proxy( {}, { @@ -274,7 +289,7 @@ export function createStagedOperationMap( dispatch(property, operationArguments(argumentsValue)) }, }, - ) as ScheduledOperations + ) as { [operation: string]: (argumentsValue?: Record) => Result } } function createReferenceProxy( diff --git a/src/repository.ts b/src/repository.ts index 9d0611b..0e7296d 100644 --- a/src/repository.ts +++ b/src/repository.ts @@ -767,6 +767,21 @@ export class Repository { }) for (const reminder of input.intents.reminders) { + if (reminder.cancel === "all") { + await connection.run( + `DELETE FROM ${this.table("reminders")} + WHERE instance_id = ? AND COALESCE(message_operation, operation) = ?`, + [turn.instance.id, reminder.operation], + ) + continue + } + if (reminder.cancel === "one") { + await connection.run( + `DELETE FROM ${this.table("reminders")} WHERE instance_id = ? AND operation = ?`, + [turn.instance.id, reminder.name], + ) + continue + } const existing = await connection.get<{ id: string; run_at_ms: number | bigint }>( `SELECT id, run_at_ms FROM ${this.table("reminders")} WHERE instance_id = ? AND operation = ?`, @@ -1154,6 +1169,15 @@ export class Repository { }) } + async remindersForInstance(instanceId: string): Promise { + return this.settings.database.connection((connection) => + connection.all( + `SELECT * FROM ${this.table("reminders")} WHERE instance_id = ? ORDER BY operation`, + [instanceId], + ), + ) + } + async findInstanceByIdentity( actorType: string, actorId: string, @@ -1750,8 +1774,8 @@ export class Repository { async enqueueReminder( reminder: ReminderRow, options: { nowMilliseconds?: number } = {}, - ): Promise { - await this.settings.database.transaction(async (connection) => { + ): Promise { + return this.settings.database.transaction(async (connection) => { const now = options.nowMilliseconds ?? (await connection.nowMilliseconds()) const claimed = await connection.get( `SELECT reminders.*, instances.actor_type, instances.actor_id @@ -1760,6 +1784,13 @@ export class Repository { WHERE reminders.id = ? AND reminders.status = 'scheduled' AND reminders.claimed_by = ?`, [reminder.id, reminder.claimed_by], ) + const surviving = + !claimed && + (await connection.get<{ id: string }>( + `SELECT id FROM ${this.table("reminders")} WHERE id = ?`, + [reminder.id], + )) + if (!claimed && !surviving) return false if (!claimed) throw new LostActivation("reminder claim no longer matches") await this.enqueueInTransaction(connection, { actorType: claimed.actor_type, @@ -1791,6 +1822,7 @@ export class Repository { claimed.claimed_by, ], ) + return true }) } diff --git a/src/runtime.ts b/src/runtime.ts index 26ed144..8af5fee 100644 --- a/src/runtime.ts +++ b/src/runtime.ts @@ -117,6 +117,7 @@ import type { MessageContext, MessageStatus, SnapshotOptions, + ScheduledReminder, } from "./types.js" import { SolidObjectsTestHelper } from "./test-helper.js" import { waitFor, Worker } from "./worker.js" @@ -165,6 +166,21 @@ type CommitActionHandler = ( context: CommitActionContext, ) => unknown | Promise +function scheduledReminderOf(row: ReminderRow): ScheduledReminder { + const operation = row.message_operation ?? row.operation + const name = row.operation + return { + name, + operation, + key: name === operation ? null : name.slice(operation.length + 1), + runAtMilliseconds: Number(row.run_at_ms), + intervalMilliseconds: row.interval_ms === null ? null : Number(row.interval_ms), + missedPolicy: row.missed_policy, + status: row.status, + handle: { name }, + } +} + export class SolidObjectsRuntime { readonly settings readonly repository @@ -727,6 +743,10 @@ export class SolidObjectsRuntime { definition: registered.definition, actorId: reference.actorId, state, + readReminders: async () => + instance === undefined + ? [] + : (await this.repository.remindersForInstance(instance.id)).map(scheduledReminderOf), }) const stateBefore = stableJson(actorState(actor, registered.definition.stateKeys)) const intentCount = actor.intentCount() @@ -779,6 +799,12 @@ export class SolidObjectsRuntime { definition: registered.definition, actorId: options.actorId, state, + ...(instance === undefined + ? {} + : { + readReminders: async () => + (await this.repository.remindersForInstance(instance.id)).map(scheduledReminderOf), + }), }) return readonlyCopy({ actorType: options.actorType, @@ -1117,6 +1143,8 @@ export class SolidObjectsRuntime { definition, actorId: turn.message.actor_id, state: deepCopy(state), + readReminders: async () => + (await this.repository.remindersForInstance(turn.instance.id)).map(scheduledReminderOf), }) } catch (error) { throw new ActorSetupFailed(error) @@ -1421,7 +1449,8 @@ export class SolidObjectsRuntime { if (!actor.operations.has(dispatchOperation)) { throw new UnknownOperation(`unknown reminder operation ${JSON.stringify(dispatchOperation)}`) } - await this.repository.enqueueReminder(reminder, options) + if (!(await this.repository.enqueueReminder(reminder, options))) return + this.wakeUp("actors") this.emitInstrumentation("reminder.enqueued", { reminderId: reminder.id, diff --git a/src/types.ts b/src/types.ts index b74d217..8414e59 100644 --- a/src/types.ts +++ b/src/types.ts @@ -1,9 +1,26 @@ +import type { ReminderStatus } from "./reminder-administration.js" + export type JsonPrimitive = null | boolean | number | string export type JsonValue = JsonPrimitive | JsonValue[] | { [key: string]: JsonValue } export type JsonObject = { [key: string]: JsonValue } export type EffectHandle = { readonly id: string } +export type ReminderHandle = { readonly name: string } + +export interface ScheduledReminder { + readonly name: string + readonly operation: string + readonly key: string | null + readonly runAtMilliseconds: number + readonly intervalMilliseconds: number | null + readonly missedPolicy: "all" | "latest" + readonly status: ReminderStatus + readonly handle: ReminderHandle +} + +export type ReminderReader = () => Promise + export type SerializedError = { name: string message: string diff --git a/test/actor-operations.types.ts b/test/actor-operations.types.ts index dbfe073..fae5562 100644 --- a/test/actor-operations.types.ts +++ b/test/actor-operations.types.ts @@ -1,3 +1,4 @@ +import type { ReminderHandle } from "../src/types.js" import { expectTypeOf } from "vitest" import { Actor, @@ -41,11 +42,11 @@ export class ChatRun extends ParentRun { start() { const operations = this.schedule({ at: new Date(0), key: "watchdog" }) - expectTypeOf(operations.recoverIfStuck({ generation: 1 })).toEqualTypeOf() + expectTypeOf(operations.recoverIfStuck({ generation: 1 })).toEqualTypeOf() operations.finish() operations.optional() operations.optional({ generation: 1 }) - this.transmit().recoverIfStuck({ generation: 1 }) + expectTypeOf(this.transmit().recoverIfStuck({ generation: 1 })).toEqualTypeOf() this.transmit().finish() this.transmit().optional() this.emit("run_model", { onSuccess: "finish", onFailure: "failTurn" }) diff --git a/test/cloudflare/runtime.test.ts b/test/cloudflare/runtime.test.ts index 8a798d2..ea2c516 100644 --- a/test/cloudflare/runtime.test.ts +++ b/test/cloudflare/runtime.test.ts @@ -58,6 +58,38 @@ describe("Durable Objects runtime", () => { expect(await runtime().ref(Counter, "destination").with({ authorizationContext }).count).toBe(1) }) + it("cancels reminders before the alarm fires", async () => { + const reference = runtime().ref(Counter, "cancelled").with({ authorizationContext }) + await reference.arm({ at: Date.now() + 60_000 }) + expect(await remainingReminders("cancelled")).toEqual(["increment"]) + + await reference.disarm() + expect(await remainingReminders("cancelled")).toEqual([]) + + const stub = env.ACTORS.getByName(JSON.stringify(["Counter", "cancelled"])) + for (let attempt = 0; attempt < 5; attempt += 1) await runDurableObjectAlarm(stub) + expect(await runtime().ref(Counter, "cancelled").with({ authorizationContext }).count).toBe(0) + }) + + it("reads its own schedule inside a durable object", async () => { + const reference = runtime().ref(Counter, "reads-schedule").with({ authorizationContext }) + expect(await reference.readArmed()).toBeNull() + + await reference.arm({ at: Date.now() + 60_000 }) + + expect(await reference.readArmed()).toEqual({ name: "increment", interval: null }) + }) + + it("cancels one keyed reminder and every key of an operation", async () => { + const reference = runtime().ref(Counter, "keyed-cancel").with({ authorizationContext }) + await reference.armKeyed({ at: Date.now() + 60_000, keys: ["a", "b", "c"] }) + await reference.disarmKey({ key: "b" }) + expect(await remainingReminders("keyed-cancel")).toEqual(["increment:a", "increment:c"]) + + await reference.disarmAll() + expect(await remainingReminders("keyed-cancel")).toEqual([]) + }) + it("retains a recoverable accepted message after caller timeout", async () => { const reference = runtime().ref(Counter, "delayed") const message = await reference.send @@ -84,3 +116,13 @@ describe("Durable Objects runtime", () => { ) }) }) + +async function remainingReminders(actorId: string): Promise { + const stub = env.ACTORS.getByName(JSON.stringify(["Counter", actorId])) + return runInDurableObject(stub, (_object, state) => + state.storage.sql + .exec<{ name: string }>("SELECT name FROM reminders ORDER BY name") + .toArray() + .map((row) => row.name), + ) +} diff --git a/test/cloudflare/worker.ts b/test/cloudflare/worker.ts index c6c82f9..7afff63 100644 --- a/test/cloudflare/worker.ts +++ b/test/cloudflare/worker.ts @@ -67,6 +67,29 @@ export class Counter extends Actor { this.schedule({ at: new Date(options.at) }).increment() } + armKeyed(options: { at: number; keys: string[] }): void { + for (const key of options.keys) { + this.schedule({ at: new Date(options.at), key }).increment() + } + } + + async readArmed(): Promise<{ name: string; interval: number | null } | null> { + const found = await this.reminder("increment") + return found ? { name: found.name, interval: found.intervalMilliseconds } : null + } + + disarm(): void { + this.unschedule("increment") + } + + disarmKey(options: { key: string }): void { + this.unschedule("increment", { key: options.key }) + } + + disarmAll(): void { + this.unscheduleAll("increment") + } + forward(options: { target: string }): void { this.sendTo(Counter.ref(options.target)).increment() } diff --git a/test/fixtures/actor-operations-consumer.mts b/test/fixtures/actor-operations-consumer.mts index c3cba74..497e449 100644 --- a/test/fixtures/actor-operations-consumer.mts +++ b/test/fixtures/actor-operations-consumer.mts @@ -3,6 +3,7 @@ import { type ActorReference, type ScheduledOperationsFor, type EffectOptions, + type ReminderHandle, } from "solid-objects" import type { ScheduledOperationsFor as CoreScheduledOperationsFor } from "solid-objects/core" @@ -26,11 +27,14 @@ export class ChatRun extends ParentRun { } start(): void { const operations = this.schedule({ at: new Date(0), key: "watchdog" }) - const result: void = operations.recoverIfStuck({ generation: 1 }) - void result + const handle: ReminderHandle = operations.recoverIfStuck({ generation: 1 }) + this.unschedule(handle) + this.unschedule("finish", { key: "watchdog" }) + this.unscheduleAll("finish") operations.finish() operations.optional() - this.transmit().recoverIfStuck({ generation: 1 }) + const transmitted: void = this.transmit().recoverIfStuck({ generation: 1 }) + void transmitted this.emit("run_model", { onFailure: "finish" }) // @ts-expect-error operation typo operations.recoverIfStcuk({ generation: 1 }) diff --git a/test/reminder-cancellation.test.ts b/test/reminder-cancellation.test.ts new file mode 100644 index 0000000..c579481 --- /dev/null +++ b/test/reminder-cancellation.test.ts @@ -0,0 +1,404 @@ +import { mkdtemp, rm } from "node:fs/promises" +import { tmpdir } from "node:os" +import { join } from "node:path" +import { afterEach, describe, expect, it } from "vitest" +import { Actor } from "../src/actor.js" +import { sqlite } from "../src/database/sqlite.js" +import { configure, type SolidObjectsRuntime } from "../src/runtime.js" +import type { ReminderHandle, ScheduledReminder } from "../src/types.js" + +class Subscription extends Actor { + static override readonly actorType = "cancel-subscriptions" + + status = "trialing" + expirations = 0 + handle: ReminderHandle | null = null + + startTrial(): void { + this.handle = this.schedule({ at: new Date(Date.now() + 3_600_000) }).trialExpired() + } + + startRecurring(): void { + this.handle = this.schedule({ + at: new Date(Date.now() - 1_000), + everyMilliseconds: 60_000, + }).trialExpired() + } + + convertByName(): void { + this.status = "active" + this.unschedule("trialExpired") + } + + convertByHandle(): void { + this.status = "active" + if (this.handle) this.unschedule(this.handle) + } + + cancelThenReschedule(): void { + this.unschedule("trialExpired") + this.schedule({ at: new Date(Date.UTC(2031, 0, 1)) }).trialExpired() + } + + cancelThenFail(): void { + this.unschedule("trialExpired") + throw new Error("turn failed") + } + + stopAllTrials(): void { + this.unscheduleAll("trialExpired") + } + + async readTrial(): Promise { + return (await this.reminder("trialExpired")) ?? null + } + + async readAfterStagedSchedule(): Promise { + this.schedule({ at: new Date(Date.UTC(2031, 0, 1)) }).trialExpired() + return (await this.reminder("trialExpired"))?.runAtMilliseconds ?? null + } + + async readAfterStagedCancel(): Promise { + this.unschedule("trialExpired") + return (await this.reminder("trialExpired")) !== undefined + } + + cancelUnknown(): void { + this.unschedule("noSuchOperation") + } + + cancelAllUnknown(): void { + this.unscheduleAll("noSuchOperation") + } + + cancelBadHandle(): void { + // @ts-expect-error a malformed handle exercises the runtime check + this.unschedule({ nope: "x" }) + } + + trialExpired(): void { + this.expirations += 1 + this.status = "expired" + } +} + +class Observed extends Actor { + static override readonly actorType = "cancel-observed" + + armed = false + seenOnActivate: string | null = null + + protected override async onActivate(): Promise { + this.seenOnActivate = (await this.reminder("ping"))?.name ?? null + } + + arm(): void { + this.schedule({ at: new Date(Date.UTC(2030, 0, 1)) }).ping() + } + + get armedName(): Promise { + return this.reminder("ping").then((found) => found?.name ?? null) + } + + ping(): void {} + + armDue(): void { + this.schedule({ at: new Date(Date.now() - 1_000) }).ping() + } +} + +class Shipment extends Actor { + static override readonly actorType = "cancel-shipments" + + dispatch({ carrierIds }: { carrierIds: string[] }): void { + for (const id of carrierIds) { + this.schedule({ at: new Date(Date.now() + 3_600_000), key: id }).chaseCarrier({ + carrierId: id, + }) + } + this.schedule({ at: new Date(Date.now() + 3_600_000) }).audit() + } + + shipped({ carrierId }: { carrierId: string }): void { + this.unschedule("chaseCarrier", { key: carrierId }) + } + + stopChasing(): void { + this.unscheduleAll("chaseCarrier") + } + + async pendingKeys(): Promise<(string | null)[]> { + return (await this.reminders("chaseCarrier")).map((status) => status.key).sort() + } + + chaseCarrier(_options: { carrierId: string }): void {} + audit(): void {} +} + +let runtime: SolidObjectsRuntime | undefined + +afterEach(async () => { + await runtime?.close() + runtime = undefined +}) + +function configuredRuntime(path: string): SolidObjectsRuntime { + runtime = configure({ + database: sqlite({ path }), + authorizeMessage: () => true, + authorizeQuery: () => true, + pollingIntervalMilliseconds: 1, + syncPollingIntervalMilliseconds: 1, + maxAttempts: 1, + }) + runtime.register(Subscription) + runtime.register(Shipment) + runtime.register(Observed) + return runtime +} + +async function start(): Promise { + runtime = configure({ + database: sqlite({ path: ":memory:" }), + authorizeMessage: () => true, + authorizeQuery: () => true, + pollingIntervalMilliseconds: 1, + syncPollingIntervalMilliseconds: 1, + maxAttempts: 1, + }) + runtime.register(Subscription) + runtime.register(Shipment) + runtime.register(Observed) + await runtime.install() + return runtime +} + +async function reminderNames(started: SolidObjectsRuntime): Promise { + const rows = await started.settings.database.connection((connection) => + connection.all<{ operation: string }>( + `SELECT operation FROM solid_objects_reminders ORDER BY operation`, + ), + ) + return rows.map((row) => row.operation) +} + +describe("reminder cancellation", () => { + it("returns a handle naming the reminder", async () => { + const started = await start() + const reference = Subscription.ref("alice") + await reference.startTrial() + + expect(await reference.handle).toEqual({ name: "trialExpired" }) + expect(await reminderNames(started)).toEqual(["trialExpired"]) + }) + + it("cancels by name", async () => { + const started = await start() + const reference = Subscription.ref("alice") + await reference.startTrial() + await reference.convertByName() + + expect(await reminderNames(started)).toEqual([]) + expect(await reference.status).toBe("active") + }) + + it("cancels by handle", async () => { + const started = await start() + const reference = Subscription.ref("alice") + await reference.startTrial() + await reference.convertByHandle() + + expect(await reminderNames(started)).toEqual([]) + }) + + it("stops a recurring reminder", async () => { + const started = await start() + const reference = Subscription.ref("alice") + await reference.startRecurring() + expect(await started.reminderScheduler().runOnce()).toBe(1) + await reference.convertByName() + + expect(await reminderNames(started)).toEqual([]) + expect(await started.reminderScheduler().runOnce()).toBe(0) + }) + + it("cancelling an absent reminder is not an error", async () => { + const started = await start() + await Subscription.ref("alice").convertByName() + + expect(await reminderNames(started)).toEqual([]) + }) + + it("a failed turn cancels nothing", async () => { + const started = await start() + const reference = Subscription.ref("alice") + await reference.startTrial() + await expect(reference.cancelThenFail()).rejects.toThrow() + + expect(await reminderNames(started)).toEqual(["trialExpired"]) + }) + + it("cancel then schedule in one turn leaves the new time", async () => { + const started = await start() + const reference = Subscription.ref("alice") + await reference.startTrial() + await reference.cancelThenReschedule() + + const rows = await started.settings.database.connection((connection) => + connection.all<{ run_at_ms: number | bigint }>( + `SELECT run_at_ms FROM solid_objects_reminders`, + ), + ) + expect(rows).toHaveLength(1) + expect(Number(rows[0]!.run_at_ms)).toBe(Date.UTC(2031, 0, 1)) + }) + + it("reads an armed reminder", async () => { + await start() + const reference = Subscription.ref("alice") + await reference.startRecurring() + + expect(await reference.readTrial()).toEqual({ + name: "trialExpired", + operation: "trialExpired", + key: null, + runAtMilliseconds: expect.any(Number), + intervalMilliseconds: 60_000, + missedPolicy: "latest", + status: "scheduled", + handle: { name: "trialExpired" }, + }) + }) + + it("reads nothing when no reminder is armed", async () => { + await start() + expect(await Subscription.ref("alice").readTrial()).toBeNull() + }) + + it("sees a schedule staged earlier in the same turn", async () => { + await start() + expect(await Subscription.ref("alice").readAfterStagedSchedule()).toBe(Date.UTC(2031, 0, 1)) + }) + + it("sees a cancel staged earlier in the same turn", async () => { + await start() + const reference = Subscription.ref("alice") + await reference.startTrial() + + expect(await reference.readAfterStagedCancel()).toBe(false) + }) + + it("lists every key of one operation", async () => { + await start() + const reference = Shipment.ref("truck") + await reference.dispatch({ carrierIds: ["a", "b", "c"] }) + + expect(await reference.pendingKeys()).toEqual(["a", "b", "c"]) + }) + + it("does not report a one-shot that already fired", async () => { + const started = await start() + const reference = Observed.ref("one") + await reference.armDue() + expect(await reference.armedName).toBe("ping") + + expect(await started.reminderScheduler().runOnce()).toBe(1) + await started.worker().runUntilIdle() + + expect(await reference.armedName).toBeNull() + }) + + it("reads the schedule from an activation hook", async () => { + const directory = await mkdtemp(join(tmpdir(), "reminder-hook-")) + const path = join(directory, "hook.sqlite3") + try { + const first = configuredRuntime(path) + await first.install() + await Observed.ref("one").arm() + await first.close() + + // A second runtime activates the actor fresh, so onActivate runs again + // with the reminder already armed. + const second = configuredRuntime(path) + await second.install() + expect(await Observed.ref("one").seenOnActivate).toBe("ping") + await second.close() + runtime = undefined + } finally { + await rm(directory, { recursive: true, force: true }) + } + }) + + it("reads the schedule from a snapshot projection", async () => { + const started = await start() + const reference = Observed.ref("one") + await reference.arm() + + const snapshot = await started.snapshot(Observed.ref("one")) + + expect(snapshot.armedName).toBe("ping") + }) + + it("refuses an unknown operation instead of cancelling nothing", async () => { + const started = await start() + const reference = Subscription.ref("alice") + await reference.startTrial() + + await expect(reference.cancelUnknown()).rejects.toThrow() + await expect(reference.cancelAllUnknown()).rejects.toThrow() + + expect(await reminderNames(started)).toEqual(["trialExpired"]) + }) + + it("rejects a malformed handle", async () => { + const started = await start() + await expect(Subscription.ref("alice").cancelBadHandle()).rejects.toThrow() + expect(await reminderNames(started)).toEqual([]) + }) + + it("cancels one key and leaves its siblings", async () => { + const started = await start() + const reference = Shipment.ref("truck") + await reference.dispatch({ carrierIds: ["a", "b", "c"] }) + await reference.shipped({ carrierId: "b" }) + + expect(await reminderNames(started)).toEqual(["audit", "chaseCarrier:a", "chaseCarrier:c"]) + }) + + it("cancels a reminder migrated before message_operation existed", async () => { + const started = await start() + const reference = Subscription.ref("alice") + await reference.startTrial() + // A row written before the keyed-reminder migration carries no + // message_operation, and its name is still the operation. + await started.settings.database.connection((connection) => + connection.run(`UPDATE solid_objects_reminders SET message_operation = NULL`), + ) + + await reference.stopAllTrials() + + expect(await reminderNames(started)).toEqual([]) + }) + + it("a cancel that lands on a claimed occurrence does not fail the scheduler", async () => { + const started = await start() + const reference = Subscription.ref("alice") + await reference.startRecurring() + + const claimed = await started.repository.claimReminder("test-scheduler") + expect(claimed).toBeDefined() + await reference.convertByName() + + await expect(started.repository.enqueueReminder(claimed!)).resolves.toBe(false) + expect(await started.reminderScheduler().runOnce()).toBe(0) + }) + + it("cancels every key of one operation", async () => { + const started = await start() + const reference = Shipment.ref("truck") + await reference.dispatch({ carrierIds: ["a", "b", "c"] }) + await reference.stopChasing() + + expect(await reminderNames(started)).toEqual(["audit"]) + }) +})