feat: select a wake-up adapter automatically - #55
Merged
Merged
Conversation
A commit in one process did not wake a worker process unless somebody configured an adapter, so delivery waited out the polling interval. The adapters existed and were measured. Nothing chose between them. `wakeUp` now takes a name or an adapter, as `config.cache_store` does in Rails, and defaults to `"automatic"`. Selection prefers a configured Redis URL, then PostgreSQL notifications, then polling. An unknown name throws rather than polls quietly. Selection proves the PostgreSQL path before it chooses it. It listens on a probe channel, sends one `NOTIFY` from a second connection, and waits for it to arrive. A setting that survives one round trip proves nothing, because a transaction pooler can hand the same backend to two consecutive statements, so only a delivered notification counts. `runtime.wakeUpCapability()` reports what was installed, whether it crosses processes, its measured floor, and why. The doctor reports the same record, and the polling-only warning now fires on what was installed rather than on whether a setting was set. An adapter that declares its own capability keeps it, so a configured `InProcessWakeUpAdapter` still warns. Selection runs once per runtime. The probe opens connections and waits, so callers that race for the first use share one promise rather than run one probe each. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`close()` cleared the memoised selection, so a late notify would select again and open a PostgreSQL client that nothing would ever close. The selection now survives `close()`. Every adapter ignores a notify after its own close, so a late caller is a no-op rather than a new connection. `resetForTesting()` still discards it, because a reset must select again. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Selection reached for `process.env` directly. Node defines that global and a browser does not, so the browser runtime threw before it could choose polling, and every wake-up stopped. Two browser tests caught it: the outbox drained nothing and live signals never updated. The read now goes through `globalThis.process?.env`, which is undefined in a browser rather than a throw. A test stubs the global away and holds that: without the guard it fails with "Cannot read properties of undefined (reading 'env')". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`validateWakeUp` nested the adapter check inside its non-string branch. The adapter check moves to its own function, so each branch is a guard clause. The selection test helper typed its setting as `unknown` and cast it away with `as never`, which removed the type from every caller. It takes a `WakeUpSetting` now, and the one test that supplies an invalid name casts at that call alone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Owner
Author
|
@greptileai review |
`capability` meant two things: what an adapter reports about itself and what selection installed. Polling shows the difference, because the adapter is an `InProcessWakeUpAdapter` while the installed capability is `polling` with a floor. An adapter declares `defaultCapability` now, and `runtime.wakeUpCapability()` stays the installed record. Ruby uses the same two names. `polling` also takes the interval it needs rather than the whole selection options. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A selection that cannot run is memoised, so the failure repeated on every commit. `wakeUp: "redis"` without a URL, or `"postgresql"` on a database that offers no notification channel, wrote one error per message. It writes one error per runtime now, under its own event name, because a failure to select is not the same as a failure to notify. The notify path is a try/catch rather than a promise chain, so the caught error carries its own type and no annotation widens it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Owner
Author
|
@greptileai review |
A requested adapter that the environment cannot provide claimed to cross processes. `wakeUp: "postgresql"` on a database with no notification channel threw, and a Redis URL without the `redis` package threw, so the failure surfaced as a repeated error after each commit rather than as a choice the operator could read. Each case now polls, logs `solid_objects.wake_up.unavailable` once, and records the reason in the capability, so the doctor warns. That is what the capability record exists for. Only a name that does not exist is still refused, because a typo cannot be honoured at all. Test types no longer annotate values as `unknown`. The intercepting connection takes its parameter types from `DatabaseConnection`, and the warning collectors name the shape they read. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Ruby suite covers this and the TypeScript suite did not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Ruby suite covers in_process opting out, a Redis URL winning on any database, a database that cannot answer, and a name refused when the configuration is built. The TypeScript suite did not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Owner
Author
|
@greptileai review |
The Ruby suite covers the matching case and the TypeScript suite did not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Owner
Author
|
@greptileai review |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ports cardmagic/solid-objects-ruby#77 to this runtime.
Why
A commit in one process did not wake a worker process unless somebody
configured an adapter, so delivery waited out the polling interval, up to the
one-second
idlePollingIntervalMillisecondsdefault. The adapters existed andwere measured. Nothing chose between them.
One setting, not two
wakeUptakes a name or an adapter, asconfig.cache_storeaccepts:redis_cache_storeor a cache object in Rails:An unknown name throws a
TypeErrornaming the accepted values, rather thanpolling quietly.
configure()throws it, so a typo fails at boot. A typo thatsilently costs a second of latency is the failure this change exists to remove.
A name that exists but that the environment cannot provide is different. It
polls, logs
solid_objects.wake_up.unavailableonce, and records the reason inthe capability, so the doctor warns rather than claim a cross-process wake-up
that cannot happen. That covers
"postgresql"on a database with nonotification channel,
"redis"withoutSOLID_OBJECTS_REDIS_URL, and a RedisURL without the
redispackage.Selection
SOLID_OBJECTS_REDIS_URLselects Redis, on any database.LISTEN, unless a probe notification fails to arrive.The probe proves delivery
LISTENdoes not survive a transaction pooler such as PgBouncer, so the path isprobed before it is chosen. A setting that survives one round trip proves
nothing: a pooler can hand the same backend to two consecutive statements, so
the read-back succeeds while
LISTENstill has no session affinity.Three points about the shape:
NOTIFYcomes from the database pool, not from the listening client.Notifying through the listener would pass whenever a pooler reused one
backend, which is the false positive the probe exists to catch.
wake every waiting role in the deployment once.
polling, which costs latency and not correctness. The probe closes its
connection, so selection leaves nothing open.
Reporting the choice
The doctor reports the same record as a
wakeUpcheck:passwhen it crossesprocesses,
warnwith the reason when it does not.An adapter that declares its own
capabilitykeeps it. A configuredInProcessWakeUpAdapterreportsin_processand does not cross processes, sothe doctor warns and the polling-only warning still fires. That warning now
guards on
capability.crossesProcessesrather than on whetherwakeUpwas set,which is what it meant to ask. An adapter that declares no capability is
somebody else's and is still assumed to cross processes, as before.
Selection runs once
runtime.wakeUpAdapter()memoises the selection promise, assigned before thefirst
await, so callers that race for the first use share one probe. The Rubyport needed a mutex for the same reason; here the promise is the lock. Without
it, eight racing callers ran eight probes and opened sixteen connections.
Behaviour change
PostgreSQL applications that configure nothing now use notifications. They
gain cross-process wake-up, one dedicated listening client outside the pool, and
one
NOTIFYper commit.wakeUp: "in_process"keeps the old behaviour.RepositoryandEffectRecoveryCoordinatornow take awakeUpAdapterresolver, because the adapter is no longer known when they are constructed.
The selection survives
runtime.close(). Clearing it would let a late notifyselect again and open a PostgreSQL client that nothing would ever close. Every
adapter ignores a notify after its own close, so a late caller is a no-op.
resetForTesting()still discards it, because a reset must select again.Tests
test/wake-up-selection.test.tsis new, 24 tests: a configured adapter wins andkeeps its own capability, an adapter that reports none is recorded as
configured, an unknown name throws, a database without a channel reports its
floor, PostgreSQL selects notifications when a probe arrives and polls when it
does not, the probe notifies from the database rather than the listener and
closes itself, the pooled fallback warns once, racing callers select once, a closed
runtime does not select again, selection works where no
processglobal exists,and the doctor reports the installed adapter.
Each one failed first. Reverting each fix gave the stated failure:
test/polling-loop.test.tsgains the other half of that: a configuredInProcessWakeUpAdaptermust still warn. Under the oldwakeUpConfiguredguard it failed, because any configured adapter suppressed the warning.
test/postgresql.test.tsgains two tests against a real server: automaticselection reports
postgresql_notifywith the probe reason, and a listenerwakes from a
NOTIFYsent on a second connection, which a pooled session couldnot do.
Validation
vitest runprettier --checkcheck:parameters,check:documentation,check:browser-importstscfortsconfig.json,.examples,.cloudflare,.buildThe browser has no
processThe first version read
process.envdirectly forSOLID_OBJECTS_REDIS_URL.Node defines that global and a browser does not, so the browser runtime threw
before it could choose polling, and every wake-up stopped. Two browser tests
caught it: the outbox drained nothing and live signals never updated.
The read goes through
globalThis.process?.envnow, which is undefined in abrowser rather than a throw. A test stubs the global away and holds that.
Without the guard it fails with
Cannot read properties of undefined (reading 'env'), and the browser suite fails withReceived: 0.This is the one place where the Ruby port gave no warning.
ENVis always therein Ruby, so the Ruby code reads it plainly.
Parity with the Ruby pull request
Same scope: one setting taking a name or an adapter, the same four names, the
same selection order, the same delivery probe on its own channel, the same
capability record, the same doctor check, the same warning guard, and one
selection per process.
The two suites now cover the same cases. Three details differ, and
docs/parity.mdrecords each. The pooled-session warning is emitted once perruntime here and once per process in Ruby, because this runtime supports several
runtimes in one process. A configured adapter must implement
watch,notify,and
closehere, while Ruby requiressignalandwaitand treatswatchandstopas optional, which is each runtime's own adapter contract. A selectionthat a later edit of the settings makes impossible is reported once here and
raised in Ruby; both refuse an unknown name when the configuration is built, so
this only reaches code that changes the setting afterwards.
Ruby needs a mutex to select once; here the memoised promise is the lock.
What this does not do
No MySQL notifier is shipped. MySQL has no
LISTEN/NOTIFYequivalent in anyrelease, 8.4 LTS and 9.x included.
GET_LOCKhands the lock to one waiterrather than broadcasting, and binlog tailing is separate infrastructure rather
than a session primitive. MySQL still polls, and the capability now says so and
reports its floor rather than leaving an operator to infer it.
🤖 Generated with Claude Code