Skip to content

feat(java)!: the Java provider on state-to-wasm flag modules - #702

Open
nicklasl wants to merge 15 commits into
nextfrom
java-provider
Open

nicklasl wants to merge 15 commits into
nextfrom
java-provider

Conversation

@nicklasl

@nicklasl nicklasl commented Nov 19, 2026 •

Copy link
Copy Markdown
Member

Draft pending review. This ports the Java provider to state-to-wasm flag modules, as #700
did for Go. It targets next and lands as one reviewed merge. All eight phases are landed, plus
a self-review fix round (58aab3fc).

What this does

Rewrites the Java provider on per-credential compiled flag modules: fetch the credential's
module from the module CDN, run it on Endive (the JVM-native wasm runtime, Chicory's
successor), and swap generations as flags change. The plan of record is
openfeature-provider/java/plans/ — one commit per phase, mirroring the Go reference
implementation.

Phase Status
1 — runtime spike (Endive 1.2.0 runs threaded modules: shared memory, atomics, wasm-to-wasm filter linking, re-entrant ack; ~22 µs/eval compiled) ✅
2 — state-to-wasm protos + shared fixtures ✅
3 — host runtime (internal.host) ✅
4 — module CDN fetcher (internal.cdn) ✅
5 — flag-log shipper (internal.ingest) + TestBackend ✅
6 — engine: module generations + swap ✅
7 — provider public API on the engine; the old gRPC stack is deleted here, as in #700 ✅
8 — build/CI/release/docs; Java joins PROVIDERS ✅

Verification

  • Harness (real pom, protoc and fixtures): 93 tests green, 5 e2e skipped without credentials.
    The provider suite is the go provider_test/value_test set plus the old Java suite's
    assertions (FlagResolverServiceTest 405/415/400/500 + context extraction,
    ApplyFlagsErrorHandlingTest, RequiredEncryptionTest, config validation).
  • docker build --target all . green: the new openfeature-provider-java.{base,build,test,lint, test_e2e} stages pass on maven:3.9-eclipse-temurin-17, wired into all; test_e2e is wired
    into e2e and runs against the production CDN + ingestion in CI (same-repo PRs get secrets).
  • The e2e set ports go's: non-sticky, materialized segments with/without a store, sticky via the
    hosted store.
  • The branch is merged with next (through 54676410, go's 0.30.0-rc1 release + the rc
    numbering change); the release-please manifest conflict resolved to go 0.30.0-rc1 + java
    0.23.4.
  • Interrupt-safety round (35402dac, from andreas-karlsson's review): Endive aborts module
    calls when the calling thread is interrupted, so an already-interrupted thread now fails fast
    at the resolver's entry check (InterruptedCallException, flag preserved); cleanup paths
    (suspend-discard, close's final flush and destroy) run uninterruptibly — an interrupt there
    would read as "closed" and leak a suspended process, or drop buffered logs; mid-call aborts
    and traps are reported through a new reportTrap hook for observability (generation recovery
    is the planned follow-up for both providers); Engine.close drains its loops
    (shutdown + bounded awaitTermination) instead of shutdownNow-interrupting an in-flight
    poll or flush. Five new tests pin each behavior, including an end-to-end discard-under-interrupt
    leak check and a drain test that fails if any poll is interrupted during close.
  • Self-review round (my diff pass + two independent agent reviews — bug-hunt and security):
    304 Not-Modified is now the quiet steady state (was a stack-trace warn every poll, with a
    regression test); evaluations report OpenFeature reasons (TARGETING_MATCH…, as Go) instead of
    raw proto enum names; instances default to one per processor; CDN fetches are deadline-bounded
    (30 s polls, start bounded by its remaining budget) and poll/flush run on separate threads;
    generation retirements left ForkJoinPool.commonPool(); absent struct fields zero-fill as Go's
    decodeTo; empty remote-store variant records mean "no assignment"; latency reports are
    best-effort; the shaded jar's relocations are asserted in make build (wired into CI's all)
    and the orphaned failsafe config is gone. Security review: credentials, crypto, Docker secrets,
    and parsers all clean — parity with Go and the old stack, with encryption now mandatory.
  • Release: release-please gains the java package (release-type: maven, extra-files
    README.md + Version.java, manifest at 0.23.4); the first release from next cuts
    0.24.0-rc.1.

Notes

  • Public API kept where sensible: constructor shapes, the MaterializationStore
    interface, FlagResolverService, Prometheus/OpenMetrics metrics. Removed/renamed per plan 07:
    ChannelFactory and all gRPC config; resolverPoolSize → instances(n) (default one per
    processor); enableApplyDedup/disableExposureCollection → exposureMode(DEDUPLICATED | ALL | NONE); useWasmInterpreter → interpreter(true) engine option.
  • Typed evaluations return error codes + the caller's default (go semantics) instead of
    throwing OpenFeature exceptions; initialize() throws on terminal startup and on
    initialization timeout, while the engine keeps retrying and flips the state to READY on
    recovery.
  • The new module accepts applies without send_time (the old stack swallowed those);
    rejected applies (e.g. rotated secret) are still swallowed and logged.
  • No gRPC anywhere: flag logs ship over HTTP to ingestion.confidence.dev with the module's
    log token as a bearer credential; the remote materialization store speaks hand-encoded
    protobuf over HTTP (wire-verified against go's test bytes).
  • Mockito's inline mock maker cannot self-attach on modern JDKs; surefire now loads
    byte-buddy-agent via an explicit -javaagent.
  • Dependency changes: com.dylibso.chicory → run.endive:runtime + run.endive:compiler
    1.2.0; protobuf 4.33.2 paired with proto-google-common-protos 2.65.0; gRPC, Guava and the
    Chicory AOT plugin are gone from the pom (the shaded jar shrinks accordingly).

nicklasl and others added 12 commits November 18, 2026 16:19
Ten plans porting the Go reference implementation (origin/go-provider) to
Java, one reviewed PR per phase, gated by a Chicory spike.

Co-Authored-By: Xirp Agent (glm-5-3) <xirp-agent@users.noreply.ghe.spotify.net>
45/45 checks on macOS, linux/arm64 and linux/amd64: shared memory,
atomics in both engines, wasm-to-wasm filter linking, re-entrant
ack_logs, dedupe, suspend/resume, and 22us/eval on the runtime
compiler. Plans updated to run.endive coordinates.

Co-Authored-By: Xirp Agent (glm-5-3) <xirp-agent@users.noreply.ghe.spotify.net>
Point the plans at next (go merged as #700), record the one-PR landing
strategy, and fold in the phase-2 outcome notes.

Co-Authored-By: Xirp Agent (glm-5-3) <xirp-agent@users.noreply.ghe.spotify.net>
protobuf 4.33.2 with proto-google-common-protos 2.65.0 (the pairing
state-to-wasm builds with), the four provider proto groups only, no
gRPC generation. gRPC, guava and the Chicory AOT stack leave the pom;
the dedupe filter module is embedded as a resource and the shared
testdata is pinned by TestdataTest. The old source stays until the
provider commit, as the go merge did.

Co-Authored-By: Xirp Agent (glm-5-3) <xirp-agent@users.noreply.ghe.spotify.net>
Co-Authored-By: Xirp Agent (glm-5-3) <xirp-agent@users.noreply.ghe.spotify.net>
internal.host runs threaded flag modules on Endive 1.2.0: module
validation (imports, exports, shared host.memory), a capped shared
memory, a lazily filled instance pool, resolve with suspend/resume
through a materialization store (every abandoned process discarded,
allocation failures surfaced as errors), evaluate, apply, logs with a
re-entrant ack, filters (constants as host functions, the dedupe
filter's memory outliving generations), metrics, and close. Tests port
the go host's module, envelope and resolver suites; 23 green.

Co-Authored-By: Xirp Agent (glm-5-3) <xirp-agent@users.noreply.ghe.spotify.net>
internal.cdn fetches a credential's ClientModule from the module CDN,
decrypts the nonce||ct||tag artifact with AES-256-GCM, and polls with
ETags accepted only after the module installs. 404 and 408/429/5xx are
retryable, other 4xx and artifact failures terminal, surfaced as an
exception taxonomy the engine can branch on. The e2e fetch self-skips
without credentials; test-e2e runs the *E2ETest set by name.

Co-Authored-By: Xirp Agent (glm-5-3) <xirp-agent@users.noreply.ghe.spotify.net>
internal.ingest forwards serialized WriteFlagLogsRequests verbatim to
/v1/flagLogs:write with the batch's log token as a bearer credential,
from a bounded queue that drops and counts instead of blocking the
write_logs callback. Retries network, 429 and 5xx three times with
200ms-400ms backoff; close drains bounded by its deadline.
internal.testutil.TestBackend fakes the module CDN and ingestion for the
engine and provider tests. Tests port the go shipper suite.

Co-Authored-By: Xirp Agent (glm-5-3) <xirp-agent@users.noreply.ghe.spotify.net>
internal.engine owns the fetcher, the log shipper, the dedupe filters
and the current module generation: startup retries within a budget
(terminal on rejected credentials and invalid first modules, a
recoverable timeout that keeps polling and fires onReady), a poll loop
that swaps in new modules without dropping evaluations (a generation is
unpublished, then write-locked forever; in-flight requests drain), a
timed flush, and a close that honors its deadline while evaluations are
in flight and cleans up afterwards. Tests port the go engine suite
against TestBackend; also fixes a null If-None-Match handling bug in
TestBackend.

Co-Authored-By: Xirp Agent (glm-5-3) <xirp-agent@users.noreply.ghe.spotify.net>
Rebuilds OpenFeatureLocalResolveProvider, FlagResolverService and the
materialization stores on the engine; deletes the gRPC stack and its
tests, re-ported against the module. Ports go fb866c6 (+0ce35258,
870eb77, c7f9121).

Co-Authored-By: Xirp Agent (glm-5-3) <xirp-agent@users.noreply.ghe.spotify.net>
Docker stages, PROVIDERS, the lint target, release-please (maven, rc
from next), the flag-module README/CLAUDE.md/STICKY_RESOLVE.md, and the
e2e set against the production CDN. Ports go 42610a2 + b8df818.

Co-Authored-By: Xirp Agent (glm-5-3) <xirp-agent@users.noreply.ghe.spotify.net>
Steady-state 304s, reason mapping, instance and fetch bounds, thread
and retirement hygiene, decode semantics, and the shaded-jar check.

Co-Authored-By: Xirp Agent (glm-5-3) <xirp-agent@users.noreply.ghe.spotify.net>

@andreas-karlsson andreas-karlsson left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interrupt handling: Endive aborts a module call when the calling thread is interrupted, which can leave the shared memory inconsistent. Four small changes inline. Making a poisoned generation recoverable (after a mid-call interrupt or any trap) is a planned follow-up for both providers, not something for this PR.

Why it matters. From the Endive 1.2.0 sources and bytecode: the interpreter checks Thread.isInterrupted() at every call and backwards branch, the compiler emits the same check (Shaded.checkInterruption) for every call and call_indirect and in each compiled function, and memory.atomic.wait throws WasmInterruptedException when interrupted. So an interrupt can abort a call anywhere, in both modes, possibly with the module's lock held or a log buffer half written, and the instance goes back to the pool. Interrupts are routine on the JVM: Future.cancel(true), Resilience4j's TimeLimiter, invokeAll with a timeout, shutdownNow() at deploys. Go doesn't have this problem: module calls never get the caller's context (callCtx), so a call always runs to completion.

Planned follow-up, for both Go and Java. A trap or an interrupt that lands inside a call poisons the generation: log an error, don't flush (a flush could block on a held lock, or ship a half-written buffer), and rebuild from the compiled module with a new memory and new instances. Go has the same gap for traps today, so we'll do both together, with a test per provider. Nothing in this PR needs to anticipate it; Resolver already keeps the machineFactory.

🤖 Generated with Claude Code

}
ModuleInstance inst = claim();
try {
return call.run(inst);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Entry check, and log what still comes out. Two small changes in with():

  1. If the thread is already interrupted, don't enter wasm. Fail with an interrupt the caller can see: either throw InterruptedException (clearing the flag, the JDK convention) and let the provider turn it into an error result with the flag set again, or keep the flag set and throw an unchecked exception. The module is untouched, so this is an ordinary error. It covers the common case, a thread that was already interrupted when it called us.
  2. A WasmInterruptedException (or any trap) that still comes out of call.run means the call was aborted midway, and the module's state may be inconsistent. Log it at error level saying so. Fixing it is the follow-up (see the review comment); the log line lets us see whether it happens in the wild.

The wait for an instance in claim() should stay interruptible: no module call is running there.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — the entry check throws an unchecked InterruptedCallException with the flag left set, and claim() stays interruptible. Traps and mid-call aborts now go to a reportTrap hook the engine wires to an error log; the test drives the abort through an apply-filter host function that sets the flag mid-call, on both the compiler and the interpreter. Closing that poisoned resolver actually hangs on the held lock, so the test just leaves it unclosed — presumably exactly what the generation-recovery follow-up is for.

}

/** Discards a suspended process on any instance, even one that must be waited for. */
private List<Throwable> discard(int id) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cleanup calls must not be interruptible. If the thread is interrupted here, claim() turns the InterruptedException into a ClosedException, which this method reads as "the resolver is closed and has freed the process". The suspended process leaks. The same applies to the final flush and destroy_instance in close() (lines 176 and 182): an interrupt there drops buffered logs.

For these three, take the flag off for the duration (boolean interrupted = Thread.interrupted()), wait for an instance uninterruptibly, make the call, and set the flag again in finally. This is what InstanceSlots.awaitClaim in state-to-wasm does. Go's equivalent is the context.WithoutCancel in its discard.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done via a withUninterruptibly that strips the flag for the duration and restores it in finally — discard() uses it, and close() strips around the final flush and destroy. The test reproduces the leak you described end-to-end: the store interrupts the read and returns an oversized resume record, and the suspended process would leak without it.

}
long deadline = System.nanoTime() + timeout.toNanos();
if (loops != null) {
loops.shutdownNow();

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shutdownNow() interrupts our own module calls. If a timed flush or a poll is running, this interrupts it, and with Endive that can abort the flush midway, with the consequences described in the review comment. Suggest shutdown() plus the closed flag the loops already see, then awaitTermination, so a running flush or poll finishes. A fetch in progress is already bounded by its deadline.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — shutdown() plus awaitTermination bounded by the close deadline, so an in-flight poll or flush runs out instead of being interrupted. The test holds a fetch in flight server-side and pins the no-interrupt part with a logger mock asserting no poll failure got logged; the drained 304 stays quiet thanks to the earlier self-review fix.

nicklasl and others added 3 commits November 20, 2026 12:00
An interrupted thread never enters the module; cleanup (discard, final
flush, destroy) runs uninterruptibly so suspended processes cannot leak
and buffered logs cannot drop; traps and mid-call aborts are reported;
the engine drains its loops on close instead of interrupting them.

Co-Authored-By: Xirp Agent (glm-5-3) <xirp-agent@users.noreply.ghe.spotify.net>
The phase plans did their job; the PR history keeps them.

Co-Authored-By: Xirp Agent (glm-5-3) <xirp-agent@users.noreply.ghe.spotify.net>
@nicklasl
nicklasl marked this pull request as ready for review November 20, 2026 16:06

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants