Repository navigation
Add ios.partWindow to set how many part tasks iOS holds - #10
Closed
elliottkember wants to merge 127 commits into
Closed
elliottkember wants to merge 127 commits into
elliottkember wants to merge 127 commits into
Conversation
Diana is the only consumer of this library. This PR removes the API that Diana does not use: - Scoped listeners. Diana's test mock does not permit them. - The `android.maxRetries` option. Diana never set it. The library owns the retry policy. - The iOS `getUploadStatus` function. Its one use was a consumer workaround. - A dead TODO in the types. The PR keeps `android.addNotificationListener`. Diana uses it to route notification taps (`uploadNotificationEpics.ts:35`). An earlier usage check had missed the JS wrapper name. The PR renames the `customUploadId` option to `id`. There is no reason for the `custom` prefix. The PR also makes iOS `startUpload` idempotent for a consumer-supplied id. A second call with the same id resolves with that id. It does not start a second upload. Android already has this behavior through `ExistingWorkPolicy.KEEP`. This change made `getUploadStatus` unnecessary. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Diana builds the same notification text object for each `startUpload` call. This PR moves that text to a one-time `configure()` call. The library writes the configuration to SharedPreferences. Thus a worker that WorkManager relaunches with no JS running builds its notification with the same text. If the app never calls `configure()`, the library uses its default text. The per-upload `android` options decrease to one flag: `noNotification`. A `TODO` in Diana's `backgroundUploadUtils.ts` suggested this change. Built and reviewed with a multi-agent workflow: a six-dimension adversarial review. Every confirmed finding was fixed inside its slice. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This PR moves the merged chunked-upload work onto master. It replaces #39, which conflicted. The stacked PRs #32–#35 were reviewed and merged, but their stack was rooted on `dylan/v9-3-configure`, so their four squash commits landed on that branch and not on master. This branch carries those same four commits, cherry-picked onto the current master. The tree at the tip is byte-identical to the merged stack. There are no new changes, and there are no conflicts: the branch is a fast-forward of master. - Chunked JS surface — types, codegen spec, chunkPlan (#32) - Android chunked-upload engine (#33) - iOS chunked-upload engine (#34) - Global cap, example app, docs, v9.0.0 (#35) Prefer "Rebase and merge", so the four commits keep their identities on master. After the merge, master is v9.0.0 and gets the `v9.0.0` tag. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
**This is a fix for the current version, not v10** On iOS, `uploadTask(with:fromFile:)` on a background session throws an Objective-C exception when it cannot read the file. Swift cannot catch that exception, so the app dies. In Diana this is Sentry DIANA-19VW: about 2,500 crashes from 1,700 users since August. It usually happens at launch. The app checks that the file exists, and then a finished upload deletes that file before iOS opens it. This change wraps the call in a small Objective-C helper that catches the exception and returns it to Swift as an error. Before: a missing file ends the app. After: `startUpload` rejects with "Cannot read file at …". Any concurrent calls with the same id get the same rejection. A chunked part that cannot be enqueued stalls with `errorKind: 'file'`. ## What to look at - The podspec. `RNBGUCatchException.h` is now the one public header, so Swift can see it through the module umbrella. It imports only Foundation, so a plain Obj-C `@import react_native_background_upload;` still compiles. - `startUpload` in `RNBackgroundUpload.swift`. Parked same-id calls now store both resolve and reject, so a failure reaches all of them. ## Decisions and trade-offs - **A chunked part that fails to enqueue stalls.** A failed part-file build is handled the same way. The upload keeps its bytes, and the next `startUpload` resumes it. - **The rejection uses the existing "RN Uploader" code.** The thrown error is `NSURLErrorCannotOpenFile`, so `errorKind(for:)` classifies it as `'file'`. - **Version 9.0.1 is set, but no tag is created.** Tagging happens at release. ## Test Plan - A standalone macOS program created a background session and uploaded a missing file. It raised "NSInvalidArgumentException: Cannot read file at …", which matches the Sentry event. With the helper the exception was caught. Without it the process terminated. - All iOS Swift sources type-check against the iOS 15.1 SDK, with React's two promise typedefs stubbed. Unmodified master also passes this check. - `yarn lint-root`, `yarn typecheck` and `yarn test` pass with 37 tests. - Not verified: a real CocoaPods build and a device run. Pod install must pick up the two new files and put the public header in the umbrella. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Fixes the top Android crash on Diana 110.3: `ForegroundServiceDidNotStopInTimeException` (openspace Sentry DIANA-FT0, 280 users). ## What breaks Android 15 gives `dataSync` foreground services six hours per day while the app is in the background. When the budget runs out the system calls the service's `onTimeout` and waits a few seconds for `stopSelf`. If that never comes, it kills the process. WorkManager 2.9 does not implement `onTimeout`, so a capture upload that outlives the budget takes the app down. Every event is on Android 15 or newer with the app in the background. ## What changes `androidx.work` goes from 2.9.1 to 2.12.0. Since 2.10, WorkManager answers the timeout: it stops each worker of the timed-out type with `STOP_REASON_FOREGROUND_SERVICE_TIMEOUT` and stops the service. The stopped workers re-run, `setForeground` is refused because the budget is spent, and `isForegroundStartDenied` already handles that by continuing the upload without foreground priority. The budget resets when the user opens the app. The library's minSdk floor becomes 24 with this version. Ours is 29. ## Not changed The raw upload worker still retries transport failures with no network constraint, so an offline worker spends the same six hours a real upload would. The worker carries a comment that WorkManager constraints were unreliable, so that stays a separate decision. ## Verification `:react-native-background-upload:compileDebugKotlin` and `testDebugUnitTest` pass locally on the example app, 84 tests. Not run on a device: reproducing needs an upload that sits in the background for six hours on an Android 15 phone. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
The public surface becomes a durable mutation registry modeled on TanStack Query's setMutationDefaults + mutate. A consumer calls define() once per request kind at boot, with a request builder, an optional response parser, and onSuccess/onError handlers. Call sites pass only variables to mutate(). The native queue owns durability, retry, and delivery. JS: src/registry.ts (define, mutate, descriptor validation, header merge, vars cap, ids), src/delivery.ts (journal replay after configure(), eventId dedupe, wait-for-mutate ordering, handler routing, ack after the handler's promise, 30 s warning, unhandled-key reporting), src/index.ts (createUploadClient), src/types.ts. 113 tests plus type tests. Codegen spec: enqueue, pause, resume, cancel, setWifiOnly, updateHeaders, synchronous getRequests, onState/onProgress/onAttempt/onSettled emitters. Removed: startUpload, startChunkedUpload, cancelUpload, removeUpload, getAllUploads, and the v9 per-outcome emitters. Native: both modules stub the new methods with E_NOT_IMPLEMENTED so the package compiles and CI passes alone. The v9 engines stay in place for the Android and iOS slices to wire up. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Review findings on the JS layer, each with a test: - Outcomes of one id now deliver in order, one handler at a time, through a per-id promise lane. Different ids stay concurrent. - A cancelled outcome acks before the definition lookup, so an entry whose key was renamed still frees its bytes. - A settled event without a string key, kind and id is dropped with one warning per eventId, neither acked nor emitted (a v9-shaped journal). - mutate() resolves with the entry id it tracked, not native's return. - Nested descriptor objects (retry, accept, android, part range) reject unknown keys and wrong value shapes, so a typo cannot silently fall back to the transient default. - Header merge matches names without regard to case; the descriptor's spelling and value win. - A throwing state listener is caught and warned, and does not block the other listeners or the ack. - CHANGELOG lists the removed UploadId type. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A DELETE, or a POST whose meaning is in the URL, has no body. Diana has several of these today (delete comment, delete attachment, convert photo, watchers). The validator now allows at most one of data, form, file, and none is valid. Docs and the test follow. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
vars is now any object or null, and data is any JSON-serializable value. Generated OpenAPI request types and DTOs have optional fields, object-typed values, and nullable strings, none of which satisfy a recursive Json type. TypeScript cannot prove serializability for those shapes, so mutate() validates at runtime: a cycle, a function, a Date, or a value that does not serialize to an object is rejected with a named error. The 4 KB cap stays. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Native enqueue() is one synchronous disk write, so a promise that never
settles is a native bug. mutate() now races it against a timer, 10 s by
default (configure({ enqueueTimeoutMs })). On timeout it rejects with an
error naming the key and id and logs a warning, and delivery for that id
is not held open. If native did persist the entry, its outcome still
reaches the handlers, and a same-id retry resumes rather than duplicates.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ative contract
From the JS interface review. The 4 KB vars cap was a leftover from the
old ctx field; the generated-ops design puts the request body in vars, so
the default is 1 MB and configure({ maxVarsBytes }) sets it. Meta gains
deliveries, native's count of how many times an outcome was delivered, so
an app can decide a poison policy; the library never gives up on its own.
The response parser receives vars, so a decoder that needs an id runs in
the parser path, where a throw is a terminal onError, not a replay.
The codegen spec comments now pin the contract the native slices
implement: enqueue resolves after every staged copy is on disk and rejects
with E_RUNNING, E_FILE_MISSING, or E_STORAGE; same-id rules for a changed
body; the exact set of rows getRequests returns; cancel of an unknown id
is a no-op; ackEvents is void and idempotent; updateHeaders bumps a header
generation; attempt events are live-only. RequestRow gains nextAttemptAt.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Findings from the Android and iOS review that land on the JS side:
- vars and data cross the bridge as JSON strings (varsJson, dataJson).
React Native on iOS drops null-valued object keys, so an object form
loses fields such as { status: null }. A "null" body is a real body; a
bodiless request omits data.
- A GET with a body is rejected at mutate(). DELETE with a body stays.
- E_INVALID joins the enqueue rejection codes for input native cannot
send: non-http(s) URL, bad header names or values, GET with a body,
parts that do not tile the moved file.
- deliveries counts deliveries that reached a JS listener; an outcome
journaled with no listener starts at 0 and is not emitted live.
- A different url or method is a different body. attempts count the
current generation. cancel of a settled entry also forgets its
unacknowledged outcomes. updateHeaders also replaces same-named part
headers. A paused entry past expiresAt settles at resume.
- AttemptEvent.outcome narrows to completed | error; pause, cancel, and
supersede emit no attempt event.
- The watchdog text says a timeout means native did not answer, and that
staging a large file body takes time.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…e written Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Android implements the v10 contract pinned in the codegen spec. The v9 chunked manifest store generalizes into QueueStore: one durable entry per mutate() (id, key, vars, descriptor, staged body, state, attempts, bytes, expiresAt, header generation, deliveries) written tmp+fsync+rename before any attempt. BodyStaging writes JSON and multipart bodies to library files, copies single-file bodies, and moves chunked sources as v9 did. WorkManager runs one worker per entry. WorkerOps applies the retry table from the plan: accept rules with bodyIncludes, transient network/5xx/408/ 429 with jittered backoff and nextAttemptAt, 401/403 parking with a header generation that updateHeaders bumps, per-request exempt lists, expiry at expiresAt with bytes kept. The transfer semaphore (4) and chunked window (3) stay. Pause gates the queue; cancel settles a live entry and forgets a settled one; ack is void and idempotent and forgets only a matching generation. The journal writes every outcome before onSettled emits it; onState carries a full RequestRow, onProgress is throttled 1 s / 10 min, onAttempt is live-only. RequestIndex backs the synchronous getRequests. First launch imports v9 journal entries as legacy rows and cancels v9 work. Logic lives in JVM-testable classes (QueueController, WorkerOps, EnqueueRules, EntryTransitions); the module and workers are thin shells. 217 unit tests, including crash-mid-write cases for the store. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Reliability: begin() applies an own-generation journal record instead of re-running, so a crash between the journal write and the store update cannot double-send. A failed journal write on settle holds the record in memory and retries; nothing is acked that was not written. cancel() stops the work in a finally block, journals before it saves, and applies an existing record of its generation rather than adding a second outcome. Listening state and the deliveries count are decided under one journal lock, and a live settle goes to the listener that drained, not the newest module instance. Contract: vars and data arrive as JSON text; "null" is a body; GET with a body rejects E_INVALID. Attempt events are completed or error only; pause, cancel, and supersede emit none. A settled entry's cancel forgets its unacked outcomes. attempts reset only on reopen. Under pause only an accepted response settles. A same-id enqueue over a legacy row adopts its v9 manifest. The response body cap applies while streaming. A prune never deletes an eventId a row or a new record names. Rows carry live bytesSent. Platform: a headless run stopped at JobScheduler's 10-minute limit takes the transient path with backoff. Header validation messages carry the name and offset, never the value. README documents the headless limit and the allowBackup requirement. Simplification: one EntryRun attempt path shared by simple and chunked transfers; classifyFailure is the only failure rule; dead code removed. Tests: 289 (was 217), with a TransferHost seam for the run loop. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The test cleared the listener with stopListening(listener), but from the second iteration on the listener was the previous iteration's owner, so nothing was cleared. When the settle won the race, the journal stamped one live delivery and the drain counted a second one. CI failed about one run in five. Now each iteration clears whatever listener is set and asserts that none is set before the race starts. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
When the cancel record was journaled but the entry save then failed, cancel() rejected with E_STORAGE. The cancel had already happened: the worker stopped, the cancelled outcome went to JS, and the ack, the next cancel(), or the boot sweep applies the record. A rejection told JS that nothing changed. Now cancel() resolves in that case, as iOS does. It rejects only when the journal write fails, and then nothing changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
iOS implements the same contract over a background URLSession. QueueStore generalizes the v9 chunked manifest into one durable entry per mutate(), written with fsync before any task is created. BodyStaging writes JSON and multipart bodies to files, copies single-file bodies, and moves chunked sources, so every upload comes from a file as the background session requires. QueueCoordinator owns the entries: enqueue with the same-id rules and E_RUNNING/E_FILE_MISSING/E_STORAGE codes, one task per attempt keyed by (id, generation, attempt), TaskMap reconciliation at relaunch and through handleEventsForBackgroundURLSession, the retry table with delayed tasks for backoff, 401/403 parking with header generations, expiry, pause, cancel, and forget-after-ack. The journal writes before onSettled emits; onState, onProgress (throttled), and onAttempt follow the TypeScript payload types. RequestIndex backs the synchronous getRequests. First launch imports v9 journal entries as legacy rows. The pure half of the module is a SwiftPM package (ios/Package.swift) so `cd ios && swift test` runs 138 host-side tests, including crash-mid-write. The podspec excludes the package files and test sources from the pod. The example app builds for the simulator. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Reliability: reconcile applies unacked same-generation records to a live row before the task loop, so a failed entry save after the journal write cannot re-issue a settled request. deliveries counts only deliveries that reached a JS listener: journaled at 0 with no listener, not emitted live. A chunked part with a pending replay holds its slot for the grace period. A completion that lands before reconcile still advances the attempt ordinal. A part-file build error settles error/file only when the blob is missing or short; otherwise it refills after backoff. The background completion handler releases as soon as the awaited replay lands, and the grace timer only closes the wait it opened. Contract: vars and data arrive as JSON text; NSNull handling for data is gone; E_INVALID covers non-http(s) URLs, bad header names or values, and GET with a body. url and method are part of the body fingerprint. attempts reset only on reopen. A paused entry past expiresAt settles at resume; the expiry check runs after classification. A same-id mutate on a waiting retry retries now. updateHeaders patches part headers. Rows carry live bytesSent; legacy rows report 0/0. Attempt events are completed or error. Simplification: dead fields removed (TaskMap.Meta.accept, isChunkedPart, fileUnreadable, lifetimeMs, descriptorJSON, allDormantManifests). Tests: 170 (was 138), including relaunch and pending-replay cases. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Owner decision, matching Android: a cancel whose journal write fails rejects with E_STORAGE and changes nothing; the caller may call again. The hold-and-retry path stays for settle outcomes only. A cancel of a settled entry now forgets atomically: the id directory is set aside by one rename, the journal files are deleted, and a failure puts the row back and rejects. A crash between the two steps is finished or undone at launch. 178 tests. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A background URLSession raises an Objective-C exception, not an error, when uploadTask(with:fromFile:) cannot read the file. Swift cannot catch it, so the app ends. v9.0.1 fixed this for the v9 code (Sentry DIANA-19VW). This applies the same guard to the v10 transport. The transport now throws when the session cannot open the file. Each caller decides what that means: - A simple entry whose staged body is gone settles error 'file', as the missing-body check before it already does. - A simple entry whose body still exists is unreadable for now (data protection while the device is locked). It goes back to queued and is issued again after a backoff. - A chunked part follows the failed part-file rule: a short blob settles error 'file'; otherwise the window refills after a backoff. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The example app is the device test harness. It is rewritten on the v10
API: definitions at module scope (JSON POST, multipart, chunked with
chunkPlan, bodiless DELETE, no-vars GET), configure() with a headers
provider, a queue list from getRequests() and the state and progress
feeds, an attempt log, and controls for every step of the two device
scripts: cancel, pause, resume, wifiOnly, updateHeaders with a typed
value, same-id resume and replace, a short expiresAt, and a section of
outcomes journaled before this launch for the kill-and-relaunch check.
The local Express server answers by path segment (401, 404, 409, 503,
slow, oversize) and writes chunked parts at their offsets. The stale
Podfile.lock is regenerated for RN 0.84.1, the AppDelegate wires the
background completion handler, and the Android manifest declares and
requests POST_NOTIFICATIONS. The example README carries the device
script in UI terms.
Library: createUploadClient({ native }) accepts a fake TurboModule, and
react-native-background-upload/src/testing exports createFakeNative(),
an in-memory Spec with settle(), seeded rows, recorded calls, and ack
tracking, so a consumer tests definitions and handlers against the real
registry and delivery. CI gains a macOS job that runs swift test.
Docs: CHANGELOG 10.0.0 covers all slices; README gets an "Upgrading from
v9" section, iOS platform notes, a "Testing your definitions" section,
and a corrected AppDelegate snippet (@import for .m, header search path
for .mm). Version 10.0.0.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both changes come from the Diana readiness audit. Diana's Wi-Fi setting
applies to captures only, and Diana pauses captures only while field
note traffic keeps flowing. A queue-wide gate cannot express either.
RequestDescriptor.wifiOnly overrides the queue setting for that entry;
an entry that does not set it follows setWifiOnly, so toggling the
setting still moves queued captures. Both natives evaluate the
constraint per attempt and persist the per-entry value.
pause(scope) and resume(scope) take { keys?: string[] }. No keys means
the global gate, as before. Keys add to or remove from a persisted set.
An entry is paused when the gate is on or its key is in the set; each
entry that moves emits one state event; a scoped resume does not free
an entry the other gate still holds; an entry enqueued into a paused
scope starts paused. Pause produces no outcome and no attempt event.
The fake native in src/testing models both. Tests: JS 189, Android 307,
iOS 201.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The promotion PR says the script measures the time from mutate() to the first send, which decides the deferred in-process fast path. The step now exists, on both platforms, using the log clocks the harness prints. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A chunked upload keeps 3 part tasks in the background session and creates the next one as each finishes. iOS starts a task created while the app is in the background at its own discretion, so later parts can wait for the system. ios.partWindow lets a caller set the window per request. A window that covers every part creates all part tasks at mutate(). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QFiCVD9pHXvcx712ubCLpR
Owner
Author
|
Opened on the wrong repo; moved to openspacelabs. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
A chunked upload keeps 3 part tasks in the iOS background session. It creates the next part task when one finishes, and that usually happens while the app is in the background. Apple's docs say the system starts a task created in the background at its own discretion and ignores
isDiscretionary = false. So after the first 3 parts, a large file can wait for the system, for example until the device charges.This adds
ios: { partWindow }to the request descriptor. It sets how many part tasks the session holds for that entry. A window that covers every part creates all part tasks atmutate(), which is usually in the foreground. The default stays 3, so current callers see no change.Before: a 40-part capture had 3 part tasks in the session, and parts 4 to 40 were created later in the background.
After, with
partWindow: 40: all 40 part tasks are in the session aftermutate().What to look at
ChunkedCoordinator.refillreads the entry's window in place of the constant.QueueEntry.partWindowis optional, so entries saved by 10.0.0 decode with the default.Decisions and trade-offs
ios. Android runs parts in a worker, not a background session. An unbounded window there would only add sockets. Android ignores the field.partWindowwithoutpartsis rejected in JS. It would do nothing, and the library rejects fields that would be dropped in silence.mutate()replaces the window, and the next refill uses it. It does not refill at once.Test Plan
swift testinios/: 210 tests pass. New tests cover a window of every part, a window of 1, a relaunch that keeps the window, parser rejects, and decoding an entry saved without the field.yarn jest: 191 tests pass, with new tests for forwarding and validation.tscis clean.🤖 Generated with Claude Code
https://claude.ai/code/session_01QFiCVD9pHXvcx712ubCLpR