Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 26 additions & 10 deletions bench/lifecycle.exs
Original file line number Diff line number Diff line change
@@ -1,16 +1,18 @@
# MIX_ENV=test ORTEX_BUILD=true mix run bench/lifecycle.exs [runs] [load_dispose_cycles]
# MIX_ENV=test ORTEX_BUILD=true mix run bench/lifecycle.exs [runs] [load_dispose_cycles] [bounded_runs]
# RSS includes BEAM/allocator retention. This is a diagnostic, not a leak proof.
runs = System.argv() |> Enum.at(0, "100000") |> String.to_integer()
cycles = System.argv() |> Enum.at(1, "500") |> String.to_integer()
bounded = System.argv() |> Enum.at(2, "0") |> String.to_integer()
if bounded < 0, do: raise(ArgumentError, "bounded run count must be non-negative")
if runs < 10 or cycles < 10, do: raise(ArgumentError, "counts must be at least ten")
path = Path.expand("../test/fixtures/xgboost.onnx", __DIR__)
bytes = File.read!(path)
{%{rows: rows, expected: expected}, _} = Code.eval_file(Path.rootname(path) <> "_cases.ex")
input = Nx.tensor(rows, type: :f32) |> Nx.backend_transfer(Ortex.Backend)
options = [intra_threads: 1, intra_op_spinning: false]

score = fn model ->
{output} = Ortex.run(model, input)
score = fn model, run_options ->
{output} = Ortex.run(model, input, run_options)
actual = output |> Nx.backend_transfer(Nx.BinaryBackend) |> Nx.to_flat_list()

unless length(actual) == length(expected) and
Expand All @@ -27,37 +29,51 @@ sample = fn phase, count ->
_ -> nil
end

IO.inspect(%{phase: phase, count: count, rss_bytes: rss, beam_bytes: :erlang.memory(:total)})
IO.inspect(%{
phase: phase,
count: count,
rss_bytes: rss,
beam_bytes: :erlang.memory(:total),
processes: :erlang.system_info(:process_count)
})
end

model = Ortex.load(path, options)
for _ <- 1..1000, do: score.(model)
Enum.each(1..1000, fn _ -> score.(model, []) end)
sample.(:warmup, 1000)

for i <- 1..runs do
score.(model)
Enum.each(1..runs, fn i ->
score.(model, [])
if rem(i, max(div(runs, 10), 1)) == 0, do: sample.(:repeated_run, i)
end
end)

# Each load is owned by a fresh short-lived process so resources become eligible
# for destruction at a known boundary. Alternate both copying constructors.
for i <- 1..cycles do
Enum.each(1..cycles, fn i ->
Task.async(fn ->
loaded =
if rem(i, 2) == 0,
do: Ortex.load(path, options),
else: Ortex.load_from_memory(bytes, options)

score.(loaded)
score.(loaded, [])
end)
|> Task.await(60_000)

if rem(i, max(div(cycles, 10), 1)) == 0, do: sample.(:load_dispose, i)
end)

if bounded > 0 do
Enum.each(1..bounded, fn i ->
score.(model, timeout: 5000)
if rem(i, max(div(bounded, 10), 1)) == 0, do: sample.(:bounded_run, i)
end)
end

IO.inspect(%{
runs: runs,
load_dispose_cycles: cycles,
bounded_runs: bounded,
constructors: [:file, :memory],
note: "Golden scores checked on every call; RSS is diagnostic, not a leak assertion."
})
61 changes: 39 additions & 22 deletions docs/integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,10 +37,13 @@ writes to another repository. The broad original PR is superseded by the stack.
| 27 | One-attempt native cancellation controls |
| 28 | Managed deadlines and caller-exit cancellation |
| 29 | Dirty-scheduler measurement and policy |
| 30 | Final lifecycle/managed-run soak and review ledger |

Each PR targets the preceding branch. Merge or retarget in order; the integration
branch is for validation rather than a replacement broad PR. CI on each branch
checks that its intermediate state works independently.
Each PR targets the preceding branch. Merge in order with merge commits to
preserve ancestry. If using squash/rebase merges, rebase and retarget the remaining
stack and rerun its checks before proceeding. The integration branch combines
validation rather than replacing the small PRs. CI checks each intermediate state.
Task commits are unsigned because the local signing agent cancelled requests.

## Integration gates

Expand Down Expand Up @@ -87,24 +90,36 @@ redundant native clone changed median latency from 121 to 85 microseconds for a
reused resource and 207 to 166 microseconds from BinaryBackend. These are local
diagnostics, not cross-platform latency guarantees or a raw-binary zero-copy claim.

`bench/lifecycle.exs` validates golden scores on each repeated run and each model
loaded by a short-lived owner process. A local Apple silicon run exercised 100,000
runs plus 1,000 alternating file/memory load/disposal cycles per runtime:
`bench/lifecycle.exs` checks golden scores through ordinary runs, alternating
file/memory load-disposal owners, and optional bounded runs. The final Apple
silicon run used 100,000 ordinary runs, 1,000 load/disposal cycles, and 100,000
managed bounded calls on each runtime:

| Runtime | Warmup RSS | Maximum run RSS | Final load/disposal RSS |
| --- | --- | --- | --- |
| 1.17.3 | 104,005,632 | 104,759,296 | 103,596,032 |
| 1.24.4 | 109,608,960 | 110,346,240 | 109,297,664 |
```sh
ORTEX_BUILD=true MIX_ENV=test mix run bench/lifecycle.exs 100000 1000 100000
```

| Runtime | Warmup RSS | Maximum ordinary-run RSS | Final disposal RSS | Bounded RSS min/max | Final bounded RSS |
| --- | --- | --- | --- | --- | --- |
| 1.17.3 | 105,185,280 | 105,742,336 | 104,677,376 | 102,825,984 / 104,153,088 | 103,038,976 |
| 1.24.4 | 107,577,344 | 108,003,328 | 107,003,904 | 105,299,968 / 106,496,000 | 105,299,968 |

These byte counts include BEAM and allocator retention. The bounded samples do
not show sustained growth for this synthetic CPU workload; they do not prove the
absence of leaks in other graphs, providers, or runtime versions. Run this script
against any production-representative fixtures before changing a consumer.
Sampled BEAM process counts stayed at 124 throughout. The harness uses
`Enum.each`, discarding per-call results; collecting comprehensions in an earlier
diagnostic retained an accumulator and distorted heap/native-memory observations.
Temporary lifetime counters independently showed controls being reclaimed and
were removed before the final run. No production native code changed for this
harness correction.

Byte counts include BEAM and allocator retention. These samples show no sustained
growth for the exercised synthetic CPU workload, including managed worker/control
ownership; they do not prove leak freedom for other graphs, providers, or runtime
versions. Repeat with production-representative fixtures before consumer rollout.

The separate global-pool VM exercises **both** file and memory constructors:
inherited pools, rejected ignored settings, explicit independent pools, unchanged
scores, and resource survival through application restart. The precompiled check
runs these same gates with cargo/rustc replaced by failing stubs.
runs these same gates with cargo/rustc/protoc replaced by failing stubs.

Separate runtime-log VMs configure info and warning levels before the one-time
environment initialization. They assert per-session log IDs from both constructors,
Expand All @@ -122,15 +137,17 @@ application restarts. Legacy positional loading keeps its CPU defaults.
CI covers API 17/runtime 1.17.3, API 17/runtime 1.24.4, API 24/runtime 1.24.4,
both Alpine architectures, all seven NIF targets, and complete checksum/package
assembly. PR checks and workflow artifacts are the current evidence; a test count
alone does not prove those platform or constructor combinations passed.

## Remaining work
alone does not prove those platform or constructor combinations passed. The final
validation PR records exact integration run links without requiring a release tag.
Runtime cancellation is cooperative and may finish after the caller deadline.
Accelerator runtime execution remains unvalidated without its libraries/hardware.

The initial runtime/build foundation is not the whole capability backlog. Continue
with separate focused PRs for:
## Deferred scope

- Per-run profiling toggles require API 25 and remain deferred beyond the API
17/24 runtime matrix; session profiling is available now.
The stack covers the selected CPU/API 17–24 modernization, session/environment
controls, I/O values, metadata/graph inspection, observability, cancellation, and
Nx support decisions. Per-run profiling toggles require API 25 and remain outside
the tested runtime matrix; session profiling is available now.

GPU/NN primitives remain a later or measurement-dependent stage: shared weights
and pooling, AOT compilation, I/O binding, device allocation, and adapters. Keep
Expand Down
Loading