Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 52 additions & 2 deletions .github/workflows/checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -170,9 +170,59 @@ jobs:
path: dist/*-${{ matrix.target }}.tar.gz
retention-days: 14
if-no-files-found: error
bundle-check:
needs: [policy, hygiene, source, msrv, alpine]
if: needs.policy.outputs.scope == 'full'
runs-on: ubuntu-24.04
timeout-minutes: 25
env:
MIX_ENV: docs
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0
persist-credentials: false
- uses: erlef/setup-beam@54075bcc5e249e4758d363f27d099f55d843f124 # v1.24.1
with:
elixir-version: '1.20.4'
otp-version: '29.1.1'
- uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
with:
python-version: '3.14.10'
- uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
pattern: nif-*
merge-multiple: true
path: dist
- name: Exercise complete release assembly without publication
run: |
python scripts/release.py validate
make deps docs inventory
version=$(python scripts/release.py validate)
tar -czf "dist/docs-$version.tar.gz" -C doc .
python scripts/package.py assemble --all
python scripts/release.py verify
previous=$(git describe --tags --abbrev=0 --match 'v[0-9]*' HEAD^ 2>/dev/null || true)
if test -n "$previous"; then
python scripts/release.py notes --from "$previous" --output dist/RELEASE_NOTES.md
else
python scripts/release.py notes --output dist/RELEASE_NOTES.md
fi
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: release-bundle
path: |
dist/ortex-*
dist/docs-*
dist/source.cdx.json
dist/artifacts.json
dist/SHA256SUMS
dist/RELEASE_NOTES.md
retention-days: 14
if-no-files-found: error
validated:
if: always()
needs: [policy, hygiene, source, msrv, alpine]
needs: [policy, hygiene, source, msrv, alpine, bundle-check]
runs-on: ubuntu-24.04
timeout-minutes: 5
steps:
Expand All @@ -185,7 +235,7 @@ jobs:
jobs = json.loads(os.environ['RESULTS'])
scope = jobs['policy']['outputs'].get('scope')
for name, job in jobs.items():
allowed = {'success', 'skipped'} if name == 'alpine' and scope == 'pr' else {'success'}
allowed = {'success', 'skipped'} if name in {'alpine', 'bundle-check'} and scope == 'pr' else {'success'}
if job['result'] not in allowed:
raise SystemExit(f"{name}: {job['result']}")
PY
Expand Down
51 changes: 51 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
name: Publish approved Hex release
on:
release:
types: [published]
permissions:
contents: read
concurrency:
group: publish-${{ github.event.release.tag_name }}
cancel-in-progress: false
jobs:
publish:
if: github.repository == 'RiverFinancial/ortex' && vars.ORTEX_HEX_PUBLISH == 'true'
runs-on: ubuntu-24.04
timeout-minutes: 20
environment: release
permissions:
contents: read
attestations: read # Verify the previously attested artifacts before publishing.
id-token: write # Exchange the protected job identity for a package-scoped Hex token.
env:
MIX_ENV: docs
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ github.event.release.tag_name }}
fetch-depth: 0
persist-credentials: false
- uses: erlef/setup-beam@54075bcc5e249e4758d363f27d099f55d843f124 # v1.24.1
with:
elixir-version: '1.20.4'
otp-version: '29.1.1'
- uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
with:
python-version: '3.14.10'
- name: Verify the approved release bundle
env:
GH_TOKEN: ${{ github.token }}
RELEASE_TAG: ${{ github.event.release.tag_name }}
run: |
python scripts/release.py validate --tag "$RELEASE_TAG"
gh release download "$RELEASE_TAG" --repo RiverFinancial/ortex --dir dist
for artifact in dist/*; do
gh attestation verify "$artifact" --repo RiverFinancial/ortex
done
python scripts/release.py verify
make deps
- name: Publish package and documentation with Hex workload identity
env:
RELEASE_TAG: ${{ github.event.release.tag_name }}
ORTEX_HEX_PUBLISH: ${{ vars.ORTEX_HEX_PUBLISH }}
run: python scripts/release.py publish --tag "$RELEASE_TAG"
64 changes: 64 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
name: Stage release
on:
push:
tags: ['v*']
permissions:
contents: read
concurrency:
group: release-${{ github.ref }}
cancel-in-progress: false
jobs:
validate:
# actionlint 1.7.12 does not yet parse GitHub's $/ reusable-workflow syntax.
uses: ./.github/workflows/checks.yml # zizmor: ignore[self-repository]
with:
scope: full
audit:
uses: ./.github/workflows/audit.yml # zizmor: ignore[self-repository]
verify-release:
needs: [validate, audit]
runs-on: ubuntu-24.04
timeout-minutes: 10
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0
persist-credentials: false
- uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
with:
python-version: '3.14.10'
- uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
name: release-bundle
path: dist
- name: Verify the tag and complete tested bundle
env:
RELEASE_TAG: ${{ github.ref_name }}
run: |
python scripts/release.py validate --tag "$RELEASE_TAG"
python scripts/release.py verify
stage:
if: github.repository == 'RiverFinancial/ortex' && vars.ORTEX_STAGE_RELEASES == 'true'
needs: verify-release
runs-on: ubuntu-24.04
timeout-minutes: 10
environment: release
permissions:
contents: write # Create only the fork's draft release and upload its bundle.
id-token: write # Authenticate build provenance attestation.
attestations: write # Record provenance for the completed release bundle.
steps:
- uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
name: release-bundle
path: dist
- uses: actions/attest-build-provenance@4d101475d8b20a2381f78447822ac1eab6504dd8 # v4.2.2
with:
subject-path: dist/*
- name: Stage the draft release
env:
GH_TOKEN: ${{ github.token }}
RELEASE_TAG: ${{ github.ref_name }}
run: |
gh release create "$RELEASE_TAG" --repo RiverFinancial/ortex --draft --verify-tag --notes-file dist/RELEASE_NOTES.md
gh release upload "$RELEASE_TAG" dist/* --repo RiverFinancial/ortex
7 changes: 7 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ export ORT_DYLIB_PATH ?= $(shell $(PYTHON) scripts/install_runtime.py path 2>/de
.PHONY: test.system test.package test.models test.tooling coverage bench fixtures package
.PHONY: test.alpine
.PHONY: setup.audit audit setup.rust
.PHONY: inventory release.notes

help: ## Show development commands
@awk 'BEGIN {FS = ":.*## "} /^[a-zA-Z0-9_.-]+:.*## / {printf " %-22s %s\n", $$1, $$2}' $(MAKEFILE_LIST)
Expand Down Expand Up @@ -88,3 +89,9 @@ setup.rust: ## Install the pinned Rust compiler, formatter, and Clippy

audit: ## Report Rust advisories and retired Hex dependencies
$(PYTHON) scripts/audit.py

inventory: ## Write the source dependency inventory (consumer-supplied ORT excluded)
MIX_ENV=prod $(PYTHON) scripts/inventory.py

release.notes: ## Render conventional-commit notes and assembled artifact hashes
$(PYTHON) scripts/release.py notes $(ARGS)
96 changes: 49 additions & 47 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,64 +1,66 @@
# Ortex

`Ortex` is a wrapper around [ONNX Runtime](https://onnxruntime.ai/) (implemented as
bindings to [`ort`](https://github.com/pykeio/ort)). Ortex leverages
[`Nx.Serving`](https://hexdocs.pm/nx/Nx.Serving.html) to easily deploy ONNX models
that run concurrently and distributed in a cluster. Ortex also provides a storage-only
tensor implementation for ease of use.
Ortex wraps [ONNX Runtime](https://onnxruntime.ai/) for Elixir through Rust and
supports numeric tensor transfer and inference with Nx.Serving.

ONNX models are a standard machine learning model format that can be exported from most ML
libraries like PyTorch and TensorFlow. Ortex allows for easy loading and fast inference of
ONNX models using different backends available to ONNX Runtime such as CUDA, TensorRT, Core
ML, and ARM Compute Library.
This is the RiverFinancial fork. Pin a reviewed commit from
https://github.com/RiverFinancial/ortex when consuming unreleased work; the
foundation stack does not publish a new Hex release.

## Examples
## Development

TL;DR:
Review `mise.toml`, run `mise trust` and `mise install`, then:

```elixir
iex> model = Ortex.load("./models/resnet50.onnx")
#Ortex.Model<
inputs: [{"input", "Float32", [nil, 3, 224, 224]}]
outputs: [{"output", "Float32", [nil, 1000]}]>
iex> {output} = Ortex.run(model, Nx.broadcast(0.0, {1, 3, 224, 224}))
iex> output |> Nx.backend_transfer() |> Nx.argmax
#Nx.Tensor<
s64
499
>
```sh
mise exec -- make setup
mise exec -- make doctor
mise exec -- make help
```

Inspecting a model shows the expected inputs, outputs, data types, and shapes. Axes with
`nil` represent a dynamic size.

To see more real world examples see the `examples` folder.
Supported BEAM pairs are Elixir 1.19 / OTP 28 and Elixir 1.20 / OTP 29. Rust 1.92
is the minimum. The source NIF currently retains ort rc.8 / Rustler 0.29; their
migration is separate feature work. Nx versions from 0.6.2 up to 1.0 are allowed
(excluding 1.0), with 0.13.1 locked for normal development.

### Serving

`Ortex` also implements `Nx.Serving` behaviour. To use it in your application's
supervision tree consult the `Nx.Serving` docs.
CPU dynamic loading is the development default. ONNX Runtime is installed
separately by `make setup`. Make reads its recorded path; set ORT_DYLIB_PATH to
use another library. When consuming this source package, configure the mode
explicitly in your application's config:

```elixir
iex> serving = Nx.Serving.new(Ortex.Serving, model)
iex> batch = Nx.Batch.stack([{Nx.broadcast(0.0, {3, 224, 224})}])
iex> {result} = Nx.Serving.run(serving, batch)
iex> result |> Nx.backend_transfer() |> Nx.argmax(axis: 1)
#Nx.Tensor<
s64[1]
[499]
>
import Config
config :ortex, Ortex.Native, features: ["ort/load-dynamic"]
```

## Installation
The consumer supplies ORT_DYLIB_PATH. Execution providers require a matching
runtime and explicit Cargo features; they are not inferred from the host OS.

`Ortex` can be installed by adding `ortex` to your list of dependencies in `mix.exs`:
## Checks and examples

```elixir
def deps do
[
{:ortex, "~> 0.1.10"}
]
end
```sh
make check
make test.system
make test.package
make test.models
make coverage
make docs
make test.alpine
make bench ARGS=5000
```

You will need [Rust](https://www.rust-lang.org/tools/install) for compilation to succeed.
Library tests and doctests use normal ExUnit discovery under test/. System,
package, and model suites are documented in [testing](docs/testing.md). XGBoost,
Iris, and MNIST have prepared reference inputs and outputs; for example, with
ORT_DYLIB_PATH set:

```sh
MIX_ENV=test mix run examples/run.exs xgboost
```

Numeric outputs use Ortex.Backend for storage. Transfer to a computation backend
before arithmetic. See [examples](examples/README.md) and
[Nx.Serving](https://hexdocs.pm/nx/Nx.Serving.html) for usage.

[Development](docs/development.md), [CI policy](docs/ci.md), and
[release procedures](docs/releasing.md) describe the supported environment,
validation matrices, artifacts, and publishing gates.
19 changes: 17 additions & 2 deletions ci/containers/alpine.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM hexpm/elixir:1.20.4-erlang-29.1.1-alpine-3.24.2@sha256:ad851f40ce103dcb4ad56f23877d99473ef5c013e09b9e57921ffe877ac6d6a9
FROM hexpm/elixir:1.20.4-erlang-29.1.1-alpine-3.24.2@sha256:ad851f40ce103dcb4ad56f23877d99473ef5c013e09b9e57921ffe877ac6d6a9 AS build
RUN apk add --no-cache build-base git bash python3 rust=1.96.1-r0 cargo=1.96.1-r0 ca-certificates && \
apk add --no-cache \
--repository=https://dl-cdn.alpinelinux.org/alpine/edge/main \
Expand All @@ -11,4 +11,19 @@ COPY mix.exs mix.lock ./
RUN mix local.hex --force && mix local.rebar --force && mix deps.get --check-locked
COPY . .
RUN mix compile --warnings-as-errors
CMD ["sh", "-ec", "mix test && python3 scripts/fixtures.py && mix test test/models --include model && python3 scripts/scenario.py && python3 scripts/package.py test --mode source && python3 scripts/package.py test --mode prebuilt && if test -n \"${TARGET:-}\"; then python3 scripts/package.py nif --target \"$TARGET\"; fi"]
RUN mix test && python3 scripts/fixtures.py && mix test test/models --include model && \
python3 scripts/scenario.py && python3 scripts/package.py test --mode source

FROM hexpm/elixir:1.20.4-erlang-29.1.1-alpine-3.24.2@sha256:ad851f40ce103dcb4ad56f23877d99473ef5c013e09b9e57921ffe877ac6d6a9
RUN apk add --no-cache git bash python3 ca-certificates && \
apk add --no-cache \
--repository=https://dl-cdn.alpinelinux.org/alpine/edge/main \
--repository=https://dl-cdn.alpinelinux.org/alpine/edge/community \
--repository=https://dl-cdn.alpinelinux.org/alpine/edge/testing \
onnxruntime=1.24.4-r1
WORKDIR /app
ENV MIX_ENV=test ORT_DYLIB_PATH=/usr/lib/libonnxruntime.so.1.24.4
COPY --from=build /app /app
COPY --from=build /root/.mix /root/.mix
RUN ! command -v cargo && ! command -v rustc
CMD ["sh", "-ec", "python3 scripts/package.py test --mode prebuilt && if test -n \"${TARGET:-}\"; then python3 scripts/package.py nif --target \"$TARGET\"; fi"]
7 changes: 6 additions & 1 deletion docs/ci.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# CI policy

Checks runs all hygiene checks and the library, model, system, and package suites
on three PR combinations. Main and the nightly use seven native combinations
on three PR combinations. Main and the nightly use eight native combinations
(pairwise BEAM/Nx/runtime boundaries and five host platforms), plus both Alpine
architectures. This is an explicit selection, not a Cartesian product.
The `ci:full` PR label opts into the complete matrix before merge.
Expand Down Expand Up @@ -46,3 +46,8 @@ Alpine edge repositories, whose transitive package selection is mutable. These
images establish compatibility, not bit-for-bit reproducibility. glibc libraries
are tested separately from musl. Published native artifacts include only the NIF;
the consumer supplies ONNX Runtime.

Full sweeps also assemble and verify the complete seven-target bundle, docs,
source inventory, and release notes. The staging workflow consumes that tested
bundle. Alpine prebuilt installation runs in a separate final image with neither
Cargo nor rustc installed.
3 changes: 2 additions & 1 deletion docs/development.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# Development

Use `mise install`, then `mise exec -- make setup` and `mise exec -- make doctor`.
Review mise.toml and run `mise trust`, then `mise install`,
`mise exec -- make setup`, and `mise exec -- make doctor`.
Rustup reads the pinned rust-toolchain.toml. GNU Make 3.81 or newer is sufficient;
Windows developers can use Git Bash with Make available on PATH.

Expand Down
Loading
Loading