From 4da541345ecb4ec16d8b1733eb564699b325db3e Mon Sep 17 00:00:00 2001 From: said Date: Mon, 28 Sep 2026 12:41:02 +0100 Subject: [PATCH 1/5] codex: restructure the PRIMA example guide Lead with what the example does, the five solvers, a two-command quick start, and the key files; explain the build with a diagram before the full script; and show COBYLA, the progress callback stopping early, and the optional f/nf outputs. Every snippet was run against a fresh build. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 4 + docs/user/examples/fortran/prima-wrapper.md | 286 ++++++++++++-------- 2 files changed, 171 insertions(+), 119 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9751f5495..87cdb1671 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,10 @@ release tags add a leading `v` to the package version. ## Unreleased +- The PRIMA example guide now starts with a quick start and its key files, + explains the build before showing it, and adds examples for COBYLA, the + progress callback with early termination, and the optional `f` and `nf` + outputs. - The README and the documentation homepage now open with a short terminal demo that builds a Fortran module and calls it from Python, followed by links to Colab, installation, and the real-library examples. diff --git a/docs/user/examples/fortran/prima-wrapper.md b/docs/user/examples/fortran/prima-wrapper.md index 30af113a0..9f9561fa1 100644 --- a/docs/user/examples/fortran/prima-wrapper.md +++ b/docs/user/examples/fortran/prima-wrapper.md @@ -2,91 +2,84 @@ title: Build and Validate PRIMA with PRIK audience: users, advanced users prerequisites: arrays, callbacks, packaging -related: ../../guide/callbacks.md, ../../reference/cli-commands.md +related: ../../guide/callbacks.md, ../../guide/optional-arguments.md, ../../reference/cli-commands.md status: maintained publication: reviewed --- # Build and Validate PRIMA with PRIK -This example builds the checked-in [libPRIMA](https://github.com/libprima/prima) -Fortran sources once and wraps five derivative-free solvers as one Python -extension. Its numerical tests exercise Python callbacks and check known -solutions. +This example turns [PRIMA](https://github.com/libprima/prima), the modern +Fortran implementation of Powell's derivative-free optimization solvers, into +one Python extension. It compiles PRIMA once into a static library, asks PRIK +to generate bindings only for the five solvers, and links those bindings +against that library. Your objective function is an ordinary Python callable. -### What this example shows +### What you get -- Select five `module::procedure` entrypoints while retaining the callback - declarations their signatures need. -- Link a PRIK wrapper to a prebuilt static Fortran archive without compiling - the native sources twice. -- Call the solvers from Python, including optional callbacks and optional - arguments inside callback interfaces. +One extension, `prik_prima`, with exactly five solvers: -You should already be comfortable with NumPy arrays, Python callables, and -building a local Fortran extension. - ---- +| Module | Solver | Problem type | +| --- | --- | --- | +| `uobyqa_mod` | `uobyqa` | Unconstrained | +| `newuoa_mod` | `newuoa` | Unconstrained | +| `bobyqa_mod` | `bobyqa` | Bound constraints | +| `lincoa_mod` | `lincoa` | Linear constraints | +| `cobyla_mod` | `cobyla` | Nonlinear constraints | -## Versions used +Each solver takes your objective as a Python callback and updates a NumPy `x` +in place. Every optional PRIMA argument stays optional in Python, including a +progress callback that can stop the solver early. -| Component | Version / source | -| --- | --- | -| PRIK | current repository checkout | -| PRIMA | [libprima/prima commit `1d76fb88`](https://github.com/libprima/prima/tree/1d76fb88aeffb427cd17ed1e9d0d3b34f414913f) | -| Python | 3.12 in the dedicated CI job | -| NumPy | 2.5.1 | -| SciPy (optional comparison) | 1.18.0 in CI | -| Native compilers | GNU Fortran 13 + GCC 13 in CI; compatible local compilers work | +--- -The source snapshot lives under `examples/fortran/prima/native/`; the build -does not download PRIMA. +## Quick start -## Tested platforms +From a PRIK checkout with PRIK installed, and with CMake and GNU Fortran on +`PATH` (see [Set up a clean environment](#set-up-a-clean-environment)): -The Real Libraries Portability workflow builds and runs the numerical suite -with Python 3.12 on: +```bash +source examples/fortran/prima/build_all.sh +python3 -m pytest -q examples/fortran/prima/tests +``` -| Operating system | Architectures | Native toolchain | -| --- | --- | --- | -| Linux | x86-64, ARM64 | GNU Fortran 13 + GCC 13 | -| macOS | Intel, ARM64 | GNU Fortran 13 + GNU GCC 13 | +The first command builds the extension and puts it on `PYTHONPATH` for this +shell; use `source`, not `bash`, so that setting survives. The second runs the +example's tests. Then try the solvers from Python as shown in +[Use the generated API](#use-the-generated-api). --- -## 1. Prepare the repository and toolchain +## Key files -Clone PRIK, create a virtual environment, and install the Python tools used by -the dedicated CI job: +Everything lives under +[`examples/fortran/prima/`](../../../../examples/fortran/prima/): -```bash -git clone https://github.com/PyNumLab/prik.git -cd prik -python3 -m venv .venv -. .venv/bin/activate -python3 -m pip install --upgrade pip -python3 -m pip install -e ".[qa]" "numpy==2.5.1" -``` - -Install CMake and GNU Fortran separately. On Ubuntu: +| File | What it does | +| --- | --- | +| [`export_symbols.txt`](../../../../examples/fortran/prima/export_symbols.txt) | Names the five `module::procedure` solvers PRIK exposes. | +| [`sources.txt`](../../../../examples/fortran/prima/sources.txt) | Lists the 55 PRIMA sources, used by both CMake and PRIK. | +| [`CMakeLists.txt`](../../../../examples/fortran/prima/CMakeLists.txt) | Compiles those sources into the static library `libprimaf.a`. | +| [`build_prik.sh`](../../../../examples/fortran/prima/build_prik.sh) | Runs the three build steps below. | +| [`build_all.sh`](../../../../examples/fortran/prima/build_all.sh) | Runs `build_prik.sh` and adds the extension to `PYTHONPATH`. | +| [`tests/test_solvers.py`](../../../../examples/fortran/prima/tests/test_solvers.py) | Checks every solver and the callbacks. | +| [`native/`](../../../../examples/fortran/prima/native/) | The PRIMA source snapshot; the build downloads nothing. | -```bash -sudo apt-get update -sudo apt-get install --yes cmake gcc gfortran -gfortran --version -``` +--- -All remaining commands run from the repository root in this shell with the -virtual environment active. The runnable project lives under -[`examples/fortran/prima/`](../../../../examples/fortran/prima/). +## How the build works ---- +PRIMA is compiled once. PRIK reads the same sources to learn the solvers' +interfaces, but it generates bindings only for the five names in +`export_symbols.txt` and links them to the library CMake already built: -## 2. Build the PRIK wrapper +```text +55 PRIMA sources ─┬─ cmake ──────────────────────> libprimaf.a ─┐ + │ │ + └─ prik generate --pyi ──> contract/ ─ prik ──┴─> prik_prima +``` -The build script compiles PRIMA into `libprimaf.a`, selects five public -procedures for the generated `.pyi` contract, and links the wrapper against -that archive: +`build_prik.sh` runs those three steps: ```bash @@ -127,107 +120,162 @@ python3 -m prik "$PRIMA_BUILD_ROOT/contract/__init__.pyi" \ --jobs 2 ``` -CMake compiles the 55 native sources once. PRIK analyzes those same sources -with matching real-precision and integer-kind settings, then links its -generated wrapper to the archive. - -For normal use, source the convenience entrypoint: - -```bash -source examples/fortran/prima/build_all.sh -``` +| Step | Command | Result | +| --- | --- | --- | +| 1. Compile PRIMA | `cmake` | `libprimaf.a` and its Fortran module files | +| 2. Generate the contract | `prik generate --pyi` with `--export-symbols` | One `.pyi` file per solver module, plus the callback interfaces they use | +| 3. Build the extension | `prik` with `--native-link-item archive:…` | `prik_prima`, linked to `libprimaf.a` without recompiling PRIMA | -It builds the extension, exports its directory on `PYTHONPATH`, and records -the temporary build directory in `PRIMA_BUILD_ROOT` for this shell. +Steps 1 and 2 use the same `PRIMA_REAL_PRECISION=64` and +`PRIMA_INTEGER_KIND=0`, so the contract describes the library that was +actually compiled. Everything is written to the temporary `PRIMA_BUILD_ROOT` +directory, not to the repository. --- -## 3. Use the generated Python API - -The public Python API has exactly these entries: +## Use the generated API -| Module | Solver | -| --- | --- | -| `bobyqa_mod` | `bobyqa` | -| `cobyla_mod` | `cobyla` | -| `lincoa_mod` | `lincoa` | -| `newuoa_mod` | `newuoa` | -| `uobyqa_mod` | `uobyqa` | +Every solver follows the same pattern: the objective receives the current point +and writes its value into `result`; the solver updates `x` in place. Pass `x` +as a Fortran-ordered `float64` array and integer options as `np.int32`. -For example, UOBYQA minimizes a two-variable quadratic whose known minimum is -at `(1, -2)`. After building the extension, run this in Python: +UOBYQA minimizes a quadratic whose minimum is at `(1, -2)`: ```python import numpy as np import prik_prima -x = np.asfortranarray(np.array([3.0, 0.0], dtype=np.float64)) - def objective(values, result): result[...] = (values[0] - 1.0) ** 2 + (values[1] + 2.0) ** 2 +x = np.asfortranarray(np.array([3.0, 0.0], dtype=np.float64)) prik_prima.uobyqa_mod.uobyqa(objective, x, maxfun=np.int32(100)) -np.testing.assert_allclose(x, [1.0, -2.0], atol=2e-3, rtol=0) -print(x) +print(x) # [ 1. -2.] ``` -The callback writes the objective value into `result`; the solver updates `x` -in place. The assertion checks the result against the known minimum. +`newuoa`, `bobyqa`, and `lincoa` take the same objective. `cobyla` handles +nonlinear constraints, so its callback also receives a constraint array, and +its second argument is the number of constraints: ---- +```python +def objective_and_constraints(values, result, constraints): + objective(values, result) -## 4. Run the complete test suite +x = np.asfortranarray(np.array([3.0, 0.0], dtype=np.float64)) +prik_prima.cobyla_mod.cobyla(objective_and_constraints, np.int32(0), x, maxfun=np.int32(100)) +print(x.round(3)) # [ 1. -2.] +``` -After the build finishes, run: +**Progress callback.** Pass `callback_fcn` to follow each iteration. Setting +`terminate[...] = True` stops the solver early; here UOBYQA stops after 8 +evaluations instead of 22: -```bash -python3 -m pytest -q examples/fortran/prima/tests +```python +def progress(values, f, nf, tr, cstrv, nlconstr, terminate): + print(f"evaluation {nf}: f = {f:.3g}") + if f < 1e-6: + terminate[...] = True + +x = np.asfortranarray(np.array([3.0, 0.0], dtype=np.float64)) +prik_prima.uobyqa_mod.uobyqa(objective, x, maxfun=np.int32(100), callback_fcn=progress) +``` + +Arguments that a solver does not use arrive as `None`; for example, `cstrv` and +`nlconstr` are `None` in UOBYQA's callback. + +**Final value and evaluation count.** `f`, `nf`, and `info` are optional +Fortran outputs. Pass rank-zero arrays to receive them; omitting them keeps them +absent to PRIMA: + +```python +f = np.zeros((), dtype=np.float64) +nf = np.zeros((), dtype=np.int32) +x = np.asfortranarray(np.array([3.0, 0.0], dtype=np.float64)) +prik_prima.uobyqa_mod.uobyqa(objective, x, f=f, nf=nf, maxfun=np.int32(100)) +print(float(f), int(nf)) ``` -The suite checks a numerical result for each of the five exposed solvers, -exact API selection, and callback behavior when optional arguments are -present or omitted. It is not an exhaustive solver-option or constraint -suite. The [test file](../../../../examples/fortran/prima/tests/test_solvers.py) -shows each solver case and checks COBYLA's optional progress callback. +The full signatures, with every optional argument, are in the generated +contract under `$PRIMA_BUILD_ROOT/contract/`. --- -## 5. Run focused examples +## Run the tests + +```bash +python3 -m pytest -q examples/fortran/prima/tests +``` + +The suite checks each solver against the known minimum, that the extension +exposes exactly the five solvers, and the progress callback with optional +arguments present or omitted. It is not an exhaustive solver-option or +constraint suite. -After building the extension, run one solver test or the optional SciPy -comparison: +To compare COBYLA with SciPy's, which also comes from PRIMA: ```bash -python3 -m pytest -q examples/fortran/prima/tests/test_solvers.py::test_lincoa_minimizes_a_quadratic python3 -m pip install "scipy==1.18.0" python3 -m pytest -q examples/fortran/prima/tests/test_solvers.py::test_cobyla_agrees_with_scipy_on_a_quadratic ``` -SciPy's COBYLA also uses PRIMA, so this is a cross-interface comparison; the -known minimizer remains the independent numerical check. To test your own -problem, add a case beside the checked-in tests and run it with pytest. - -- Solver and callback examples → - [`test_solvers.py`](../../../../examples/fortran/prima/tests/test_solvers.py) -- Reviewed API selection → - [`export_symbols.txt`](../../../../examples/fortran/prima/export_symbols.txt) -- Copyable build script → - [`build_prik.sh`](../../../../examples/fortran/prima/build_prik.sh) -- Project instructions → - [`examples/fortran/prima/README.md`](../../../../examples/fortran/prima/README.md) +To test your own problem, add a case beside the checked-in tests. --- +## Set up a clean environment + +Clone PRIK, create a virtual environment, and install the Python tools used by +the dedicated CI job: + +```bash +git clone https://github.com/PyNumLab/prik.git +cd prik +python3 -m venv .venv +. .venv/bin/activate +python3 -m pip install --upgrade pip +python3 -m pip install -e ".[qa]" "numpy==2.5.1" +``` + +Install CMake and GNU Fortran separately. On Ubuntu: + +```bash +sudo apt-get update +sudo apt-get install --yes cmake gcc gfortran +gfortran --version +``` + +Run the example's commands from the repository root with the virtual +environment active. + +## Versions used + +| Component | Version / source | +| --- | --- | +| PRIK | current repository checkout | +| PRIMA | [libprima/prima commit `1d76fb88`](https://github.com/libprima/prima/tree/1d76fb88aeffb427cd17ed1e9d0d3b34f414913f) | +| Python | 3.12 in the dedicated CI job | +| NumPy | 2.5.1 | +| SciPy (optional comparison) | 1.18.0 in CI | +| Native compilers | GNU Fortran 13 + GCC 13 in CI; compatible local compilers work | + +## Tested platforms + +The Real Libraries Portability workflow builds and runs the numerical suite +with Python 3.12 on: + +| Operating system | Architectures | Native toolchain | +| --- | --- | --- | +| Linux | x86-64, ARM64 | GNU Fortran 13 + GCC 13 | +| macOS | Intel, ARM64 | GNU Fortran 13 + GNU GCC 13 | + ## Troubleshooting - Confirm that `cmake` and `gfortran` are available on `PATH`. -- Use `source examples/fortran/prima/build_all.sh`; executing it in a child - shell does not preserve the exported `PYTHONPATH`. +- Use `source examples/fortran/prima/build_all.sh`; running it with `bash` + starts a child shell, so the exported `PYTHONPATH` is lost. - SciPy is optional. The COBYLA comparison skips if it is not installed. - Run one failing solver test with `-vv -s` to see its output. ---- - ## Source provenance The files under [`examples/fortran/prima/native/`](../../../../examples/fortran/prima/native/) From 503ebd8a3f02136ded886eac4b6bbaa764046022 Mon Sep 17 00:00:00 2001 From: said Date: Mon, 28 Sep 2026 12:50:15 +0100 Subject: [PATCH 2/5] codex: polish the PRIMA guide after review Say that prik_prima imports in the same shell after the quick start, name when the static-archive pattern applies, point to the generated contracts after the build steps, and give the COBYLA example a real constraint (x[1] >= -1, minimum (1, -1)) using PRIMA's constraint <= 0 convention. Co-Authored-By: Claude Opus 5.5 --- docs/user/examples/fortran/prima-wrapper.md | 28 +++++++++++++-------- 1 file changed, 18 insertions(+), 10 deletions(-) diff --git a/docs/user/examples/fortran/prima-wrapper.md b/docs/user/examples/fortran/prima-wrapper.md index 9f9561fa1..edf8b6a6d 100644 --- a/docs/user/examples/fortran/prima-wrapper.md +++ b/docs/user/examples/fortran/prima-wrapper.md @@ -45,8 +45,10 @@ python3 -m pytest -q examples/fortran/prima/tests The first command builds the extension and puts it on `PYTHONPATH` for this shell; use `source`, not `bash`, so that setting survives. The second runs the -example's tests. Then try the solvers from Python as shown in -[Use the generated API](#use-the-generated-api). +example's tests. + +After this, you can `import prik_prima` in the same shell and call the solvers +as shown in [Use the generated API](#use-the-generated-api). --- @@ -71,7 +73,10 @@ Everything lives under PRIMA is compiled once. PRIK reads the same sources to learn the solvers' interfaces, but it generates bindings only for the five names in -`export_symbols.txt` and links them to the library CMake already built: +`export_symbols.txt` and links them to the library CMake already built. The +same pattern works whenever a large Fortran library is already compiled as a +static archive and you want to expose only a few of its entry points to +Python: ```text 55 PRIMA sources ─┬─ cmake ──────────────────────> libprimaf.a ─┐ @@ -131,6 +136,10 @@ Steps 1 and 2 use the same `PRIMA_REAL_PRECISION=64` and actually compiled. Everything is written to the temporary `PRIMA_BUILD_ROOT` directory, not to the repository. +The generated contracts live in `$PRIMA_BUILD_ROOT/contract/`, one `.pyi` +file per solver module. Open them to see each solver's exact Python signature, +including every optional argument. + --- ## Use the generated API @@ -154,16 +163,18 @@ print(x) # [ 1. -2.] ``` `newuoa`, `bobyqa`, and `lincoa` take the same objective. `cobyla` handles -nonlinear constraints, so its callback also receives a constraint array, and -its second argument is the number of constraints: +nonlinear constraints: its callback also fills a `constraints` array, each +entry meaning `constraint <= 0`, and its second argument `m_nlcon` is the +number of constraints. Requiring `x[1] >= -1` moves the minimum to `(1, -1)`: ```python def objective_and_constraints(values, result, constraints): objective(values, result) + constraints[0] = -1.0 - values[1] # x[1] >= -1, written as -1 - x[1] <= 0 x = np.asfortranarray(np.array([3.0, 0.0], dtype=np.float64)) -prik_prima.cobyla_mod.cobyla(objective_and_constraints, np.int32(0), x, maxfun=np.int32(100)) -print(x.round(3)) # [ 1. -2.] +prik_prima.cobyla_mod.cobyla(objective_and_constraints, np.int32(1), x, maxfun=np.int32(200)) +print(x.round(3)) # [ 1. -1.] ``` **Progress callback.** Pass `callback_fcn` to follow each iteration. Setting @@ -195,9 +206,6 @@ prik_prima.uobyqa_mod.uobyqa(objective, x, f=f, nf=nf, maxfun=np.int32(100)) print(float(f), int(nf)) ``` -The full signatures, with every optional argument, are in the generated -contract under `$PRIMA_BUILD_ROOT/contract/`. - --- ## Run the tests From 487d3b1da14b65c430e0acd32310573ae84affec Mon Sep 17 00:00:00 2001 From: said Date: Mon, 28 Sep 2026 12:57:54 +0100 Subject: [PATCH 3/5] codex: tighten the PRIMA guide's static-archive sentence Co-Authored-By: Claude Opus 5.5 --- docs/user/examples/fortran/prima-wrapper.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/docs/user/examples/fortran/prima-wrapper.md b/docs/user/examples/fortran/prima-wrapper.md index edf8b6a6d..5526dbda8 100644 --- a/docs/user/examples/fortran/prima-wrapper.md +++ b/docs/user/examples/fortran/prima-wrapper.md @@ -73,10 +73,9 @@ Everything lives under PRIMA is compiled once. PRIK reads the same sources to learn the solvers' interfaces, but it generates bindings only for the five names in -`export_symbols.txt` and links them to the library CMake already built. The -same pattern works whenever a large Fortran library is already compiled as a -static archive and you want to expose only a few of its entry points to -Python: +`export_symbols.txt` and links them to the library CMake already built. Use +this pattern for any Fortran library built as a static archive when Python +needs only a few of its entry points: ```text 55 PRIMA sources ─┬─ cmake ──────────────────────> libprimaf.a ─┐ From 716e842507ce7188141cb7f93488887cc3b51080 Mon Sep 17 00:00:00 2001 From: said Date: Mon, 28 Sep 2026 13:17:19 +0100 Subject: [PATCH 4/5] codex: restructure the Open MPI mpi_f08 tutorial Add a three-step "How this works" overview after the opening example, say early that prik_mpi.py is deliberately minimal (MPI_INT only, no receive status), state that PRIK needs Open MPI's source and build trees and move building Open MPI to the end, and show the key allreduce edit and a Comm excerpt inline with the full contract and module in collapsible sections. Every displayed file is still checked against its fixture. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 5 + docs/user/tutorials/openmpi-f08.md | 166 +++++++++++++++++++++-------- 2 files changed, 125 insertions(+), 46 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 87cdb1671..15dbc6922 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,11 @@ release tags add a leading `v` to the package version. ## Unreleased +- The Open MPI `mpi_f08` tutorial now explains its three steps up front, says + early that its mpi4py-style layer is deliberately minimal, states that it needs + Open MPI's source and build trees, and moves building Open MPI to the end. The + full edited contract and Python module are collapsible, with the key edits + shown inline. - The PRIMA example guide now starts with a quick start and its key files, explains the build before showing it, and adds examples for COBYLA, the progress callback with early termination, and the optional `f` and `nf` diff --git a/docs/user/tutorials/openmpi-f08.md b/docs/user/tutorials/openmpi-f08.md index 4b61c6a00..ebf81ed84 100644 --- a/docs/user/tutorials/openmpi-f08.md +++ b/docs/user/tutorials/openmpi-f08.md @@ -59,15 +59,33 @@ if rank == 0: You build two APIs: the **wrapped API** (`prik_openmpi_f08`), generated by PRIK, and the **Python API** (`prik_mpi.py`), a few lines of Python on top of -it. In the measured setup, both had lower median times than mpi4py for small +it. The Python API is deliberately minimal: it sends only `MPI_INT` buffers and +reports no receive status, to show the pattern rather than all of mpi4py. In +the measured setup, both had lower median times than mpi4py for small `Allreduce` calls; see [Compare call times](#compare-call-times) for the other operations and the test environment. +## How this works + +1. **Select** the 18 `mpi_f08` names the program needs, out of hundreds + ([step 1](#1-choose-what-to-wrap)). +2. **Generate, then edit, a `.pyi` contract.** PRIK writes the Fortran + signatures as they are; the edits make them Pythonic: `ierror` becomes an + exception, `count` comes from the NumPy buffer, and output arguments become + return values ([steps 2-4](#2-generate-the-contract)). +3. **Add a thin Python layer**, `prik_mpi.py`, that gives the familiar + `MPI.COMM_WORLD` and `comm.Send(...)` style ([step 5](#5-add-the-python-api)). + ## What you need - Linux or macOS, Python 3.10 or later, NumPy, and PRIK; see [Installation](../getting-started/installation.md). - GNU Fortran and GCC of the same version; CI uses version 13. +- **Open MPI's source and build trees**, configured with the `mpi_f08` + bindings. PRIK reads the `mpi_f08` Fortran sources, which an installed Open + MPI does not ship. If you do not have them, follow + [Build Open MPI from source](#build-open-mpi-from-source) first; it also + builds a matching mpi4py for the comparison. - The tutorial's files, from the PRIK repository: [`mpi_exports.txt`](https://github.com/PyNumLab/prik/blob/main/tests/fortran/assumed_types/end_to_end/fixtures/contracts/openmpi/mpi_exports.txt), [`mpi_f08.pyi`](https://github.com/PyNumLab/prik/blob/main/tests/fortran/assumed_types/end_to_end/fixtures/contracts/openmpi/mpi_f08.pyi), @@ -88,44 +106,14 @@ for file in \ done ``` -## Install Open MPI and mpi4py - -Build [Open MPI 5.0.11](https://www.open-mpi.org/software/ompi/v5.0/) from -source and keep its source and build trees, which PRIK reads. From your -working directory: +Point two variables at Open MPI's trees, and put that Open MPI's `mpifort` and +`mpirun` on `PATH`: ```bash -OMPI_ROOT="$HOME/openmpi-5.0.11" -TUTORIAL_DIR="$PWD" -mkdir -p "$OMPI_ROOT/source" "$OMPI_ROOT/build" "$OMPI_ROOT/toolchain" -ln -sf "$(command -v gfortran-13)" "$OMPI_ROOT/toolchain/gfortran" -ln -sf "$(command -v gcc-13)" "$OMPI_ROOT/toolchain/gcc" -export PATH="$OMPI_ROOT/toolchain:$PATH" -curl -fsSLO https://download.open-mpi.org/release/open-mpi/v5.0/openmpi-5.0.11.tar.bz2 -tar -xjf openmpi-5.0.11.tar.bz2 -C "$OMPI_ROOT/source" --strip-components=1 -cd "$OMPI_ROOT/build" -../source/configure --prefix="$OMPI_ROOT/install" --enable-mpi-fortran=usempif08 CC=gcc FC=gfortran -make -j2 && make install -cd "$TUTORIAL_DIR" -export PATH="$OMPI_ROOT/install/bin:$PATH" -export LD_LIBRARY_PATH="$OMPI_ROOT/install/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" -export DYLD_LIBRARY_PATH="$OMPI_ROOT/install/lib${DYLD_LIBRARY_PATH:+:$DYLD_LIBRARY_PATH}" -OMPI_SRC="$OMPI_ROOT/source" -OMPI_BUILD="$OMPI_ROOT/build" +OMPI_SRC="$HOME/openmpi-5.0.11/source" +OMPI_BUILD="$HOME/openmpi-5.0.11/build" ``` -Build mpi4py against the same Open MPI; both checks should report 5.0.11: - -```bash -MPI4PY_BUILD_MPICC="$OMPI_ROOT/install/bin/mpicc" \ - python3 -m pip install --no-cache-dir --no-binary=mpi4py mpi4py==4.1.2 -mpirun --version -python3 -c 'from mpi4py import MPI; print(MPI.Get_library_version())' -``` - -For Open MPI 4.1.8, which CI also tests, change `5.0.11` to `4.1.8` and `v5.0` -to `v4.1`. - ## 1. Choose what to wrap `mpi_exports.txt` selects the names the program needs from `mpi_f08`'s @@ -199,6 +187,35 @@ Replace the published module with the edited one you downloaded: cp mpi_f08.pyi contract/mpi_f08.pyi ``` +The downloaded `mpi_f08.pyi` is the generated contract after these edits. +Compare `allreduce` with the generated version in step 2: + +```python +@raises(status="ierror", success=0) +@bind("MPI_Allreduce") +@native_call([Arg(0), Arg(1), Int32(Arg(1).size), Arg(2), Arg(3), Arg(4), Hidden("ierror", Int32)]) +def allreduce( + sendbuf: Annotated[AnyNative[Flat], ReadOnly], + recvbuf: AnyNative[Flat], + datatype: Mpi_Datatype, + op: Mpi_Op, + comm: Mpi_Comm, +) -> None: ... +``` + +Each function still calls the Fortran routine it names; the edits change only +its Python signature: + +| Edit | Effect in Python | +| --- | --- | +| `@bind("MPI_Send")` on `def send` | The Python name differs from the Fortran name. | +| `Int32(Arg(0).size)` | The count comes from the buffer, so the caller does not pass it. | +| `Return("rank", 0)` | The output argument becomes the return value. | +| `Hidden("ierror", Int32)` with `@raises(status="ierror", success=0)` | A nonzero error code raises an exception. | + +
+The complete edited mpi_f08.pyi + ```python from prik.contracts import Annotated, AnyNative, Arg, Flat, Hidden, Int32, ReadOnly, Return, bind, native_call, raises @@ -319,15 +336,7 @@ __all__ = [ ] ``` -Each function still calls the Fortran routine it names; the edits change only -its Python signature: - -| Edit | Effect in Python | -| --- | --- | -| `@bind("MPI_Send")` on `def send` | The Python name differs from the Fortran name. | -| `Int32(Arg(0).size)` | The count comes from the buffer, so the caller does not pass it. | -| `Return("rank", 0)` | The output argument becomes the return value. | -| `Hidden("ierror", Int32)` with `@raises(status="ierror", success=0)` | A nonzero error code raises an exception. | +
## 4. Build the wrapped API @@ -346,8 +355,33 @@ generated code is compiled; the extension links to the installed Open MPI. ## 5. Add the Python API -`prik_mpi.py` gives the wrapped API mpi4py's `Comm` object. It is kept small: -one datatype, `MPI_INT`, and no receive status. +`prik_mpi.py` imports the wrapped API as `_mpi` and gives it mpi4py's `Comm` +object. Each method forwards to one wrapped function, adding the datatype and +the communicator handle: + +```python +class Comm: + """A communicator, with the methods mpi4py spells for it.""" + + def __init__(self, handle): + self.handle = handle + + def Get_rank(self): + return _mpi.comm_rank(self.handle) + + def Allreduce(self, sendbuf, recvbuf, op=SUM): + _mpi.allreduce(sendbuf, recvbuf, _INT, op, self.handle) + + +COMM_WORLD = Comm(_mpi.mpi_comm_world) + +# Like mpi4py, MPI starts when this module is imported and stops at exit. +_mpi.init() +atexit.register(_mpi.finalize) +``` + +
+The complete prik_mpi.py ```python @@ -411,6 +445,8 @@ _mpi.init() atexit.register(_mpi.finalize) ``` +
+ ## 6. Run it ```bash @@ -455,3 +491,41 @@ median of five runs was: `Get_rank` does almost nothing, so its time is the overhead of a call, which PRIK can still reduce. + +## Build Open MPI from source + +Do this once if you do not already have Open MPI's source and build trees. +It builds [Open MPI 5.0.11](https://www.open-mpi.org/software/ompi/v5.0/) and +keeps both trees, which PRIK reads. From your working directory: + +```bash +OMPI_ROOT="$HOME/openmpi-5.0.11" +TUTORIAL_DIR="$PWD" +mkdir -p "$OMPI_ROOT/source" "$OMPI_ROOT/build" "$OMPI_ROOT/toolchain" +ln -sf "$(command -v gfortran-13)" "$OMPI_ROOT/toolchain/gfortran" +ln -sf "$(command -v gcc-13)" "$OMPI_ROOT/toolchain/gcc" +export PATH="$OMPI_ROOT/toolchain:$PATH" +curl -fsSLO https://download.open-mpi.org/release/open-mpi/v5.0/openmpi-5.0.11.tar.bz2 +tar -xjf openmpi-5.0.11.tar.bz2 -C "$OMPI_ROOT/source" --strip-components=1 +cd "$OMPI_ROOT/build" +../source/configure --prefix="$OMPI_ROOT/install" --enable-mpi-fortran=usempif08 CC=gcc FC=gfortran +make -j2 && make install +cd "$TUTORIAL_DIR" +export PATH="$OMPI_ROOT/install/bin:$PATH" +export LD_LIBRARY_PATH="$OMPI_ROOT/install/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}" +export DYLD_LIBRARY_PATH="$OMPI_ROOT/install/lib${DYLD_LIBRARY_PATH:+:$DYLD_LIBRARY_PATH}" +OMPI_SRC="$OMPI_ROOT/source" +OMPI_BUILD="$OMPI_ROOT/build" +``` + +Build mpi4py against the same Open MPI; both checks should report 5.0.11: + +```bash +MPI4PY_BUILD_MPICC="$OMPI_ROOT/install/bin/mpicc" \ + python3 -m pip install --no-cache-dir --no-binary=mpi4py mpi4py==4.1.2 +mpirun --version +python3 -c 'from mpi4py import MPI; print(MPI.Get_library_version())' +``` + +For Open MPI 4.1.8, which CI also tests, change `5.0.11` to `4.1.8` and `v5.0` +to `v4.1`. From 7ed3bc3cc405cea1a04dc201f9aa56fdfbf9d529 Mon Sep 17 00:00:00 2001 From: said Date: Mon, 28 Sep 2026 13:39:00 +0100 Subject: [PATCH 5/5] update CHANGELOG.md --- CHANGELOG.md | 15 +++------------ 1 file changed, 3 insertions(+), 12 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 15dbc6922..752ea5246 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,18 +7,9 @@ release tags add a leading `v` to the package version. ## Unreleased -- The Open MPI `mpi_f08` tutorial now explains its three steps up front, says - early that its mpi4py-style layer is deliberately minimal, states that it needs - Open MPI's source and build trees, and moves building Open MPI to the end. The - full edited contract and Python module are collapsible, with the key edits - shown inline. -- The PRIMA example guide now starts with a quick start and its key files, - explains the build before showing it, and adds examples for COBYLA, the - progress callback with early termination, and the optional `f` and `nf` - outputs. -- The README and the documentation homepage now open with a short terminal - demo that builds a Fortran module and calls it from Python, followed by links - to Colab, installation, and the real-library examples. +- Improve the Open MPI `mpi_f08` tutorial +- Improve the PRIMA example guide +- The README and the documentation homepage now open with a short terminal demo. - The repository root holds fewer files. Contributor notes moved to `.github/CONTRIBUTING.md`, the pre-push hook to `tools/githooks/` (activate it with `git config core.hooksPath tools/githooks`), and the documentation theme