Repository navigation
ci: use CPU torch on GitHub runners to stop random CUDA-preload segfault - #807
Merged
Merged
Conversation
The cu130 torch build pinned in poetry.lock sometimes segfaults in torch._preload_cuda_lib on GPU-less runners (seen on master 7cac0c3 and Dependabot PRs #804/#805). After poetry install, reinstall the same torch/torchvision versions from the CPU wheel index and assert the build has no CUDA. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hananel-Hazan
added a commit
that referenced
this pull request
Oct 2, 2026
Conflicts came from #807. pythonpackage.yml stays deleted (this branch replaces it); both jobs in python-app.yml now run .github/scripts/use_cpu_torch.sh after poetry install. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hananel-Hazan
added a commit
that referenced
this pull request
Oct 19, 2026
The image's CUDA torch build segfaulted on the GPU-less runner while preloading CUDA (run 39191323281), the same cause as #807. The test container swaps in the CPU build of the same version first; the image is otherwise as built. Checked locally: 7 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hananel-Hazan
added a commit
that referenced
this pull request
Oct 23, 2026
… 3.13, README benchmark in every simulator (#811) * repro: seed GPU runs, fix broken MNIST/Breakout examples, Docker on Python 3.13 - Examples seeded only the GPU generator on GPU runs, so --seed did not make runs repeatable; torch.manual_seed is now unconditional (checked by a test). - supervised_mnist crash, reservoir scored on training images, conv_mnist had no test phase, --plot could not be turned off, loops ran one extra sample. - Breakout: map removed BreakoutDeterministic-v4 id; ANN file re-saved as a state_dict and loaded with weights_only=True. - Dockerfile back on python:3.13-slim; Dependabot no longer bumps its minor version; CI builds the image and runs test/repro inside it. - Add BindsNET-only scaling benchmark script. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * benchmark: run the README method in every simulator; document the old mismatch benchmark.py could not run (missing experiments package) and gave each simulator different input, none of them the README's U(0,100) Hz Poisson, all-to-all N(0,1) weights. Rewritten so BindsNET, BRIAN2, brian2genn, PyNEST and ANNarchy all get that network, one process per simulator (NEST 3.10 and ANNarchy 5.0 corrupt memory when loaded together). Run here for BindsNET, BRIAN2 2.9.0, NEST 3.10.0 and ANNarchy 5.0 CPU at n=250 and 500; brian2genn and ANNarchy GPU not run. DATA.md and README corrected. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: measured REPRODUCING.md results; benchmark result files Every Measured output cell was produced by its command at commit f392a65 (Threadripper 2990WX, RTX 2070): eth_mnist 0.75/0.76 (CPU), batch_eth_mnist 0.81/0.83 (GPU), supervised_mnist 0.64/0.64 (CPU), conv_mnist 0.12/0.14 (CPU, accuracy falls with training; adaptive threshold keeps rising), reservoir 53.00 % (CPU), Breakout mean reward 5.4 over 10 episodes, identical in two runs. Scaling benchmark times for BindsNET, BRIAN2, NEST and ANNarchy, with thread settings stated and not tuned. ANNarchy back on its default of one thread (all 64 threads made it slower here). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: publish.yml reads the version from [project] [tool.poetry] has no version since 0.3.4, so a release-triggered publish would fail its tag check with KeyError. Test added. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: state that results reproduce with the release they were run on Research package: results can change between releases and hardware, so a publication should name its release (version DOI). The 2018 benchmark script (6a1168d) did not implement the paper's described setup, so it cannot regenerate the figure; the long per-simulator table moves to CHANGELOG only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: limit pytest to test/ Ignored folders (logs/) can hold git worktrees, whose duplicate test modules broke collection with 17 import-mismatch errors. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: reword reproducibility docs and changelog to lead with what works Same facts, each stated from what the code does now. Also corrects the benchmark.py docstring, which still said BRIAN2, NEST and ANNarchy had not been run, and two REPRODUCING.md phrases that went beyond the checks done. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: docker job tests with CPU torch inside the image The image's CUDA torch build segfaulted on the GPU-less runner while preloading CUDA (run 39191323281), the same cause as #807. The test container swaps in the CPU build of the same version first; the image is otherwise as built. Checked locally: 7 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: record one_spike reorder measurement; 0.3.5 keeps current order Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Tests on GitHub runners crash at random with
Fatal Python error: Segmentation faultinsidetorch._preload_cuda_lib(torch/__init__.py:348), before any test runs. The runners have no GPU, butpoetry.lockinstalls the CUDA 13.0 build of torch. Seen on master7cac0c3and on Dependabot PRs #804 and #805 (lock-only bumps of tornado/virtualenv, unrelated to torch).Fix
.github/scripts/use_cpu_torch.shruns afterpoetry installin both test workflows. It reads the installed torch/torchvision versions from package metadata (without importing torch), reinstalls those same versions from the CPU wheel index, and fails the job if the resulting torch still reports CUDA.Local installs and
poetry.lockare unchanged.🤖 Generated with Claude Code