Repository navigation
Conversation
Investigation prototype behind an experimental trajectory_sharing option of compile_sampler / compile_detector_sampler (statevector strategy only). Shots that agree on the f-bit part of every rotation sign (and, with postselection, of every postselected sign) share one lane with a shot count and a range in a key-sorted order. Lanes split binomially at measurements, and a lane whose postselected record is 1 moves to a dead table without amplitudes. Tables have static capacities: the loop runs in segments, and the host doubles a full table or shrinks the lane table to the live lanes between segments. Grouping sorts on the host (numpy's radix sort is an order of magnitude faster than XLA's on CPU), with random tie-breaking so that every shot's outputs stay independent. The measurement kernels are split into weight and projection helpers so that both executors share them. Without the option, sample_program is called exactly as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sharing_stats.py counts distinct histories and states per action on the clifft-bench components, breakdown.py splits a benchmark call into its stages, shared_breakdown.py times the shared executor's stages, and benchmark.py compares one lane per shot with shared trajectories. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Pruning now XORs the detector reference bits into the postselection test, so use_detector_reference_sample no longer kills the shots the user keeps. - Rare outcomes: whether any shot of a lane sees the rarer outcome is decided with choose_outcome's 2^-64 resolution, then the first such shot and the rest are drawn exactly. jax.random.binomial alone cut that chance to a float32 grid, so a lone shot never saw an outcome of probability below about 1e-7. - Every lane slot draws from its own key, so table sizes, resumes and segments no longer change the samples (now tested bitwise). - Pruned shots report False after their pruning record. - The dead table holds one entry per shot and never grows; the sticky lane capacity lives in a Sharing object owned by the sampler; the unused device sort is gone; sharing_stats is bounded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The exact rare-outcome draw now runs its first-hit and binomial draws only when some lane hit, and reconstruct finds each shot's lane with a scatter and a running maximum in place of sorting the one-per-shot dead table. The trajectory benchmark script is renamed benchmark_sharing.py so that it no longer shadows scripts/statevector/benchmark.py. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
scripts/trajectory/results.md: how much sharing the clifft-bench components allow, where a sample call spends its time, how the prototype handles static shapes, the review fixes, and the speedups of the shared executor and end to end (78x on coherent_d5_r5, 2x to 3x on the cultivation and coherent d3 circuits, none on distillation). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
investigation.md replaces scripts/trajectory/results.md and adds a section on whether Tsim's representation helps trajectory sharing, measured with two new sharing_stats.py options (--mode measurements for raw records, --noiseless) and a new structure.py (noise compression, direct outputs and tape structure per workload). The answer: yes, mostly through the classical/quantum split of the ZX reduction. Postselecting on the direct detectors before any quantum simulation multiplies the sharing by 34x to 195x on the cultivation circuits and removes all noise-driven branching on coherent_d3_r3, and random measurement outcomes become classical bits (noiseless cultivation: 2 trajectories in 16,384 shots in both sampling modes). Sampling detectors rather than records matters mainly because it makes that postselection possible. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The simplified sampler reads every output off the final symbols instead of writing records at measurement or Record actions, so the shared executor can no longer prune at the action that writes a postselected record. - Prune each postselected output at the action that sets its last branch bit, or at the first measurement for an output of f-bits only, and key lanes on the f-parts of the postselected output rows. - Mark a pruned shot's outputs False when they need a branch bit set after its pruning action (`ready` replaces `slot_action`). - Keep the weight and projection helpers the shared kernels need in shared.py, so execute.py stays as upstream. - Port sharing_stats.py and structure.py to output rows, and the pruning test to the new `ready` order. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
run_all.sh waits for an idle machine (summed %CPU, since macOS's load average sits near 5 on an idle desktop), then times Clifft, Tsim on all threads and Tsim on one thread with one lane per shot and with trajectory sharing. report.py merges the JSON lines into a speed table and checks acceptance and observable rates of Tsim against Clifft by a two-proportion z-test. Workloads follow clifft-bench's manifest; SymFT's d=7 circuit runs with and without postselection. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: de01bc1f41
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| sticky = sharing.lane_capacity if segment >= num_actions else 0 | ||
| capacity = max(_capacity(2 * groups), sticky) |
There was a problem hiding this comment.
When grouping keys are mostly unique, groups approaches the batch size and this allocation reserves the next power of two above twice that many lanes—between 2× and 4× the batch. However, _estimate_batch_size() still budgets memory for only one statevector lane per shot using half the available memory. For wide, low-sharing components, an automatically selected batch can therefore allocate roughly all to twice the available memory and fail with an OOM even though the non-sharing executor fits; account for this capacity multiplier when selecting the batch size or grow from a smaller initial table.
Useful? React with 👍 / 👎.
| def _dispatch(f_batch: np.ndarray, indices: list[int], n_valid: int) -> None: | ||
| self._key, subkey = jax.random.split(self._key) | ||
| jax_out = np.asarray( | ||
| sample_program(self._program, jnp.asarray(f_batch), subkey) | ||
| sample_program(self._program, jnp.asarray(f_batch), subkey, **sharing) |
There was a problem hiding this comment.
When a non-direct masked detector prunes a shared lane and either reference-sample option is enabled, this call returns partially written rows but no per-row prune status. _sample_batches_with_postselection therefore does not add these rows to was_discarded, and its later detector/observable reference XORs reintroduce set bits into fields that reconstruct deliberately cleared after pruning. Discarded rows can consequently report spurious later detections or observables instead of False; propagate the shared executor's discarded-row mask or exclude those rows from reference postprocessing.
Useful? React with 👍 / 👎.
☂️ Code Coverage
Overall Coverage
New Files
Modified Files
|
This branch has not been deployed
1 participant
Summary
Stacked on #196 (base
statevector-sampler). This PR adds an experimentaltrajectory_sharing=Trueoption tocompile_samplerandcompile_detector_samplerforstrategy="statevector", along with the investigation behind it.Shots whose quantum trajectories agree can share one state vector. In the clifft-bench circuits, the only per-shot input to the amplitudes is the sign of a rotation, and that sign depends only on noise bits. Once the direct detectors are postselected, the surviving shots share a handful of trajectories. One state vector per shot does 700× to 12,000× more array work than needed.
How
src/tsim/statevector/shared.pyworks:choose_outcome's 2^-64 resolution. The first such shot and the rest are then drawn exactly.Without the option,
sample_programis called exactly as before.Also in this PR:
investigation.md(at the branch root) covers:Its numbers were measured on an older
statevector-samplerbase (883b125).scripts/trajectory/: sharing statistics, stage breakdowns and a sharing benchmark.scripts/clifft_comparison/: a Tsim/Clifft comparison harness on the clifft-bench workloads and SymFT's d=7 circuit.run_all.shwaits for an idle machine, then times Clifft, Tsim on all threads and Tsim on one thread.report.pywrites a speed table and compares acceptance and observable rates with Clifft's using a two-proportion z-test.Performance (same machine, Apple M4 Pro)
How to read the table:
scripts/clifft_comparison/run_all.sh, with Clifft 0.11.0. "1 thread" meansNPROC=1.What the numbers show:
tsim-perfaddresses this.How this was verified
test/unit/statevector/test_shared.py):test/integration/test_trajectory_sharing.py):Test plan
pytest: 1569 passedpyright src test: 0 errorsKnown limitations, left for follow-ups:
🤖 Generated with Claude Code