Skip to content

perf(cache): batch VAL range maintenance and use full cache-line strides - #669

Merged
chetan-rathore merged 2 commits into
ARM-software:mainfrom
cdwilde:perf/630-batched-cache-maintenance
Nov 9, 2026
Merged

chetan-rathore merged 2 commits into
ARM-software:mainfrom
cdwilde:perf/630-batched-cache-maintenance

Conversation

@cdwilde

@cdwilde cdwilde commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Batch the non-Linux VAL range cache maintenance: one DC operation per cache line and a single completion barrier per range.

  • Per review, the implementation lives in VAL rather than the PALs. val_pe_cache_clean_range(), val_pe_cache_clean_invalidate_range() and val_pe_cache_invalidate_range() call assembly helpers in val/src/AArch64/SystemReg.S.
  • val_clean_dcache_range() (DC CVAC) and val_clean_inv_dcache_range() (DC CIVAC) are new. They reuse the existing do_dcache_maintenance_by_mva macro, which already backs val_inv_dcache_range() (DC IVAC).
  • The macro decodes CTR_EL0.DminLine as 4 << DminLine bytes. The old C loops used 2 <<, which visited every line twice.
  • It aligns the start down to a line boundary and covers every line overlapping [start, start + length). It returns immediately for a zero length and issues one DSB SY at the end, with no per-line DSB/ISB.
  • No PAL interface changes. The single-address val_data_cache_ops_by_va() and the Linux range behavior are unchanged.

Fixes #630.

Validation

  • Executed the assembled SystemReg.S helpers in an AArch64 emulator (Unicorn): 135 cases across all three operations, aligned and unaligned starts, and lengths from 0 to 8 MB. Each case checks the exact DC address sequence, one DSB SY and no ISB.
  • Integrated into a full SBSA baremetal build (-Werror) and ran the complete suite on a pre-silicon emulation platform, with our local replacement for val_pe_cache_clean_range() removed. All 222 rule results match the baseline run, and the ACS summary is unchanged.

@cdwilde

cdwilde commented Oct 28, 2026

Copy link
Copy Markdown
Contributor Author

SBSA compliance assessment

My assessment is that this is a cache-maintenance correctness/performance change, not a relaxation of an SBSA requirement. It uses the architectural cache-line stride and batches completion barriers while retaining the range-maintenance completion guarantee. The existing single-address API and Linux range behavior remain unchanged.

The 112 compiled AArch64 behavioral cases check addresses and operation/barrier counts with architectural side effects mocked; they are not a demonstration of physical cache/coherency behavior. The two UEFI range bodies were cross-compiled in isolation, not as full UEFI applications. Custom non-Linux PALs must implement the new range API before integrating the updated VAL.

These are contributor assessments, not an Arm compliance determination. This PR alone does not establish platform SBSA compliance. The applicable BSA and SBSA checks still need to be run, with remaining failures and waivers handled separately. Before claiming the affected checks are validated on the target platform, the fix needs to be integrated, its replaced local wrappers disabled/removed, and the affected checks rerun as part of the applicable suite.

Validation so far is limited to the targeted compilation and compiled-code behavioral checks described in the PR. No full target-platform compliance rerun has been performed.

@SrikarJosyula

Copy link
Copy Markdown
Contributor

Hi @cdwilde,

Thanks for the PR! The changes look good overall.

We have one design suggestion: since pal_pe_data_cache_ops_range() is identical across the ACPI, DT, and BM PALs and uses architectural cache operations, could we move the common implementation into VAL?

We could place the required assembly helpers in SystemReg.S and call them from the VAL range APIs, reusing existing helpers where possible. This would avoid maintaining three copies while preserving the single completion barrier and the updated range-boundary handling.

cdwilde and others added 2 commits November 5, 2026 06:55
Introduce a PAL range operation for baremetal and both UEFI PALs. Decode CTR_EL0.DminLine in four-byte words, issue one DC per line and one final DSB SY. Keep single-address operations and Linux range behavior unchanged. Handle empty, unaligned and top-of-address-space ranges without end-address overflow.

Cross-compiled the changed baremetal/VAL units using upstream -Wall -Werror recipes and both UEFI range bodies in isolation. Executed 112 compiled AArch64 range cases, checking operation addresses and barrier counts. Full UEFI application builds were not performed.

Fixes ARM-software#630

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 64fb407d-eb53-484a-bc49-606a91214662
Per review, replace the three identical PAL range implementations with
VAL assembly helpers in SystemReg.S. val_clean_dcache_range() and
val_clean_inv_dcache_range() reuse the existing
do_dcache_maintenance_by_mva macro already used by val_inv_dcache_range():
one DC op per CTR_EL0.DminLine cache line from the aligned start address
to the end of the range, then a single DSB SY. The val_pe_cache_*_range()
APIs call them directly, so no new PAL API is needed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: aa96196c-fd99-476b-a579-8de0fe7f65e7
@cdwilde
cdwilde force-pushed the perf/630-batched-cache-maintenance branch from 368e2e6 to 20e5775 Compare November 5, 2026 18:36
@cdwilde cdwilde changed the title perf(cache): batch range maintenance and use full cache-line strides perf(cache): batch VAL range maintenance and use full cache-line strides Nov 5, 2026
@cdwilde

cdwilde commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @SrikarJosyula. Done in 20e5775, rebased on main. The PAL copies and the new PAL API are gone. val_pe_cache_*_range() now call SystemReg.S helpers: I added val_clean_dcache_range() and val_clean_inv_dcache_range(), which reuse the existing do_dcache_maintenance_by_mva macro behind val_inv_dcache_range(). That keeps one DC per line, the start alignment and a single DSB SY per range. Validation is in the updated description.

@SrikarJosyula

Copy link
Copy Markdown
Contributor

Looks good! We’ll run the changes through our regression and let you know if any updates are needed.

@chetan-rathore
chetan-rathore merged commit 548c9cc into ARM-software:main Nov 9, 2026
29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

High Impact Enhancement Request for Pre-silicon: Batch cache maintenance in val_pe_cache_*_range() (one DC op per line, one DSB per range)

3 participants