Skip to content

Probes: own a probe on a rank face by one rank only #1945

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sbryngelson merged 2 commits into from
Oct 8, 2026
Merged

Probes: own a probe on a rank face by one rank only #1945

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
dc81fac
Probes: own a probe on a rank face by one rank only
sbryngelson Oct 6, 2026
c5bfb0e
Probes: give a face probe to the lower rank; add a 2-rank regression …
sbryngelson Oct 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 20 additions & 6 deletions src/simulation/m_data_output.fpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1547,7 +1547,7 @@ contains
G_damaged = 0._wp

if (n == 0) then
if ((probe(i)%x >= x_cb(-1)) .and. (probe(i)%x <= x_cb(m))) then
if (f_probe_owned(probe(i)%x, x_cb(-1), x_cb(m), 1)) then
Comment thread
sbryngelson marked this conversation as resolved.
do s = -1, m
distx(s) = x_cb(s) - probe(i)%x
if (distx(s) < 0._wp) distx(s) = 1000._wp
Expand Down Expand Up @@ -1665,8 +1665,8 @@ contains
end do
end if

if ((probe(i)%x >= x_cb(-1)) .and. (probe(i)%x <= x_cb(m))) then
if ((probe(i)%y >= y_cb(-1)) .and. (probe(i)%y <= y_cb(n))) then
if (f_probe_owned(probe(i)%x, x_cb(-1), x_cb(m), 1)) then
if (f_probe_owned(probe(i)%y, y_cb(-1), y_cb(n), 2)) then
do s = -1, m
distx(s) = x_cb(s) - probe(i)%x
if (distx(s) < 0._wp) distx(s) = 1000._wp
Expand Down Expand Up @@ -1750,9 +1750,9 @@ contains
end if
end if
else
if ((probe(i)%x >= x_cb(-1)) .and. (probe(i)%x <= x_cb(m))) then
if ((probe(i)%y >= y_cb(-1)) .and. (probe(i)%y <= y_cb(n))) then
if ((probe(i)%z >= z_cb(-1)) .and. (probe(i)%z <= z_cb(p))) then
if (f_probe_owned(probe(i)%x, x_cb(-1), x_cb(m), 1)) then
if (f_probe_owned(probe(i)%y, y_cb(-1), y_cb(n), 2)) then
if (f_probe_owned(probe(i)%z, z_cb(-1), z_cb(p), 3)) then
do s = -1, m
distx(s) = x_cb(s) - probe(i)%x
if (distx(s) < 0._wp) distx(s) = 1000._wp
Expand Down Expand Up @@ -2007,4 +2007,18 @@ contains

end subroutine s_finalize_data_output_module

!> Half-open ownership lo < v <= hi, so a probe on a rank face is sampled once, by the rank holding the cell the serial code
!! samples (left of the face); the first rank in a direction also owns its lower face.
logical function f_probe_owned(v, lo, hi, dir)

real(wp), intent(in) :: v, lo, hi
integer, intent(in) :: dir
logical :: first

first = num_procs == 1
if (.not. first) first = proc_coords(dir) == 0
f_probe_owned = v <= hi .and. (v > lo .or. (first .and. v >= lo))

end function f_probe_owned

end module m_data_output
162 changes: 162 additions & 0 deletions tests/B227B20E/golden-metadata.txt
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

33 changes: 33 additions & 0 deletions tests/B227B20E/golden.txt
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions toolchain/mfc/test/cases.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1193,6 +1193,9 @@ def alter_ppn(dimInfo):
cases.append(define_case_d(stack, "2 MPI Ranks", {}, ppn=2))
if ARG("rdma_mpi"):
cases.append(define_case_d(stack, "2 MPI Ranks -> RDMA MPI", {"rdma_mpi": "T"}, ppn=2))
if len(dimInfo[0]) == 1:
# 32 cells split 16/16, so the rank face is exactly x = 0.5: a probe there must be sampled by one rank, not summed
cases.append(define_case_d(stack, "2 MPI Ranks -> Probe on rank face", {"m": 31, "probe_wrt": "T", "fd_order": 1, "num_probes": 1, "probe(1)%x": 0.5}, ppn=2))

def alter_ib(dimInfo, six_eqn_model=False, viscous=False, num_fluids=1):
for slip in [True, False]:
Expand Down
Loading