Merge from Ruochun/DEM-Engine

AGENTS.md

@@ -58,6 +58,11 @@ DEM-Engine is a performance-sensitive CUDA/C++ DEM solver with a Chrono-like pub
 - Keep Python bindings in sync when changing public C++ APIs that should be exposed to Python.
 - Preserve backward compatibility unless the requested change explicitly breaks it.
 
+## Asynchronous Output
+
+- Background writers must own complete snapshots or have explicit synchronization protecting every shared input until use finishes. Include counts, mappings, output settings and mesh caches in the ownership audit, not just device/host arrays. Follow [the asynchronous-output design](docs/developer/async-output.rst).
+- Keep completion waits needed before reading or deleting output files; snapshot safety does not imply file completion.
+
 ## Tests And Demos
 
 - Prefer modular tests for targeted behavior changes. They should be deterministic, self-checking, and small enough to run as normal build targets.

VERSION

@@ -1 +1 @@
-3.0.14
+3.0.18

cmake/PrepareJitify.cmake

@@ -0,0 +1,30 @@
+# Copy the selected Jitify header without changing the third-party checkout. On MSVC, the optional upstream
+# fallback needs the same DbgHelp serialization as DEME's vendored variant. Reject unfamiliar fallback layouts
+# rather than silently producing an unprotected Windows build.
+function(deme_prepare_jitify input_header output_header msvc_build)
+	set_property(DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS "${input_header}")
+	file(READ "${input_header}" jitify_source)
+	if(msvc_build AND NOT jitify_source MATCHES "DEME serializes DbgHelp")
+		set(demangle_signature "inline std::string demangle_native_type(const std::type_info& typeinfo) {")
+		string(FIND "${jitify_source}" "${demangle_signature}" signature_offset)
+		if(signature_offset EQUAL -1)
+			message(FATAL_ERROR "Cannot add MSVC DbgHelp protection to ${input_header}; use DEME's vendored Jitify header.")
+		endif()
+		# This signature also occurs in the non-MSVC implementation. Guard the inserted code explicitly so the
+		# generated header remains usable on other platforms with no locking or extra includes there.
+		set(demangle_guard "\n#ifdef _MSC_VER
+    // DEME serializes DbgHelp across all reflected types and worker threads.
+    static std::mutex dbghelp_mutex;
+    std::lock_guard<std::mutex> lock(dbghelp_mutex);
+#endif
+")
+		string(REPLACE "${demangle_signature}" "${demangle_signature}${demangle_guard}" jitify_source "${jitify_source}")
+		set(jitify_source "#ifdef _MSC_VER\n#include <mutex>\n#endif\n${jitify_source}")
+		# COPYONLY keeps the final header's timestamp stable when configure is rerun without content changes.
+		set(patched_header "${output_header}.msvc-input")
+		file(WRITE "${patched_header}" "${jitify_source}")
+		configure_file("${patched_header}" "${output_header}" COPYONLY)
+	else()
+		configure_file("${input_header}" "${output_header}" COPYONLY)
+	endif()
+endfunction()

docs/README.md

@@ -84,37 +84,58 @@ cmake --build build-docs --target docs
 Normal HTML builds use the committed `docs/python/reference.rst` and therefore
 do not import DEME or require a CUDA GPU.
 
-Regenerate that file after changing pybind11 signatures or docstrings. First
-build and stage the Python extension:
+Regenerate that file after changing pybind11 signatures or docstrings. From the
+repository root, activate the Python environment used to build the extension
+and run this complete sequence. Doxygen and Sphinx must also be available:
 
 ```bash
+# Build and stage the current Python extension.
 cmake -S . -B build-python \
     -DDEME_BUILD_PYTHON=ON \
     -DCMAKE_BUILD_TYPE=Release
 cmake --build build-python --target _deme --parallel
 cmake --install build-python \
     --component python \
     --prefix /tmp/deme-python-reference
-```
-
-Then run the generator with the same Python interpreter used to build the
-extension:
 
-```bash
+# Generate the reference using the same Python interpreter.
 PYTHONPATH=/tmp/deme-python-reference \
-    /usr/bin/make -C docs python-reference
-```
-
-Finally, rebuild the site:
+    /usr/bin/make -C docs python-reference PYTHON="$(command -v python)"
 
-```bash
+# Rebuild the documentation (requires Doxygen and Sphinx).
 /usr/bin/make -C docs html
+
+# Review the generated changes.
+git diff -- docs/python/reference.rst
 ```
 
 Do not edit `docs/python/reference.rst` directly. Improve the corresponding
 docstring in `src/DEM/python/bindings.cpp`, rebuild the extension, and
 regenerate the page.
 
+## Publish to GitHub Pages
+
+In **Settings → Secrets and variables → Actions → Variables**, create or update
+the repository variable `DOCS_PUBLISH_BRANCH` to the branch that should publish
+the documentation. Set it to `develop` to publish from `develop` instead of
+`DEME3_docs`. This must be a repository **variable**, not a secret or an
+environment variable: the workflow reads `vars.DOCS_PUBLISH_BRANCH`.
+
+Set **Settings → Pages → Build and deployment → Source** to **GitHub Actions**,
+and allow the selected branch in the `github-pages` environment's deployment
+branch rules under **Settings → Environments**.
+
+Commit the documentation sources, including the regenerated Python reference,
+and push to the selected branch. The **Build and publish documentation** workflow
+builds and publishes the site automatically. Keep generated `docs/_build/`
+files out of Git.
+
+An unset variable or a branch mismatch causes push and manual runs to skip the
+build and deployment. Changing the variable alone does not trigger a build;
+push a new commit or, when **Run workflow** is available, select the matching
+branch and run it manually. Pull requests build for validation without deploying.
+See [hosting options](developer/hosting.rst) for the full setup details.
+
 ## Preview on the same machine
 
 Run a local HTTP server from the repository root:

docs/developer/architecture.rst

@@ -33,6 +33,9 @@ synchronization. Avoid adding device-wide synchronization to hot paths. When
 cross-device data is transferred, preserve the lifetime of the source storage
 until the destination operation has completed.
 
+File-output ownership, current snapshot coverage and remaining host-memory risks
+are documented in `Asynchronous output <async-output.rst>`__.
+
 Adding Python bindings
 ----------------------
 

docs/developer/async-output.rst

@@ -0,0 +1,198 @@
+Asynchronous output: ownership and synchronization
+==================================================
+
+Scope and current status
+------------------------
+
+This page describes file-output ownership and synchronization requirements.
+It separates protections already implemented from the contract intended for all
+writers. Asynchronous output does not make the solver API safe for arbitrary
+concurrent calls from multiple application threads.
+
+Current writer ownership
+------------------------
+
+.. list-table::
+   :header-rows: 1
+   :widths: 23 42 35
+
+   * - Writer
+     - Background inputs
+     - Protection and remaining limitations
+   * - ``WriteContactFile``
+     - Owned ``ContactInfoContainer``, copied output flags and wildcard names.
+     - Snapshot completed before return. Subsequent dynamics cannot change the
+       writer's inputs. Includes the CSV fallback for binary output and the
+       potential-pair convenience API.
+   * - ``WriteAnalyticalFile``
+     - Owned vector of transformed analytical components; copied domain bounds
+       and tessellation resolution.
+     - Snapshots inputs before launching the writer.
+   * - ``WriteSphereFile``
+     - Shared dT host arrays and output metadata, accessed through the solver.
+     - CSV, binary fallback and VTK still depend on those inputs remaining
+       unchanged until the writer finishes.
+   * - ``WriteClumpFile``
+     - Shared dT host arrays and output metadata; copied numeric precision.
+     - CSV and binary fallback still depend on shared inputs remaining unchanged.
+   * - ``WriteMeshFile``
+     - Shared host arrays, mesh caches and output metadata; some options copied.
+     - VTK, STL and PLY do not yet own complete snapshots. Refreshing a cache
+       before launch does not isolate it from later mutation.
+
+See the implementations in `APIPublic.cpp <../../src/DEM/APIPublic.cpp>`__, the
+host readers in `dT.cpp <../../src/DEM/dT.cpp>`__, and the snapshot-only contact
+formatter in `ContactOutput.hpp <../../src/DEM/utils/ContactOutput.hpp>`__.
+
+Execution and synchronization
+-----------------------------
+
+The solver has one pending output thread. Each file-output method first calls
+``WaitForPendingOutput()``, which joins the previous writer. Output jobs therefore
+serialize with each other; they do not each run on an independent concurrent
+writer thread.
+
+The shared-array methods then download the necessary GPU values into the solver's
+existing host storage and launch a thread capturing ``this``. Formatting and disk
+writing take place on that thread. Capturing ``this`` copies a pointer, not the
+arrays. Local vectors or string streams constructed by the writer do not protect
+the shared inputs while the writer is constructing them.
+
+The snapshot methods instead finish reading solver state on the calling thread
+and hand owned data to the background thread. Contact output performs filtering,
+ID resolution and coordinate transformations while constructing this snapshot;
+CSV formatting and disk I/O remain asynchronous.
+
+``DoDynamics()`` does not join pending output. ``DoDynamicsThenSync()`` synchronizes
+the simulation workers, not the file writer. Some mutating operations, including
+``Update()`` and the mesh-deformation APIs, explicitly wait for output. Those are
+local safeguards, not a general lock around every host access or mutation.
+
+Data categories and treatment
+-----------------------------
+
+Device arrays
+~~~~~~~~~~~~~
+
+File writers use CPU-readable data downloaded before launch. Once a transfer has
+completed, changing a separate device allocation does not itself change its host
+mirror. This is why overlapping GPU dynamics and output can work when host inputs
+remain stable. Transfer completion alone does not guarantee that the host mirror
+will remain unchanged afterward.
+
+Shared host mirrors
+~~~~~~~~~~~~~~~~~~~
+
+The host portions of ``DualArray`` objects belong to the solver. A later download,
+host-side setter, resize or storage replacement can change the same memory a
+writer is reading. Resizing can also invalidate references or pointers. A host
+mirror is not an output-job snapshot merely because it is separate from device
+memory. Shared storage requires protection until the last reader finishes.
+
+Counts, mappings and configuration
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Counts, owner/geometry mappings, family filters, output flags, wildcard names and
+coordinate-conversion parameters are part of the output's inputs too. Arrays and
+metadata must describe the same frame. Copying a count alone cannot protect an
+array that can be overwritten or reallocated; copying an array alone cannot
+protect interpretation through mutable mappings or settings.
+
+Contact output resolves these dependencies before launch and copies its
+column configuration. Its background formatter has no solver access. The shared
+sphere, clump and mesh writers still consult solver metadata while formatting.
+
+Mesh caches and setup geometry
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+``WriteMeshFile()`` synchronizes deformed mesh caches before starting its writer,
+but the writer still reads solver-owned meshes and owner transforms. Treat vertex
+and connectivity vectors, patch data and any output attributes as dependencies.
+Setup-time data may be shared only if its lifetime and immutability are guaranteed
+for the entire write. Do not infer that guarantee merely because a vector usually
+stays unchanged during dynamics. Direct mutation of exposed setup/cache objects
+also needs to be considered when reviewing a writer.
+
+Job-owned snapshots
+~~~~~~~~~~~~~~~~~~~
+
+A complete snapshot owns the values needed to interpret and serialize a frame.
+It must not retain pointers, references or views into mutable solver storage.
+Contact output holds an independently populated container through a const shared
+pointer; analytical output moves its component vector into the writer. These
+jobs keep their storage alive until formatting and writing finish.
+
+Contact snapshot construction
+-----------------------------
+
+``WriteContactFile()`` calls ``generateContactInfo()`` before launching the
+writer and captures the resulting owned container, output flags and wildcard
+names. Both normal CSV and the binary-to-CSV fallback use the same snapshot-only
+formatter. Invalid contact types are rejected during snapshot construction.
+
+Remaining risks and caller guidance
+-----------------------------------
+
+Sphere, clump and mesh output have not yet been converted to complete snapshots.
+Their shared inputs can cause inconsistent frames, invalid indexing or invalid
+memory access if changed concurrently. This is an ownership risk identified from
+the implementation, not a claim that every ordinary dynamics call reproduces a
+failure in each writer. Stable owner counts and GPU-only updates can leave their
+host inputs undisturbed, and some APIs already wait before mutation.
+
+Until those writers are isolated, callers can explicitly finish output before
+resuming dynamics or performing operations that could change its shared inputs:
+
+.. code-block:: cpp
+
+   solver.WriteMeshFile("frame.vtk");  // Same precaution for sphere/clump output.
+   solver.WaitForPendingOutput();
+   solver.DoDynamicsThenSync(duration);
+
+Contact and analytical output do not need that wait solely to protect their
+snapshots from subsequent dynamics. All asynchronous outputs still need a
+completion wait before the application reads, checks, deletes or otherwise
+consumes the destination file. An owned snapshot guarantees input stability;
+it does not make the file complete when the API returns.
+
+Required contract and follow-up work
+------------------------------------
+
+Every background writer must either own its complete inputs or have explicit
+synchronization protecting all shared inputs for their entire use. The preferred
+follow-up is to give sphere, clump and mesh output complete owned snapshots,
+including any metadata needed for serialization. These writers currently retain
+the shared-storage behavior described above.
+
+Build snapshots while the relevant solver state is stable, then pass only those
+snapshots to standalone formatting code. Preserve file layouts, filtering,
+coordinate conventions and format fallbacks. Snapshot creation adds foreground
+CPU work, but should leave formatting and disk writing asynchronous and avoid
+unnecessary device-wide synchronization in dynamics. Assess memory use for large
+outputs, including snapshot and formatting buffers.
+
+Ownership does not solve every output concern. File-open/write failures and
+exceptions on the background thread need a separate error-propagation policy;
+the snapshot mechanism does not provide one. Completion waits, solver lifetime and the
+supported application-thread calling model also remain part of the contract.
+
+Regression coverage
+-------------------
+
+`DEMTest_ContactOutputSnapshot.cpp <../../src/demo/ModularTests/DEMTest_ContactOutputSnapshot.cpp>`__
+uses a POSIX FIFO to block the writer opening its destination until after dynamics
+changes the contact list. It compares the delayed output byte-for-byte with a
+reference from the requested frame. Coverage includes contact-list growth, CSV,
+binary fallback with potential pairs, force filtering to zero rows and contact
+wildcards. The FIFO test skips on Windows; the production snapshot mechanism is
+platform-independent.
+
+The test deliberately has no completion wait between queuing the delayed contact
+output and advancing dynamics. Its other waits are required to finish the
+reference file before reading it and to join the delayed writer after draining
+the FIFO. Removing those waits would not be a valid test of snapshot safety.
+
+Future writer conversions should receive equivalent delayed-output tests covering
+their mutable arrays, metadata and caches. Do not use sleeps as the only mechanism
+for provoking a race, and do not treat contact-test success as coverage of the
+remaining shared-array writers.

docs/index.rst

@@ -33,6 +33,7 @@ kernels.
    :caption: Development
 
    developer/architecture
+   developer/async-output
    developer/type-codes
    developer/documentation
    developer/packaging