Document concurrent access limits for persistent caches

[terminalchai]

Persistent-cache documentation describes reuse across processes without explaining concurrent access limits. Users sharing CLOUDPATHLIB_LOCAL_CACHE_DIR can encounter incomplete cached downloads or OverwriteNewerLocalError when another worker is downloading the same object.

Add a guide covering cache isolation, cleanup modes, environment-variable precedence, external coordination, and streaming cache fallback. Include an executable thread-pool example that reads the same object in two tasks with separate temporary caches. Link the guide from the README, client settings, and docs navigation; update HISTORY.md.

Addresses the documentation request in #522, which has maintainer support. This PR references the issue without a closing keyword because runtime locking and timestamp behavior remain open topics.

Validation:

• Reproduced OverwriteNewerLocalError with two LocalS3Client instances sharing a cache and a paused partial download.
• Executed the exact example with local S3 mocks. Same-object reads succeeded; each task used a distinct cache; explicit settings overrode both deliberately conflicting cache environment variables; caches were removed after each read.
• Built the docs with MkDocs after regenerating the root-backed pages. Verified rendered guide text, code, navigation and links. Existing unrelated link warnings remain.
• Black and git diff --check pass.
• Codex reviewed the documentation and found no remaining factual/API/example defects after its minor suggestions were applied. Review used supplied text because its nested shell runner was unavailable.
• Documentation-only change; the runtime test suite was not rerun, and no live cloud backends were accessed.

Contributor checklist:

• Read CONTRIBUTING.md
• Existing issue Cache timestamps are wrong while downloading #522 and maintainer-supported documentation request
• Based on latest master
• Reproduced the reported behavior and verified the documented workaround
• Guidance covers shared cache behavior across providers
• Manual live cloud backend testing (docs example verified with local mocks)
• Relevant local validation passes
• Updated HISTORY.md under UNRELEASED

Prepared with AI assistance; the documentation and executable example were reviewed before submission.

[pjbull]

I think that this is not quite right from an advice perspective.

• We should lead with recommending implementers architect their app with path-level parallelism. A shared persistent cache is safe when each worker uses a different cloud object. One Client is safe to share across threads. Each process should construct its own client pointed at the same local_cache_dir.
• If that first recommendation is not a good fit, keep a separate cache per worker, and the external lock, for the case where workers read or write the same object.
• Make sure to add the caveat that submitting a CloudPath to a process pool drops _client on pickle, so the child uses the default client; implements should construct clients in the subprocess or set the default client before forking.
• Extend the cache-key warning past providers: the same bucket and key on two accounts or endpoints share a local file, and HTTP keys ignore the host.
• Link this page from the caching guide (docs/docs/script/caching.py and the notebook), which still says a persistent cache is safe across processes. Mention the same limit on the local_cache_dir docstrings for S3, GCS, Azure, and HTTP.

[pjbull]

This makes a private cache the default for independent work. Independent work on different objects can share one client and one persistent cache.

Suggested lead, before the same-file section:

Parallelize by cloud object. Workers that read or write different objects can share one persistent cache. Give each thread its own CloudPath; the client can be shared. In a process pool, construct the client inside the worker with the same local_cache_dir and file_cache_mode="persistent". Do not pass a Client or CloudPath into the pool.

Move the TemporaryDirectory example under "same file". For the shared case, use different URIs and a client created once:

def read_in_worker(uri):
    return client.CloudPath(uri).read_bytes()

if __name__ == "__main__":
    client = S3Client(local_cache_dir="cache", file_cache_mode="persistent")
    paths = ["s3://bucket/first.csv", "s3://bucket/second.csv"]
    with ThreadPoolExecutor(max_workers=4) as executor:
        for data in executor.map(read_in_worker, paths):
            print(len(data))

[pjbull]

For a process pool, create the client inside the worker and reuse the cache directory. A new directory per worker is only needed when two workers use the same object. Pickling a CloudPath removes _client, and the child then builds the default client.

[pjbull]

Move this up so it is the normal case, not a footnote. Add that names also collide across accounts, endpoints, and HTTP hosts: the cache key is the path with the scheme removed, and for HTTP it is the URL path with the host removed.

[pjbull]

Point this summary at shared-cache, different-object work first. "Worker-specific caches" should be the same-object fallback.

[pjbull]

Link the rendered docs page, https://cloudpathlib.drivendata.org/stable/concurrent_cache_access/, the same way the streaming notes link to the docs site.