Repository navigation
Fix append mode to create missing cloud files - #599
Merged
pjbull merged 2 commits intoOct 6, 2026
Merged
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #599 +/- ##
========================================
- Coverage 96.1% 95.3% -0.8%
========================================
Files 30 30
Lines 2977 2977
========================================
- Hits 2861 2839 -22
- Misses 116 138 +22
🚀 New features to boost your workflow:
|
pjbull
requested changes
Oct 5, 2026
| def test_read_missing_file_still_raises(rig, mode): | ||
| p = rig.create_cloud_path("missing.txt") | ||
|
|
||
| with pytest.raises(FileNotFoundError): |
Member
There was a problem hiding this comment.
This should be the more specific CloudPathFileNotFoundError, no?
| ## UNRELEASED | ||
|
|
||
| - Added streaming I/O via `FileCacheMode.streaming` for S3, Azure Blob Storage, Google Cloud Storage, and HTTP/HTTPS. In this mode `CloudPath.open()` returns the standard `io` file objects (`BufferedReader`/`BufferedWriter`/`TextIOWrapper`) backed by ranged reads and multipart uploads instead of the local cache, so only the bytes you read are downloaded and written data is uploaded as you write it. Read streams are seekable; append and update modes fall back to the cache and remove the cache file on close. `buffering` has its builtin `open()` meaning and sets the ranged-request size (default 5 MiB). A new `streaming_max_concurrency` client option (default 4) bounds background read-ahead and part uploads per stream; the buffer size, concurrency, and multipart part size can also be set with the `CLOUDPATHLIB_STREAMING_BUFFER_SIZE`, `CLOUDPATHLIB_STREAMING_MAX_CONCURRENCY`, and `CLOUDPATHLIB_[<PROVIDER>_]STREAMING_PART_SIZE` environment variables. Opening a file for append or update in streaming mode warns that it falls back to the cache. `copy`/`rename`/`replace` stream between clients, `fspath` raises `CloudPathNotImplementedError`, and failures raise `cloudpathlib.exceptions` types including the new `CloudPathStreamingError`. See the [Streaming I/O](https://cloudpathlib.drivendata.org/stable/streaming_io/) docs. (PR [#535](https://github.com/drivendataorg/cloudpathlib/pull/535)) | ||
| - Fixed append mode (`a`, `ab`, `a+`, and `ab+`) to create missing cloud files, matching Python file behavior. Appending to existing files still preserves their content. (Issue [#585](https://github.com/drivendataorg/cloudpathlib/issues/585)) |
Member
There was a problem hiding this comment.
I think this is more of a "changed" than a "fixed"
Assert CloudPathFileNotFoundError for missing-file read modes and classify append creation as a Changed changelog entry. Addresses review comments 4185731244 and 4185809794 on PR drivendataorg#599.
terminalchai
had a problem deploying
to
reviewed-live-cloud
October 5, 2026 20:26 — with 
GitHub Actions
Error
GitHub Actions
Error
terminalchai
had a problem deploying
to
reviewed-live-cloud
October 5, 2026 20:26 — with
GitHub Actions
Error
GitHub Actions
Error
terminalchai
had a problem deploying
to
reviewed-live-cloud
October 5, 2026 20:26 — with
GitHub Actions
Error
GitHub Actions
Error
pjbull
approved these changes
Oct 5, 2026
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Allow
CloudPath.open()in append mode to create a missing cloud file, matching Python's local-file behavior. The shared missing-file guard now rejects read modes only. Existing append content is preserved and uploads continue through the existing close handler.Closes #585.
Validation:
CloudPathFileNotFoundError; the four missing-file read cases passed.1846ae2): 2,165 passed, 127 skipped. All 120 regression cases also pass withCLOUDPATHLIB_FILE_CACHE_MODE=streaming, confirming the new append/update cache fallback works. Before that upstream merge, the file I/O and caching suites passed all 540 cases.git diff --checkpasses.--target-version py310because the installed version does not recognize the repository's Python 3.15 target.Added a user-facing entry under
HISTORY.md's UNRELEASED heading.Prepared with AI assistance. Issue #585 exists but does not yet have maintainer sign-off; the contribution guide requests sign-off before external contributions proceed. Opening as a draft pending maintainer sign-off and any requested live-backend validation.