Skip to content

fix(nwis): index format_response's frame without set_index(inplace=True) - #460

Merged
thodson-usgs merged 1 commit into
DOI-USGS:mainfrom
thodson-usgs:fix/nwis-set-index-inplace
Oct 29, 2026
Merged

thodson-usgs merged 1 commit into
DOI-USGS:mainfrom
thodson-usgs:fix/nwis-set-index-inplace

Conversation

@thodson-usgs

Copy link
Copy Markdown
Collaborator

Summary

pandas 3.1.0 (released 10/2026) deprecates set_index(..., inplace=True) with a Pandas4Warning, which subclasses DeprecationWarning. nwis.format_response calls it for every frame with a datetime column, so the 3.13 and 3.14 test jobs on main fail test_nested_calls_emit_one_warning on the second warning. The 3.10 jobs install pandas 2.x and pass.

 df = df.copy(deep=False)
-df.set_index(keys, inplace=True)
+levels = [df.pop(key) for key in keys]
+df.index = (
+    pd.MultiIndex.from_arrays(levels) if len(levels) > 1 else pd.Index(levels[0])
+)

The in-place call was deliberate: on pandas 2 with copy-on-write off, set_index without it copies every column. Popping the key columns off the shallow copy is what set_index(inplace=True) does internally, so that property is kept without the deprecated keyword.

Evidence

  • Before: pandas 3.1.0, Python 3.13 (as in CI, e.g. run 40888458898):
    FAILED tests/nwis_test.py::TestDeprecationWarnings::test_nested_calls_emit_one_warning - assert 2 == 1
    Pandas4Warning: The inplace keyword in DataFrame.set_index is deprecated ... (PDEP-8)
    
    After: full offline suite, 1757 passed, no Pandas4Warning, on pandas 3.1.0 (Python 3.13), 2.3.3 (3.12) and 3.0.5 (3.12).
  • format_response before and after, on pandas 3.1.0 and 2.3.3 (single-site, multi-site, multi_index=False, GeoDataFrame, string datetimes, peaks):
    frames and indexes: identical (assert_frame_equal, check_exact; assert_index_equal, exact)
    warnings:           pandas 3.1: 2 Pandas4Warning -> none;  pandas 2.3: none -> none
    peak heap / frame:  pandas 3.1: 0.83 -> 0.83;              pandas 2.3: 1.12 -> 1.12
    

Merge Danger

Door: two-way

Blast Radius: none

format_response returns the same frames. The only visible difference is that pandas 3.1 users no longer see the warning.

🤖 Generated with Claude Code

pandas 3.1.0 deprecates the inplace keyword of set_index with a
Pandas4Warning, a DeprecationWarning subclass. Every nwis getter that
returns a time-indexed frame emitted it, and the 3.13 and 3.14 CI jobs
failed test_nested_calls_emit_one_warning on the second warning; the
3.10 jobs install pandas 2 and passed.

The in-place call was deliberate: without it, set_index copies every
column when copy-on-write is off. Popping the key columns off the
shallow copy is what the in-place call did internally, so the frames
are identical and peak memory is unchanged on pandas 2.3.3 and 3.1.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@thodson-usgs
thodson-usgs marked this pull request as ready for review October 29, 2026 14:40
@thodson-usgs
thodson-usgs merged commit c926c70 into DOI-USGS:main Oct 29, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant