[林SO (Linxiushen)]

Why are these changes needed?

On pandas 3.1, tabular preprocessing emits Pandas4Warning when concatenating/resetting rows, dropping uninformative columns, or deriving weekday/day-of-year features. Workflows that treat deprecations as errors fail before estimator fitting. Twelve regression cases fail on unchanged 12b0f15e4b2c00fe119d9224ed5333f8fc6bab34 under that warning policy.

Assign the results of reset_index and drop, and use the new datetime accessors when available, with a fallback for older supported pandas versions. Keep the existing generated feature names. Reset Series sample weights before assigning them to fit_kwargs, so both references keep the same aligned object.

Thirteen new cases cover categorical concat, constant/identifier column removal, leap/year-boundary datetime values, a simulated legacy accessor surface, and DataFrame/ndarray shuffling with absent/array/Series weights. The new and adjacent preprocessing tests pass: python -m pytest test/automl/test_pandas_preprocessing.py test/automl/test_preprocess_api.py -q — 38 passed. pre-commit run --all-files and pip check also pass.

Validated locally on Windows, Python 3.12.14, with actual pandas 2.3.3 and 3.1.0: the same 38 tests pass on each at 91fe7c93e7514f86d09cf9f5bd14825249195d50. On pandas 2.3.3, the previous d0ca3b1 test shim reproduced the CI failure because the cached datetime accessor writes back to Series.dt; the follow-up adds the property setter used by that cache. The simulated legacy-accessor test remains a compatibility control, not execution on pandas 1.1.4. Other Python versions, pandas 4, the full optional-dependency suite, Spark and GPU/distributed workloads were not run. This change was prepared with AI assistance.

Related issue number

Related to #1647. This covers the tabular preprocessing paths above; the separate time-series transformer and portfolio CLI sites reported there remain outside this patch.

Checks

• I've used pre-commit to lint the changes in this PR.
• I've included any doc changes needed (no documentation change needed for this compatibility fix).
• I've added tests corresponding to the changes introduced in this PR.
• I've made sure all auto checks have passed (the current-head Build matrix is still running; local focused tests pass).

[Copilot]

🟢 Approval recommended

The narrowly scoped compatibility changes preserve preprocessing behavior, have focused regression coverage, and leave no unresolved review findings.

No open issues 🎉

What changed in this PR

This PR addresses the tabular preprocessing portion of #1647, avoiding pandas 3.1 deprecation warnings while preserving compatibility with older pandas versions and existing feature names.

Changes:

• Replace in-place index resets and column drops with reassignment.
• Prefer modern datetime accessors with legacy fallbacks, and preserve aligned sample-weight references.
• Add 13 regression cases covering preprocessing behavior with deprecation warnings treated as errors.

File	Description
test/​automl/​test_pandas_preprocessing.py	Add regression coverage for concatenation, column removal, datetime features, and shuffled weights.
flaml/​automl/​task/​generic_task.py	Reassign reset indexes while keeping sample-weight references aligned.
flaml/​automl/​data.py	Replace deprecated in-place operations and add compatible datetime accessor selection.

_{Review effort: Balanced}

[Li Jiang (thinkall)]

Reviewed the complete PR at 91fe7c9. No actionable source correctness or regression findings were identified in the tabular preprocessing scope. The independent Low review returned clean.

Native local validation passed 38 new/adjacent preprocessing tests on each of Python 3.13/pandas 3.0.5, Python 3.10/pandas 2.3.3 and Python 3.12/pandas 2.3.3, plus the 13 new helper/alignment cases on pandas 1.5.3: 127 passing executions. Actual pandas 3.1 was not run locally, and the legacy-accessor shim is not a native pandas 1.1.4 run; Spark, GPU/distributed workloads and the full optional suite were not run.

Withholding approval because actual current-head Build run 40269762380 failed in Ubuntu/Python 3.13 job 121400045420: 36 pytest failures report YAML ParserError in an MLflow run's meta.yaml. Both focused preprocessing files passed in that job, which explicitly pinned pandas to 2.3.3 before testing. I have not reproduced the corrupt file or established that the failure is unrelated to this patch. No failed CI was rerun.

Balanced Copilot review 5648428805 recommends approval, but that does not satisfy the failed actual-workflow/check gate. This is a source-clean review with the stated validation limits, not PR approval.

Posted by thinkall-agent-auto-reviewer

[Copilot]

🟢 Approval recommended

The localized compatibility changes preserve existing behavior and have focused regression coverage, with no blocking issues identified.

No open issues 🎉

_{Review effort: Balanced}

[Li Jiang (thinkall)]

Re-reviewed commits 91fe7c93e7514f86d09cf9f5bd14825249195d50 -> ee76e8a0327ded25f223d698031badc5b1825184. The only delta is main's website/yarn.lock update of follow-redirects 1.16.0 -> 1.16.1. One explicitly head-bound review-code Low execution (actual runtime gpt-6.1-sol) found no new issue in this identical inherited patch. Controller Git comparisons verify all non-lockfile tree entries are unchanged. This is a scoped merge-delta review, not a fresh clean signoff for unchanged feature code.

Version/URL/caret/SRI shape checks pass. No independent local registry/tarball verification, package smoke test or website rebuild is claimed. Previous feature findings and runtime validation limits remain preserved. The earlier 127 passing native test executions remain feature evidence, not 127 new runs here; actual pandas 3.1 was not run locally. The previous MLflow/YAML CI failure's relationship to this PR was not established.

At the 2026-10-26T12:13:44.943946+00:00 gate snapshot, actual current-head workflows were: Code formatting completed/success; Build in_progress/no conclusion; docs completed/success. Copilot has explicit Balanced Approval recommended; 0 review thread(s) remain unresolved. Gate approval is not successful CI completion, and no new PR approval is being submitted while required gates remain unmet.

Posted by thinkall-agent-auto-reviewer

[Li Jiang (thinkall)]

Overall review of the whole PR at ee76e8a0327ded25f223d698031badc5b1825184: approved.

The complete feature diff is byte-identical to the prior independent clean preprocessing review: reset/drop results are rebound, modern datetime accessors retain legacy fallbacks, and shuffled Series weights keep the correct positional alias. The current head is explicitly bound in the clean inherited-main review. I reused verified review-code Low evidence (actual runtime gpt-6.1-sol) and the 127 prior passing native executions across pandas 1.5.3, 2.3.3 and 3.0.5; these are not new executions in this cycle. Actual pandas 3.1 and the native minimum pandas 1.1.4 were not run locally. The cause of the earlier head's MLflow/YAML CI failure was not established; that uncertainty is not relabeled as an unrelated failure.

Immediately before submission, I verified the same current open, non-draft, non-self head, all review threads resolved, explicit current-head Balanced Copilot Approval recommended (review), and successful applicable check contexts. I also queried the actual Actions workflow runs directly: Build succeeded; Code formatting succeeded; docs succeeded. The deployment-only gh-release job is intentionally skipped for pull requests by its explicit event guard; it is not being counted as a passed validation.

Posted by thinkall-agent-auto-reviewer