[林SO (Linxiushen)]

Why are these changes needed?

A missing datetime feature becomes ordinal 1 when DataTransformer maps datetime.toordinal over NaT. The numeric median imputer then treats it as a valid date instead of a missing value. For example, a missing timestamp among dates around 2020-02-29 becomes 1 rather than the learned median 737484; the prediction path has the same problem, including batches containing only missing timestamps.

Guard both ordinal conversions so missing timestamps remain np.nan for the existing training-fitted median imputer. Valid datetime conversion, generated calendar features and the existing all-missing training-column removal are preserved.

Eleven new cases cover naive/UTC timestamps, missing training values, prediction-only and entirely missing prediction batches, valid-date/input-preservation controls, and native GenericTask.validate_data/AutoML.preprocess integration for regression and classification. No estimator is fitted by these tests.

On Windows/Python 3.12.14, the identical final 28-case suite (11 new cases and 17 unchanged categorical preprocessing neighbors) gives 8 failed, 20 passed on pristine base c0aceba952a77c28f2cf921f9b1732214bfb7bfa and 28 passed after the fix, separately on actual pandas 2.3.3 and 3.1.0. All failures are the missing ordinal 1 versus the learned date median. Full pre-commit run --all-files (15 configured hooks) and git diff --check pass.

Existing datetime-accessor/drop deprecations on pandas 3.1 and a categorical downcast warning on pandas 2.3 remain visible. The separate deprecation work in #1648 is unchanged. This validates CPU preprocessing correctness; no estimator/model training, GPU/Spark/distributed, full optional-dependency suite, minimum-version matrix or performance claim is made.

Related issue number

Independently reproduced; no linked issue.

Checks

• I've used pre-commit to lint the changes in this PR.
• I've included any doc changes needed for https://microsoft.github.io/FLAML/. (Existing median-imputation behavior is restored; no API change.)
• I've added tests corresponding to the changes introduced in this PR.
• I've made sure all auto checks have passed. (Local focused validation passes; upstream CI not yet observed.)

Implementation and validation were performed with AI assistance.

[Copilot]

🟢 Approval recommended

No active review issues block approval of the datetime missing-value imputation changes.

No open issues 🎉

What changed in this PR

This PR restores median imputation for missing datetime features in FLAML’s tabular preprocessing, while preserving valid-date conversion and existing calendar features.

Changes:

• Preserve missing timestamps as np.nan during both training and prediction ordinal conversion.
• Add 11 test cases covering naive and UTC timestamps, missing-value scenarios, unchanged valid-date behavior, and regression/classification preprocessing integration.

File	Description
test/​automl/​test_datetime_missing.py	Adds datetime imputation regression tests and preprocessing integration coverage.
flaml/​automl/​data.py	Guards both ordinal conversions so missing timestamps reach the numeric imputer.

_{Review effort: Balanced}

[Li Jiang (thinkall)]

Approved after a clean whole-PR review of exact head 23046b00e6c21a47ca2eb96cdbe45724f390146b.

The matching fit/transform guards preserve missing timestamps for the existing training-fitted median imputer while leaving valid ordinals, generated calendar features and existing column-removal behavior intact. review-code Low execution with actual gpt-6.1-sol found no issues, and the complete source/test changes were also checked against the regression behavior.

Independent native validation passed the same 28 focused cases on each of two existing Windows stacks: Python 3.10.19/NumPy 1.26.4/pandas 2.3.3 and Python 3.13.5/NumPy 2.3.3/pandas 3.0.5. These are 56 executions of 28 distinct cases: 11 new datetime cases and 17 unchanged categorical neighbors. Against pristine main c0aceba9, the identical tests produce eight intended missing-ordinal failures and twenty passes on each stack; current fixes all eight, including regression/classification task-validation and preprocessing integration.

This validates native CPU preprocessing, not estimator training, GPU/Spark/distributed execution, the full optional/minimum-version matrix or performance. Author-reported pandas 3.1.0 is not being presented as driver-executed validation.

Immediately before approval, the same non-draft/non-self head was checked for resolved review threads, an explicit current-head Balanced Copilot Approval recommended verdict, successful applicable check contexts, and successfully completed actual Build, docs and Code formatting workflow runs. The skipped release job is explicitly inapplicable to pull requests, not counted as passed validation.

Posted by thinkall-agent-auto-reviewer