Repository navigation
fix(automl): enable limit_resource on LRL1Classifier (#1612) - #1634
Abhinav Maurya (Fire162) wants to merge 6 commits into
Conversation
LRL1Classifier omitted limit_resource = True, causing budgeted training in AutoML.fit to bypass resource and timeout limits. This adds limit_resource = True matching LRL2Classifier and unit tests verifying that resource limits are applied during fit.
There was a problem hiding this comment.
🟡 Changes recommended
The new invocation test fails on Windows and needs to skip platforms without resource limiting.
1 open issue
What changed in this PR
Enables existing resource limiting for L1 logistic regression, addressing budget overruns reported in #1612.
Changes:
- Sets
LRL1Classifier.limit_resource = True, matching L2. - Adds tests for the flag, budgeted fitting, and limiter invocation.
| File | Description |
|---|---|
test/automl/test_model.py |
Adds L1 resource-limiting regression tests. |
flaml/automl/model.py |
Enables resource limiting for LRL1Classifier. |
Review effort: Balanced
|
|
||
|
|
||
| def test_lrl1_resource_limit_called(monkeypatch): | ||
| import flaml.automl.model as model_mod |
Li Jiang (thinkall)
left a comment
There was a problem hiding this comment.
Overall review of the complete PR: changes requested for a deterministic Windows test failure.
The LRL1 resource-limit flag matches LRL2 and reuses the existing production platform guard. The added invocation assertion does not respect that guard. Running the two new tests on Windows produced 1 passed and 1 failed: test_lrl1_resource_limit_called asserts a call even though resource is unavailable and BaseEstimator.fit intentionally bypasses the limiter. Please make that test capability-aware; do not remove the production guard. The review-code harness and current Balanced Copilot review agree on this blocker.
Posted by thinkall-agent-auto-reviewer
| X, y = make_classification(50, 5, random_state=0) | ||
| lr = LRL1Classifier() | ||
| lr.fit(X, y, budget=1.0) | ||
| assert len(called) > 0, "limit_resource was not invoked for LRL1Classifier" |
There was a problem hiding this comment.
This assertion fails on supported Windows platforms. model_mod.resource is None there, so BaseEstimator.fit deliberately does not enter limit_resource, regardless of the new LRL1 flag. I reproduced the failure with python -m pytest test/automl/test_model.py::test_lrl1 test/automl/test_model.py::test_lrl1_resource_limit_called -q (1 passed, 1 failed). Skip the invocation-specific case when resource is unavailable, or assert the appropriate call/no-call behavior for each capability while retaining the cross-platform fit smoke test.
Posted by thinkall-agent-auto-reviewer
Li Jiang (thinkall)
left a comment
There was a problem hiding this comment.
Re-reviewed commits 4d9553cefbb14233782ccfd928264bfde5ca739c -> 6d5a3a774153cf7836d651b90b73ad48e33694bb. The only delta is main's website/yarn.lock update of follow-redirects 1.16.0 -> 1.16.1. One explicitly head-bound review-code Low execution (actual runtime gpt-6.1-sol) found no new issue in this identical inherited patch. Controller Git comparisons verify all non-lockfile tree entries are unchanged. This is a scoped merge-delta review, not a fresh clean signoff for unchanged feature code.
Version/URL/caret/SRI shape checks pass. No independent local registry/tarball verification, package smoke test or website rebuild is claimed. Previous feature findings and runtime validation limits remain preserved.
Changes remain required. The existing Windows test blocker remains: test_lrl1_resource_limit_called still requires a limiter call when resource is unavailable and production deliberately bypasses that limiter. The test and production guard are unchanged by this merge. The prior Windows result was one passed and one failed; this cycle does not claim to rerun that native test. The original changes-requested review remains applicable.
At the 2026-10-26T12:13:44.943946+00:00 gate snapshot, actual current-head workflows were: Build in_progress/no conclusion; Code formatting completed/success; docs completed/success. Copilot has Balanced review without an Approval recommended verdict; 2 review thread(s) remain unresolved. Gate approval is not successful CI completion, and no new PR approval is being submitted while required gates remain unmet.
Posted by thinkall-agent-auto-reviewer

Why are these changes needed?
In
flaml/automl/model.py,SKLearnEstimator.fit()checksgetattr(self, "limit_resource", None)before wrapping execution in thelimit_resourcecontext manager. WhileLRL2Classifierdeclareslimit_resource = True,LRL1Classifieromitted this attribute.Because
lrl1is included in the default classification estimator list, trainingLRL1Classifierwith a specifiedtime_budgetbypassed resource and timeout limiting entirely, causing long-running fits to overrun budgets significantly on wide or many-class datasets.This change:
limit_resource = TruetoLRL1Classifierinflaml/automl/model.py, aligning it withLRL2Classifier.test/automl/test_model.pyverifying thatLRL1Classifier.limit_resourceisTrue, fit respects budget constraints, andlimit_resourceis invoked during execution.Related issue number
Closes #1612
Checks