Conversation
…lbacks pl.Trainer was constructed with a hard-coded callbacks=[...] list followed by **trainer_kwargs, so an explicit fit(callbacks=[...]) collided and raised: TypeError: Trainer() got multiple values for keyword argument 'callbacks' The docs (config_system.md, training_and_evaluation.md) document callbacks as a Lightning trainer kwarg forwarded from fit(), and logger= already gets this override-merge treatment a few lines below -- callbacks had no equivalent path, so there was no supported way to attach a custom callback (LR logging, Optuna pruning, gradient accumulation schedulers, etc). Pop callbacks from trainer_kwargs and append it to the built-in EarlyStopping/ModelCheckpoint/ModelSummary list, mirroring how logger is already handled. This addresses one of several independent defects filed together in OpenTabular#452; the others (seed_context no-op, corrupted state after a failed fit, NODE data-aware init leaking validation data, fit(random_state=) being overridden by the constructor, predict(device=) being a no-op, and profile(dry_run=True) not fully restoring state) are unrelated code paths and are left to separate fixes. Addresses OpenTabular#452 (callbacks portion only) Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Calling
fit(callbacks=[...]), exactly asdocs/core_concepts/config_system.mdanddocs/core_concepts/training_and_evaluation.mddocument as a supported Lightning passthrough, crashes:One of several independent defects filed together in #452 — this PR addresses only the
callbacks=collision, since the others touch unrelated code paths (seeding, fit-failure state corruption, NODE initialization,predict(device=),profile(dry_run=True)).Root cause
deeptab/models/_mixins/fit.py'spl.Trainer(...)construction hard-codescallbacks=[early_stop_callback, checkpoint_callback, ModelSummary(...)]and then spreads**trainer_kwargsafter it. Anycallbacks=intrainer_kwargscollides with the hard-coded keyword argument.logger=a few lines below already handles this correctly by popping the user's value out oftrainer_kwargsand merging it in;callbackshad no equivalent handling, so there is no way to attach a custom callback (LR monitors, Optuna pruning, gradient-accumulation schedulers) despite the docs saying otherwise.Fix
Mirrors the existing
logger=trainer_kwargs.pop("logger", ...)pattern immediately below it. When nocallbacks=is passed, behavior is unchanged (empty list unpacks to nothing). When one is passed, it's appended after the built-ins instead of colliding with them.Testing
test_fit_accepts_user_supplied_callbacksintests/test_models.py, following the existingMLPRegressor/regression_data/FIT_KWARGSfixtures used throughout that file. It fits with an explicitcallbacks=[LearningRateMonitor()]and asserts both the user's callback and the built-in callbacks are present onmodel._trainer.callbacks.python3 -m py_compilepasses on both changed files.callbacks=" and "user passes nothing" cases — confirmed no collision and no change to default behavior.lightning/torchare not installed here) — please runpytest tests/test_models.py -k callbacksin CI/a configured dev environment to confirm.Addresses #452 (callbacks portion only — not closing, since the issue bundles 6 additional unrelated defects)
🤖 Generated with Claude Code