Conversation
AbstractFileStore.newTagManager() always built the TagManager for the main branch, while snapshotManager(), changelogManager() and AbstractFileStoreTable.tagManager() honor options.branch(). On a branch table this made ExpireSnapshotsImpl miss branch tags, so the manifest list and data files of a tagged snapshot were deleted; it also created automatic tags and _SUCCESS files under the main tag directory and made the rollback procedures fail to find branch tags. Pass options.branch() instead. Main tables are unaffected because options.branch() is "main" there. Generated-by: Claude Code
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
fix #9867
AbstractFileStore.newTagManager()usedDEFAULT_MAIN_BRANCH, whilesnapshotManager()andAbstractFileStoreTable.tagManager()honoroptions.branch(). Passoptions.branch(); main tables are unaffected.ExpireSnapshotsImplnow protects tagged snapshots instead of deleting their manifest list and data files.rollback_to/rollback_to_as_latestand Sparkrollbackprocedures find branch tags._SUCCESSfiles land in the branchtag/directory.StaticFromTagStartingScanner); the main-branch default dates from [Feature] Support Flink read / write data branch #3029.Tests
PrimaryKeySimpleTableTest#testExpireSnapshotsKeepsBranchTag: tags a branch snapshot, expires to one snapshot, asserts store and table tag directories match and the tag stays readable. Without the fix the directories differ and, past that assertion, the tag read hitsFileNotFoundExceptionon the manifest list.mvn -pl paimon-core -Dtest=PrimaryKeySimpleTableTest test— 136 tests passed.mvn -pl paimon-core clean install(checkstyle, spotless, enforcer, surefire) — 5437 tests, 0 failures; onlyPostgresqlCatalogTest(needs Docker) could not start locally.