Skip to content

fix(lake-formation-tag-sync): match views when syncing LF tags - #2804

Draft
vitaliano-atlan wants to merge 1 commit into
atlanhq:mainfrom
vitaliano-atlan:csa-614-lftag-views
Draft

vitaliano-atlan wants to merge 1 commit into
atlanhq:mainfrom
vitaliano-atlan:csa-614-lftag-views

Conversation

@vitaliano-atlan

Copy link
Copy Markdown

Summary

Lake Formation tag sync now tags views. Before this change, every view in a tagged schema was skipped and the run still reported success.

CSVProducer writes every table-level row with type Table, because the Lake Formation feed has no table type. Lake Formation lists views as tables too. The package imports in update-only mode, and AssetBatch matches on type plus qualifiedName, so a row that points at an existing View never matches and is skipped.

The asset importer already supports this case through assets_table_view_agnostic. This change turns it on for the package. A Table row that is not found as a Table is then matched as a View, MaterializedView or SnowflakeDynamicTable.

Changes

  • LakeTagSynchronizer: pass assetsTableViewAgnostic = true to AssetImportCfg.
  • LakeTagSynchronizerTest: create View vw1 with Column vcol1, and assert that both are tagged.
  • lftag_association_1.json: add a sch.vw1 record with no type field, the same shape as a real feed.

Testing

I ran LakeTagSynchronizerTest with -PpackageTests against an internal sandbox tenant.

Without the change, the import skips the view and validateViewTagged fails:

WARN  aim.Importer -  ... skipped asset: Table::default/minisql/1791322652/db_test/sch/vw1
WARN  aim.Importer - Total assets skipped : 1
validateViewTagged -> java.lang.AssertionError: expected [public] but found [null]

With the change, all 6 tests pass and every batch logs Total assets skipped : 0. I also opened the test view in the UI before teardown. It showed the mapped custom metadata values.

validateViewColumnTagged passes with and without the change. Column rows are already typed Column, so view columns were matched before. Only the view row itself was lost.

Not in this PR

  • Skipped assets still do not affect the run status (CSVReader.someFailure counts failures only).
  • One connection-map key still resolves to exactly one connection.
  • A Lake Formation database name that differs from the Redshift schema name still misses.

Linear: CSA-614

🤖 Generated with Claude Code

Lake Formation lists views as tables and the tag feed carries no table
type, so every table-level row is written as Table. In update-only mode
AssetBatch matches on type plus qualifiedName, so rows for existing
views were skipped while the run still reported success.

Turn on assetsTableViewAgnostic so a Table row that is not found as a
Table matches a View, MaterializedView or SnowflakeDynamicTable.
Extend LakeTagSynchronizerTest with a view and a view column.

Refs CSA-614

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant