Skip to content

HIVE-29819: Fix Iceberg COW UPDATE for null identity partition column filters #6763

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kasakrisz merged 1 commit into from
Sep 14, 2026
Merged

HIVE-29819: Fix Iceberg COW UPDATE for null identity partition column filters #6763

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions ...rg/iceberg-handler/src/test/queries/positive/update_iceberg_cow_null_identity_partition.q
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
-- SORT_QUERY_RESULTS

Comment thread
kasakrisz marked this conversation as resolved.
set hive.explain.user=false;

drop table if exists iceberg_cow_partitioned;

create external table iceberg_cow_partitioned (
index int,
string_col string,
boolean_col boolean,
str_col string,
tinyint_col int
) partitioned by spec(str_col, tinyint_col)
stored by iceberg
tblproperties ('write.update.mode'='copy-on-write');

insert into iceberg_cow_partitioned partition (str_col, tinyint_col)
values (1, 'a', true, null, 1);

explain update iceberg_cow_partitioned set str_col = 'UPDATED NULLS' where str_col is null;
update iceberg_cow_partitioned set str_col = 'UPDATED NULLS' where str_col is null;

select * from iceberg_cow_partitioned;

-- Disable vectorization

set hive.vectorized.execution.enabled=false;

insert into iceberg_cow_partitioned partition (str_col, tinyint_col)
values (2, 'b', false, null, 2);

explain update iceberg_cow_partitioned set str_col = 'UPDATED NULLS' where str_col is null;
update iceberg_cow_partitioned set str_col = 'UPDATED NULLS' where str_col is null;

select * from iceberg_cow_partitioned;
6 changes: 3 additions & 3 deletions iceberg/iceberg-handler/src/test/results/positive/iceberg_colstats_folded.q.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,12 +141,12 @@ STAGE PLANS:
TableScan
alias: ice_folded
filterExpr: ((id > 4) and (p = 'a')) (type: boolean)
Statistics: Num rows: 2 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 2 Data size: 376 Basic stats: COMPLETE Column stats: COMPLETE
Filter Operator
predicate: (id > 4) (type: boolean)
Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
Statistics: Num rows: 1 Data size: 4 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 1 Data size: 188 Basic stats: COMPLETE Column stats: COMPLETE
Group By Operator
aggregations: count()
minReductionHashAggr: 0.4
Expand Down
8 changes: 4 additions & 4 deletions iceberg/iceberg-handler/src/test/results/positive/iceberg_colstats_granularity.q.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ STAGE PLANS:
TableScan
alias: ice_p
filterExpr: (p = 'a') (type: boolean)
Statistics: Num rows: 2 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 2 Data size: 376 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: id (type: int), 'a' (type: string)
outputColumnNames: _col0, _col1
Expand Down Expand Up @@ -156,7 +156,7 @@ STAGE PLANS:
TableScan
alias: ice_p
filterExpr: (p = 'a') (type: boolean)
Statistics: Num rows: 2 Data size: 8 Basic stats: COMPLETE Column stats: PARTIAL
Statistics: Num rows: 2 Data size: 376 Basic stats: COMPLETE Column stats: PARTIAL
Select Operator
expressions: id (type: int), 'a' (type: string)
outputColumnNames: _col0, _col1
Expand Down Expand Up @@ -198,7 +198,7 @@ STAGE PLANS:
TableScan
alias: ice_p
filterExpr: (p = 'a') (type: boolean)
Statistics: Num rows: 2 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 2 Data size: 376 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: id (type: int), 'a' (type: string)
outputColumnNames: _col0, _col1
Expand Down Expand Up @@ -275,7 +275,7 @@ STAGE PLANS:
TableScan
alias: ice_p
filterExpr: (p = 'a') (type: boolean)
Statistics: Num rows: 2 Data size: 8 Basic stats: COMPLETE Column stats: COMPLETE
Statistics: Num rows: 2 Data size: 376 Basic stats: COMPLETE Column stats: COMPLETE
Select Operator
expressions: id (type: int), 'a' (type: string)
outputColumnNames: _col0, _col1
Expand Down
4 changes: 2 additions & 2 deletions iceberg/iceberg-handler/src/test/results/positive/iceberg_insert_into_partition.q.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -267,7 +267,7 @@ STAGE PLANS:
TableScan
alias: ice_parquet_int
filterExpr: (pcol = 2) (type: boolean)
Statistics: Num rows: 2 Data size: 182 Basic stats: COMPLETE Column stats: PARTIAL
Statistics: Num rows: 2 Data size: 190 Basic stats: COMPLETE Column stats: PARTIAL
Select Operator
expressions: strcol (type: string), intcol (type: int), 3 (type: int)
outputColumnNames: _col0, _col1, _col2
Expand Down Expand Up @@ -374,7 +374,7 @@ STAGE PLANS:
TableScan
alias: ice_parquet_int
filterExpr: (pcol = 2) (type: boolean)
Statistics: Num rows: 2 Data size: 182 Basic stats: COMPLETE Column stats: PARTIAL
Statistics: Num rows: 2 Data size: 190 Basic stats: COMPLETE Column stats: PARTIAL
Select Operator
expressions: strcol (type: string), intcol (type: int), 4 (type: int)
outputColumnNames: _col0, _col1, _col2
Expand Down
4 changes: 2 additions & 2 deletions iceberg/iceberg-handler/src/test/results/positive/iceberg_insert_overwrite_partition.q.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ STAGE PLANS:
TableScan
alias: ice_parquet_int
filterExpr: (pcol = 2) (type: boolean)
Statistics: Num rows: 2 Data size: 182 Basic stats: COMPLETE Column stats: PARTIAL
Statistics: Num rows: 2 Data size: 190 Basic stats: COMPLETE Column stats: PARTIAL
Select Operator
expressions: strcol (type: string), intcol (type: int), 1 (type: int)
outputColumnNames: _col0, _col1, _col2
Expand Down Expand Up @@ -164,7 +164,7 @@ STAGE PLANS:
TableScan
alias: ice_parquet_int
filterExpr: (pcol = 2) (type: boolean)
Statistics: Num rows: 2 Data size: 376 Basic stats: COMPLETE Column stats: PARTIAL
Statistics: Num rows: 2 Data size: 384 Basic stats: COMPLETE Column stats: PARTIAL
Select Operator
expressions: strcol (type: string), intcol (type: int), 1 (type: int)
outputColumnNames: _col0, _col1, _col2
Expand Down
4 changes: 2 additions & 2 deletions iceberg/iceberg-handler/src/test/results/positive/iceberg_partition_pruner_cache_key.q.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -272,9 +272,9 @@ STAGE PLANS:
TableScan
alias: tbl_ice_pp_key
filterExpr: (a > 2) (type: boolean)
Statistics: Num rows: 8 Data size: 51110 Basic stats: COMPLETE Column stats: PARTIAL
Statistics: Num rows: 8 Data size: 32 Basic stats: COMPLETE Column stats: PARTIAL
Select Operator
Statistics: Num rows: 8 Data size: 51110 Basic stats: COMPLETE Column stats: PARTIAL
Statistics: Num rows: 8 Data size: 32 Basic stats: COMPLETE Column stats: PARTIAL
Group By Operator
aggregations: count()
minReductionHashAggr: 0.99
Expand Down
2 changes: 1 addition & 1 deletion iceberg/iceberg-handler/src/test/results/positive/iceberg_pcr_null_partition.q.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ STAGE PLANS:
TableScan
alias: ice_01
filterExpr: ds is null (type: boolean)
Statistics: Num rows: 1 Data size: 171 Basic stats: COMPLETE Column stats: PARTIAL
Statistics: Num rows: 1 Data size: 355 Basic stats: COMPLETE Column stats: PARTIAL
Select Operator
expressions: key (type: string), value (type: string), null (type: string)
outputColumnNames: _col0, _col1, _col2
Expand Down
Loading
Loading