Skip to content

Merge PR Differential privacy for aggregates - #2617

Open
ywcb00 wants to merge 52 commits into
apache:mainfrom
ywcb00:pr-2539
Open

ywcb00 wants to merge 52 commits into
apache:mainfrom
ywcb00:pr-2539

Conversation

@ywcb00

@ywcb00 ywcb00 commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Intermediate PR for merging #2539.

(If anybody knows a better way to verify the CI tests when merging larger PRs with multiple changes that were made during the merge, please inform me.)

Maya Anderson added 28 commits September 18, 2026 15:10
…ransformation matrix T internally, returning T %*% X with noise fused into a single matrix multiply.
Lets a DML script declare its session-wide differential-privacy budget once at the top, instead of always falling back to the hardcoded default.
Resolved entirely at compile time: epsilon/delta must be literals, validated in BuiltinFunctionExpression and stored on DMLProgram during HOP construction, then read by ExecutionContext.getDPBudgetAccountant().
Four federated workers simulated on localhost, a logistic regression FedAvg loop in DML where the coordinator applies dp_gaussian to the aggregated gradient, a sweep over ε ∈ {0.5, 1, 4, 8} plus a non-private baseline, and a matplotlib accuracy-vs-ε plot saved as a PNG.

Add clip_norm (default 4.0) as a script parameter. Inside the private == 1 branch, each row's gradient contribution is clipped to L2-norm less than clip_norm.
…ntical dp_laplace or dp_gaussian calls are never be merged into one execution.
Flip Builtins.DP_LAPLACE/DP_GAUSSIAN to parameterized=true so the parser
builds a ParameterizedBuiltinFunctionExpression for these calls instead of
a positional BuiltinFunctionExpression. This lets them reuse the existing
varParams-based parsing, instead of hand-unpacking expr/expr2/expr3 by position in
DMLTranslator and re-deriving parameter names from argument order in
BuiltinFunctionExpression. Laplace and Gaussian validation is merged into
one validateDpMechanism(), varying only on whether 'delta' is required,
since the two mechanisms differ by exactly that one optional parameter.
Maya Anderson and others added 23 commits September 18, 2026 15:10
…nCPInstruction, owning parse validation, processInstruction(), and the lineage-refusal — all DP instruction-level concerns now live in one file
…hich defines two global constants

	define these constants as environment variables instead
	remove unnecessary comments and shorten method headers
@codecov

codecov Bot commented Sep 18, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.91%. Comparing base (b42b641) to head (8d74228).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2617      +/-   ##
==========================================
- Coverage   74.94%   74.91%   -0.04%     
==========================================
  Files         436      436              
  Lines       25237    25237              
==========================================
- Hits        18915    18906       -9     
- Misses       6322     6331       +9     
Flag Coverage Δ
python 74.91% <ø> (-0.04%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

… parsing tests

fix(test/functions/privacy/dp/DPBuiltinDMLTest.java): use dml script files for test execution

feat(test/scripts/functions/privacy/dp/**): create dml test scripts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

1 participant