Skip to content

feat: add Gateway API foundation (GEP-1748) for Zero-Public-Exposure Ingress - #400

Draft
Ramkumar Chinchani (rchincha) wants to merge 10 commits into
Azure:mainfrom
rchincha:rchinchani/gep-1748-gateway-api
Draft

Ramkumar Chinchani (rchincha) wants to merge 10 commits into
Azure:mainfrom
rchincha:rchinchani/gep-1748-gateway-api

Conversation

@rchincha

@rchincha Ramkumar Chinchani (rchincha) commented Aug 18, 2026 •

Copy link
Copy Markdown

What type of PR is this?

feature

What this PR does / why we need it:

Implements the GEP-1748 using Azure AFD as the L7 ingress to achieve Zero-Public-Exposure Ingress solution for highly regulated customer environments.

Which issue(s) this PR fixes:

Fixes #

Requirements:

How has this code been tested

Special notes for your reviewer

@rchincha Ramkumar Chinchani (rchincha) changed the title Rchinchani/gep 1748 gateway api feat: add Gateway API controller foundation (GEP-1748) Aug 18, 2026
Ramkumar Chinchani (rchincha) added a commit to rchincha/fleet-networking that referenced this pull request Aug 18, 2026
Replace the removed Gateway API generated specification URL with the
version-pinned v1.2.1 type definitions used by this implementation.

This fixes the markdown-link-check failure in PR Azure#400 while keeping the
documentation reference aligned with the module version in go.mod.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 0d8af5d6-9aba-40a2-8efe-9bf09675667b
@codecov

codecov Bot commented Aug 18, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 86.60674% with 298 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.98%. Comparing base (76f455e) to head (05ea818).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
pkg/controllers/hub/gatewayresolver/route.go 70.83% 35 Missing and 14 partials ⚠️
pkg/controllers/hub/httproute/controller.go 65.09% 28 Missing and 9 partials ⚠️
cmd/member-net-controller-manager/main.go 32.43% 22 Missing and 3 partials ⚠️
pkg/providers/azure/frontdoor/backend_provision.go 87.56% 12 Missing and 12 partials ⚠️
pkg/controllers/hub/gateway/controller.go 89.84% 12 Missing and 8 partials ⚠️
pkg/controllers/hub/gateway/http_provisioning.go 82.10% 9 Missing and 8 partials ⚠️
pkg/controllers/hub/gatewayclass/controller.go 62.79% 12 Missing and 4 partials ⚠️
cmd/hub-gateway-controller-manager/main.go 85.98% 11 Missing and 4 partials ⚠️
pkg/controllers/hub/gateway/ownership.go 83.33% 6 Missing and 6 partials ⚠️
...kg/controllers/hub/gateway/reference_withdrawal.go 89.21% 6 Missing and 5 partials ⚠️
... and 15 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #400      +/-   ##
==========================================
+ Coverage   67.06%   76.98%   +9.92%     
==========================================
  Files          34       73      +39     
  Lines        3859     7165    +3306     
==========================================
+ Hits         2588     5516    +2928     
- Misses       1090     1301     +211     
- Partials      181      348     +167     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread pkg/annotations/afd.go Resolved
Comment thread pkg/annotations/afd.go Resolved
Comment thread pkg/annotations/afd.go Resolved
Define a fresh multi-cluster global ingress architecture that uses
GatewayClass, Gateway, and HTTPRoute while preserving the existing
networking.fleet.azure.com/v1alpha1 ServiceImport API.

Align the Fleet hub configuration-cluster model with GKE multi-cluster
Gateway guidance while retaining Fleet-specific ServiceImport support.
Document environment prerequisites, ownership, status, fail-static
behavior, repository placement, feasibility, and the phased tests-first
implementation plan.

Define optional annotations for Azure Front Door SKU, WAF attachment,
health probes, host headers, and public or Private Link connectivity.
Clarify that unknown or invalid explicit annotations must fail rather
than silently falling back to defaults.

Add separate operational guides for public LoadBalancer backends and
AFD Premium Private Link to internal LoadBalancer and PLS backends.
Identify the actor and placement for every hub, member-cluster, Azure,
WAF, PLS approval, DNS, traffic-weight, and cleanup action.

Document that the private AFD plus optional WAF plus PLS topology is
intended to support SFI-NS253, while public member LoadBalancer origins
are not that topology. Do not claim compliance until Private Link
reconciliation, enforcement, and end-to-end evidence are complete.

Pin references to the Gateway API v1.2.1 type definitions used by the
implementation and keep the public documentation free of internal URLs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 0d8af5d6-9aba-40a2-8efe-9bf09675667b
Pin Gateway API v1.2.1 to the repository's Kubernetes v0.31.1
baseline and register Gateway, HTTPRoute, ReferenceGrant, and the
unchanged Fleet ServiceImport types in a separate hub Gateway manager.

Add controller-wide Azure configuration validation, feature gating,
health and readiness endpoints, leader election, structured klog
events, and repository build targets. The foundation intentionally
registers no Azure reconcilers yet.

Add typed annotation parsing for AFD SKU, optional WAF attachment,
origin connectivity, health probes, and host headers. Reject unknown
reserved keys, malformed values, unsupported resource-group overrides,
and incompatible Private Link settings.

Add a provider-neutral normalized Gateway model with deterministic
ordering and separate validation for Gateway API backend weights and
Fleet ServiceExport per-cluster weights.

Focused tests cover manager startup and failure paths, annotations, and
the normalized model. Local race testing is deferred to CI because this
Windows host does not have gcc installed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 0d8af5d6-9aba-40a2-8efe-9bf09675667b
Install the selected Gateway API CRDs in the E2E hub and verify Gateway, HTTPRoute, ReferenceGrant, and Fleet ServiceImport interoperability. Document the managed-hub prerequisite and upgrade path.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Separate baseline networking, Traffic Manager, and current Gateway API contract coverage. Document the additional wiring required for future Azure Front Door reconciliation tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@rchincha Ramkumar Chinchani (rchincha) changed the title feat: add Gateway API controller foundation (GEP-1748) feat: add Gateway API foundation (GEP-1748) for Zero-Public-Exposure Ingress Oct 20, 2026
@rchincha

Ramkumar Chinchani (rchincha) commented Oct 24, 2026 •

Copy link
Copy Markdown
Author

https://github.com/rchincha/fleet-networking/blob/rchinchani/gep-1748-gateway-api/docs/howtos/gateway-api-afd-configuration.md

Controllers for the AFD private HTTP feature

Cluster Controller manager Relevant controllers / responsibility
Hub hub-gateway-controller-manager (new, separate deployment) GatewayClass, Gateway and HTTPRoute reconcilers. Manages AFD provisioning, approval observation, readiness, withdrawal and cleanup.
Hub hub-net-controller-manager (existing) Existing InternalServiceExport / ServiceImport processing supporting backend discovery and transport.
Each member member-net-controller-manager (existing, extended) Its serviceexport-controller discovers internal load balancer / Private Link Service metadata and publishes it through InternalServiceExport. No new member controller deployment.

Both feature gates default to off:

  • Hub: --enable-afd=true
  • Member: --enable-private-link-origin-discovery=true

These are implementation/component names—not confirmation that they are currently installed in your clusters.

Category Signed Commit ID Files Changed
Docs f71c000 12
Controllers, tests and telemetry 962c2d9 93
Build artifacts, packaging and CI 765444e 16
Live-test preparation 05ea818 11
Total 4 commits 132

@rchincha
Ramkumar Chinchani (rchincha) force-pushed the rchinchani/gep-1748-gateway-api branch 3 times, most recently from 8368905 to 05ea818 Compare October 24, 2026 04:18
Comment thread test/README.md

```bash
export AZURE_RESOURCE_GROUP=<YOUR-RESOURCE-GROUP-NAME>
export AZURE_SUBSCRIPTION_ID=<YOUR-SUBSCRIPTION-ID>
# Available values for AZURE_NETWORK_SETTING are shared-vnet, dynamic-ip-allocation and peered-vnet,
# and detailed explanations for each network setting are provided in the scripts under folder "test/scripts".
export AZURE_NETWORK_SETTING=shared-vnet
export ENABLE_TRAFFIC_MANAGER=true

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can the new AFD tests be added in a way that you just do

export AZURE_SUBSCRIPTION_ID=""
export AZURE_RESOURCE_GROUP=""
export AZURE_NETWORK_SETTING=shared-vnet
export ENABLE_TRAFFIC_MANAGER=false
export ENABLE_AZURE_FRONT_DOOR=true

and then

make e2e-setup
make e2e-tests
make e2e-cleanup

will test the existing scenarios and the new AFD scenario end to end?

This will help people validating that their new changes don't break the new AFD scenario

switch state := ptr.Deref(properties.DeploymentStatus, ""); state {
case armcdn.DeploymentStatusInProgress, armcdn.DeploymentStatusNotStarted:
return "", false, nil
case armcdn.DeploymentStatusSucceeded:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to run the test but the disabled endpoint never became DeploymentStatusSucceeded so everything after profile and endpoint are not getting created, such as route, origin groups, origin etc

Did this work for you?

Also wondering if Deployment Status Succeeded is a necessary gate. Someone on Microsoft Learn said no: https://learn.microsoft.com/en-us/answers/questions/5642906/azure-front-door-endpoint-deployment-stuck-at-nots

Gateway private HTTP proof [It] proves controller-owned private delivery, observed probes, withdrawal and cleanup [afd-private-http-proof]
/home/weiweng/fleet-networking/test/e2e/gateway_private_http_test.go:45

  [FAILED] Timed out after 600.000s.
  Expected success, but got an error:
      <*errors.errorString | 0x230d65eb4530>: 
      origin fleet-origin-e03c32ee1716ecbdb5a30cda2e9fecb3 is Missing, not an observed manual approval request
      {
          s: "origin fleet-origin-e03c32ee1716ecbdb5a30cda2e9fecb3 is Missing, not an observed manual approval request",
      }
  In [It] at: /home/weiweng/fleet-networking/test/e2e/gateway_private_http_test.go:116 @ 10/28/26 18:22:08.582

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also the Gateway status can be more specific to give more visibility to the users

For above issue, the condition is Programmed: False, and the message is "AFD private HTTP graph: Pending"

It seems that throughout the whole process, the message is either "AFD private HTTP graph: Pending" or "AFD private HTTP graph: Staging"

I wonder if the message can indicate that endpoint deployment status is "Not started"/"in progress" so we are waiting?

@rchincha
Ramkumar Chinchani (rchincha) force-pushed the rchinchani/gep-1748-gateway-api branch 12 times, most recently from 8d4e77e to 4b52ff6 Compare November 1, 2026 01:09
@rchincha
Ramkumar Chinchani (rchincha) force-pushed the rchinchani/gep-1748-gateway-api branch 3 times, most recently from aaa023b to 9bb4ff3 Compare November 2, 2026 00:38
Consolidate feature design, implementation plans, decisions, operator guidance, chart documentation and test runbooks. Document asynchronous Private Link approval, the ATM comparison and deferred live verification.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 19936d92-7c14-4bee-b1fd-70f2b4bc340d
Implement private-origin discovery and transport, Gateway API resolution and authorization, owned AFD lifecycle, full-graph readiness, withdrawal and cleanup. Include required API schemas and SDK dependencies, unit and local API-server tests, events and metrics.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 19936d92-7c14-4bee-b1fd-70f2b4bc340d
Add the separate Gateway image, default-disabled Helm packaging, member configuration, build/publication targets and Helm contracts. Verify real native amd64/arm64 images in CI, restricted nonroot entrypoints, default-disabled HTTP probes, loopback-only registry export/pull and both Linux architectures in the combined MCR manifest.

Current amd64 image build/runtime/local-registry proof, full repository-pinned lint, controller regressions, Helm contracts and workflow/ShellCheck validation pass locally. Native arm64 hosted execution and production publication remain unrun; no Azure, cluster or production-registry operations were performed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 19936d92-7c14-4bee-b1fd-70f2b4bc340d
Extend the existing Gateway E2E framework and sample workload for opt-in installed-controller private HTTP proof. Follow ATM intent/status/actual-ARM/cleanup patterns while adding private-only frontend, asynchronous approval, fresh HTTP identity/Host, probe-log and withdrawal assertions. Live execution remains deferred; existing ATM tests and E2E workflow are unchanged.

Allow 45-minute AFD convergence and cloud-cleanup waits with separate enclosing spec/cleanup deadlines and slower ARM polling. Preserve all success criteria and short controller request budgets; add offline timing-budget regression coverage.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 19936d92-7c14-4bee-b1fd-70f2b4bc340d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants