Skip to content

[Bug]: GPUCluster common name label breaks DRA validator selector #2955

Description

@ajavanma

Describe the bug

On current main (4fdfb1db7ddb87ba8969c52c634a5aeff104ab5f), setting GPUCluster.spec.daemonsets.labels["app.kubernetes.io/name"] overrides the DRA validator's pod-template label while its DaemonSet selector remains fixed. The rendered selector therefore does not match the pod template, which violates Kubernetes' DaemonSet validation requirements.

The pod-label loop protects app and app.kubernetes.io/part-of, but omits the app.kubernetes.io/name key used by this DaemonSet's selector. Other operand templates select on app, which their loops already protect.

To Reproduce

  1. Add the following common labels to a GPUCluster:

    spec:
      daemonsets:
        labels:
          app.kubernetes.io/name: gpu-platform
          team: platform
  2. Render the DRA validation state using the existing newTestDRAValidationState, sampleGPUCluster and draSupportedCatalog test helpers and getManifestObjects.

  3. Convert the rendered DaemonSet selector using metav1.LabelSelectorAsSelector and compare it with the pod-template labels.

The real renderer produces this mismatch:

spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: nvidia-dra-validator
  template:
    metadata:
      labels:
        app.kubernetes.io/name: gpu-platform
        app: nvidia-operator-validator
        team: platform

A table-driven regression fails for the name collision and combined reserved-label cases. Default labels, an ordinary custom label, and the existing app/part-of protections pass.

Expected behavior

The pod-template label required by the immutable selector should remain nvidia-dra-validator. Unrelated custom labels should remain present, and the upgrade-controller label app: nvidia-operator-validator should remain protected. The selector itself should not change.

Proposed fix: extend the existing pod-label filter to protect app.kubernetes.io/name; retain current common-label behavior on the DaemonSet's own metadata. I plan to submit the focused fix with regression coverage.

Environment (please provide the following information):

  • GPU Operator: main at 4fdfb1db7ddb87ba8969c52c634a5aeff104ab5f
  • Reproduction: repository renderer tests on macOS ARM64, Go 1.26.5
  • Kernel/container runtime/Kubernetes distribution: not applicable to the renderer reproduction
  • No live GPU deployment or API-server rejection was exercised in this initial reproduction.

Information to attach

Cluster logs and a must-gather bundle are not applicable to this source-level renderer reproduction.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions