Skip to content

About

MultiNIC Agent for Kubernetes

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

ย 

History

394 Commits

Folders and files

NameName
Last commit message
Last commit date
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

Multi Network Interface Agent

Kubernetes ๋„ค์ดํ‹ฐ๋ธŒ ๋„คํŠธ์›Œํฌ ์ž๋™ํ™” ์—์ด์ „ํŠธ

OpenStack ํ™˜๊ฒฝ์—์„œ Kubernetes ๋…ธ๋“œ์˜ ๋‹ค์ค‘ ๋„คํŠธ์›Œํฌ ์ธํ„ฐํŽ˜์ด์Šค๋ฅผ ์™„์ „ ์ž๋™์œผ๋กœ ๊ด€๋ฆฌํ•˜๋Š” Controller + Job ๊ธฐ๋ฐ˜ ์‹œ์Šคํ…œ์ž…๋‹ˆ๋‹ค.

๊ฐœ์š”

ํ•ต์‹ฌ ํŠน์ง•

  • ๋‹จ์ผ ์ด๋ฏธ์ง€, ์ด์ค‘ ์‹คํ–‰: ํ•˜๋‚˜์˜ ์ปจํ…Œ์ด๋„ˆ ์ด๋ฏธ์ง€์—์„œ Controller/Agent ๋ชจ๋“œ๋กœ ๋™์ž‘
  • ์ž๋™ํ™” ์›Œํฌํ”Œ๋กœ์šฐ: CR ์ƒ์„ฑ/์ˆ˜์ • ์‹œ ์ฆ‰์‹œ ํ•ด๋‹น ๋…ธ๋“œ์— Agent Job ์Šค์ผ€์ค„๋ง
  • ๋…ธ๋“œ๋ณ„ ๋งž์ถค ์‹คํ–‰: ๊ฐ ๋…ธ๋“œ์˜ SystemUUID ๊ฒ€์ฆ ํ›„ ๋„คํŠธ์›Œํฌ ์ธํ„ฐํŽ˜์ด์Šค ์ž๋™ ์„ค์ •
  • ์‹ค์‹œ๊ฐ„ ์ƒํƒœ ๋™๊ธฐํ™”: Job ์™„๋ฃŒ ํ›„ Controller๊ฐ€ ์ž๋™์œผ๋กœ CR status ์—…๋ฐ์ดํŠธ
  • ๋ผ์šฐํŒ… ์ถฉ๋Œ ๋ฐฉ์ง€: ์ „์—ญ ๋ผ์šฐํŒ… ์ง๋ ฌํ™”๋กœ ๋„คํŠธ์›Œํฌ ํ…Œ์ด๋ธ” ์•ˆ์ •์„ฑ ๋ณด์žฅ
  • ์„ฑ๋Šฅ ์ตœ์ ํ™”: ์•ˆ์ •์„ฑ ์šฐ์„  ๋™์‹œ์„ฑ ์ œ์–ด (๊ธฐ๋ณธ 1๊ฐœ ์ž‘์—…, ์„ค์ • ๊ฐ€๋Šฅ)

๋™์ž‘ ๋ฐฉ์‹

  1. Controller (Deployment): CR ๋ณ€๊ฒฝ์‚ฌํ•ญ์„ ์‹ค์‹œ๊ฐ„ ๊ฐ์‹œ
  2. Agent (Job): ํŠน์ • ๋…ธ๋“œ์—์„œ๋งŒ ์‹คํ–‰๋˜์–ด ๋„คํŠธ์›Œํฌ ์ธํ„ฐํŽ˜์ด์Šค ์„ค์ •
  3. ์ž๋™ ์Šค์ผ€์ค„๋ง: CR ์—…๋ฐ์ดํŠธ โ†’ ํ•ด๋‹น ๋…ธ๋“œ์šฉ Agent Job ์ƒ์„ฑ โ†’ ๋„คํŠธ์›Œํฌ ๊ตฌ์„ฑ โ†’ ์ƒํƒœ ์—…๋ฐ์ดํŠธ

๊ฒฐ์ • ์‚ฌํ•ญ(์šด์˜ ์ฒ ํ•™)

  • ๋Ÿฐํƒ€์ž„ ์ ์šฉ์€ ip ๊ธฐ๋ฐ˜์œผ๋กœ ์ฆ‰์‹œ ๋ฐ˜์˜(์ด๋ฆ„/MTU/IPv4/๋ผ์šฐํŠธ)
  • ์˜์†์„ฑ์€ OS๋ณ„ ํŒŒ์ผ โ€œ์ž‘์„ฑ๋งŒโ€ ์ˆ˜ํ–‰(์ฆ‰์‹œ netplan apply/nmcli reload ํ˜ธ์ถœ ์—†์Œ)
  • Ubuntu: netplan YAML์— match.macaddress + set-name ํฌํ•จ์œผ๋กœ ์ด๋ฆ„ ์˜์†
  • RHEL: .link(systemd-udev, ์ด๋ฆ„ ์˜์†) + .nmconnection(NetworkManager, ๊ถŒํ•œ 600) ์ž‘์„ฑ, Helm์ด /etc/systemd/network๋„ ๋งˆ์šดํŠธ
  • Preflight: UP NIC์ด๋ผ๋„ IPv4/๋ผ์šฐํŠธ/๋งˆ์Šคํ„ฐ ์†Œ์†์ด ์—†์œผ๋ฉด ํ—ˆ์šฉ; ์šฐํšŒ ํ”Œ๋ž˜๊ทธ PREFLIGHT_ALLOW_UP ์ œ๊ณต
  • ๋ผ์šฐํŒ…/๊ธฐ๋ณธ๊ฒฝ๋กœ ๋ณ€๊ฒฝ์€ ์ „์—ญ ์ง๋ ฌํ™”

ํ˜„์žฌ ๋กœ์ง ํ๋ฆ„

์‹œ์Šคํ…œ ์•„ํ‚คํ…์ฒ˜

graph TB
  %% ์ƒ๋‹จ: MGMT Cluster
  subgraph "MGMT Cluster"
    Operator[Operator]
    ViolaAPI[Viola API]
  end

  %% Biz Cluster
  subgraph "Kubernetes Cluster (Biz)"
    subgraph "CR ์ฒ˜๋ฆฌ"
      MultiNICController[MultiNIC Controller\nCR Watch]
      NodeCR[MultiNicNodeConfig CR\n๋…ธ๋“œ๋ณ„ Interface ๋ฐ์ดํ„ฐ]
    end

    subgraph "Job ์‹คํ–‰"
      Job1[Agent Job\nNode1 ๋Œ€์ƒ]
      Job2[Agent Job\nNode2 ๋Œ€์ƒ]
      Job3[Agent Job\nNode3 ๋Œ€์ƒ]
    end

    subgraph "Worker Nodes"
      Node1[Node1\nSystemUUID: b4975c5f-50bb]
      Node2[Node2\nSystemUUID: d4defd76-faa9]
      Node3[Node3\nSystemUUID: a1b2c3d4-e5f6]
    end
  end

  %% Network Interfaces
  subgraph "Network Interfaces"
    NIC1[Node1: multinic0, multinic1]
    NIC2[Node2: multinic0]
    NIC3[Node3: multinic0, multinic1, multinic2]
  end

  %% ๋ฐ์ดํ„ฐ ํ๋ฆ„
  Operator -->|์ธํ„ฐํŽ˜์ด์Šค ์ •๋ณด POST| ViolaAPI
  ViolaAPI -->|CR ์ƒ์„ฑ| NodeCR
  NodeCR -.->|Watch Event| MultiNICController
  MultiNICController -->|Node๋ณ„ Job ์Šค์ผ€์ค„๋ง| Job1
  MultiNICController -->|Node๋ณ„ Job ์Šค์ผ€์ค„๋ง| Job2
  MultiNICController -->|Node๋ณ„ Job ์Šค์ผ€์ค„๋ง| Job3
  Job1 -->|ip ์ ์šฉ + ์˜์† ํŒŒ์ผ ์ž‘์„ฑ| Node1
  Job2 -->|ip ์ ์šฉ + ์˜์† ํŒŒ์ผ ์ž‘์„ฑ| Node2
  Job3 -->|ip ์ ์šฉ + ์˜์† ํŒŒ์ผ ์ž‘์„ฑ| Node3
  Node1 -->|์ธํ„ฐํŽ˜์ด์Šค ์ƒ์„ฑ| NIC1
  Node2 -->|์ธํ„ฐํŽ˜์ด์Šค ์ƒ์„ฑ| NIC2
  Node3 -->|์ธํ„ฐํŽ˜์ด์Šค ์ƒ์„ฑ| NIC3

  %% ์Šคํƒ€์ผ(๋ Œ๋”๋Ÿฌ ๋ฏธ์ง€์› ์‹œ ์ œ๊ฑฐ ๊ฐ€๋Šฅ)
  classDef mgmt fill:#e8f5e8
  classDef controller fill:#f3e5f5
  classDef cr fill:#fff3e0
  classDef job fill:#ffecb3
  classDef node fill:#fafafa
  classDef nic fill:#ffcdd2

  class Operator,ViolaAPI mgmt
  class MultiNICController controller
  class NodeCR cr
  class Job1,Job2,Job3 job
  class Node1,Node2,Node3 node
  class NIC1,NIC2,NIC3 nic
Loading

์ฒ˜๋ฆฌ ์›Œํฌํ”Œ๋กœ์šฐ

sequenceDiagram
    participant ์šด์˜ as Config Source
    participant API as Kubernetes API
    participant ์ปจํŠธ๋กค๋Ÿฌ as Controller
    participant ์žก as Agent Job
    participant ๋…ธ๋“œ as Target Node

    ์šด์˜->>API: CR ์ƒ์„ฑ/์ˆ˜์ • (MAC, IP, CIDR, MTU)
    note over ์šด์˜,API: ์™ธ๋ถ€ API๋„ CR์„ ์ง์ ‘ ์ƒ์„ฑ/๊ฐฑ์‹ ํ•  ์ˆ˜ ์žˆ์Œ
    API-->>์ปจํŠธ๋กค๋Ÿฌ: Watch ์ด๋ฒคํŠธ ์ „๋‹ฌ
    ์ปจํŠธ๋กค๋Ÿฌ->>API: ๋…ธ๋“œ ์ •๋ณด ์กฐํšŒ(osImage/SystemUUID)
    ์ปจํŠธ๋กค๋Ÿฌ->>API: Agent Job ์ƒ์„ฑ(nodeSelector)

    API->>์žก: ๋Œ€์ƒ ๋…ธ๋“œ์—์„œ ์‹คํ–‰
    ์žก->>๋…ธ๋“œ: Preflight (MAC ํ™•์ธ, ๋ฏธ์‚ฌ์šฉ NIC ํ—ˆ์šฉ)
    ์žก->>๋…ธ๋“œ: ip ๊ธฐ๋ฐ˜ ์ ์šฉ(์ด๋ฆ„, MTU, IPv4, ๋ผ์šฐํŠธ)
    ์žก->>๋…ธ๋“œ: ์˜์† ํŒŒ์ผ๋งŒ ์ž‘์„ฑ(Ubuntu: netplan, RHEL: .link + .nmconnection)
    ์žก->>๋…ธ๋“œ: ๊ฒ€์ฆ ๋ฐ termination log ๊ธฐ๋ก

    ์žก-->>์ปจํŠธ๋กค๋Ÿฌ: ์š”์•ฝ ์ „๋‹ฌ
    ์ปจํŠธ๋กค๋Ÿฌ->>API: CR ์ƒํƒœ ์—…๋ฐ์ดํŠธ(Configured/Failed)
    ์ปจํŠธ๋กค๋Ÿฌ->>API: Job ์ •๋ฆฌ(TTL)
Loading

Agent Job ๋™์ž‘ ๋ฐ ์•ˆ์ •์„ฑ

๋„คํŠธ์›Œํฌ ๊ตฌ์„ฑ ํ”„๋กœ์„ธ์Šค

  • ์‹œ์ž‘ ์‹œ ์ •๋ฆฌ ์ˆ˜ํ–‰(RUN_MODE=job):

    • Ubuntu: /etc/netplan/9*-multinic*.yaml ๊ณ ์•„ ํŒŒ์ผ๋งŒ ์‚ญ์ œ(์ฆ‰์‹œ netplan apply๋Š” ํ˜ธ์ถœํ•˜์ง€ ์•Š์Œ)
    • RHEL: RHEL9+์—์„œ๋Š” /etc/sysconfig/network-scripts๊ฐ€ ์—†์„ ์ˆ˜ ์žˆ์œผ๋ฏ€๋กœ .nmconnection ๊ณ ์•„ ํŒŒ์ผ๋งŒ ์ •๋ฆฌํ•˜๊ณ  ๋””๋ ‰ํ„ฐ๋ฆฌ ๋ถ€์žฌ๋Š” ๋ฌด์‹œ
    • ์‹œ์Šคํ…œ ๊ธฐ๋ณธ ํŒŒ์ผ(50-cloud-init.yaml ๋“ฑ)์€ ๊ฑด๋“œ๋ฆฌ์ง€ ์•Š์Œ
    • ๋‚จ์•„์žˆ๋Š” multinic0~9 ์ธํ„ฐํŽ˜์ด์Šค๋Š” DOWN ์ƒํƒœ์ผ ๋•Œ๋งŒ altname(ens*/enp*)์œผ๋กœ rename ์‹œ๋„(์—†์œผ๋ฉด ์Šคํ‚ต)
  • ์ด๋ฆ„ ์ถฉ๋Œ ๋ฐฉ์ง€(์‚ฌ์ „ ๋ฐฐ์ •): ์‹คํ–‰ ์‹œ์ž‘ ์‹œ MACโ†’multinicX ์ด๋ฆ„์„ ๋ฏธ๋ฆฌ ๋ฐฐ์ •ํ•ด ์ค‘๋ณต ์ด๋ฆ„ ์ถฉ๋Œ์„ ์ œ๊ฑฐ

  • ๊ฒ€์ฆ ๋ฐฉ์‹ ์ „ํ™˜(์ด๋ฆ„โ†’MAC):

    • ์ ์šฉ ํ›„ ๊ฒ€์ฆ์€ ip -o link show ์ „์ฒด์—์„œ CR์˜ MAC ์กด์žฌ ์—ฌ๋ถ€๋กœ ํŒ๋‹จ(ํŠน์ • ์ด๋ฆ„์— ์˜์กดํ•˜์ง€ ์•Š์Œ)
  • ์ฒ˜๋ฆฌ ์ˆœ์„œ: "์ •๋ฆฌ โ†’ ์„ค์ •(์ ์šฉ) โ†’ ๊ฒ€์ฆ"์œผ๋กœ ์‹คํ–‰

์•ˆ์ •์„ฑ ๊ธฐ๋Šฅ

  • ๋ผ์šฐํŒ… ์ถฉ๋Œ ๋ฐฉ์ง€:

    • ์ „์—ญ mutex๋ฅผ ํ†ตํ•œ ๋ผ์šฐํŒ… ํ…Œ์ด๋ธ” ์ง๋ ฌํ™”
    • ๋™์‹œ ๋„คํŠธ์›Œํฌ ์„ค์ •์œผ๋กœ ์ธํ•œ ๋ผ์šฐํŒ… ํ…Œ์ด๋ธ” ๊ฒฝ์Ÿ ์ƒํƒœ ๋ฐฉ์ง€
    • ๋ผ์šฐํŒ… ์ž‘์—… ๋ฉ”ํŠธ๋ฆญ ์ˆ˜์ง‘ (์‹คํ–‰ ์‹œ๊ฐ„, ์„ฑ๊ณต/์‹คํŒจ์œจ)
  • ๋™์‹œ์„ฑ ์ œ์–ด ์ตœ์ ํ™”:

    • ๊ธฐ๋ณธ ์ตœ๋Œ€ ๋™์‹œ ์ž‘์—… ์ˆ˜: 1๊ฐœ (์•ˆ์ •์„ฑ ์šฐ์„ )
  • Helm values๋ฅผ ํ†ตํ•œ ์„ค์ • ๊ฐ€๋Šฅ (agent.maxConcurrentTasks)

    • ๋Œ€๊ทœ๋ชจ ํ™˜๊ฒฝ์—์„œ ๋ผ์šฐํŒ… ์ถฉ๋Œ ์ตœ์†Œํ™”

๊ถŒ์žฅ ๋ฐฐํฌ ์„ค์ • (์•ˆ์ •์„ฑ ์šฐ์„ )

# ๊ธฐ๋ณธ ์„ค์ • (์•ˆ์ •์„ฑ ์ตœ์šฐ์„ )
helm upgrade --install multinic-agent ./deployments/helm \
  -n multinic-system \
  --set image.tag=1.0.2 \
  --set agent.maxConcurrentTasks=1

# ๋Œ€๊ทœ๋ชจ ํ™˜๊ฒฝ (์„ฑ๋Šฅ ์šฐ์„ ์‹œ)
helm upgrade --install multinic-agent ./deployments/helm \
  -n multinic-system \
  --set image.tag=1.0.2 \
  --set agent.maxConcurrentTasks=3

์ˆ˜๋™ ์ „์ฒด ์ •๋ฆฌ(์˜ต์…˜):

# ์ปจํŠธ๋กค๋Ÿฌ๊ฐ€ ์ƒ์„ฑํ•˜๋Š” Job์— ํ™˜๊ฒฝ๋ณ€์ˆ˜๋กœ ์ „๋‹ฌ๋˜๋ฉด ๋ชจ๋“  multinic ํŒŒ์ผ๋งŒ ์ •๋ฆฌ
AGENT_ACTION=cleanup

๋™์ผ CIDR ๋ฉ€ํ‹ฐ NIC ์•ˆ์ „์žฅ์น˜

  • ๊ธฐ๋ณธ: ์†Œ์Šค ๊ธฐ๋ฐ˜ ์ •์ฑ… ๋ผ์šฐํŒ… + noprefixroute ์ ์šฉ โ†’ main ํ…Œ์ด๋ธ”/ens3 ๊ธฐ๋ณธ ๋ผ์šฐํŠธ ์œ ์ง€
  • ARP ํ”Œ๋Ÿญ์Šค/RPF ์™„ํ™”: arp_ignore=1, arp_announce=2, rp_filter=2 ๋ฅผ ์ธํ„ฐํŽ˜์ด์Šค ๋‹จ์œ„๋กœ ์ ์šฉ
  • ์„ค์ •๊ฐ’(ํ™˜๊ฒฝ๋ณ€์ˆ˜/Helm agent.network.*):
    • NETWORK_POLICY_ROUTING_ENABLED(default: true)
    • NETWORK_ROUTING_TABLE_BASE(default: 100), NETWORK_ROUTE_METRIC(default: 100)
    • NETWORK_NOPREFIXROUTE(default: true)
    • NETWORK_SET_ARP_SYSCTLS(default: true), NETWORK_SET_RP_FILTER_LOOSE(default: true)

ํŒจํ‚ค์ง€ ๊ตฌ์กฐ

multinic-agent/
โ”œโ”€โ”€ cmd/
โ”‚   โ”œโ”€โ”€ agent/                 # Agent Job ๋ฐ”์ด๋„ˆ๋ฆฌ
โ”‚   โ””โ”€โ”€ controller/            # Controller ๋ฐ”์ด๋„ˆ๋ฆฌ
โ”œโ”€โ”€ internal/                  # Clean Architecture
โ”‚   โ”œโ”€โ”€ domain/               # ๋„๋ฉ”์ธ ๊ณ„์ธต
โ”‚   โ”‚   โ”œโ”€โ”€ entities/         # NetworkInterface, InterfaceName
โ”‚   โ”‚   โ”œโ”€โ”€ interfaces/       # Repository, Network ์ธํ„ฐํŽ˜์ด์Šค
โ”‚   โ”‚   โ””โ”€โ”€ services/         # InterfaceNamingService, RoutingCoordinator
โ”‚   โ”œโ”€โ”€ application/          # ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜ ๊ณ„์ธต
โ”‚   โ”‚   โ””โ”€โ”€ usecases/        # ConfigureNetwork, DeleteNetwork
โ”‚   โ”œโ”€โ”€ infrastructure/       # ์ธํ”„๋ผ์ŠคํŠธ๋Ÿญ์ฒ˜ ๊ณ„์ธต
โ”‚   โ”‚   โ”œโ”€โ”€ persistence/     # NodeCR Repository (K8s CR ๊ธฐ๋ฐ˜)
โ”‚   โ”‚   โ”œโ”€โ”€ network/         # Netplan, RHEL Adapter
โ”‚   โ”‚   โ”œโ”€โ”€ metrics/         # Prometheus ๋ฉ”ํŠธ๋ฆญ ์ˆ˜์ง‘
โ”‚   โ”‚   โ””โ”€โ”€ config/         # ์„ค์ • ๊ด€๋ฆฌ
โ”‚   โ””โ”€โ”€ controller/          # Controller ๊ตฌํ˜„
โ”‚       โ”œโ”€โ”€ reconciler.go   # CR ์ฒ˜๋ฆฌ ๋กœ์ง
โ”‚       โ”œโ”€โ”€ watcher.go      # Watch ์ด๋ฒคํŠธ ์ฒ˜๋ฆฌ
โ”‚       โ””โ”€โ”€ service.go      # Controller ์„œ๋น„์Šค
โ”œโ”€โ”€ deployments/
โ”‚   โ”œโ”€โ”€ crds/               # CRD ์ •์˜ ๋ฐ ์ƒ˜ํ”Œ
โ”‚   โ””โ”€โ”€ helm/              # Helm ์ฐจํŠธ
โ””โ”€โ”€ scripts/               # ๋ฐฐํฌ ์ž๋™ํ™”

๐Ÿ”ง CRD ์„ค๊ณ„

MultiNicNodeConfig CRD ์Šคํ‚ค๋งˆ

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: multinicnodeconfigs.multinic.io
spec:
  group: multinic.io
  versions:
  - name: v1alpha1
    served: true
    storage: true
    schema:
      openAPIV3Schema:
        type: object
        properties:
          spec:
            type: object
            properties:
              nodeName:
                type: string
                description: "Target Kubernetes node name"
              instanceId:
                type: string
                description: "OpenStack Instance UUID"
              interfaces:
                type: array
                items:
                  type: object
                  properties:
                    id:
                      type: integer
                    name:
                      type: string
                    macAddress:
                      type: string
                    address:
                      type: string
                    cidr:
                      type: string
                    mtu:
                      type: integer
          status:
            type: object
            properties:
              state:
                type: string
                enum: ["Pending", "InProgress", "Configured", "Failed"]
              lastProcessed:
                type: string
              interfaceStatuses:
                type: array
                items:
                  type: object
                  properties:
                    name:
                      type: string

์˜ˆ์‹œ CR ์ ์šฉ

apiVersion: multinic.io/v1alpha1
kind: MultiNicNodeConfig
metadata:
  name: viola2-biz-worker01
  namespace: multinic-system
  labels:
    multinic.io/node-name: viola2-biz-worker01
    multinic.io/instance-id: b4975c5f-50bb-479f-9e7b-a430815ae852
spec:
  nodeName: viola2-biz-worker01
  instanceId: b4975c5f-50bb-479f-9e7b-a430815ae852
  interfaces:
    - id: 0
      name: multinic0
      macAddress: fa:16:3e:1c:1a:6e
      address: 11.11.11.37
      cidr: 11.11.11.0/24
      mtu: 1450
    - id: 1
      name: multinic1
      macAddress: fa:16:3e:0a:17:3b
      address: 11.11.11.148
      cidr: 11.11.11.0/24
      mtu: 1450

name์€ ์„ ํƒ์‚ฌํ•ญ์ด์ง€๋งŒ, ์„ค์ • ์‹œ id๋Š” name์˜ ์ธ๋ฑ์Šค๋กœ ํ•ด์„๋ฉ๋‹ˆ๋‹ค. id๋Š” 09 ๋ฒ”์œ„์ด๋ฉฐ name(multinic09)๊ณผ ๋™์ผํ•œ ์ธ๋ฑ์Šค๋กœ ๋งž์ถ”๋Š” ๊ฒƒ์„ ๊ถŒ์žฅํ•ฉ๋‹ˆ๋‹ค.

๋ฐฐํฌ ๋ฐฉ๋ฒ•

1. SSH ํŒจ์Šค์›Œ๋“œ ์„ค์ •

# deploy.sh ์Šคํฌ๋ฆฝํŠธ์—์„œ SSH_PASSWORD ์ˆ˜์ •
vi scripts/deploy.sh
# SSH_PASSWORD=${SSH_PASSWORD:-"YOUR_SSH_PASSWORD"} โ†’ ์‹ค์ œ ํŒจ์Šค์›Œ๋“œ๋กœ ๋ณ€๊ฒฝ

๋น ๋ฅธ ์‹œ์ž‘

์‚ฌ์ „ ์š”๊ตฌ์‚ฌํ•ญ

  • Kubernetes 1.24+
  • Helm 3.0+
  • kubectl
  • nerdctl (์ปจํ…Œ์ด๋„ˆ ๋Ÿฐํƒ€์ž„)

์„ค์น˜

1๋‹จ๊ณ„: ์ปจํ…Œ์ด๋„ˆ ์ด๋ฏธ์ง€ ๋ฐฐํฌ

๋ฐฉ๋ฒ• A: ๋กœ์ปฌ ์ด๋ฏธ์ง€ ์ˆ˜๋™ ๋ฐฐํฌ (Air-gap ํ™˜๊ฒฝ)

# ์‚ฌ์ „ ๋นŒ๋“œ๋œ ์ด๋ฏธ์ง€ ์‚ฌ์šฉ (๊ถŒ์žฅ)
# deployments/images/ ๋””๋ ‰ํ† ๋ฆฌ์— ์žˆ๋Š” tar ํŒŒ์ผ์„ ๋ชจ๋“  ๋…ธ๋“œ์— ๋ฐฐํฌ

# A-1: ์Šคํฌ๋ฆฝํŠธ๋กœ ๋ชจ๋“  ๋…ธ๋“œ์— ๋ฐฐํฌ (๊ถŒ์žฅ) - SSH ํŒจ์Šค์›Œ๋“œ ์‚ฌ์šฉ
NODES=(192.168.1.10 192.168.1.11 192.168.1.12)  # ์‹ค์ œ ๋…ธ๋“œ IP๋กœ ๋ณ€๊ฒฝ
for node in "${NODES[@]}"; do
    echo "Deploying to $node..."
    scp deployments/images/multinic-agent-1.0.2.tar root@$node:/tmp/
    ssh root@$node "nerdctl load -i /tmp/multinic-agent-1.0.2.tar && rm /tmp/multinic-agent-1.0.2.tar"
done

# A-2: SSH Key๋ฅผ ์‚ฌ์šฉํ•˜๋Š” ๊ฒฝ์šฐ
NODES=(192.168.1.10 192.168.1.11 192.168.1.12)  # ์‹ค์ œ ๋…ธ๋“œ IP๋กœ ๋ณ€๊ฒฝ
SSH_KEY_PATH="~/.ssh/id_rsa"  # SSH private key ๊ฒฝ๋กœ
for node in "${NODES[@]}"; do
    echo "Deploying to $node..."
    scp -i $SSH_KEY_PATH -o StrictHostKeyChecking=no deployments/images/multinic-agent-1.0.2.tar root@$node:/tmp/
    ssh -i $SSH_KEY_PATH -o StrictHostKeyChecking=no root@$node "nerdctl load -i /tmp/multinic-agent-1.0.2.tar && rm /tmp/multinic-agent-1.0.2.tar"
done

# A-3: ๊ฐœ๋ณ„ ๋…ธ๋“œ์— ์ˆ˜๋™ ๋ฐฐํฌ
scp deployments/images/multinic-agent-1.0.2.tar root@192.168.1.10:/tmp/
ssh root@192.168.1.10 "nerdctl load -i /tmp/multinic-agent-1.0.2.tar"

# A-4: ์ง์ ‘ ๋นŒ๋“œ (๊ฐœ๋ฐœ์šฉ)
nerdctl build -t multinic-agent:1.0.2 .

๋ฐฉ๋ฒ• B: Nexus Registry ์‚ฌ์šฉ (Registry ํ™˜๊ฒฝ)

# B-1: Nexus์— ์ด๋ฏธ์ง€ ํ‘ธ์‹œ (๊ด€๋ฆฌ์ž ์ž‘์—…)
nerdctl build -t multinic-agent:1.0.2 .
nerdctl tag multinic-agent:1.0.2 nexus.your-domain.com:5000/multinic-agent:1.0.2
nerdctl push nexus.your-domain.com:5000/multinic-agent:1.0.2

# B-2: ์ธ์ฆ์ด ํ•„์š”ํ•œ ๊ฒฝ์šฐ ๋กœ๊ทธ์ธ
nerdctl login nexus.your-domain.com:5000

# B-3: ๊ฐ ๋…ธ๋“œ์—์„œ ์ž๋™์œผ๋กœ ์ด๋ฏธ์ง€ Pull (Kubernetes๊ฐ€ ์ž๋™ ์ฒ˜๋ฆฌ)
# helm install ์‹œ --set image.repository=nexus.your-domain.com:5000/multinic-agent ์‚ฌ์šฉ

2๋‹จ๊ณ„: ๋„ค์ž„์ŠคํŽ˜์ด์Šค ์ƒ์„ฑ

kubectl create namespace multinic-system

3๋‹จ๊ณ„: CRD ์„ค์น˜

# MultiNicNodeConfig CRD ์„ค์น˜
kubectl apply -f deployments/crds/multinicnodeconfig-crd.yaml

# CRD ์„ค์น˜ ํ™•์ธ
kubectl get crd multinicnodeconfigs.multinic.io

4๋‹จ๊ณ„: MultiNic Agent ์„ค์น˜ (Controller ๋ฐฐํฌ)

๋กœ์ปฌ ์ด๋ฏธ์ง€ ์‚ฌ์šฉ ์‹œ:

# Controller Deployment + RBAC + ServiceAccount ์ƒ์„ฑ
helm upgrade --install multinic-agent ./deployments/helm \
  --namespace multinic-system \
  --set image.tag=1.0.2 \
  --set agent.metricsPort=18080 \
  --set agent.preflightAllowUp=false \
  --wait --timeout=300s

Nexus Registry ์‚ฌ์šฉ ์‹œ:

# Nexus Registry์—์„œ ์ด๋ฏธ์ง€ ๊ฐ€์ ธ์™€์„œ ๋ฐฐํฌ
helm upgrade --install multinic-agent ./deployments/helm \
  --namespace multinic-system \
  --set image.repository=nexus.your-domain.com:5000/multinic-agent \
  --set image.tag=1.0.2 \
  --wait --timeout=300s

# ๋‹ค๋ฅธ Registry ์˜ˆ์‹œ๋“ค:
# --set image.repository=192.168.1.50:5000/multinic-agent
# --set image.repository=nexus.company.com:8082/docker/multinic-agent

๋ฐฐํฌ ํ™•์ธ:

# Controller ์ƒํƒœ ํ™•์ธ
kubectl get pods -n multinic-system -l app.kubernetes.io/name=multinic-agent-controller

์ด ๋‹จ๊ณ„์—์„œ ์ƒ์„ฑ๋˜๋Š” ๋ฆฌ์†Œ์Šค:

  • Controller Deployment: CR ๊ฐ์‹œ ๋ฐ Agent Job ์Šค์ผ€์ค„๋ง
  • ServiceAccount + RBAC: Job ์ƒ์„ฑ ๊ถŒํ•œ ์„ค์ •
  • ์ž๋™ํ™” ์‹œ์ž‘: ์ด์ œ CR ์ƒ์„ฑ ์‹œ ์ž๋™์œผ๋กœ Agent Job ์‹คํ–‰

์—…๊ทธ๋ ˆ์ด๋“œ

# ์ฐจํŠธ ์—…๊ทธ๋ ˆ์ด๋“œ
helm upgrade multinic-agent ./deployments/helm \
  --namespace multinic-system \
  --set image.tag=1.0.3 \
  --wait --timeout=300s

์ œ๊ฑฐ

# ์ฐจํŠธ ์ œ๊ฑฐ
helm uninstall multinic-agent -n multinic-system

# CRD ์ œ๊ฑฐ (์„ ํƒ์‚ฌํ•ญ)
kubectl delete crd multinicnodeconfigs.multinic.io

# ๋„ค์ž„์ŠคํŽ˜์ด์Šค ์ œ๊ฑฐ (์„ ํƒ์‚ฌํ•ญ)
kubectl delete namespace multinic-system

์›ํด๋ฆญ ๋ฐฐํฌ (์ž๋™ํ™”)

์ด Helm ์ฐจํŠธ๋Š” MultiNic Agent์˜ ๋ชจ๋“  ์ปดํฌ๋„ŒํŠธ๋ฅผ Kubernetes ํด๋Ÿฌ์Šคํ„ฐ์— ๋ฐฐํฌํ•˜๊ณ  ๊ด€๋ฆฌํ•ฉ๋‹ˆ๋‹ค.

vi ./scripts/deploy.sh

SSH_PASSWORD=${SSH_PASSWORD:-"๋ฐฐํฌ ๋Œ€์ƒ ssh password ์ž…๋ ฅ"}

์ €์žฅ ํ›„ deploy.sh ์‹คํ–‰
# ์ž๋™ ๋ฐฐํฌ ์‹คํ–‰
./scripts/deploy.sh

๋ฐฐํฌ ์Šคํฌ๋ฆฝํŠธ ๊ธฐ๋Šฅ:

  • ํ•„์ˆ˜ ๋„๊ตฌ ํ™•์ธ (nerdctl, helm, kubectl, sshpass)
  • ์ด๋ฏธ์ง€ ๋นŒ๋“œ (nerdctl build)
  • ๋ชจ๋“  ๋…ธ๋“œ์— ์ด๋ฏธ์ง€ ๋ฐฐํฌ (scp + nerdctl load)
  • CRD ์„ค์น˜ (kubectl apply)
  • Helm ์ฐจํŠธ ๋ฐฐํฌ (helm upgrade --install)
  • ๋ฐฐํฌ ์ƒํƒœ ํ™•์ธ
  • ๋ผ์šฐํŒ… ์ง๋ ฌํ™” ์„ค์ • ์œ ์ง€, ๋ฉ”ํŠธ๋ฆญ ์ˆ˜์ง‘ ํ™œ์„ฑํ™”

๋ฐฐํฌ ์™„๋ฃŒ ํ™•์ธ

1. Controller ์ƒํƒœ ํ™•์ธ

# Controller Pod ์‹คํ–‰ ํ™•์ธ
kubectl get pods -n multinic-system -l app.kubernetes.io/name=multinic-agent-controller

# Controller ๋กœ๊ทธ ํ™•์ธ
kubectl logs -n multinic-system -l app.kubernetes.io/name=multinic-agent-controller

2. ์ƒ˜ํ”Œ CR ํ…Œ์ŠคํŠธ

# ์ƒ˜ํ”Œ CR ์ ์šฉ
kubectl apply -n multinic-system -f deployments/crds/samples/

# CR ์ƒํƒœ ํ™•์ธ
kubectl get multinicnodeconfigs -n multinic-system

# ์ƒ์„ฑ๋œ Job ํ™•์ธ
kubectl get jobs -n multinic-system -l app.kubernetes.io/name=multinic-agent

3. ์„ฑ๊ณต ํ™•์ธ ๋ฐฉ๋ฒ•

# CR ์ƒํƒœ๊ฐ€ "Configured"์ธ์ง€ ํ™•์ธ
kubectl get multinicnodeconfigs -n multinic-system -o custom-columns=NAME:.metadata.name,STATE:.status.state

์˜ˆ์ƒ ์„ฑ๊ณต ๊ฒฐ๊ณผ

root@bastion:~/multinic-agent# kubectl get multinicnodeconfigs -n multinic-system -o custom-columns=NAME:.metadata.name,STATE:.status.state
NAME                  STATE
viola2-biz-master03   Configured

โš™๏ธ ๊ณ ๊ธ‰ ์„ค์ • ์˜ต์…˜

Helm Values ์„ค์ • (values.yaml)

๊ธฐ๋ณธ ์„ฑ๋Šฅ ์„ค์ •

# ๋™์‹œ์„ฑ ์ œ์–ด (์•ˆ์ •์„ฑ vs ์„ฑ๋Šฅ ๊ท ํ˜•)
agent:
  maxConcurrentTasks: 1        # ๊ธฐ๋ณธ๊ฐ’: ์•ˆ์ •์„ฑ ์šฐ์„  (1-10 ๊ถŒ์žฅ)

# ์ด๋ฏธ์ง€ ์„ค์ •
image:
  repository: multinic-agent
  tag: "1.0.2"
  pullPolicy: IfNotPresent

# ์ปจํŠธ๋กค๋Ÿฌ Job ๋ณด์กด ์„ค์ •
controller:
  jobTTLSeconds: "3600"
  jobDeleteDelaySeconds: "1800"
  metricsPort: "9090"

RHEL ํ™˜๊ฒฝ SELinux (์ˆ˜๋™ ๊ถŒ์žฅ)

SELinux enforcing ํ™˜๊ฒฝ์—์„œ๋Š” ์ƒ์„ฑ๋œ ํŒŒ์ผ์— ๋Œ€ํ•ด restorecon์„ ๊ถŒ์žฅํ•ฉ๋‹ˆ๋‹ค.

restorecon -Rv /etc/NetworkManager/system-connections
restorecon -Rv /etc/systemd/network

๋ฆฌ์†Œ์Šค ๋ฐ ๋ณด์•ˆ ์„ค์ •

# Pod ๋ฆฌ์†Œ์Šค ์ œํ•œ
resources:
  limits:
    cpu: 500m
    memory: 512Mi
  requests:
    cpu: 100m
    memory: 128Mi

# ๋ณด์•ˆ ์ปจํ…์ŠคํŠธ
securityContext:
  runAsNonRoot: false          # ๋„คํŠธ์›Œํฌ ์„ค์ •์„ ์œ„ํ•ด root ๊ถŒํ•œ ํ•„์š”
  privileged: true             # ํ˜ธ์ŠคํŠธ ๋„คํŠธ์›Œํฌ ์ ‘๊ทผ ํ•„์š”

ํ™˜๊ฒฝ๋ณ„ ๋ฐฐํฌ ์˜ˆ์‹œ

๊ฐœ๋ฐœ/ํ…Œ์ŠคํŠธ ํ™˜๊ฒฝ (๋น ๋ฅธ ๋ฐ˜๋ณต)

helm upgrade --install multinic-agent ./deployments/helm \
  --namespace multinic-system \
  --set agent.maxConcurrentTasks=3 \
  --set image.pullPolicy=Always \
  --set resources.limits.cpu=1000m \
  --set resources.limits.memory=1Gi

ํ”„๋กœ๋•์…˜ ํ™˜๊ฒฝ (์•ˆ์ •์„ฑ ์ตœ์šฐ์„ )

helm upgrade --install multinic-agent ./deployments/helm \
  --namespace multinic-system \
  --set agent.maxConcurrentTasks=1 \
  --set resources.limits.cpu=500m \
  --set resources.limits.memory=512Mi

๋ชจ๋‹ˆํ„ฐ๋ง ๋ฐ ๋กœ๊น…

๋ฉ”ํŠธ๋ฆญ ํ™•์ธ (Prometheus ์—ฐ๋™์‹œ)

# ๋ผ์šฐํŒ… ์ž‘์—… ๋ฉ”ํŠธ๋ฆญ ํ™•์ธ
curl http://localhost:8080/metrics | grep routing

# ์˜ˆ์ƒ ๋ฉ”ํŠธ๋ฆญ:
# routing_operation_duration_seconds_sum
# routing_operation_duration_seconds_count  
# routing_operation_total{operation="configure",result="success"}

๋กœ๊ทธ ๋ชจ๋‹ˆํ„ฐ๋ง

# Controller ๋กœ๊ทธ (CR ์ฒ˜๋ฆฌ ๊ณผ์ •)
kubectl logs -n multinic-system -l app.kubernetes.io/name=multinic-agent-controller -f

# Agent Job ๋กœ๊ทธ (์‹ค์ œ ๋„คํŠธ์›Œํฌ ์„ค์ •)
kubectl logs -n multinic-system -l app.kubernetes.io/name=multinic-agent-job -f

About

MultiNIC Agent for Kubernetes

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages