Add report transceiver command: live NIC and transceiver hardware inventory for Linux nodes and SONiC switches #2876

Category: feature | Scope: Large

Summary

Add a command osism report transceiver that collects, live over SSH, the
pluggable transceivers and the surrounding hardware of every physical network
port on OSISM-managed Linux nodes and SONiC switches, and prints the
result as one report (table, CSV or JSON).

Per physical port the report contains:

• the transceiver (presence, form factor, vendor, part number, serial, revision,
  firmware where the module exposes one, optional DOM readings),
• the port state (admin up/down, link up/down, speed, FEC, MTU, MAC),
• the hardware carrying the port: NIC vendor, model, driver and firmware on a
  node; platform, HWSKU, ASIC and SONiC version on a switch,
• the mapping between the interface name on the device and the interface name
  in NetBox.

Devices are selectable by NetBox attributes (site, device role, rack, tag, ...)
and by name. The goal is a detailed, current hardware view of each device
without logging in to it.

Motivation

There is no way today to answer "which optics are plugged in where" across a
deployment. The information is needed for spare-part planning, RMA handling,
vendor/firmware compatibility checks before upgrades, and for debugging link
problems, where both ends of a cable have to be compared. Today that means
running ethtool -m on each node and show interfaces transceiver on each
switch by hand, and translating between the names used on the device and the
names used in NetBox.

Analysis of the current code base

Existing report commands

osism/commands/report.py already contains four live, SSH-based reports
(report memory, report lldp, report bgp, report status). They all follow
the same pattern, which the new command can build on but cannot reuse as is:

No command in the repository produces CSV. --format {table,json} plus a
repeatable --column exists in osism/commands/redfish.py and is the closest
convention. Logging goes to stderr (osism/main.py), so stdout can carry
machine-readable output safely.

Existing SONiC SSH path

osism/commands/sonic.py, SonicCommandBase:

• _get_device_from_netbox() finds the device by name, then by the custom
  field inventory_hostname.
• _get_ssh_connection_details() takes management.ip / management.username
  from local_context_data, falls back to get_device_oob_ip()
  (osism/tasks/conductor/netbox.py), and defaults the user to admin.
• _create_ssh_connection() uses paramiko with the operator key and
  /share/known_hosts.
• sonic show <host> <cmd> proves that the login shell accepts SONiC CLI
  commands directly.

_get_config_context() fails when a device has no local_context_data. The
report must not depend on that: a switch that was never synced still has
transceivers. It should resolve the address from the OOB IP or the primary IP
on its own.

Existing NetBox selection

• osism/tasks/conductor/sonic/device.py:get_devices() selects SONiC switches
  with NETBOX_FILTER_CONDUCTOR_SONIC (default
  status=active, tag=managed-by-metalbox) and a role in
  DEFAULT_SONIC_ROLES (osism/tasks/conductor/sonic/constants.py).
• get_nb_device_query_list_sonic() documents the supported filter keys
  (site, region, site_group, location, rack, tag, status) and
  the fact that NetBox only accepts location_id / rack_id, so names are
  resolved to IDs first.
• utils.nb is a lazily initialised pynetbox client that CLI commands use
  directly (console.py, sonic.py). It is None when NetBox is not
  configured, which is a supported setup for the node-only reports.

Existing interface name mapping

osism/tasks/conductor/sonic/interface.py already translates between NetBox
and SONiC names:

• convert_netbox_interface_to_sonic(interface, device) maps Eth1/1,
  Eth1/1/1 (breakout), Eth1(Port1) and EthernetN to the SONiC name
  EthernetN, using the HWSKU from the sonic_parameters custom field and
  get_port_config(hwsku) (/etc/sonic/port_config/<hwsku>.ini, installed
  into the image by the Containerfile).
• convert_sonic_interface_to_alias() is the reverse direction.
• On failure convert_netbox_interface_to_sonic() returns the input name
  unchanged, so a caller has to check the result against ^Ethernet\d+$ to
  detect an unmapped interface.

Nothing comparable exists for Linux nodes. The only link the code relies on is
the MAC address: osism/tasks/conductor/ironic.py creates Ironic ports from
interface.mac_address of every enabled, non-management NetBox interface.

Runtime environment

The CLI image contains openssh-client, paramiko, pynetbox, tabulate and the
SONiC port_config files. Nothing has to be added to the image. On the target
side the nodes need python3 (present, Ansible requires it), ethtool,
iproute2, pciutils; devlink and lldpctl are used when available.

Proposed command

osism report transceiver [NAME ...]
    [--type {all,node,switch}] [--include-unmanaged]
    [--include-vf] [--include-infiniband]
    [-l PATTERN | --limit PATTERN]
    [--site SLUG] [--role SLUG] [--rack NAME] [--location NAME]
    [--region SLUG] [--site-group SLUG] [--tenant SLUG] [--tag SLUG]
    [--manufacturer SLUG] [--device-type SLUG] [--platform SLUG]
    [--status STATUS] [--filter KEY=VALUE]
    [--interface PATTERN] [--present] [--oper-status {up,down}]
    [--format {table,csv,json}] [--column NAME] [--long] [--output FILE]
    [--dom] [--parallel N] [--timeout SECONDS]

Examples:

# everything known, default columns
osism report transceiver

# all leaf switches of one site as CSV
osism report transceiver --site fra1 --role leaf --format csv --output optics.csv

# one node, full hardware view as JSON
osism report transceiver compute-042 --format json

# name patterns, only ports with a module plugged in
osism report transceiver 'leaf-1*' 'spine-*' --present

# all Ceph nodes via an Ansible pattern, with optical power readings
osism report transceiver -l ceph-osd --dom --long

Registration in setup.cfg: report transceiver = osism.commands.report:Transceiver.

Device selection

1. NetBox available (the normal case): query dcim.devices with
   status=active (overridable with --status) plus the given filters.
   • Typed options map to NetBox device filters: site, role, region,
     site_group, tenant, tag, manufacturer, device_type, platform
     take slugs; rack and location take names and are resolved to IDs the
     way get_nb_device_query_list_sonic() does it.
   • Every option is repeatable. Values of one option are combined the way
     NetBox combines a repeated query parameter, different options are ANDed.
   • --filter KEY=VALUE (repeatable) passes any other NetBox device filter
     through unchanged, for example --filter cf_maintenance=false.
2. Classification of each device found:
   • switch: role slug in DEFAULT_SONIC_ROLES and matching
     NETBOX_FILTER_CONDUCTOR_SONIC, i.e. the same set sonic sync and
     sonic list operate on. Those are the switches the operator key is
     deployed to. A switch with a SONiC role that does not match
     NETBOX_FILTER_CONDUCTOR_SONIC is skipped, also when it is named
     explicitly; the skip is logged as a warning in that case so it is not
     silent. --include-unmanaged lifts the restriction: every selected
     device with a role in DEFAULT_SONIC_ROLES is then queried.
   • node: the device's hostname (get_device_hostname(): custom field
     inventory_hostname, else name) is a host in the Ansible inventory.
   • anything else (PDUs, unmanaged switches, firewalls, ...) is skipped and
     listed at debug level.
3. Name filter: positional NAME arguments are shell-style patterns
   (fnmatch), matched against the NetBox device name and the inventory
   hostname. A device is selected when any pattern matches. Matching happens
   client-side, so it does not depend on the lookup expressions of a specific
   NetBox version.
4. --limit PATTERN is the Ansible host pattern known from the other report
   commands (groups, ranges, exclusions). It is evaluated against the
   inventory and therefore only ever selects nodes.
5. --type restricts the result to nodes or switches.
6. NetBox not configured (utils.nb is None): the command still works
   for nodes from the inventory, with --limit and NAME. NetBox filters are
   rejected with a clear error, switches cannot be discovered, and the NetBox
   interface column stays empty.

Collection

One SSH session per device; devices are processed in parallel with a
ThreadPoolExecutor (--parallel, default 10). --timeout (default 60 s)
bounds one device. All NetBox calls happen in the main thread before and after
the SSH fan-out, so the pynetbox session is not shared between threads and
NETBOX_MAX_CONNECTIONS is respected.

Each session runs a small, stdlib-only Python collector that is piped to the
remote python3 - over stdin and prints one JSON document. The collector only
gathers raw command output and sysfs/DB values; all parsing and normalisation
happens on the manager. That keeps the remote part trivial, makes every parser
unit-testable against captured fixtures, and a --debug run can dump the raw
document for a bug report.

Linux nodes

Connection as in the existing report commands (OPERATOR_USER, operator key,
resolve_host_with_fallback()), with BatchMode=yes so a missing key fails
instead of blocking a worker thread. The collector runs under sudo
(ethtool -m needs root; report memory already relies on passwordless sudo).

Physical ports are the entries of /sys/class/net/* that have a device
link and type 1 (Ethernet). Bonds, bridges, VLANs and other virtual devices
never have a device link. Switchdev representors are skipped.

Two groups of ports are left out by default and can be added on request:

• --include-vf: SR-IOV virtual functions (device/physfn exists). A VF row
  carries its own state, MAC and driver and names its physical function
  (interface.parent); the transceiver belongs to the physical function and
  is not repeated on the VF.
• --include-infiniband: InfiniBand ports (type 32). Link state and rate
  come from /sys/class/infiniband/<dev>/ports/<n>/{state,rate}; module
  EEPROM access depends on the driver and is best effort.

Each row has interface.kind (physical, vf, infiniband) so the added
rows can be told apart and filtered.

ethtool -m prints Key : Value lines; JSON output for it only exists in very
recent ethtool releases, so the text form is parsed. A port without a cage
(RJ45, backplane) or with an empty cage makes ethtool -m fail; that is a
normal result (transceiver.present = false), not an error.

A transceiver firmware version only exists for CMIS modules (QSFP-DD, OSFP,
some QSFP28/SFP-DD) and is only printed by newer ethtool versions. SFF-8472 and
SFF-8636 modules have a vendor revision but no firmware field. The report shows
what the module exposes and leaves the field empty otherwise.

SONiC switches

Address and user: local_context_data.management.{ip,username} when present,
else OOB IP (get_device_oob_ip()), else primary IP; user defaults to admin.
Authentication with the operator key, as in SonicCommandBase.

The data is read from the SONiC Redis databases instead of parsing the
human-readable show tables:

A port without an entry in TRANSCEIVER_INFO has no module plugged in. Only
front-panel ports (Ethernet*) are reported; the management port eth0 has no
cage.

The field names in TRANSCEIVER_INFO differ between community SONiC releases
and Enterprise SONiC (for example manufacturer vs. vendor_name,
hardware_rev vs. vendor_rev). The normaliser therefore works with a list of
candidate keys per output field instead of fixed names.

First implementation step: capture the raw collector output on one switch
per supported platform family and SONiC release in use
(DEFAULT_SONIC_VERSION is 4.5.0, the firmware prefix points at Enterprise
SONiC) and on one node per NIC family (Mellanox/NVIDIA, Intel, Broadcom), and
commit the captures as test fixtures. The exact access method on the switch
(sonic-db-dump -n <DB> -y -k '<pattern>', sonic-db-cli, or the
swsscommon Python binding from inside the collector) and the exact key names
have to be confirmed there; they were not verified against a live device for
this issue.

Interface name mapping

The NetBox interfaces of all selected devices are fetched in bulk
(dcim.interfaces.filter(device_id=[...]), chunked), skipping mgmt_only,
virtual and LAG interfaces. Each report row records the NetBox interface name
and how it was matched (mapping), so a wrong or missing match is visible.

SONiC, in this order:

1. port-config: run convert_netbox_interface_to_sonic() over the device's
   NetBox interfaces and index the results by SONiC name. Needs
   sonic_parameters.hwsku and the matching port_config file. Results that
   do not look like EthernetN are discarded (the function returns its input
   on failure).
2. alias: the live alias of the port equals the NetBox interface name.
3. name: the NetBox interface is already called EthernetN.

Linux, in this order:

1. mac: permanent MAC of the port equals a MAC of the NetBox interface
   (mac_address, and the mac_addresses list on NetBox 4.2 and later). This
   is the same key Ironic port creation uses.
2. name: kernel name or one of its altnames equals the NetBox interface
   name.
3. label: kernel name equals the NetBox interface label.
4. lldp (optional, when LLDP data is available): the LLDP neighbour
   (switch, port) equals the cable peer of exactly one NetBox interface.

Rows without a match get mapping = none. NetBox interfaces without a live
port are reported per device as unmatched_netbox_interfaces (JSON) and as a
warning (table/CSV); both directions point at documentation drift.

When a NetBox interface is matched and has a cable, its peer device and peer
interface are added to the row. That makes it possible to put the optics on
both ends of a link side by side.

Data model

One record per physical port. Field names below are the JSON keys; CSV and
table use the same names as flat columns (transceiver.vendor becomes
transceiver_vendor).

device                name, inventory_hostname, kind (node|switch), address,
                      site, rack, role, device_type, manufacturer,
                      serial, system_vendor, system_model,
                      # switch only
                      hwsku, platform, asic, os_version, firmware{bios,cpld,onie}
interface             name (as on the device), kind, parent, alias,
                      netbox_name, mapping,
                      netbox_peer_device, netbox_peer_interface,
                      lldp_neighbor, lldp_neighbor_port,
                      admin_status, oper_status, speed, duplex, autoneg, fec,
                      mtu, mac, permanent_mac, port_type, bond
nic (node only)       pci_address, vendor, model, subsystem_vendor,
                      subsystem_model, revision, serial, driver,
                      driver_version, firmware_version, firmware{...}
transceiver           present, form_factor, connector, vendor, vendor_oui,
                      part_number, serial, revision, date_code,
                      firmware_active, firmware_inactive, compliance,
                      cable_length, wavelength
transceiver.dom       temperature, voltage, lanes[{rx_power, tx_power, tx_bias}]
  (only with --dom)

Speeds are normalised to Mbit/s on both device kinds.

Output

• --format table (default): tabulate with psql style like the sibling
  commands, a compact default column set (device, interface, NetBox interface,
  admin, oper, speed, form factor, vendor, part number, serial, revision,
  firmware, NIC model, NIC firmware) and the usual footer (date, devices,
  ports, modules present).
• --long shows all columns, --column NAME (repeatable) selects columns
  explicitly, following osism/commands/redfish.py.
• --format csv: csv module, header row, flat columns, no footer.
• --format json: nested per device, so one device's entry is the complete
  hardware view:

  {
    "generated_at": "2026-11-11T12:00:00Z",
    "devices": [
      {"name": "...", "kind": "switch", "...": "...",
       "interfaces": [{"name": "Ethernet0", "netbox_name": "Eth1/1",
                       "transceiver": {"present": true, "...": "..."}}],
       "unmatched_netbox_interfaces": []}
    ],
    "failed": [{"name": "...", "error": "timeout"}]
  }

• --output FILE writes the report to a file instead of stdout.
• Row filters applied after collection: --interface PATTERN (matches the
  device-side or the NetBox name), --present, --oper-status.
• Rows are sorted by device name and natural interface order (Ethernet2
  before Ethernet10).

Error handling

• Unreachable device, timeout, authentication failure, unparsable output: the
  device goes to the failed list, the others are still reported, a summary
  warning is logged at the end (as the existing report commands do).
• A missing optional tool (devlink, lldpctl, lspci) leaves the dependent
  fields empty and is logged once per device at debug level.
• Exit code: 1 when selection fails (inventory, NetBox) or no device
  matches; 0 when at least the selection worked. --strict turns any failed
  device into exit code 1, for use in scripts.
• The command is read-only on the devices and in NetBox.

Implementation outline

Suggested order, each step a reviewable pull request:

1. Fixture capture on real hardware, parsers and data model with tests.
2. Node collection end to end (inventory selection, table/CSV/JSON).
3. SONiC collection.
4. NetBox selection filters and name patterns.
5. Interface mapping, cable peers, unmatched interfaces.
6. DOM readings, user documentation.

Acceptance criteria

• osism report transceiver without arguments reports every physical port
  of all managed nodes and SONiC switches, collected live.
• Each row carries port state (admin, oper, speed), transceiver identity
  (vendor, part number, serial, revision, firmware where exposed) and, for
  nodes, NIC vendor, model, driver and firmware; switch rows carry
  platform, HWSKU, ASIC and SONiC version.
• Ports without a module are reported with present = false and do not
  count as failures.
• --site, --role, --rack, --location, --region, --site-group,
  --tenant, --tag, --manufacturer, --device-type, --platform,
  --status and --filter narrow the selection; options are repeatable
  and combinable.
• Positional name patterns and --limit select devices by name; --type
  restricts to nodes or switches.
• SONiC switches outside NETBOX_FILTER_CONDUCTOR_SONIC are skipped with
  a warning, also when named explicitly, and are queried with
  --include-unmanaged.
• SR-IOV virtual functions and InfiniBand ports are absent by default and
  appear with --include-vf / --include-infiniband, marked by
  interface.kind.
• Per-device failures leave the exit code at 0 with a warning;
  --strict turns them into exit code 1.
• Every row shows the NetBox interface name and the mapping method, or
  none; NetBox interfaces without a live port are reported.
• --format table|csv|json, --column, --long and --output work;
  CSV and JSON on stdout contain no log lines.
• Devices are queried in parallel; one unreachable device neither blocks
  nor aborts the report and is listed as failed.
• Without NetBox the command still reports nodes from the inventory and
  rejects NetBox filters with a clear message.
• Parsers are covered by unit tests using captured output from at least
  one node per NIC family in use and one switch per supported SONiC
  platform family.

Decisions

1. Command name: osism report transceiver.
2. Exit code on partial failure: 0 with a warning, like the sibling
   report commands; --strict makes any failed device exit 1.
3. Unmanaged SONiC switches: a switch that has a SONiC role but does not
   match NETBOX_FILTER_CONDUCTOR_SONIC is not queried, not even when named
   explicitly. --include-unmanaged overrides that.
4. Virtual functions and InfiniBand ports: not reported by default,
   available through --include-vf and --include-infiniband. Switchdev
   representors stay excluded.

Out of scope

• Writing the collected data back to NetBox (inventory items / modules) and
  flagging differences between NetBox and reality (speed, type). The report
  is the prerequisite; both are natural follow-ups.
• Switches that do not run SONiC, and out-of-band collection through the BMC.
  osism manage redfish list <host> NetworkAdapters already covers the
  Redfish view for nodes that are powered off.
• Persisting reports, an API endpoint, or a frontend view.
• Moving the four existing report commands to the shared SSH/selection
  helpers. Worth doing afterwards, but not part of this change.
• User documentation in osism/osism.github.io, to follow once the command
  is merged.