feat(component): support ssh in tmt tests by dmcilvaney · Pull Request #422 · microsoft/azure-linux-dev-tools

Daniel McIlvaney (dmcilvaney) · 2026-10-29T22:42:24Z

No description provided.

Copilot

🟡 Changes recommended

The new TMT/SSH guidance lacks required rendered-content test assertions covering connection flags, host-key settings, and structured-report instructions.

1 open issue

Add rendered-content assertions for TMT/SSH guidance · New

What changed in this PR

This PR extends azldev component test to run TMT tests on existing SSH guests and adds structured results with retained logs and JUnit reports across provisioner modes.

Changes:

Adds SSH connection flags and validation, with optional candidate RPM installation for connect mode.
Retains per-run artifacts, reports native exit codes, and handles output-write failures.
Adds local and SSH end-to-end coverage, updates documentation, and keeps virtual coverage explicitly opt-in.

File	Description
`scenario/tmt_virtual_e2e_test.go`	Checks KVM availability before setup; clarifies quarantine.
`scenario/tmt_local_e2e_test.go`	Covers local results, artifacts, and interruption handling.
`scenario/tmt_connect_e2e_test.go`	Covers SSH provisioning, host trust, and reporting.
`scenario/testdata/tmt-root/main.fmf`	Adds a root-level smoke plan.
`scenario/testdata/tmt-root/.fmf/version`	Declares the root-plan fixture’s fmf tree.
`scenario/testdata/tmt-local/tests.fmf`	Defines local test-result cases.
`scenario/testdata/tmt-local/run.sh`	Exercises local reporting and cancellation scenarios.
`scenario/testdata/tmt-local/plans.fmf`	Defines reporting and lifecycle-failure plans.
`scenario/testdata/tmt-local/.fmf/version`	Declares the local fixture’s fmf tree.
`scenario/testdata/tmt-connect/tests.fmf`	Defines SSH pass, fail, and error tests.
`scenario/testdata/tmt-connect/run.sh`	Exercises host trust and guest package installation.
`scenario/testdata/tmt-connect/plans.fmf`	Defines SSH test-selection plans.
`scenario/testdata/tmt-connect/helper.sh`	Checks remote helper transfer and elevated execution.
`scenario/testdata/tmt-connect/.fmf/version`	Declares the SSH fixture’s fmf tree.
`scenario/internal/containertest/containers.go`	Supports named Docker networks.
`scenario/internal/cmdtest/cmdtest.go`	Passes network selections to container tests.
`scenario/docker/Dockerfile.tmt-ssh`	Provides a disposable SSH guest image.
`magefiles/magescenario/magescenario.go`	Selects all TMT end-to-end tests without enabling virtual coverage.
`internal/app/azldev/testresult.go`	Defines structured test-run results.
`internal/app/azldev/command.go`	Propagates JSON output-write failures.
`internal/app/azldev/command_test.go`	Tests failed and short JSON writes.
`internal/app/azldev/cmds/component/tmtrunner.go`	Manages retained artifacts, transcripts, and exit codes.
`internal/app/azldev/cmds/component/tmtrunner_internal_test.go`	Tests artifact collection and capture failures.
`internal/app/azldev/cmds/component/test.go`	Adds connect mode and integrates structured reporting.
`internal/app/azldev/cmds/component/test_internal_test.go`	Updates runner and command-argument assertions.
`internal/app/azldev/cmds/component/connect_internal_test.go`	Tests connection validation and environment preservation.
`internal/app/azldev/agentskill/content/build-component.md.tmpl`	Adds SSH testing and reporting guidance.
`go.mod`	Makes the SSH cryptography dependency direct.
`docs/user/reference/cli/azldev_component.md`	Updates the component-test command summary.
`docs/user/reference/cli/azldev_component_test.md`	Documents connection flags and retained results.
`docs/user/explanation/tmt-tests.md`	Explains SSH workflows and artifact semantics.
`docs/developer/how-to/testing.md`	Documents TMT coverage and virtual-test opt-in.

_{Review effort: Balanced}

Copilot

🔵 Needs a closer look

CSV mode forwards child stdout into the report before the header and artifact rows, contaminating machine-readable output; child stdout should be redirected to stderr for CSV as well.

No open issues 🎉

1 previously missed issue

In code that hasn't changed since last review

Prevent child stdout from contaminating CSV reports

internal/app/azldev/cmds/component/tmtrunner.go:211

-O csv also requests machine-readable results, but this branch only protects JSON output. In CSV mode, any child stdout is forwarded to env.ReportFile() before RunFuncWithExtraArgs writes the CSV header and artifact rows, contaminating the report with runner/setup output. Redirect child stdout to stderr for CSV as well so downstream CSV consumers receive only the structured results.

1 resolved since last review

Add rendered-content assertions for TMT/SSH guidance

_{Review effort: Balanced}

Copilot

🟡 Changes recommended

Privileged connect tasks can create root-owned, mode-0600 artifacts that subsequent unprivileged rsync collection cannot retrieve because the existing SSH permission repair is skipped.

1 open issue

Repair permissions before pulling root-owned artifacts over SSH · New

_{Review effort: Balanced}

+	args = append(args, "provision", "--how", settings.Provision, "--become")
+	if settings.Provision == ProvisionConnect {
+		args = append(args, "--guest", settings.Connection.Guest, "--port", strconv.Itoa(settings.Connection.Port),
+			"--user", settings.Connection.User, "--key", settings.Connection.Key)


Add native connect provisioning with explicit SSH inputs, inherited host-key policy, optional candidate RPMs, and shared native reporting. Document guest-writable work directories and cover argument construction and validation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Exercise native SSH provisioning against a separate disposable Azure Linux container, including host-key rejection, dependency and RPM installation, and native reports. Run alongside existing TMT end-to-end coverage. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Assert rendered connection flags, host-key settings, and structured report guidance. Include the same TMT guidance in the no-lockfile skill. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Separate SSH validation and key resolution after the TMT package extraction. Keep command descriptions outside the constructor and format migrated regression tests. Validation: Go 1.26.0 with GOTOOLCHAIN=local; TESTING_DISABLE_NETWORK_TESTS=1 mage all check all passed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI balanced review requested due to automatic review settings October 29, 2026 22:42

Daniel McIlvaney (dmcilvaney) changed the title ~~Damcilva/add ssh to tmt~~ feat(component): support ssh in tmt tests Oct 29, 2026

Copilot started reviewing on behalf of Daniel McIlvaney (dmcilvaney) October 29, 2026 22:42

View session

Copilot AI reviewed Oct 29, 2026

View reviewed changes

Comment thread internal/app/azldev/agentskill/content/build-component.md.tmpl Resolved

Copilot AI balanced review requested due to automatic review settings November 2, 2026 19:38

Copilot started reviewing on behalf of Daniel McIlvaney (dmcilvaney) November 2, 2026 19:39

View session

Copilot AI reviewed Nov 2, 2026

View reviewed changes

Daniel McIlvaney (dmcilvaney) force-pushed the damcilva/add_ssh_to_tmt branch from 02e41c4 to ef59d0f Compare November 2, 2026 21:31

Daniel McIlvaney (dmcilvaney) changed the base branch from main to damcilva/tmt-report-artifacts November 2, 2026 21:31

Copilot AI balanced review requested due to automatic review settings November 2, 2026 21:31

Copilot started reviewing on behalf of Daniel McIlvaney (dmcilvaney) November 2, 2026 21:32

View session

Copilot AI reviewed Nov 2, 2026

View reviewed changes

Daniel McIlvaney (dmcilvaney) and others added 4 commits November 2, 2026 14:33

test(agentskill): cover TMT SSH guidance in both modes

5addf8f

Assert rendered connection flags, host-key settings, and structured report guidance. Include the same TMT guidance in the no-lockfile skill. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Daniel McIlvaney (dmcilvaney) force-pushed the damcilva/add_ssh_to_tmt branch from ef59d0f to 00b5702 Compare November 2, 2026 22:35

Navigation Menu

Repository navigation

feat(component): support ssh in tmt tests - #422

feat(component): support ssh in tmt tests#422
Daniel McIlvaney (dmcilvaney) wants to merge 4 commits into
microsoft:damcilva/tmt-report-artifactsfrom
dmcilvaney:damcilva/add_ssh_to_tmt

Daniel McIlvaney (dmcilvaney) commented Oct 29, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Daniel McIlvaney (dmcilvaney) commented Oct 29, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants