feat(component): publish native TMT reports and artifact paths

.github/workflows/test.yml

@@ -74,8 +74,8 @@ jobs:
       fail-fast: false
       matrix:
         host: [ubuntu-latest]
-        # Add 'tmt' once the quarantined readiness regression is fixed.
-        target: [component-idempotency, image, full-render]
+        # The tmt target runs local coverage; virtual provisioning remains quarantined.
+        target: [component-idempotency, image, full-render, tmt]
     runs-on: ${{ matrix.host }}
     permissions:
       contents: read

docs/developer/how-to/testing.md

@@ -46,16 +46,16 @@ general end-to-end suite with:
 mage e2e
 ```
 
-The GitHub Actions workflow runs three E2E shards in parallel:
+The GitHub Actions workflow runs four E2E shards in parallel:
 
 - `mage e2e -target=component-idempotency` runs update idempotency and subset render
   idempotency.
 - `mage e2e -target=image` runs Azure Linux image construction.
 - `mage e2e -target=full-render` runs the full Azure Linux render idempotency
   check, which dominates the suite runtime.
+- `mage e2e -target=tmt` runs local TMT reporting; virtual provisioning stays
+  quarantined unless explicitly enabled.
 
-`mage e2e -target=tmt` runs the temporarily quarantined virtual TMT
-regression; add it to the CI matrix when the readiness fix lands.
 
 ### Azure Linux image build
 
@@ -70,20 +70,26 @@ container. Only the resolved source revision and checked qcow2 are written to a
 host artifact mount. The privileged container is restricted to this
 image-construction test; ordinary scenario and e2e containers remain unchanged.
 
-### Virtual TMT provisioning regression
+### TMT tests
 
-`mage e2e -target=tmt` builds the current Azure Linux image, then runs the real
-`azldev component test` flow with the pinned TMT/testcloud runtime and a tiny
-test-only probe RPM. The non-privileged TMT runner maps `/dev/kvm`, uses private
-session-libvirt state, and retains diagnostics under `out/tmt-virtual-e2e/`.
-Production and test dependency pins share
-`internal/tmt/requirements.txt`, which Dependabot tracks.
+```bash
+mage e2e -target=tmt
+```
+
+Local reporting runs in a disposable, unprivileged AZL4 container without KVM.
+Do not use the local provisioner directly on your development host: it installs
+packages and executes tests there. Diagnostics remain under `out/tmt-local-e2e/`.
+
+Virtual provisioning is quarantined pending the testcloud readiness fix. Set
+`TESTING_ENABLE_TMT_VIRTUAL_E2E=1` with the same command to reproduce it. Once
+enabled, it skips if `/dev/kvm` is missing and checks access before building an
+image; access errors fail the test. Image construction uses a **privileged**
+container. Virtual diagnostics remain under `out/tmt-virtual-e2e/`.
 
-This regression is temporarily quarantined while the known testcloud readiness
-failure is fixed. The normal `mage e2e` suite compiles the test but skips it;
-`mage e2e -target=tmt` explicitly enables and executes the known-red path. Remove the
-temporary environment guard once the provisioning fixes land so the standard
-E2E job enforces the regression.
+Remove the temporary environment guard when the virtual test is fixed. Both
+tests share the production pins in
+`internal/tmt/requirements.txt`, which Dependabot tracks. Broad `mage e2e` also runs
+the privileged image-builder test independently of the virtual quarantine.
 
 ## Mutation Testing
 

docs/user/explanation/tmt-tests.md

@@ -143,6 +143,59 @@ The same technique works locally: `createrepo_c` the built RPMs, then
 (dnf accepts a local directory as the `--repofrompath` baseurl) resolves conflicts
 for you instead of a hand-rolled `--rpm` filter.
 
+## Structured reports and diagnostic files
+
+Use JSON output to locate retained reports and logs:
+
+```bash
+azldev component test example --provision virtual \
+  --image-path ./image.qcow2 --rpm ./example.rpm --plan /plans/smoke \
+  --work-dir ./test-artifacts -O json >artifacts.json
+status=$?
+```
+
+The output lists attempted catalog entries; execution stops at the first failure.
+JSON and CSV keep diagnostics on stderr. Example (paths are illustrative):
+
+```json
+[
+  {
+    "test": "smoke",
+    "exitCode": 0,
+    "artifactDir": "/work/test-artifacts/azldev-test-123",
+    "junit": ["001-smoke/native/plans/smoke/report/azldev-junit/junit.xml"]
+  }
+]
+```
+
+`artifactDir` is a fresh directory containing `setup.log`, each invocation's
+`runner.log`, and its native TMT run directory. JUnit paths are relative to it;
+copy the whole directory to retain diagnostics outside a container. Test-created
+payloads may retain restrictive permissions; report discovery does not change them.
+Discovery is best-effort: unreadable or invalid artifact paths produce warnings
+and are omitted, without changing the command's execution result.
+It matches the `report/azldev-junit/junit.xml` path suffix, not report provenance:
+test data or copied sources containing the same layout can also appear in `junit`.
+Consumers publishing these files should account for that limitation; discovery
+does not override TMT's exit status.
+
+TMT's native JUnit reporter is added without replacing existing reporters.
+Its default mapping is lossy (`info` becomes skipped; `warn` and `pending`
+become errors); native results and detailed logs remain in the artifact directory.
+Early failures or interruption can leave no JUnit. Available artifacts are
+returned alongside the command error, not reconstructed after execution.
+
+azldev exits **0 on success, 1 on failure**, including TMT's all-skipped and
+no-results exits. `exitCode` records the native TMT value, or `null` before
+startup/on signal termination. Capture/output failures can fail azldev even if
+TMT exited zero. Validation can fail before any artifact row exists.
+
+`--dry-run` previews commands without execution or artifact creation; rows have
+an empty directory/report list and null exit code. Cancellation bounds waiting
+for inherited output pipes, not guest teardown. SIGKILL cannot guarantee output.
+Artifacts persist until the caller removes them. Logs and stderr can contain
+sensitive test output or credentials; restrict access before sharing.
+
 ## Where plans and tests can live
 
 Because `source` pins the **plan** and `discover` resolves the **tests**, the two

internal/app/azldev/agentskill/agentskill_test.go

@@ -153,6 +153,17 @@ func TestBuildComponentSkillDocumentsRPMDevBumpspecPreview(t *testing.T) {
 	assert.NotContains(t, doc, "rpmdev-bumpspec` or RPM macros")
 }
 
+func TestBuildComponentSkillDocumentsTMTArtifactsInBothModes(t *testing.T) {
+	for _, withoutLockfile := range []bool{false, true} {
+		doc, err := agentskill.NewCatalog(withoutLockfile).SkillDocument("azldev-build-component", testParams())
+		require.NoError(t, err)
+		assert.Contains(t, doc, "-O json")
+		assert.Contains(t, doc, "relative JUnit paths")
+		assert.Contains(t, doc, "early failures may have logs but no JUnit")
+		assert.Contains(t, doc, "disposable Azure Linux 4 test container")
+	}
+}
+
 func TestComponentTomlSkillDocumentsRPMDevBumpspecPreview(t *testing.T) {
 	doc, err := agentskill.SkillDocument("azldev-comp-toml", testParams())
 	require.NoError(t, err)

internal/app/azldev/agentskill/content/build-component.md.tmpl

@@ -38,6 +38,15 @@ Prefer `comp render` for quick overlay verification; use `comp diff-sources` to
 exact overlay effect (it fetches sources once, applies overlays to a copy, and diffs the
 two trees). Builds can be slow — set generous timeouts.
 
+For components with TMT tests, `azldev component test <name> --image-path <qcow2>
+--rpm <candidate.rpm> --plan /plans/smoke --work-dir <artifacts> -O json` retains
+logs and available native JUnit reports. JSON rows give the test, TMT exit code,
+artifact directory, and relative JUnit paths. Keep the command's nonzero exit
+status; early failures may have logs but no JUnit. Artifacts persist until removed.
+Do not use
+`--provision local` on a development host: it installs packages and executes
+tests on that machine; use a disposable Azure Linux 4 test container instead.
+
 Finalize with `azldev comp update -p <name>` before opening a PR (see the
 `azldev-update-component` skill).
 

internal/app/azldev/agentskill/content/withoutlockfile/build-component.md.tmpl

@@ -38,6 +38,15 @@ Prefer `comp render` for quick overlay verification; use `comp diff-sources` to
 exact overlay effect (it fetches sources once, applies overlays to a copy, and diffs the
 two trees). Builds can be slow — set generous timeouts.
 
+For components with TMT tests, `azldev component test <name> --image-path <qcow2>
+--rpm <candidate.rpm> --plan /plans/smoke --work-dir <artifacts> -O json` retains
+logs and available native JUnit reports. JSON rows give the test, TMT exit code,
+artifact directory, and relative JUnit paths. Keep the command's nonzero exit
+status; early failures may have logs but no JUnit. Artifacts persist until removed.
+Do not use
+`--provision local` on a development host: it installs packages and executes
+tests on that machine; use a disposable Azure Linux 4 test container instead.
+
 Finalize with `azldev comp refresh-upstream-commit -p <name>` before opening a PR (see the
 `azldev-refresh-upstream-commit` skill).
 

internal/app/azldev/cmds/component/test.go

@@ -72,7 +72,9 @@ azldev creates or reuses an isolated Python environment under --work-dir and
 installs TMT with virtual-provisioner support there. python3 must be available
 on the host; git is required for azldev's own catalog clone unless
 '--source-dir' is used, though the selected plan's 'discover' step may still
-invoke git itself regardless of '--source-dir'.`,
+invoke git itself regardless of '--source-dir'.
+
+Reports and logs stay under --work-dir. Use -O json for artifact paths and TMT exit codes, including on failure.`,
 		Example: `  # Build the component first
   azldev component build buildah
 
@@ -86,7 +88,7 @@ invoke git itself regardless of '--source-dir'.`,
     --rpm ./base/out/rpms/rpm-base/buildah-tests-1.29.0-5.x86_64.rpm`,
 		Args: cobra.ExactArgs(1),
 		RunE: azldev.RunFuncWithExtraArgs(func(env *azldev.Env, args []string) (interface{}, error) {
-			return nil, tmt.RunComponentTests(env, args[0], options)
+			return tmt.RunComponentTests(env, args[0], options)
 		}),
 	}
 
@@ -107,7 +109,8 @@ func registerComponentTestFlags(cmd *cobra.Command, options *tmt.Options) {
 		&options.Tests, "test", "t", nil, "Mapped TMT test name to run (may be repeated; defaults to all)",
 	)
 	cmd.Flags().StringVar(
-		&options.WorkDir, "work-dir", "", "Directory for cloned metadata and TMT artifacts (default: current directory)",
+		&options.WorkDir, "work-dir", "", "Parent for the reusable TMT venv and fresh retained test artifacts "+
+			"(default: current directory)",
 	)
 	_ = cmd.MarkFlagDirname("work-dir")
 	cmd.Flags().StringVar(&options.Provision, "provision", tmt.ProvisionVirtual,