Add UnitTestEx.Aspire: .NET Aspire multi-host (inter-domain) testing support - #106
Merged
Merged
Conversation
Assesses feasibility of supporting .NET Aspire's DistributedApplicationTestingBuilder alongside the existing WebApplicationFactory-based intra-domain testing, as a new opt-in UnitTestEx.Aspire package (net8.0+). Covers the two-tier model, WireMock.Net as the cross-process equivalent of MockHttpClientFactory, per-component mocking options, and the Playwright/UI extension point. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Eric Sibly <eric.sibly@avanade.com>
Adds section 9 analyzing which TesterBase<TSelf> fluent methods are safe to reuse for a Tier 2 Aspire tester vs which would silently no-op (all Replace*/Mock*/ReplaceHttpClientFactory funnel through the non-virtual TesterBase.ConfigureServices). Proposes a minimal, backward-compatible SupportsServiceConfiguration guard so these fail loudly instead of silently doing nothing. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Eric Sibly <eric.sibly@avanade.com>
…ions - Flip the proposed capability property from protected to a public virtual getter, so extension methods (UnitTestEx's own, companion packages, or consumers) can defensively check it, mirroring the existing public IsHostInstantiated flag. - Audit existing TesterBase extension methods across the ServiceBus and Azure Functions companion packages: none touch DI, so none need guarding. Note that the IServiceCollection extensions in core ExtensionMethods.cs (ReplaceSingleton/Scoped/Transient/etc.) are automatically shielded already, since they can only run from inside a ConfigureServices callback that is now itself guarded. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Eric Sibly <eric.sibly@avanade.com>
) Adds a public virtual SupportsServiceConfiguration property (defaults to true) to TesterBase, and guards the internal ConfigureServices funnel method to throw NotSupportedException when it is false. This is Phase 1 of the phased plan tracked in #105 for supporting .NET Aspire multi-host (inter-domain) testing alongside the existing WebApplicationFactory-based single-host model. A future Aspire-based tester does not run in-process, so DI service configuration/replacement (ReplaceSingleton/ReplaceScoped/ReplaceTransient and their Keyed/Mock variants, ReplaceHttpClientFactory) cannot be supported; without this guard such calls would be silently queued and never applied. The property is public so extension methods (in this repo or consumers) can proactively check the capability. - Add tests validating default behavior is unaffected, and that a tester overriding the property to false throws. - Update CHANGELOG.md. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Eric Sibly <eric.sibly@avanade.com>
chullybun
force-pushed
the
chullybun-aspire-multi-host-testing
branch
from
September 22, 2026 17:59
25c014d to
1f796fb
Compare
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The changelog version mismatch remains unresolved, along with the noted guard and exception-documentation follow-ups.
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 1
What changed in this PR
Adds a backward-compatible TesterBase capability guard for unsupported service configuration in future Aspire multi-host testing.
Changes:
- Adds
SupportsServiceConfigurationand a fail-fastNotSupportedException. - Adds Xunit coverage for supported and guarded behavior.
- Updates Aspire design documentation and the changelog.
| File | Summary | Final review comments |
|---|---|---|
tests/UnitTestEx.Xunit.Test/Other/SupportsServiceConfigurationTest.cs |
Tests default and guarded behavior. | No findings. |
src/UnitTestEx/Abstractions/TesterBase.cs |
Adds the capability property and configuration guard. | Nit (2 votes): Include actionable alternatives in the exception. Nit (1 vote): Change “where” to “when” in the documentation. |
docs/design/aspire-multi-host-testing.md |
Documents the Aspire testing design. | Nit (1 vote): Clarify that direct configuration entry points remain outside the guard, or handle them. |
CHANGELOG.md |
Records the enhancement. | Moderate (1 vote): Resolve the v5.12.0 changelog entry versus the current 5.11.1 package version. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
- Include actionable, Tier-2-appropriate guidance in the NotSupportedException thrown by TesterBase.ConfigureServices (point callers at configuring the resource's environment/configuration surface or mocking the HTTP boundary, e.g. WireMock.Net.Aspire), instead of just referencing the SupportsServiceConfiguration flag. - Change "Thrown where" to "Thrown when" in the XML doc comment. - Clarify in the design doc (section 9) that TestSetUp.ConfigureServices is a separate, tester-implementation-level entry point that is not, and does not need to be, routed through the new guard. - Bump Common.targets Version to 5.12.0 to match the CHANGELOG.md entry added for this change, resolving the version mismatch flagged by review. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Signed-off-by: Eric Sibly <eric.sibly@avanade.com>
…esterBase<TSelf> Re-examined the Phase 2 base-class decision after a direct challenge: is TesterBase<TSelf> actually the right thing for AspireTesterBase to inherit, or does it just produce a large surface of methods that read right, compile fine, and blow up at runtime? Verified against the real code: - TesterBase<TSelf> contributes only DI-reach-in members (~25 Replace*/Mock*/ReplaceHttpClientFactory overloads, ScopedType/Type, UseScopedTypeSetUp) - nothing else. - HttpTesterBase (the class implementing Tier 1's entire HTTP request/response/assertion engine) already depends only on the non-generic TesterBase in its constructor, not TesterBase<TSelf>. It's a working example of exactly where the tier boundary sits. Conclusion: AspireTesterBase should inherit the non-generic TesterBase directly, not TesterBase<TSelf>. This is a compile-time-safe alternative to guarding away DI methods at runtime - they simply won't exist on the type. Changes: - Section 8: sketch updated to AspireTesterBase<TAppHost, TSelf> : TesterBase, with explicit throwing Services/Configuration overrides. - Section 9: retitled and rewritten to explain the corrected inheritance decision, cite the HttpTesterBase evidence, and reframe SupportsServiceConfiguration as defense-in-depth rather than the primary safety mechanism for AspireTesterBase itself. - Section 11: phasing updated to add extracting HttpTesterBase's HttpClient source (currently hard-coded to TestServer) behind a small seam as a prerequisite step before prototyping AspireTesterBase. No code changes - design doc only. The already-shipped SupportsServiceConfiguration guard on TesterBase remains valid and useful as defense-in-depth for future extension methods. Signed-off-by: Eric Sibly <eric.sibly@avanade.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
… in design doc Further discussion questioned whether inheriting the DI-heavy TesterBase (or TesterBase<TSelf>) for a Tier 2 (Aspire) tester was right at all, even with a runtime guard - proposed instead extracting a new TesterBaseCore containing only the genuinely host-agnostic members, with TesterBase staying as-is on top of it, and the new Tier 2 base inheriting TesterBaseCore directly. This makes the runtime SupportsServiceConfiguration guard (shipped earlier in this branch) unnecessary: its entire premise was "a Tier 2 tester will inherit ~25 DI-flavoured methods it can't honor, so make them throw instead of silently no-op." Under the TesterBaseCore split a Tier 2 tester never inherits those methods in the first place, so there is nothing left to guard. Keeping the flag around would be speculative infrastructure for an abandoned design. Code changes: - src/UnitTestEx/Abstractions/TesterBase.cs: removed the SupportsServiceConfiguration property and the guard clause in ConfigureServices; restored to its pre-guard shape. - Removed tests/UnitTestEx.Xunit.Test/Other/SupportsServiceConfigurationTest.cs (tested only the reverted guard). - CHANGELOG.md: removed the v5.12.0 entry (no functional change remains). - Common.targets: reverted <Version> from 5.12.0 back to 5.11.1. Design doc changes (docs/design/aspire-multi-host-testing.md): - Section 8: AspireTesterBase sketch now inherits TesterBaseCore (not TesterBase), implementing a new IHttpClientSource interface instead of overriding Services/Configuration to throw - those members simply don't exist on the type. - Section 9: rewritten to explain the TesterBaseCore extraction, the concrete member split (table cross-checked against every member on TesterBase today), the IHttpClientSource interface as the answer to "common but doesn't fit the base class", and why the previously shipped runtime guard is now unnecessary. - Section 11: phasing updated to lead with the TesterBaseCore + IHttpClientSource extraction as the core-package prerequisite. - TL;DR: updated to describe the compile-time (not runtime) tier boundary. Verified: full solution builds; xUnit 60/60 pass (62 before, minus the 2 removed guard tests). Signed-off-by: Eric Sibly <eric.sibly@avanade.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
chullybun
changed the title
…Phase 1 of #105) Splits TesterBase into a new host-agnostic TesterBaseCore base class plus the existing DI-specific TesterBase, so a future multi-host (e.g. Aspire) tester can reuse the common testing engine without inheriting single-host/DI-only capabilities it cannot support. - Add TesterBaseCore (Abstractions) holding all host-agnostic members: logging, JSON serialization, SharedState, host-reset plumbing, pre/post-run actions, etc. TesterBase now inherits it and keeps only Services, Configuration, ConfigureServices, AdditionalConfiguration and CreateHttpRequest*. - Add IHttpClientSource interface so HttpTesterBase-derived testers can build their HttpClient without a hardcoded TestServer; ApiTesterBase implements it. This is the reuse seam for a future Aspire-based tester. - Widen the HTTP-response assertor/expectations chain (ExpectationsArranger, ExpectationsBase(T), AssertArgs, ErrorExpectations, ExceptionExpectations, HttpResponseMessageExpectations, LoggerExpectations, ValueExpectations, HttpResponseMessageAssertor(Base)(T), TesterExtensionsConfig. UpdateValueFromHttpResponseMessage) from TesterBase to TesterBaseCore. - Rename the ResetHost() override hook to OnResetHost() in ApiTesterBase, FunctionTesterBase and GenericTesterCore to avoid a name clash with the new public TesterBaseCore.ResetHost(). - Breaking change: HttpTester, HttpTester<TValue>, ControllerTester<TController> and the HttpTesterBase* constructors now take (TesterBaseCore, IHttpClientSource) instead of (TesterBase, TestServer). Nothing in the test suite constructs these directly. - Fix JsonElementComparerResult.ToString() to join differences with a literal '\n' instead of AppendLine()/Environment.NewLine, so its output is deterministic across platforms (was failing MSTest on Windows due to CRLF vs. the LF-only literals checked into the test file). Signed-off-by: Eric Sibly <eric.sibly@avanade.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Remove unnecessary `new` on TesterBase.ResetHost(bool) (CS0109); the differing parameter list from TesterBaseCore.ResetHost() means there was nothing to hide. - Fix XML doc `<see cref="TesterBase.X"/>` references (CS1574) that broke when SetUp, UserName, JsonSerializer, JsonComparerOptions, ResetHost(), OnHostStartUp, PreRunActions, PostRunBeforeExpectationsActions and PostRunAfterExpectationsActions moved to TesterBaseCore in Phase 1. Re-pointed each to TesterBaseCore.X in TesterBaseT.cs, HttpTesterBaseT.cs, HttpTesterBaseT2.cs, TesterArgs.cs and TestSetUp.cs. Verified: full solution build is warning-free for these categories (remaining NETSDK1138/CS8785 warnings are pre-existing net6.0 tooling noise, unrelated to this change). Full test suite passes 100% (xUnit 60/60, NUnit 115/115, MSTest 95/95 across net8/9/10). Signed-off-by: Eric Sibly <eric.sibly@avanade.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- New UnitTestEx.Aspire package with AspireTesterBase/AspireTester<TAppHost>/static AspireTester factory for testing .NET Aspire distributed applications end-to-end. - Aspire test suite (Xunit/NUnit/MSTest) multi-targets net8.0/9.0/10.0, orchestrating the real tests/UnitTestEx.Api sample via a single AppHost pinned to net8.0 (Aspire's DCP orchestrator cannot launch a multi-targeted project resource without an explicit --framework arg; see microsoft/aspire#2962). Higher-TFM test project slots can safely reference the lower-TFM AppHost since NuGet ProjectReference TFM compatibility is asymmetric. - HttpTesterBase.CreateRequest fix: honour the underlying HttpClient's BaseAddress so Aspire-issued clients (with resource-discovered base addresses) work correctly. - CI: add explicit Aspire MSTest/NUnit/Xunit test steps (dotnet test auto-iterates all target frameworks in a multi-targeted test project without needing a manual loop). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…s a real readiness gate A plain AddProject resource has no health check by default, so it's reported 'Healthy' as soon as the OS process starts - not once Kestrel has actually bound its endpoint(s). On a slower/cold CI machine this raced ahead of endpoint allocation, causing 'Endpoint '' for resource 'api' not found' when CreateHttpClient tried to resolve the default https/http endpoint. WithHttpHealthCheck closes that race. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…g on launchSettings.json auto-detection AddProject's implicit endpoint detection from launchSettings.json proved unreliable for a multi-targeted referenced project (UnitTestEx.Api targets net8.0/9.0/10.0): it worked on Windows locally but failed on Linux CI with 'Could not create HTTP health check for resource api as no endpoint was found matching one of the specified names: https, http'. Declaring a single explicit https endpoint (launchProfileName: null + WithHttpsEndpoint) removes the dependency on that detection entirely and is deterministic across platforms. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…add timeout to WaitForResourceAsync Linux CI does not fully support 'dotnet dev-certs https --trust', so the ASP.NET Core dev cert used by the 'api' resource's https endpoint is not OS-trusted there. Both AspireTesterBase.CreateHttpClient() and the WithHttpHealthCheck probe resolve their HttpClient via the same DistributedApplication DI container's IHttpClientFactory, so a failing TLS handshake would affect both - and with no cap on the health check's retry loop plus no timeout on WaitForResourceAsync, this manifested as an indefinite hang rather than a clean failure. - AppHost.cs: ConfigureHttpClientDefaults with a permissive ServerCertificateCustomValidationCallback for this dev/test-only AppHost (never shipped). - AspireTesterTest.cs (all 3 frameworks): pass a 60s timeout to WaitForResourceAsync so any future genuine problem fails fast instead of hanging. Verified: all 9 combinations (Xunit/NUnit/MSTest x net8.0/9.0/10.0) pass locally against the real UnitTestEx.Api. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…iting indefinitely A resource that never becomes healthy should fail the test quickly, not hang it forever. Added AspireTesterBase.DefaultWaitForResourceTimeout (60s) as the default when no timeout is passed; callers can still opt into an indefinite wait via Timeout.InfiniteTimeSpan. Removed the now-redundant explicit per-call timeouts from the 3 AspireTesterTest.cs files since the library default covers it. Verified: all 9 combinations (Xunit/NUnit/MSTest x net8.0/9.0/10.0) pass locally. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ceTimeout doc comment Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
….Api reference GetTargetPath is only invoked by Visual Studio's build engine (not dotnet build/CLI, which is why CI and prior local dotnet build verifications never surfaced this). A multi-targeted project reference only exposes GetTargetPath on its inner (single-TFM) build, not its outer cross-targeting wrapper, so VS fails to resolve it unless a single TFM is pinned via SetTargetFramework - the standard, documented fix for referencing a multi-targeted project from an AppHost (matches the framework already selected via --framework at runtime). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…-log relay in test output The underlying DistributedApplication's own ILogger pipeline (Aspire.Hosting.* bootstrap messages, DCP startup, and the console-log relay it forwards per-resource, e.g. '<AppHost>.Resources.<name>') was logging at Information by default, drowning out UnitTestEx's own formatted request/response tester output (which is written directly via Implementor.WriteLine and is unaffected by this). Added AspireTesterBase.MinimumLogLevel (defaults to Warning) which sets LoggerFilterOptions.MinLevel on the testing builder before it is built; settable before first use if a consumer wants the extra verbosity back (e.g. while diagnosing why a resource never became healthy). Verified: all 9 combinations (Xunit/NUnit/MSTest x net8.0/9.0/10.0) pass locally with clean, focused output. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ction
Adds IHttpClientSource.GetRequestLogMessages(requestId) as an optional seam
(default returns null, preserving existing WAF-based behaviour unchanged)
that HttpTesterBase now prefers over TestSharedState-based correlation.
AspireTesterBase implements it by capturing each resource's own forwarded
logging (the '{ApplicationName}.Resources.{resourceName}' category Aspire
already relays to the AppHost) via a dedicated ILoggerProvider, independent
of the MinimumLogLevel console-noise filter. ResourceLoggerService.GetAllAsync/
WatchAsync are confirmed unpopulated under DistributedApplicationTestingBuilder,
so this capture provider is the reliable path.
Per-request correlation uses a captured-line-count window (snapshot before
send, diff after response) since there is no shared in-process request-id
correlation across a real, separate resource process.
…rBase style
Previously captured resource log lines only had ANSI codes and Aspire's own
'{lineNumber}: {timestamp}Z ' prefix stripped, discarding the embedded
timestamp entirely and leaving the raw two-physical-line console shape
('level: category[eventId]' header + indented message).
Now parses and reassembles each logical entry (header + one or more message/
continuation lines) into the same '{timestamp} {level}: {message} [{category}]'
format that Tier 1's LoggerBase already produces, re-using Aspire's own
embedded timestamp (converted to local time) rather than the capture time.
Also fixes a real bug uncovered along the way: an entry only completed once
a subsequent header line arrived to terminate it, so the most recently
logged entry in any given request/wait window (often the interesting one)
was silently dropped. Pending entries are now force-flushed before each
buffer read (and on provider Dispose as a final safety net).
…ier 2 testers
Reason(string) writes an arbitrary context note ('REASON >' block) to the
test output, for both TesterBase<TSelf> (single in-process host) and
AspireTesterBase<TAppHost, TSelf> (multi-host).
WaitAndLog(string reason, TimeSpan duration) writes a '====' delimited
block (reason + timeout), waits for the duration, then reports whatever was
logged during that window under the same 'LOGGING >' heading used elsewhere
- useful when a test needs to wait on background/inter-resource activity
that isn't tied to a specific HTTP request/response.
The 'what counts as elapsed log output' behaviour is supplied via a new
protected virtual TesterBaseCore.DrainElapsedLogMessages():
- Tier 1 default: any SharedState logging not attributed to a specific HTTP
request (i.e. background/hosted service logging).
- Tier 2 (Aspire) override: combines each resource's captured logging
(via ResourceLogCaptureProvider) across ALL resources since the last
drain, each line prefixed with its owning resource name, since background
activity may occur in any resource.
It set LoggerFilterOptions.MinLevel on the AppHost's testing builder, but unlike Tier 1 testers (which call ClearProviders() before adding their own capture provider, leaving MinLevel as the sole governing filter), the Aspire AppHost keeps its own default console logger active. That provider's effective level is governed by Aspire's own internal logging configuration, not by our MinLevel fallback rule, so setting this property had no observable effect on the console noise it was meant to control. The exemption rule that always lets our own ResourceLogCaptureProvider see everything (regardless of the AppHost's own console noise level) is retained. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…t (Tier 1 and Aspire)
Tier 1 (UnitTestEx.*.Test): Reason() writes the expected 'REASON >' block; WaitAndLog() blocks for
approximately the requested duration, surfaces logging performed via the host's DI-resolved ILogger
during the wait window under 'LOGGING >', and correctly excludes stale logging that occurred before the
wait started ('None.').
Aspire (UnitTestEx.Aspire.*.Test): a new Reason_And_WaitAndLog_AggregatesResourceLogs test verifies the
same behaviour plus the Aspire-specific DrainElapsedLogMessages override - a background request against
the 'api' resource fired mid-wait is captured and prefixed with '[api]' in the aggregated LOGGING > output.
A small SpyTestFrameworkImplementor (duplicated per test project, matching this repo's established
per-framework test file convention) wraps the tester's own implementor so output still reaches the test
framework while also being captured for assertions.
Verified: full Tier 1 suites (63/118/98 Xunit/NUnit/MSTest) and full Aspire net10.0 suites (3/3 each,
9/9 total) all pass; full solution build is clean (0 errors, no new warnings).
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ost console noise - Suppress Aspire AppHost's default ILoggerProvider console output by default (opt-in via new EnableHostDiagnostics()); prevents raw AppHost/DCP/resource noise from duplicating UnitTestEx's own curated log output. - Fix genuine log-duplication bug: unify the per-request (.Http().Run()) and multi-resource (Wait()) log drains onto a single shared watermark so a resource log line is reported exactly once, never twice. - Merge the resource-name attribution into the existing category bracket, e.g. '[UnitTestEx.Api.Controllers.PersonController (api)]', instead of a separate trailing '[api]' bracket. - Rename WaitAndLog(reason, duration) to Wait(reason = null, duration = null) across TesterBaseCore/TesterBaseT/AspireTesterBase; add TesterBaseCore.DefaultWaitDuration (1s) used when duration is omitted. - Default reason text changed to 'No reason specified' when none supplied. - Simplify WAIT output to a single 'WAIT (duration) >' line with dashed dividers (removed '====' clutter). - Rename ReasonAndWaitAndLogTest -> ReasonAndWaitTest (Xunit/NUnit/MSTest) and update all call sites/assertions accordingly. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Fix WireMock.Net scenario state-machine bug in WithSequenceAsync (SetStateTo: null resets scenario to start rather than leaving state unchanged). - Redesign sequence semantics to exactly mirror Tier 1's MockHttpClientRequest: sequence length is an exact invocation count, Times() disallowed alongside WithSequenceAsync, and overrunning the sequence fails loudly (synthetic guard mapping returns 500, VerifyAsync throws MockHttpClientException). - Add JsonResource equivalents (WithJsonResourceBody/Async/WithJsonResource) across AspireHttpMockRequest/Response/ResponseSequenceItem, matching Tier 1 parity, plus [StringSyntax(Json)] on raw JSON string parameters. - Add pathsToIgnore support to WithJsonBody via WireMock.Net's native JsonPartialMatcher (strips named properties from the match pattern and switches matcher), for cases like an unpredictable ETag. - Add end-to-end tests covering sequence exhaustion and pathsToIgnore (positive + negative match) against a real WireMock.Net container. - Document the finalized sequence and pathsToIgnore semantics, including divergences from Tier 1, in the design doc. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Introduces IHttpMockClient/IHttpMockRequest/IHttpMockRequestBody/IHttpMockResponse/ IHttpMockResponseSequence/IHttpMockResponseSequenceItem/IHttpMockedRequest so test-authoring code can be written once and used identically against Tier 1's in-process Moq-based MockHttpClient* and Tier 2/3's out-of-process WireMock.Net-based AspireHttpMock*. Each interface exposes a minimal abstract core, with convenience overloads provided as default interface methods composed from that core. Retrofits both MockHttpClient* and AspireHttpMock* to implement the shared interfaces (async-wrapping Tier 1's native synchronous members for compatibility - non-breaking). Fixes a real cross-tier semantic gap surfaced by the retrofit: MockHttpClientRequest's WithAnyBody() previously required a non-null body and so could never match a body-less request (e.g. GET), unlike Aspire's true-wildcard behavior and the interface's documented contract. Tier 1 now matches with or without a body, aligning both tiers; updated UriAndAnyBody/DefaultHttpClient tests across the Xunit/NUnit/MSTest projects accordingly. Adds HttpMockInterfaceTest.cs (Tier 1) exercising every DIM extra via interface-typed variables, and HttpMock_InterfaceCoreMembers_TierSpecificAdaptations (Aspire) exercising the hand-written, tier-specific core-member adaptations. Adds HttpMockSharedConfig.cs and a HttpMock_SharedInterface_ConfiguresIdenticallyAcrossTiers proof test on both tiers. Updates docs/design/aspire-multi-host-testing.md with a new section documenting the shared interfaces and the WithAnyBody() alignment rationale. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Adds a new 'Aspire multi-host testing' section covering: when to reach for Tier 2 (AspireTester) over the existing in-process Tier 1 testers, a basic Http() usage example, and a worked example of mocking multiple external dependencies (email/auth/erp) each as their own isolated AddWireMock(...) container resource via HttpMock(resourceName) - highlighting that the fluent stubbing API is shared with Tier 1's MockHttpClientFactory via the IHttpMock* interfaces. Also updates the intro TOC and Examples section. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The design was captured in the tracking issue, so docs/design/aspire-multi-host-testing.md (and the now-empty docs/ folder) is removed. All in-code XML doc / comment references to it (AspireHttpMockClient, AspireTesterBase, IHttpMockClient, AspireTesterTest) are rephrased to drop the dangling pointer while keeping the surrounding explanation intact. Also adds Aspire's own official guidance as a verbatim quote to the README's new 'Aspire multi-host testing' section, reinforcing that the two-tier split (WebApplicationFactory for single-project/in-memory/mocked-dependency testing vs. AspireTester for genuine multi-host interaction) isn't a UnitTestEx-specific compromise. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…L/JSON config gap between tiers - New internal HttpMockResourceConfig helper parses the same mock.unittestex YAML/JSON schema as Tier 1's native MockHttpClient.WithRequestsFromResource, but is implemented purely against IHttpMockClient/IHttpMockRequest/IHttpMockResponse/IHttpMockResponseSequenceItem, so it works identically against Tier 2/3's AspireHttpMockClient without either tier reimplementing the parsing. - Added two IHttpMockClient DIMs: WithRequestsFromResourceAsync<TAssembly> and WithRequestsFromResourceAsync(string, Assembly?, CancellationToken). - Tier 1's existing native WithRequestsFromResource is left completely unchanged to avoid any regression risk; this is purely additive. - One intentional, documented behavioral difference: where a request entry omits body entirely, the shared loader matches any body (WithAnyBody) rather than Tier 1's native 'no body at all' semantics, since the shared interface has no way to express the latter. - Added Xunit test coverage on Tier 1 (HttpMockInterfaceTest) and Tier 2/3 (AspireTesterTest) with matching embedded YAML resources. - Documented the new capability in the README's YAML/JSON configuration and Aspire multi-host testing sections. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Introduced UnitTestEx.Aspire to support .NET Aspire multi-host testing with a familiar API. Added AspireHttpClientMock for HttpClient mocking. Updated Common.targets to version 5.12.0.
…cher, and Aspire AppHost helpers - Renamed JsonComparerMatcher to JsonElementComparerMatcher for UnitTestEx naming consistency; fixed it to honor configured JsonElementComparerOptions instead of always using JsonElementComparer.Default. - Added WireMockConsole.RunAsync to UnitTestEx.Aspire, encapsulating the self-hosted WireMock.Net console boilerplate (PORT env var, settings, custom-matcher registration, graceful shutdown) so a consumer's MockHost Program.cs is a one-liner. - Added UnitTestExAspireExtensions (AddMockHostProject / WithMockHostEnvironment) to remove the run-mode/publish-mode null-handling boilerplate from AppHost projects wiring up self-hosted mock resources. - Added tests/UnitTestEx.Aspire.MockHost as the copy-paste template project referenced by the README, using the above helpers; added it to the solution under /tests. - Fixed a latent null-argument risk in the sample AppHost (WithEnvironment received a null EndpointReference in publish mode) via the new WithMockHostEnvironment helper. - Made Humanizer package references PrivateAssets to avoid leaking them downstream. - Renamed WaitAndLog/Reason test coverage to Delay-based naming (ReasonAndDelayTest) for consistency with the Delay/Wait unification. - Updated README's Aspire multi-host testing section and CHANGELOG to match. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
chullybun
changed the title
… request ApiTesterBase.CreateHttpClient routed through GetTestServer() -> HostExecutionWrapper() -> SharedState.Reset() on every call. Since CreateHttpClient is invoked by HttpTesterBase.SendAsync per-request (i.e. potentially after an Expect*()-style call, such as CoreEx's ExpectEvents(), has already registered request-scoped state keyed by the HttpTester's RequestId), that state was silently wiped immediately before the request was sent - causing correctly-matched expectations to be reported as failed. CreateHttpClient now resolves the WebApplicationFactory's TestServer directly, without going through the resetting wrapper. GetTestServer() itself is untouched (still resets) for its own construction-time callers (Controller<T>(), Http(), Http<TResponse>()), where the reset-before-use semantics are correct. Added RequestStateDataTest as regression coverage: registers request-scoped state, confirms it is still present at post-run. Verified it fails against the prior behavior and passes with the fix. Also: - Documented that Playwright can drive Tier 2/3 Aspire resources directly (README footnote only - no new API, per discussion on issue #105). - Added UnitTestEx.Aspire to nuget-publish.ps1's project list. - Reformatted WithMockHostEnvironment's signature (no functional change). - Folded the fix into the CHANGELOG's existing (unreleased) v5.12.0 entry, since the regression was introduced by this same branch's work. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ric subclass Mirrors the existing TesterBase / TesterBase<TSelf> pattern so extension methods can target AspireTesterBase without needing TAppHost/TSelf. - AspireTesterBase (non-generic) now sits directly on TesterBaseCore and holds everything not dependent on TAppHost/TSelf: GetDistributedApplicationAsync (public), HttpMock, Http(), WaitForResourceAsync, HostDiagnosticsEnabled, DrainElapsedLogMessages, ResourceLogCaptureProvider/ResourceLogger/ ResourceHttpClientSource, and the DisposeAsync/DisposeAsyncCore pair. - AspireTesterBase<TAppHost, TSelf> (new file AspireTesterBaseT.cs) keeps only the fluent (TSelf-returning) configuration methods and the TAppHost-dependent CreateBuilderAsync() override. - Added ConfigureBuilderActions (protected property) and IsDistributedApplicationBuilding (protected guard) on the base so the generic subclass's WithResourceEnvironment/EnableHostDiagnostics can still enforce their pre-build-only invariant. - Fixed dangling XML doc <see cref> references left pointing at members that moved to the other class as part of the split. Verified: full solution build (0 errors) and UnitTestEx.Aspire.Xunit.Test 15/15 passing on net8.0/net9.0/net10.0. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
… JsonElementComparerMatcher to HttpMock namespace - Renamed TesterBase/AspireTesterBase's Reason() to Checkpoint(), which now also flushes (writes) any elapsed log messages since the last Checkpoint()/Delay() call, mirroring Delay() minus the actual wait. Both Checkpoint() and Delay() now stay silent (no 'LOGGING >' section) when there is nothing new to report, instead of writing 'None.'. - Added AspireTesterBase.WaitForResourceAsync(string[], TimeSpan?) to wait for multiple named resources concurrently; converted 4 existing sequential-wait call sites in the Xunit Aspire test suite to use it. - Moved JsonElementComparerMatcher (+ its exception type) from the UnitTestEx.Aspire namespace into UnitTestEx.Aspire.HttpMock, since it is tightly coupled to the HttpMock feature (registered by WireMockConsole, consumed by AspireHttpMockRequest). - Renamed ReasonAndDelayTest.cs/class to CheckpointAndDelayTest across all three Tier 1 test projects. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…itForResourceAsync(string[]) - CHANGELOG.md: describe the final Checkpoint()/Delay()/WaitForResourceAsync(string[]) API under v5.12.0, since the earlier Reason()->Checkpoint() rename and the CreateHttpClient SharedState-reset bug were both introduced and resolved entirely within this unreleased PR and were never part of a shipped release. - AspireTesterBase.WaitForResourceAsync(string[], TimeSpan?) now de-duplicates the supplied resource names before waiting, so a caller passing the same resource twice does not wait on it concurrently more than once. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…tart BeforeStart(Func<DistributedApplication, Task>) registers a callback invoked after the DistributedApplication is built but before any resource is started, closing the gap where GetDistributedApplicationAsync() atomically builds and starts in one step. This enables use cases such as running database migrations/seeding or clearing a cache against a pre-existing (e.g. AddConnectionString) resource before dependent project resources come online, avoiding a race between test setup and Aspire resource start-up. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Aspire's own DistributedApplication.GetConnectionStringAsync (and
GetEndpoint/CreateHttpClient) testing extensions throw
InvalidOperationException ('The application must be started before
resolving endpoints or connection strings') until the host's
IHostApplicationLifetime.ApplicationStarted has fired - which has not
happened yet inside a BeforeStart callback (invoked after BuildAsync but
before StartAsync).
GetConnectionStringAsync resolves a resource's connection string directly
against the DistributedApplicationModel instead, bypassing that guard. This
works for resources whose connection string does not depend on a
dynamically-allocated endpoint - e.g. a static, externally-hosted dependency
added via the AppHost's own AddConnectionString(name) - which is exactly the
primary BeforeStart use case (running migrations/seeding against such a
resource before dependent project resources start).
Also corrects the BeforeStart XML docs, which previously (and incorrectly)
suggested calling app.GetConnectionStringAsync(name) directly.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
… extension methods - AfterStart mirrors BeforeStart: runs after app.StartAsync() succeeds, before the tester returns the app; a throwing callback aborts start-up and disposes the app, same as BeforeStart. - OnHostStartUp() is now sealed on AspireTesterBase - consumers should use BeforeStart/AfterStart instead of the single-host-oriented hook. - WaitForResourceAsync(DistributedApplication, ...) moved from static methods on AspireTesterBase to extension methods on UnitTestExAspireExtensions (Aspire.Hosting namespace) for discoverability alongside AddProject etc. - Corrected BeforeStart/AfterStart doc wording: reentrant calls into GetDistributedApplicationAsync() deadlock/hang, they do not recursively kick off a second build. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…eding Exposes the same fluent AspireHttpMockClient API as AspireTesterBase.HttpMock, but directly against a started DistributedApplication with no AspireTester required. Lets an AppHost.cs itself pre-seed default request/response stubs (after 'await app.StartAsync()') so an exploratory/manual run of the AppHost - outside of any UnitTestEx test - does not fail against an un-stubbed external dependency. - New Aspire.Hosting.HttpMock(this DistributedApplication, string, string?, JsonElementComparerOptions?) extension method in UnitTestExAspireExtensions. - New Xunit test proving it configures the same underlying WireMock.Net resource as the tester's own HttpMock. - README: new 'Pre-seeding stubs from AppHost.cs' subsection under Aspire multi-host testing. - CHANGELOG updated. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- AspireTesterBase<TAppHost, TSelf>.ErrorWhenLogContains(LogLevel, exclude, include) registers a
continuous, streaming check that fails a test the moment any resource logs at/above a given
LogLevel (default Error), checked whenever log activity is drained (Checkpoint/Delay/Http).
Enabled by default at LogLevel.Error; pass LogLevel.None to opt out. Supports wildcard exclude
(known noise, always wins) and include (narrows what is checked) patterns; each call fully
replaces the prior configuration rather than merging with it.
- Added ExpectLogNotContains expectation (Tier 1), mirroring ExpectLogContains, to confirm log
output does not contain specified text.
- Added Person/test/log/{level} test-support endpoint to the sample API to exercise the above.
- Updated README and CHANGELOG for both features.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Delay() previously pre-drained and discarded any backlog captured before it started, to avoid misattributing it to the delay's own window. For Aspire testers this is the only mechanism resource console logs ever reach test output, so the discard caused genuine, permanent loss of log lines that were still visible to AssertLogContains (a raw, watermark-bypassing scan) but never printed by any later Checkpoint()/Delay(). WriteDelay now reports this backlog via WriteElapsedLogMessages() under its own preceding LOGGING > section (ahead of the delay's own DELAY (...) > marker), instead of silently dropping it via DrainElapsedLogMessages(). Silent when there is nothing to report. Updated the Tier 1 CheckpointAndDelayTest mirrors (Xunit/MSTest/NUnit) to assert the corrected behavior.
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Unresolved critical API-compatibility and concurrent log-drain findings, along with additional behavior issues, require changes before approval.
Review effort: Lite
Findings: 6
Open (6)
Resolved since last review (1)
DrainElapsedLogMessages and ResourceHttpClientSource.GetNewResourceLogMessagesAsync each derived their returned entries via buffer.Skip(lastCount).ToArray() but then separately read buffer.Count to advance the watermark. A resource log enqueued between the two reads could advance the watermark past an entry that was never included in newEntries, silently and permanently skipping it from all future drains/checks. Both now take a single buffer.ToArray() snapshot and derive newEntries and the advanced watermark from that same array, so the watermark can never advance past anything that wasn't also returned/checked. Addresses automated PR review feedback on AspireTesterBase.cs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
UnitTestEx.Aspire, a second, opt-in testing tier alongside the existingWebApplicationFactory-based tier (ApiTester/FunctionTester/GenericTester, etc.). Where the existing tier hosts a single system in-process for deep intra-domain testing (DI replacement, mockedHttpClients),UnitTestEx.Aspirespins up an entire .NET AspireAppHost- every project/container/executable resource it declares - as real, separately-hosted OS processes wired together with Aspire's actual service discovery, for proving genuine inter-domain interaction between services. Resolves #105.See the README's Aspire multi-host testing section for full usage.
What's included
AspireTester/AspireTesterBase<TAppHost, TSelf>- launches a realAppHost(DistributedApplicationTestingBuilderunder the hood), waits for resource readiness, and exposesHttp(resourceName)for calling any hosted resource by name.TesterBaseCoreholds only genuinely host-agnostic members; Tier 1'sTesterBasekeeps its full DI-replacement surface (ReplaceSingleton/ReplaceScoped/etc.) on top of it, whileAspireTesterBaseinheritsTesterBaseCoredirectly. A Tier 2 tester simply doesn't have DI-replacement methods to call, rather than having them throw at runtime.BeforeStart/AfterStarthooks -AspireTesterBase<TAppHost, TSelf>.BeforeStart(Func<DistributedApplication, Task>)runs after theDistributedApplicationis built but before any resource starts (e.g. DB migrations/seeding against a pre-existing resource), paired withGetConnectionStringAsyncfor resolving connection strings at that point.AfterStart(Func<DistributedApplication, Task>)runs once all resources have started - the natural place forWaitForResourceAsynccalls.OnHostStartUp()issealedonAspireTesterBasein favour of these two hooks.HttpMock- a sharedIHttpMockClient/IHttpMockRequest/IHttpMockResponseinterface is implemented by both Tier 1'sMockHttpClient(Factory)and Tier 2'sAspireHttpMockClient(backed by a real WireMock.Net resource), so the same fluent stubbing API - and the same YAML/JSON config schema viaWithRequestsFromResourceAsync- works against either tier.Aspire.Hosting.HttpMock(this DistributedApplication, ...)exposes the same API directly against a startedDistributedApplicationwith noAspireTesterrequired, so anAppHost.cscan pre-seed default stubs for exploratory/manual runs.WireMock.Net.Aspirepackage's container resource), so no Docker/Podman is required.UnitTestEx.Aspireships:JsonElementComparerMatcher- a custom WireMock.NetIMatcherdelegating JSON body matching to UnitTestEx's ownJsonElementComparer, giving genuine semantic parity (dates/GUIDs/numbers) with Tier 1's JSON matching.WireMockConsole.RunAsync- encapsulates all the self-hosted console boilerplate (port binding, settings, custom-matcher registration, graceful shutdown), reducing a consumer's mock hostProgram.csto one line. Builds anAspireWireMockServerSettings(a plain, unsealedWireMockServerSettingssubclass) and registers aWireMockRequestResponseLoggerthat logs every genuine stubbed request/response pair through the resource's own console output, giving parity with Tier 1'sMockHttpClientHandlerlogging - both now have an independently configurable log level (AspireWireMockServerSettings.RequestResponseLogLevel,MockHttpClientFactory.LogLevel/UseLogLevel(LogLevel)), defaulting toInformationandDebugrespectively, orNoneto disable.AddMockHostProject/WithMockHostEnvironmentAppHostextensions - remove the run-mode/publish-mode null-handling boilerplate needed since a mock resource must never appear in a published deployment manifest.AspireTesterBase<TAppHost, TSelf>captures each resource's console log output and exposes it via:Checkpoint(string reason)/Delay(TimeSpan, string? reason)- report anything logged since the lastCheckpoint/Delaycall (staying silent when there's nothing new), so background/hosted-service activity between assertions is still visible in test output.Delayno longer silently discards pre-existing backlog that arrives just before it starts - it now reports it under its own preceding section instead of losing it.ErrorWhenLogContains(LogLevel, string[]? exclude, string[]? include)- a continuous, streaming check that fails the test the moment any resource logs at or above the given level (defaultError, enabled by default); each call replaces rather than merges prior configuration.AssertLogContains(string, string?)/AssertLogNotContains(string, string?)- one-shot assertions against every captured log entry so far, across all resources or scoped to one.ResetLogs()- discards captured log entries/watermarks, for tests that deliberately reuse a host via a shared fixture.ExpectLogNotContainsexpectation, mirroring the existingExpectLogContains.tests/UnitTestEx.Aspire.MockHost- the copy-paste template project referenced by the README (not a shipped package - there's nothing to install).tests/UnitTestEx.Aspire.AppHostplus parity test suites (UnitTestEx.Aspire.MSTest.Test/.NUnit.Test/.Xunit.Test) exercising both tiers across all three test frameworks.Testing
dotnet build UnitTestEx.sln- 0 errors.Co-authored-by: Copilot App 223556219+Copilot@users.noreply.github.com