[zacharycmontoya]

Motivation

The tests assert that the Datadog SDKs consistently emit the correct set of span attributes, span (resource) names, and span statuses when OTel semantics is enabled. This update asserts the behaviors introduced in DataDog/dd-trace-java#12765, with known follow-up work needed for akka-http and play frameworks.

Changes

Overall, this PR runs the test cases in test_otel_http_semantics.py against dd-trace-java. Several other changes were needed to accomplish this:

• Removes the _span_is_sampled check used in the OTEL_SEMANTICS_OTLP_SAMPLING_RULES. This check was intended to assert that spans were sampled, but this is unneeded because OTLP spans that are exported from the SDK, by definition, are sampled. This is not the case with MsgPack spans where all spans are sent to the local agent
• Adds DD_METRICS_OTEL_ENABLED=true to the set of environment variables for the OTEL_SEMANTICS_OTLP_TRACE_METRICS scenario
• Updates the GET /make_distant_call in all Java weblog applications to accept an optional method parameter, which is tested in the OTel HTTP Semantics test cases

Workflow

1. ⚠️ Create your PR as draft ⚠️
2. Work on you PR until the CI passes
3. Mark it as ready for review
   • Tests, manifest, weblog are modified -> you'll need a review from system-tests-reviewers: ask to one of youre co-worker familiar with the tested feature.
   • Framework is modified, or non obvious usage of it -> get a review from system-tests-core (slack)

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Notes

This is intended to fail in the pull request CI because the changes are not merged, but they can be run locally with dev branches of dd-trace-java following the binaries.md documentation

[github-actions]

CODEOWNERS have been resolved as:

utils/build/docker/java/play/app/SystemTestsAkkaHttpServerProvider.scala  @DataDog/system-tests-reviewers
manifests/java.yml                                                      @DataDog/system-tests-reviewers
tests/test_otel_http_semantics.py                                       @DataDog/system-tests-reviewers
utils/_context/_scenarios/otel_semantics.py                             @DataDog/system-tests-core
utils/build/docker/java/akka-http/src/main/scala/com/datadoghq/akka_http/AppSecRoutes.scala  @DataDog/system-tests-reviewers
utils/build/docker/java/akka-http/src/main/scala/com/datadoghq/akka_http/Main.scala  @DataDog/system-tests-reviewers
utils/build/docker/java/jersey-grizzly2/src/main/java/com/datadoghq/jersey/MyResource.java  @DataDog/system-tests-reviewers
utils/build/docker/java/play/app/controllers/AppSecController.scala     @DataDog/system-tests-reviewers
utils/build/docker/java/play/conf/application.conf                      @DataDog/system-tests-reviewers
utils/build/docker/java/play/conf/routes                                @DataDog/system-tests-reviewers
utils/build/docker/java/play/pom.xml                                    @DataDog/system-tests-reviewers
utils/build/docker/java/ratpack/src/main/java/com/datadoghq/ratpack/Main.java  @DataDog/system-tests-reviewers
utils/build/docker/java/resteasy-netty3/src/main/java/com/datadoghq/resteasy/MyResource.java  @DataDog/system-tests-reviewers
utils/build/docker/java/spring-boot-3-native/src/main/java/com/datadoghq/springbootnative/WebController.java  @DataDog/system-tests-reviewers
utils/build/docker/java/spring-boot/src/main/java/com/datadoghq/system_tests/springboot/App.java  @DataDog/system-tests-reviewers
utils/build/docker/java/vertx3/pom.xml                                  @DataDog/system-tests-reviewers
utils/build/docker/java/vertx3/src/main/java/com/datadoghq/vertx3/Main.java  @DataDog/system-tests-reviewers
utils/build/docker/java/vertx4/src/main/java/com/datadoghq/vertx4/Main.java  @DataDog/system-tests-reviewers

[datadog-prod-us1-6]

 

✨ Unblock PR with BitsAI

❌ Errors

Your PR has failed checks. Please review the issues below and take necessary action before merging.

🚦 20 Pipeline jobs failed

Testing the test | System Tests (java, dev) / End-to-end #2 / spring-boot-3-native 2 — ❌ 40 tests failed · 🔧 Needs a code fix, caused by this PR

View more details · View in GitHub Actions

❌ tests.test_otel_http_semantics.Test_OtelSemantics_OTLP_Spans_Http_ErrorStatusConfiguration.test_client_error_statuses_config_excludes_500[spring-boot-3-native] from system_tests_suite

AssertionError: could not correlate a SERVER span to request FMDTBCEILSALSMFABMIRBIFVQWSIWFVSJIDY by user agent; the correlated OTLP payloads contained 0 SERVER spans
assert []

self = <tests.test_otel_http_semantics.Test_OtelSemantics_OTLP_Spans_Http_ErrorStatusConfiguration object at 0x7f0e6068e4b0>

    def test_client_error_statuses_config_excludes_500(self) -> None:
>       span = _client_span(self.response)

tests/test_otel_http_semantics.py:703: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
...

❌ tests.test_otel_http_semantics.Test_OtelSemantics_OTLP_Spans_Http_ErrorStatusConfiguration.test_client_error_statuses_config_overrides[spring-boot-3-native] from system_tests_suite

AssertionError: could not correlate a SERVER span to request IZKKERFMGOWLVKRKAYBSSLUNPCIBOHVTQTOP by user agent; the correlated OTLP payloads contained 0 SERVER spans
assert []

self = <tests.test_otel_http_semantics.Test_OtelSemantics_OTLP_Spans_Http_ErrorStatusConfiguration object at 0x7f0e6068f8f0>

    def test_client_error_statuses_config_overrides(self) -> None:
>       span = _client_span(self.response)

tests/test_otel_http_semantics.py:694: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
...

❌ tests.test_otel_http_semantics.Test_OtelSemantics_OTLP_Spans_Http_ErrorStatusConfiguration.test_server_error_statuses_config_excludes_500[spring-boot-3-native] from system_tests_suite

AssertionError: could not correlate a SERVER span to request UZMRDPOOEZXOQQEJLLHDGNDGPKICOYDTFUSU by user agent; the correlated OTLP payloads contained 0 SERVER spans
assert []

self = <tests.test_otel_http_semantics.Test_OtelSemantics_OTLP_Spans_Http_ErrorStatusConfiguration object at 0x7f0e6068fc80>

    def test_server_error_statuses_config_excludes_500(self) -> None:
        """The configured range replaces the OTel rule, it does not add to it.
    
        The other half of precedence, and the half that was untested. With the range configured
        to 200 only, a 500 is outside it, so a tracer that hardcodes the 5xx threshold and merely
...

↳ and 37 more — View all

Testing the test | System Tests (php, dev) / End-to-end #1 / apache-mod-7.0-zts 1 — ❌ 1 test failed · 🔧 Needs a code fix, caused by this PR

View more details · View in GitHub Actions

❌ tests.debugger.test_debugger_expression_language.Test_Debugger_Expression_Language.test_expression_language_access_variables[apache-mod-7.0-zts] from system_tests_suite

AssertionError: assert 'Probes did not reach INSTALLED status' is None
 +  where 'Probes did not reach INSTALLED status' = <built-in method join of str object at 0x7f01d142b720>(['Probes did not reach INSTALLED status'])
 +    where <built-in method join of str object at 0x7f01d142b720> = '\n'.join
 +    and   ['Probes did not reach INSTALLED status'] = <tests.debugger.test_debugger_expression_language.Test_Debugger_Expression_Language object at 0x7f01c8405ca0>.setup_failures

self = <tests.debugger.test_debugger_expression_language.Test_Debugger_Expression_Language object at 0x7f01c8405ca0>

    def test_expression_language_access_variables(self):
>       self._assert(expected_response=200)

...

Testing the test | System Tests (php, dev) / End-to-end #1 / apache-mod-7.1-zts 1 — ❌ 2 tests failed · 🔧 Needs a code fix, caused by this PR

View more details · View in GitHub Actions

❌ tests.debugger.test_debugger_expression_language.Test_Debugger_Expression_Language.test_expression_language_access_variables[apache-mod-7.1-zts] from system_tests_suite

AssertionError: assert 'Probes did not reach INSTALLED status' is None
 +  where 'Probes did not reach INSTALLED status' = <built-in method join of str object at 0x7f9e8362b720>(['Probes did not reach INSTALLED status'])
 +    where <built-in method join of str object at 0x7f9e8362b720> = '\n'.join
 +    and   ['Probes did not reach INSTALLED status'] = <tests.debugger.test_debugger_expression_language.Test_Debugger_Expression_Language object at 0x7f9e7948c950>.setup_failures

self = <tests.debugger.test_debugger_expression_language.Test_Debugger_Expression_Language object at 0x7f9e7948c950>

    def test_expression_language_access_variables(self):
>       self._assert(expected_response=200)

...

❌ tests.debugger.test_debugger_probe_snapshot.Test_Debugger_Method_Probe_Snaphots_With_SCM.test_log_method_snapshot[apache-mod-7.1-zts] from system_tests_suite

AssertionError: assert 'Probes did not reach INSTALLED status' is None
 +  where 'Probes did not reach INSTALLED status' = <built-in method join of str object at 0x7f1509a2b720>(['Probes did not reach INSTALLED status'])
 +    where <built-in method join of str object at 0x7f1509a2b720> = '\n'.join
 +    and   ['Probes did not reach INSTALLED status'] = <tests.debugger.test_debugger_probe_snapshot.Test_Debugger_Method_Probe_Snaphots_With_SCM object at 0x7f14f38238c0>.setup_failures

self = <tests.debugger.test_debugger_probe_snapshot.Test_Debugger_Method_Probe_Snaphots_With_SCM object at 0x7f14f38238c0>

    def test_log_method_snapshot(self):
>       self._assert()

...

View all 20 failed jobs.

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 0beaf76 | Docs | View more details | Give us feedback!}