Repository navigation
test: Fix metrics-related timeouts in instrumentation tests and stabilize 16 KB emulator job - #228
Conversation
KishanPRao
left a comment
There was a problem hiding this comment.
Mocking metrics / turning them off fixes the metrics timeouts and handledException.
The description needs updating, since the issue wasn't the upload taking longer.
Reports were stuck behind stalled metrics calls on the same queue.
The PR only fixes part of the Sauce flakiness. The native crash report tests and dumpWithoutCrash are unchanged and will likely still fail. On the latest run, Android 14 passed only on attempt 3 of 3.
Attempts 1 and 2 failed fatalCrashIsRecoveredAndIngestedExactlyOnce:
- Job (green): https://github.com/backtrace-labs/backtrace-android/actions/runs/38459153198/job/115732238656
- Attemp 1: https://app.eu-central-1.saucelabs.com/tests/500fbf578e884dc79c119cd30fc9361d
- Attempt 2: https://app.eu-central-1.saucelabs.com/tests/c082024ecf0e41b2856bd44cecbce128
"More stable emulator image": can we prove this? We might need more CI runs to confirm.
The description needs to be updated to cover just the metrics fixes, or the PR needs to be extended to fix the remaining flakes.
(And a BT ticket hasn't been linked here)
Fixes the metrics-related timeouts in the instrumentation tests. The library metrics tests and
ExampleInstrumentedTest.handledExceptionfailed for the same reason.Root cause
Metrics and report uploads share a single sender thread. When
metrics.enable()runs, it sends startup events to the real events endpoint right away. On Sauce devices and CI those calls could stall. Any report submitted afterwards was stuck in the queue behind them, so tests waiting on a report upload timed out.Example app instrumentation tests (
example-app)MainActivity.EXTRA_DISABLE_METRICSintent extra.ExampleInstrumentedTestsets it, so metrics stay off and can't hold up the reports under test. None of these tests assert on metrics.handledExceptionhardening:Ok, so a failed upload shows its real reason instead of an empty rxIdLibrary tests (
backtrace-library)BacktraceClientMetricsTest,BacktraceClientSummedEventTest,BacktraceClientUniqueEventTest: route startup metrics events to mock handlers insetUp(), so no test reaches the real events endpoint.CI: 16 KB qualification job
google_apis_ps16kimage bumped from API 35 to API 36. The API 35 image crashed system_server mid-run (NullPointerExceptioninReferenceQueueDaemon, a runtime bug unrelated to our code).Ref: BT-7865