fix(bedrock): do not report a guardrail activation on every call - #4487
linhongyu510 wants to merge 1 commit into
Conversation
is_guardrail_activated ended with
response.get("amazon-bedrock-guardrailAction") != "NONE"
When no guardrail is configured Bedrock omits that key, so .get() returns
None and None != "NONE" is True. Every ordinary response was counted as a
guardrail activation and gen_ai.bedrock.guardrail.activation was
incremented on every Bedrock call. Default the key to "NONE".
Also guard the two guardrail_activation.add() call sites: when metrics are
disabled the SDK sets the counter to None, and a genuine activation raised
AttributeError. The span attributes are still set in that case.
Fixes traceloop#4471
|
Understand this PR’s impact Explore downstream dependencies and potential security impact with Blast Radius. No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (2)
Included review availability: Your plan provides up to 8 included reviews per hour; 7 remain after this review. 📝 WalkthroughWalkthroughThe Bedrock instrumentation now treats missing guardrail actions as inactive. Guardrail metric updates are skipped when metrics are disabled. Regression tests cover inactive responses, activations, metric counts, and disabled metrics. ChangesBedrock guardrail handling
Priority: ➖ Normal Estimated code review effort: 2 (Simple) | ~10 minutes Change: Bug fix · Severity of issue fixed: Medium 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Warning Comment |
|
Closing as a duplicate — I opened this without checking for existing PRs first. #4475 (2026-09-16) already fixes the false-positive activation with the same one-line default. It doesn't yet cover the second half of #4471 (the |
What
is_guardrail_activatedended with:When no guardrail is configured, Bedrock omits that key, so
.get()returnsNone, andNone != "NONE"isTrue. Every ordinary response — from users with no guardrail configured at all — was treated as a guardrail activation, andgen_ai.bedrock.guardrail.activationwas incremented on every Bedrock call. Fixes #4471.This bites the
conversepath in particular:guardrail_conversecallsis_guardrail_activated(response)directly, and a normal converse response has noamazon-bedrock-guardrailActionkey.Fix
Two changes in
guardrail.py:Default the key to
"NONE"so an absent key is not an activation:Guard the two
guardrail_activation.add()call sites againstNone. The issue also notes a crash: when metrics are disabled,_instrument()sets every counter toNone, but the two activation call sites dereferenced it unconditionally, so a genuine activation raisedAttributeError: 'NoneType' object has no attribute 'add'. The guard skips the counter while still setting the span attributes (span attributes are independent of the metrics toggle).+5/-3in one source file.Reproduction (offline, no AWS)
Detection, before the fix:
Metrics-disabled crash, before the fix:
Both are resolved after the change, and real activations (
GUARDRAIL_INTERVENED,stopReason == "guardrail_intervened",completionReason == "CONTENT_FILTERED") are still detected.Tests
Added
tests/test_guardrail_activation.py(pure unit, no cassette / no AWS):Load-bearing, verified by reverting each half:
"NONE"default fails the "no key → False" cases andtest_converse_without_guardrail_does_not_record_activation;Noneguard failstest_activation_with_metrics_disabled_does_not_crashwith the exactAttributeErrorfrom the issue.ruff checkclean on both files (repo config).Summary by CodeRabbit
Bug Fixes
Tests