Repository navigation
fix(webhooks): ignore opens SES marks as automated - #535
Conversation
|
Thanks, this is a great find, and the measurements against a real account made it easy to review. The One change before merging: could you drop the So that means:
Everything else, including the |
SES publishes an Open for every fetch of the tracking pixel, including
the ones mail providers make on their own. Gmail's servers fetch it a
second or two after delivery whether or not anyone reads the email, so
almost every email to Gmail was counted as opened, segments on opens
were wrong, and workflows waiting for an open resumed seconds after
sending.
SES marks every Open with `isBotEvent` ("Likely" / "Unlikely"). Measured
against a Gmail inbox, Gmail's fetch was "Likely" every time and every
real open was "Unlikely", including one a few seconds after delivery and
Apple Mail's proxy. An Open marked "Likely" now writes nothing: no
openedAt, no open count, no status change, no `email.open` event and no
workflow. The reader's own open arrives later and counts as the first
one. An Open without the mark still counts, as before.
The log line of every counted open shows its isBotEvent mark.
Refs useplunk#531
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Describe how Plunk treats the opens mail providers make on their own: Gmail can fetch the pixel on delivery, SES marks those fetches, and they are not counted. Corporate gateways that pre-fetch images are still counted unless SES marks them. The webhook event table said `email.open` and `email.click` fire the first time only; they fire on every open and click. Refs useplunk#531 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2282b7e to
9805c18
Compare
|
Thanks for the review! Done:
I folded the change into the two existing commits and force-pushed, so the commit messages no longer mention the setting either. The full suite passes locally (1415 tests), and merged with the current |
Description
Bug
SES publishes an
Openfor every fetch of the tracking pixel, including the fetches mail providers make on their own. Gmail's servers fetch the pixel a second or two after delivery, whether or not anyone reads the email, and Plunk counted that as an open. So almost every email sent to a Gmail address read as opened, segments on opens (email.opened) included contacts who never opened anything, campaign and analytics open rates were inflated, and workflows waiting for an open resumed seconds after sending.Root cause
case 'Open'inapps/api/src/controllers/Webhooks.tstreats every Open as a person: it stampsopenedAt, incrementsopens, moves the status toOPENEDand tracksemail.open, without reading anything frombody.open.What SES sends
Every Open (and Click) event carries
isBotEvent,"Likely"or"Unlikely"(SES event contents). I measured it on a real account (eu-north-1, one Gmail recipient, events captured from the configuration set's SNS topic):mail.timestampisBotEventLikely…Chrome/42.0.2311.135 Safari/537.36 Edge/12.246 Mozilla/5.0Unlikely…Firefox/11.0 (via ggpht.com GoogleImageProxy)LikelyUnlikely…GoogleImageProxyLikelyUnlikely…GoogleImageProxyLikelyUnlikelyMozilla/5.0Gmail's fetch was
Likelyevery time (4 of 4) and every real open wasUnlikely(4 of 4), including one under seven seconds after sending and Apple Mail's proxy. Gmail fetched the pixel again when the email was actually opened, so leaving its own fetch out does not lose the real open.Solution
isAutomaticOpen(open), exported next tonextStatus: an Open is automatic only if SES marks it exactly"Likely". Anything else (noopenblock, no mark, an unknown value) counts, as every Open did before SES added the mark.case 'Open', an automatic Open logsAutomatic open ignored …and returns 200 before anything is written: noopenedAt, noopens, no status change, nomarkStatsDirtyand noemail.openevent, so no workflow runs and noWAIT_FOR_EVENTresumes. The reader's own open arrives later and is recorded as the first one.isBotEventmark, so the signal can be watched in production.email.openandemail.clickfire the first time only (they fire on every open and click).There is no schema change, migration or extra query: an automatic Open now does less work than before (no update, no Redis write, no event insert).
Worth knowing
email.open. Past data is not corrected. There is no setting to turn it off: only opens SES marksLikelyare left out, and every other open still counts.isBotEventis AWS's classification. AWS does not publish how it is computed. Opens it marksUnlikelykeep counting, so a provider or gateway it misses behaves as today. If AWS stops sending the mark, Plunk goes back to counting every open.Unlikely, so the mark does not look reliable enough for clicks yet. That is better handled separately.WAIT_FOR_EVENTresumes on anemail.openof any email of the contact, not only the one the workflow sent, and theemail.opened/email.clickednames used in the workflow docs and placeholders are never emitted as events. Happy to open issues for those.Type of Change
fix:Bug fix (PATCH version bump)Testing
Webhooks.automaticOpen.test.ts:Likelyis automatic;Unlikely, noopenblock,open: null, no mark, and anisBotEventof"likely",true,nullor"Unknown"all count.automatic opensblock inWebhooks.sns.test.ts, with the user agents measured above: aLikelyopen leaves the row, the events and the campaign's dirty set untouched and still returns 200; anUnlikelyopen (Gmail and Apple Mail) counts once withisFirstOpen: true; the measured sequence (automatic, then the reader) records one open and one event; a redelivered automatic open is ignored too. On v0.17.0 the automatic-open cases fail by assertion, while theUnlikelycase already passes and keeps passing. The existing open tests, which send noopenblock, are unchanged and pass.next(bf6a6ef), the full suite passes too (1445).yarn lintreports no errors andyarn build --filter=apisucceeds. Prettier and ESLint are clean on the lines this PR changes.Checklist
Related Issues
Closes #531
Related: #534 (open and click rates measured against tracked emails; no overlapping code)
🤖 Generated with Claude Code