Skip to content

fix(webhooks): ignore opens SES marks as automated - #535

Merged
driaug merged 2 commits into
useplunk:nextfrom
joanproductivo:fix/531-automatic-opens
Oct 24, 2026
Merged

driaug merged 2 commits into
useplunk:nextfrom
joanproductivo:fix/531-automatic-opens

Conversation

@joanproductivo

@joanproductivo joanproductivo commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Description

Bug

SES publishes an Open for every fetch of the tracking pixel, including the fetches mail providers make on their own. Gmail's servers fetch the pixel a second or two after delivery, whether or not anyone reads the email, and Plunk counted that as an open. So almost every email sent to a Gmail address read as opened, segments on opens (email.opened) included contacts who never opened anything, campaign and analytics open rates were inflated, and workflows waiting for an open resumed seconds after sending.

Root cause

case 'Open' in apps/api/src/controllers/Webhooks.ts treats every Open as a person: it stamps openedAt, increments opens, moves the status to OPENED and tracks email.open, without reading anything from body.open.

What SES sends

Every Open (and Click) event carries isBotEvent, "Likely" or "Unlikely" (SES event contents). I measured it on a real account (eu-north-1, one Gmail recipient, events captured from the configuration set's SNS topic):

Case Event After mail.timestamp isBotEvent User agent
Gmail web open Gmail's own fetch (email not opened yet) 2.61 s Likely …Chrome/42.0.2311.135 Safari/537.36 Edge/12.246 Mozilla/5.0
Reader opens it 95.6 s Unlikely …Firefox/11.0 (via ggpht.com GoogleImageProxy)
Gmail closed in the browser Gmail's own fetch 2.24 s Likely same as above
Reader opens it 139.2 s Unlikely …GoogleImageProxy
Reader opens it right away Gmail's own fetch 2.51 s Likely same as above
Reader opens it 6.78 s Unlikely …GoogleImageProxy
Read in Apple Mail (Mail Privacy Protection on) Gmail's own fetch 2.25 s Likely same as above
Apple's proxy when the email is opened 139.0 s Unlikely Mozilla/5.0

Gmail's fetch was Likely every time (4 of 4) and every real open was Unlikely (4 of 4), including one under seven seconds after sending and Apple Mail's proxy. Gmail fetched the pixel again when the email was actually opened, so leaving its own fetch out does not lose the real open.

Solution

  • isAutomaticOpen(open), exported next to nextStatus: an Open is automatic only if SES marks it exactly "Likely". Anything else (no open block, no mark, an unknown value) counts, as every Open did before SES added the mark.
  • At the top of case 'Open', an automatic Open logs Automatic open ignored … and returns 200 before anything is written: no openedAt, no opens, no status change, no markStatsDirty and no email.open event, so no workflow runs and no WAIT_FOR_EVENT resumes. The reader's own open arrives later and is recorded as the first one.
  • The log line of every counted open now shows its isBotEvent mark, so the signal can be watched in production.
  • Docs: the tracking guide describes how automatic opens are left out, and the webhook event table no longer says email.open and email.click fire the first time only (they fire on every open and click).

There is no schema change, migration or extra query: an automatic Open now does less work than before (no update, no Redis write, no event insert).

Worth knowing

  • Behaviour change on upgrade: open rates drop to what people actually opened, and automatic fetches stop triggering workflows and Webhook steps on email.open. Past data is not corrected. There is no setting to turn it off: only opens SES marks Likely are left out, and every other open still counts.
  • isBotEvent is AWS's classification. AWS does not publish how it is computed. Opens it marks Unlikely keep counting, so a provider or gateway it misses behaves as today. If AWS stops sending the mark, Plunk goes back to counting every open.
  • Clicks are left as they are. In the same test a second click, six seconds after the reader's and from a different OS, was also marked Unlikely, so the mark does not look reliable enough for clicks yet. That is better handled separately.
  • Related but separate: WAIT_FOR_EVENT resumes on an email.open of any email of the contact, not only the one the workflow sent, and the email.opened / email.clicked names used in the workflow docs and placeholders are never emitted as events. Happy to open issues for those.

Type of Change

  • fix: Bug fix (PATCH version bump)

Testing

  • New Webhooks.automaticOpen.test.ts: Likely is automatic; Unlikely, no open block, open: null, no mark, and an isBotEvent of "likely", true, null or "Unknown" all count.
  • New automatic opens block in Webhooks.sns.test.ts, with the user agents measured above: a Likely open leaves the row, the events and the campaign's dirty set untouched and still returns 200; an Unlikely open (Gmail and Apple Mail) counts once with isFirstOpen: true; the measured sequence (automatic, then the reader) records one open and one event; a redelivered automatic open is ignored too. On v0.17.0 the automatic-open cases fail by assertion, while the Unlikely case already passes and keeps passing. The existing open tests, which send no open block, are unchanged and pass.
  • Full suite locally against Postgres 16 and Redis 7 (Node 24): 1415 passed, against 1407 on v0.17.0, where the branch starts. Merged with the current next (bf6a6ef), the full suite passes too (1445). yarn lint reports no errors and yarn build --filter=api succeeds. Prettier and ESLint are clean on the lines this PR changes.

Checklist

  • PR title follows conventional commits format
  • Code builds successfully
  • Tests pass locally
  • Documentation updated (if needed)

Related Issues

Closes #531

Related: #534 (open and click rates measured against tracked emails; no overlapping code)

🤖 Generated with Claude Code

@driaug

driaug commented Oct 24, 2026

Copy link
Copy Markdown
Member

Thanks, this is a great find, and the measurements against a real account made it easy to review. The isBotEvent check and the early return in case 'Open' are exactly what we want.

One change before merging: could you drop the IGNORE_AUTOMATIC_OPENS setting and always leave out opens SES marks as Likely? I can't think of a case where someone would want Gmail's automatic fetch counted as an open, and an open SES doesn't mark Likely still counts. So the opt-out doesn't protect against anything, and it's most of the PR's footprint: three env files, CLAUDE.md, docker-compose.yml, turbo.json, the test/setup.ts override and the env-parsing tests.

So that means:

  • remove the constant from constants.ts and the enabled parameter from isAutomaticOpen
  • revert the changes to .env.self-host.example, apps/api/.env.example, environment-variables.mdx, CLAUDE.md, docker-compose.yml, turbo.json and test/setup.ts
  • drop the self-hosting sentence about IGNORE_AUTOMATIC_OPENS from tracking.mdx, and the env cases from the tests

Everything else, including the webhooks.mdx fix (email.open and email.click do fire on every open and click), can stay as it is. Happy to merge after that.

joanproductivo and others added 2 commits October 24, 2026 18:22
SES publishes an Open for every fetch of the tracking pixel, including
the ones mail providers make on their own. Gmail's servers fetch it a
second or two after delivery whether or not anyone reads the email, so
almost every email to Gmail was counted as opened, segments on opens
were wrong, and workflows waiting for an open resumed seconds after
sending.

SES marks every Open with `isBotEvent` ("Likely" / "Unlikely"). Measured
against a Gmail inbox, Gmail's fetch was "Likely" every time and every
real open was "Unlikely", including one a few seconds after delivery and
Apple Mail's proxy. An Open marked "Likely" now writes nothing: no
openedAt, no open count, no status change, no `email.open` event and no
workflow. The reader's own open arrives later and counts as the first
one. An Open without the mark still counts, as before.

The log line of every counted open shows its isBotEvent mark.

Refs useplunk#531

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Describe how Plunk treats the opens mail providers make on their own:
Gmail can fetch the pixel on delivery, SES marks those fetches, and they
are not counted. Corporate gateways that pre-fetch images are still
counted unless SES marks them.

The webhook event table said `email.open` and `email.click` fire the
first time only; they fire on every open and click.

Refs useplunk#531

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@joanproductivo
joanproductivo force-pushed the fix/531-automatic-opens branch from 2282b7e to 9805c18 Compare October 24, 2026 16:26
@joanproductivo

Copy link
Copy Markdown
Contributor Author

Thanks for the review! Done: IGNORE_AUTOMATIC_OPENS is gone and opens SES marks Likely are always left out.

  • isAutomaticOpen no longer takes the enabled parameter, and the constant is removed from constants.ts.
  • .env.self-host.example, apps/api/.env.example, environment-variables.mdx, CLAUDE.md, docker-compose.yml, turbo.json and test/setup.ts are back to how they are on next.
  • The self-hosting sentence is dropped from tracking.mdx, and the env cases from the tests. In webhooks.mdx I also dropped "By default" from the email.open row, since there is no setting any more.

I folded the change into the two existing commits and force-pushed, so the commit messages no longer mention the setting either. The full suite passes locally (1415 tests), and merged with the current next (bf6a6ef) too (1445), along with tsc and yarn lint. I've updated the PR description to match.

@driaug
driaug merged commit 3c1af34 into useplunk:next Oct 24, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Automatic pixel fetches (Gmail image proxy, Apple Mail Privacy Protection) are counted as real opens

2 participants