Repository navigation
Fix: detect rich-message spam bypassing text-based filters - #51
Merged
Merged
Conversation
Rich messages (Message.rich_message) arrive with empty text while their payload hides in page blocks, so the AI spam monitor's text/caption-only filter never classified them and the inline-keyboard URL check (which only inspects reply_markup) missed their URL buttons. This let the 2026-10-17 adult-spam forward (t.me/base6d1ef16_bot) sit in the group for hours until a manual admin delete. - New bot.services.rich_message: duck-typed walker extracting plain text (headings, paragraphs, table cells, button labels, custom emoji alt text) and deduplicated URLs (buttons, text links, login/web-app) from rich message blocks. - Group 1 (inline keyboard spam) now also auto-deletes rich messages carrying non-whitelisted URLs, with its own notification wording. - ai_spam_monitor now includes rich text + URLs in the classified text, so URL-less rich spam still reaches the classifier and admin alerts. - 21 new tests; full suite 1320 passed, ruff + mypy clean.
Per review: no stock Telegram client can compose rich messages or their buttons, so button presence is the automation signature (same rationale as inline keyboard spam). URL-only rich content without buttons now goes to the AI monitor, mirroring how plain text links are handled, instead of being auto-deleted. - services.rich_message: new has_rich_buttons() covering block buttons and inline RichTextButton nodes. - anti_spam: has_rich_message_buttons() replaces the URL-based rich detector; notification wording updated to mention buttons. - Tests updated: whitelisted-URL buttons are still deleted; URL-only rich content is ignored by group 1.
- Walker now traverses RichBlockCaption.credit and RichTextButton.button, so text/URLs hidden in quotation credits and inline text buttons reach both the AI monitor and the button detector. - Group 1 skips is_automatic_forward messages: linked-channel auto-forwards are admin-configured and must never trigger delete+restrict against the channel/service sender. - Tests: credit extraction, inline button URL/text extraction, button-in-credit detection, auto-forward skip.
has_non_whitelisted_link() now also scans extract_rich_urls(), so probation users cannot smuggle non-whitelisted links past the filter inside rich text blocks. No behavior change for duplicate_spam: rich messages without text return before the link check there.
Blockers: - B1: feed rich text into bio-bait matching (short text-only rich spam was invisible to every layer) - B2: duplicate detection falls back to extracted rich text as the tracking key (rich-only floods were never tracked) Walker hardening: - M1: extract RichBlockListItem.label - M2: descend into copy_text/login_url/web_app/chat-picker payloads; extract_rich_urls also detects URL-shaped strings in plain text (rich blocks carry no entity metadata; dotted-hostname required to avoid flagging fragments like 'https://' alone) - M3: depth budget (64) on walkers; over-budget nesting is fail-closed for has_rich_buttons (only craftable, never client-composed) Monitor: - M5: truncate classifier input to 8000 chars - M9: light normalization (NFKC + zero-width/bidi strip) before classification Tests: e2e ai_spam_filter on rich-only updates, classify truncation, is_automatic_forward on the legacy keyboard path, login_url/web_app extraction, credit-nested button URLs, deep-nesting behavior, NO_RESTRICT template path, admin exemption, bio-bait on rich text, rich duplicate tracking. L4/L5/L15/L16 addressed.
…lete+restrict)
Per policy decision: forwarders of rich-button messages keep
delete+restrict (same as the inline-keyboard rule; the incident
spammer was a forwarder). The notification now reads 'Pesan terusan'
for forwarded messages and carries an appeal line ('Hubungi admin
grup') in both restricted and non-restricted variants.
_handle_group_spam gains an optional extra_format mapping for
template placeholders; property tests for _url_shaped_candidates
invariants (no-http => empty, well-formed candidates, no dupes).
The M3 depth budget removed the RecursionError fail-fast that used to terminate walks over infinitely-generative objects (mocks): with every attribute truthy, the walker branched ~15^64 and never finished, OOMing the suite and hanging CI. _walk/_visit now only descend into str/list/tuple or aiogram-module objects (_is_rich_node); the depth + node budgets remain as defense-in-depth for hostile aiogram-shaped payloads.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
On 2026-10-17 a spammer forwarded adult spam ("HOT 2026 DROP",
t.me/base6d1ef16_botlinks) into the group as a Telegram rich message. It sat there for hours until a manual admin delete.Root cause: rich messages arrive with empty
message.text— the whole payload lives inmessage.rich_messagepage blocks. The bot's filters only readtext/caption:ai_spam_monitor(group 7):_classifiable_text()returned"", soai_spam_filternever matched — verified in Logfire, the update was dispatched but group 7 never ran and zero classification lines were logged.inline_keyboard_spam(group 1):has_non_whitelisted_inline_keyboard_urlsonly inspectsmessage.reply_markup; richpageButtonURL buttons live in the block tree, so it completed without flagging.Changes
bot.services.rich_message: duck-typed walker extracting plain text (headings, paragraphs, table cells, button labels, custom emoji alt text, quotation/mediacreditlines, list-itemlabels) and deduplicated URLs (buttons, text links, login/web-app/copy-text payloads, inlineRichTextButtonnodes, plus URL-shaped strings in plain text — rich blocks carry no entity metadata) from rich message blocks, plushas_rich_buttons(). Depth budget (64): over-budget nesting is fail-closed for the button predicate. aiogram 3.31 already parsesMessage.rich_message; the data was available, just unread.is_automatic_forward) are skipped — admin-configured content must never trigger delete+restrict.has_non_whitelisted_link()now also scans rich block URLs, so probation users cannot smuggle links inside rich text (no behavior change for duplicate_spam's short-message gate)._classifiable_text()now includes rich text + URLs (light unicode normalization, classifier input truncated to 8000 chars), so URL-less rich spam still reaches the classifier and admin alerts.Verification
reviews/2026-10-17-rich-spam/, gitignored): verdict merge after fixes — 2 blockers (B1/B2) and mediums M1–M5/M7–M9 all addressed in this branch.tests/test_rich_message.py(+3 in neighboring suites); full suite 1353 passed,ruff checkandmypyclean.ai_spam_filterandhas_rich_message_buttons(both were False before the fix).