Skip to content

Support XML sitemaps and the Google News sitemap format - #609

Open
smoltafet wants to merge 2 commits into
kurtmckee:mainfrom
smoltafet:feat/google-news-sitemap
Open

smoltafet wants to merge 2 commits into
kurtmckee:mainfrom
smoltafet:feat/google-news-sitemap

Conversation

@smoltafet

@smoltafet smoltafet commented Oct 24, 2026 •

Copy link
Copy Markdown

Fixes #535.

Feedparser previously parsed sitemap documents to an empty result (no version, no entries). This adds sitemap support following the existing namespace-handler architecture:

  • New feedparser/namespaces/sitemap.py: a <urlset> maps to the feed and each <url> to an entry — <loc> becomes the entry link/links, <lastmod> becomes updated/updated_parsed, and <changefreq>/<priority> are stored verbatim on the entry. Sitemap index documents (<sitemapindex>/<sitemap>) are handled the same way. The parsed version is reported as sitemap.
  • The Google News extension (news:news) is exposed as a nested news dict on each entry (publication.name/publication.language, publication_date (+_parsed), title, keywords, genres, stock_tickers), and its title, publication date, and keywords are additionally mapped onto the standard entry title, published/published_parsed, and tags keys so news sitemaps work with feed-oriented consumers such as feed2fedi (the reporter's use case).
  • feedparser/mixin.py registers the namespace module and the sitemap/news namespace URIs; feedparser/namespaces/_base.py routes <url> to the sitemap entry handler only while inside a sitemap (RSS <image><url> behavior is unchanged and covered by a regression test); feedparser/api.py adds the sitemap version string.

Tests: new tests/test_sitemap.py (8 tests: plain urlset fields, news publication/mapping, optional news elements, no cross-entry leakage, sitemap index, RSS <url> regression) plus two tests/wellformed/sitemap/ fixtures. Full suite: 4309 passed, 8 skipped. Baseline (implementation stashed, new fixtures present) fails exactly the 4 new wellformed fixture cases and passes everything else (4297 passed, 8 skipped), so there are no regressions. The stale volunteer from January 2026 never produced a PR, so I went ahead; happy to adjust naming/shape to maintainer preference.

@smoltafet

Copy link
Copy Markdown
Author

Fixed the pre-commit.ci failure: the XML literals in tests/test_sitemap.py used 2 and 6 space indents, which editorconfig-checker flagged. Re-indented them to multiples of 4, the suite passes locally (4309 passed, 8 skipped), and pre-commit.ci is green on the new head.

@smoltafet

Copy link
Copy Markdown
Author

Hi @kurtmckee, could you take a look at this when you have a moment? Happy to fix up anything that needs changing. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support Google news sitemap format

1 participant