Repository navigation
fix(website): serve sitemap.xml and robots.txt from Next.js - #15
Merged
Merged
Conversation
nginx routed /sitemap.xml and /robots.txt to /app/public, where neither file exists: Next.js generates both from app/sitemap.ts and app/robots.ts. The sitemap returned 404 on stackdog.app, and robots.txt showed only Cloudflare's managed content, without the Sitemap line. Both now go to the Next.js proxy. Checked against a locally built image: /sitemap.xml returns 200 with all five pages, /robots.txt returns 200 with the Sitemap line, and public assets are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
https://stackdog.app/sitemap.xmlreturns 404, so search engines don't find/php-shell-detectionor/php-shell-quickstartthrough the sitemap.website/nginx.confsent/sitemap.xmland/robots.txttoroot /app/public, butpublic/only holdsstackdog-mark.png. Next.js generates both files fromapp/sitemap.tsandapp/robots.ts, and nginx never passed those requests to it.Live
/robots.txtis currently only Cloudflare's managed content (# BEGIN Cloudflare Managed content), without the app'sSitemap:line.Fix
robots.txtandsitemap.xmlare removed from the static-filelocationregex, so both reach the Next.js proxy. A one-line change, plus a comment.Verification
I built the website image locally (
docker build --build-arg SITE_URL=https://stackdog.app) and ran it:/sitemap.xml: 200application/xml, listing/,/docs,/php-shell-detection,/php-shell-quickstart,/contactunderhttps://stackdog.app/robots.txt: 200, includesSitemap: https://stackdog.app/sitemap.xml/stackdog-mark.pngand/php-shell-quickstart: 200, unchangedAfter deploy
Check
https://stackdog.app/robots.txt. Guess: Cloudflare's managed robots.txt merges with the origin's rather than replacing it. Not verified.🤖 Generated with Claude Code