Repository navigation
Conversation
The line modes of http_xfread1() read one byte per recv() and called select() after each byte, so a 150-byte header block cost about 300 syscalls. They now peek at what is buffered, find where the byte loop would stop, and consume exactly that, so the bytes taken from the socket and the per-call stops are unchanged. Transports that cannot peek (files, old MPTCP kernels) keep the byte loop. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com>
…t flag Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com>
…ffer entry Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com>
xroche
enabled auto-merge (squash)
November 14, 2026 18:00
macOS loopback hands sent bytes to a separate input thread, so the zero-timeout poll right after send() could see nothing. The case then read nothing and left its bytes for the next one, and xfread-lines failed on the macOS arm64 leg. A mutant that delivers the stream 5 ms late reproduces this on Linux, and passes with the wait. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The engine read response headers with one
recv()per byte and aselect()after each byte. It now peeks at the buffered bytes, finds where the old byte loop would stop, and reads exactly that much. On a synthetic crawl of 100,000 pages, CPU time falls from 78.7 s to 58.7 s (median of three runs), and system time halves.Linux 5.6 to 5.12 ignore
MSG_PEEKon MPTCP sockets and consume the data. So once the engine opens an MPTCP socket on such a kernel, or on macOS, it reads bytewise again. A self-test checks that both paths leave the same lines and the same unread bytes on random streams.