Repository navigation
RGB32 Euclidean filters: count the max distance as in range on x86 SIMD - #1637
Merged
Merged
Conversation
Check every ISA's filter_rgb32_euclidean, to_blackwhite_rgb32_euclidean and compress_rgb32_to_binary_euclidean against a scalar reference: a pixel is in range when its squared RGB distance is <= the squared limit. Most test pixels sit exactly at, one below, or one above the limit, over widths around the SIMD vector sizes and the 64-pixel binary tile width. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Default and NEON treat a pixel whose squared distance equals the squared limit as in range (sum_sqr <= limit). The SSE4.2, AVX2 and AVX512 versions used cmpgt(limit, sum_sqr), a strict <, so on x86 those pixels were out of range. With a max distance of 0, an exact color match never matched. Compare unsigned <= instead: cmpeq(min_epu32(sum_sqr, limit), sum_sqr) on SSE4.2 and AVX2, cmple_epu32 on AVX512. This covers the RGB32 filters (filter and to-black/white) and compress_rgb32_to_binary_euclidean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Gin890
force-pushed
the
claude/euclidean-boundary
branch
from
October 21, 2026 04:58
4224969 to
80e7212
Compare
Mysticial
approved these changes
Oct 21, 2026
Collaborator
|
Oh and fix the Mac build. Looks like it segfaulted. |
Collaborator
Author
|
Claude found the segfault: |
This was referenced Oct 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR changes the x86 SIMD versions of the RGB32 Euclidean color filters. Now they count a pixel at the maximum distance as in range, as the default and ARM versions do.
Before this change, the same image gave different results on x86 and on ARM. For example, with a maximum distance of 0, a pixel with the expected color was out of range on x86. The author of the filters said that a pixel at the maximum distance is in range.
I found this problem during the work on PokemonAutomation/Arduino-Source#1633. You can merge the two PRs in any sequence.
2 tests added:
SerialPrograms/Source/Kernels/ImageFilters/RGB32_EuclideanDistance/Kernels_ImageFilter_RGB32_Euclidean_Tests.cpp(Test_FilterRgb32Euclidean): compares each CPU version of the color filters with a simple reference.SerialPrograms/Source/Kernels/ImageFilters/RGB32_EuclideanDistance/Kernels_ImageFilter_RGB32_Euclidean_Tests.cpp(Test_CompressRgb32Euclidean): compares each CPU version of the binary image compression with the same reference.Most test pixels are at the maximum distance, or 1 above or below it. Without the change, the tests fail on x86 (CI run). With the change, the tests pass (CI run).
🤖 Generated with Claude Code