Repository navigation
Add SVE2 optimizations of SynetDynamicQuantizedInnerProduct16bGemmV0 - #1132
Merged
Merged
Conversation
…GemmV0. Quantize and multiply dynamic 16-bit inner products with scalable SVE2 vectors and USDOT, using the Neon GemmV0 data layout. Co-authored-by: igor.ermolaev <igor.ermolaev@altezza.org>
Choose the int8 range from p.overflow or a.overflow, and zero-fill packed B through aK. Co-authored-by: igor.ermolaev <igor.ermolaev@altezza.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an SVE2 implementation of
SynetDynamicQuantizedInnerProduct16bGemmV0, following the Neon GemmV0 algorithm and the existing SVE2 32-bit microkernel.p.overfloworAlgParam::overflow, and packed B is zero-filled throughaK.svusdot, a 12 by2 * svcntw()microkernel, and bfloat16 or float output through the shared SVE2 activations.Testing
Cross-built with
aarch64-linux-gnu-g++(-march=armv9-a+sve+sve2+i8mm+bf16) and ranTest -fi=SynetDynamicQuantizedInnerProduct16bunder qemu-aarch64 at SVE vector lengths 128, 256, and 512. All three runs finished successfully after rebasing onto the latestdev, including float, bfloat16, constant int8 B, and dynamic B.