Repository navigation
docs: SYCL-bonsai fixes from a verbatim run on an Arc B390 - #360
Merged
Merged
Conversation
Add the llama-bench command for the 16K column, quote the KLD the documented commands actually give (2.3e-3 vs the CPU reference), note the CPU reference cost, limit the FLASH_ATTN_EXT check to head size 256 (192 aborts in fattn-tile.cpp), and say what GGML_SYCL_OP_TIMING covers.
github-actions
Bot
added
the
documentation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #359 after running every command in the doc as written on an Arc B390. All steps worked except the FLASH_ATTN_EXT check, which hits the existing head size 192 abort in
fattn-tile.cpp; the check is now limited to the model's head size 256.Also adds the llama-bench command for the 16K column, quotes the KLD the documented commands give (2.3e-3, same top token 98.0%), notes the CPU reference cost (about 40 minutes, 2 GB), and narrows the
GGML_SYCL_OP_TIMINGdescription to what it reports. Docs only.