Skip to content

Gemma 4 template and other stuff - #115

Open
Husky110 wants to merge 7 commits into
LettuceAI:mainfrom
Husky110:main
Open

Husky110 wants to merge 7 commits into
LettuceAI:mainfrom
Husky110:main

Conversation

@Husky110

@Husky110 Husky110 commented Nov 1, 2026

Copy link
Copy Markdown
Collaborator

Check the commits so nothing gets lost...

Husky110 and others added 7 commits October 17, 2026 11:43
Since nvcc is no longer wrapped with sccache, the long CUDA compile phase
sends zero requests to the sccache server. With the default 600s idle
timeout the server shuts itself down mid-build, so the trailing cl.exe
objects fail with 'Timed out waiting for server startup' on the cold
restart. Setting SCCACHE_IDLE_TIMEOUT=0 keeps the server alive for the
whole build.
Expose llama.cpp's top-n-sigma sampler as a configurable advanced model
setting. Keeps only tokens within n standard deviations of the top
logit and is meant to replace top-k/top-p/min-p. Wired through session,
model and global settings resolution, the sampler chain/order editor,
provider parameter support info, feature generation and i18n (de/en).
Register a "qwen-image-2.1" stable-diffusion.cpp profile with Q3_K/Q4_K/Q6_K
diffusion variants, Qwen3-VL-8B text/vision encoders and the Qwen-Image-2.1
VAE. Supports text-to-image and multi-reference editing (up to 10 refs),
native transparency and strong in-image text. Wires up HF bundle detection
(qwen-image prefix matching, qwen3-vl-8b arch hint), the LoRA architecture
label and matcher rules.

Also handle stable-diffusion.cpp master-962 (PR #2132) turning --auto-fit
into a value-taking option: newer/unknown builds get `--auto-fit on`, older
builds keep the bare flag.
Add the top-n-sigma sampler strings (label, description, short label and
sampler-order entry) to the remaining 18 locales; previously only de/en
were covered. Also drop the Greek sigma glyph from the short label in
de/en in favour of the plain "Top-n-sigma".
Default sampling steps for the qwen-image-2.1 profile reduced from 40 to 35.
Newer Gemma4 chat templates open the thought channel in their generation
prompt when thinking is enabled, so the model's reply begins mid-thought
and never emits the open tag. The thinking parser only switched to its
mid-thought mode for the forced-reasoning toggle, so with the toggle off
the whole thought leaked into the visible content and no reasoning was
extracted.

Detect an open reasoning channel at the end of the built prompt and start
both the streaming and final parsers inside the reasoning block in that
case too, reusing the existing starting_in_reasoning machinery.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant