Issues

fix(server): assistant tool_calls with string arguments cannot be rendered by templates that require a mapping (Gemma 4)

NVIDIA/TensorRT-Edge-LLM#254 · opened

on Oct 19, 2026

fix(quantize): with tied embeddings, AWQ smoothing of a quantized lm_head rewrites embed_tokens.weight

NVIDIA/TensorRT-Edge-LLM#253 · opened

on Oct 19, 2026

Significant performance improvements for Orin (sm87) and RTX4060 (sm89) + bug fixes

NVIDIA/TensorRT-Edge-LLM#252 · opened

on Oct 18, 2026

[Bug][Runtime] GDN Tree-MTP greedy divergence at token 14 on Thor SM110; partial AR-by-depth workaround still fails 200K XHigh

NVIDIA/TensorRT-Edge-LLM#250 · opened

on Oct 16, 2026

perf: tokenizer byte-pair merge is O(n²) in the piece length, slow for long Gemma prompts

NVIDIA/TensorRT-Edge-LLM#247 · opened

on Oct 13, 2026

docs: Jetson build prerequisites omit the NVRTC headers (cuda-nvrtc-dev) and the TensorRT plugin headers

NVIDIA/TensorRT-Edge-LLM#244 · opened

on Oct 10, 2026

feat(server): tool-call parser for Gemma 4's native call syntax

NVIDIA/TensorRT-Edge-LLM#243 · opened

on Oct 10, 2026

feat(server): let tensorrt-edgellm-serve run a directory of engines built with llm_build / visual_build

NVIDIA/TensorRT-Edge-LLM#242 · opened

on Oct 10, 2026

fix(server): tool_choice "required" and forced-function choices are not enforced

NVIDIA/TensorRT-Edge-LLM#241 · opened

on Oct 10, 2026

feat(server): expose ContextCacheConfig::encoderEmbeddingCacheBudgetBytes through pybind and tensorrt-edgellm-serve

NVIDIA/TensorRT-Edge-LLM#240 · opened

on Oct 10, 2026

fix: GPU allocation failures in Tensor do not report the tensor, its size, or free device memory

NVIDIA/TensorRT-Edge-LLM#239 · opened

on Oct 10, 2026

fix(lora): insert-lora attaches the LoRA branch to the AWQ-smoothed activation (x * pre_quant_scale) of INT4 GEMMs

NVIDIA/TensorRT-Edge-LLM#237 · opened

on Oct 10, 2026

tensorrt-edgellm-serve cannot start Cosmos3-Edge on Thor: tokenizer copy fails from HF cache, and unused vae-encoder build fails (myelin map::at)

NVIDIA/TensorRT-Edge-LLM#233 · opened

on Oct 7, 2026

Cosmos3-Edge reasoner on Jetson AGX Thor (SM110): guide build runs but describes the wrong scene (vs BF16 reference)

NVIDIA/TensorRT-Edge-LLM#232 · opened

on Oct 7, 2026

2

llm_bench crashes on Gemma-4 engines due to missing PLE bindings (ple_token_embeds_0)

NVIDIA/TensorRT-Edge-LLM#230 · opened

on Oct 1, 2026

1

0.11.0 regression: spec-base engine for Qwen3.8-27B fails to build at maxInputLen 196608 (Myelin autotune requests 582.5 GiB) and OOM-kills the device at maxInputLen 98304–131072 — 0.10.1 builds the identical config cleanly

NVIDIA/TensorRT-Edge-LLM#228 · opened

on Sep 30, 2026

2

OpenAI-compatible server cannot build MTP spec-base engine for Qwen3.8-27B: `--speculative-config {"method":"mtp"}` fails in the Python builder (Myelin tactic assertion)

NVIDIA/TensorRT-Edge-LLM#223 · opened

on Sep 27, 2026

6

llm_build fails with Myelin codegen error for maxInputLen >= 229376 (and 131072 at default batch size) on Qwen3.8-27B / SM110

NVIDIA/TensorRT-Edge-LLM#222 · opened

on Sep 26, 2026

3

feat: Add image-text datasets

NVIDIA/TensorRT-Edge-LLM#221 · opened

on Sep 25, 2026

1

Decoding cannot be constrained: no per-step hook in the sampling path, and `logit_bias` cannot substitute for one

NVIDIA/TensorRT-Edge-LLM#218 · opened

on Sep 17, 2026

1

Direct export/build fails for Qwen3 VL FP8 fine-grained 2D block-FP8 checkpoints

NVIDIA/TensorRT-Edge-LLM#217 · opened

on Sep 17, 2026

2

tensorrt-edgellm-build --dense nvfp4-qdq SIGSEGVs in a libnvinfer compiler worker thread on Jetson Thor (CC 11.0) for Cosmos-Reason2-2B; --dense fp16 builds fine from the same NVFP4 checkpoint

NVIDIA/TensorRT-Edge-LLM#216 · opened

on Sep 17, 2026

10

llm_inference ignores temperature/top_p/top_k from the input JSON its own help tells you to set them in

NVIDIA/TensorRT-Edge-LLM#211 · opened

on Sep 12, 2026

2

Int4GroupwiseGemmPluginV2 fails silently when ENABLE_CUTE_DSL omits int4_fp16_gemm: "Failed to enqueue status -1" with no diagnostic

NVIDIA/TensorRT-Edge-LLM#210 · opened

on Sep 11, 2026

1

ONNX-less direct builder cannot resolve decoder tensors for multimodal cosmos3_edge, and renaming them yields a numerically wrong engine

NVIDIA/TensorRT-Edge-LLM#208 · opened

on Sep 11, 2026

Navigation Menu

Repository navigation

Issues

fix(server): assistant tool_calls with string arguments cannot be rendered by templates that require a mapping (Gemma 4)

fix(quantize): with tied embeddings, AWQ smoothing of a quantized lm_head rewrites embed_tokens.weight

Significant performance improvements for Orin (sm87) and RTX4060 (sm89) + bug fixes

[Bug][Runtime] GDN Tree-MTP greedy divergence at token 14 on Thor SM110; partial AR-by-depth workaround still fails 200K XHigh

perf: tokenizer byte-pair merge is O(n²) in the piece length, slow for long Gemma prompts

docs: Jetson build prerequisites omit the NVRTC headers (cuda-nvrtc-dev) and the TensorRT plugin headers

feat(server): tool-call parser for Gemma 4's native call syntax

feat(server): let tensorrt-edgellm-serve run a directory of engines built with llm_build / visual_build

fix(server): tool_choice "required" and forced-function choices are not enforced

feat(server): expose ContextCacheConfig::encoderEmbeddingCacheBudgetBytes through pybind and tensorrt-edgellm-serve

fix: GPU allocation failures in Tensor do not report the tensor, its size, or free device memory

fix(lora): insert-lora attaches the LoRA branch to the AWQ-smoothed activation (x * pre_quant_scale) of INT4 GEMMs

tensorrt-edgellm-serve cannot start Cosmos3-Edge on Thor: tokenizer copy fails from HF cache, and unused vae-encoder build fails (myelin map::at)

Cosmos3-Edge reasoner on Jetson AGX Thor (SM110): guide build runs but describes the wrong scene (vs BF16 reference)

llm_bench crashes on Gemma-4 engines due to missing PLE bindings (ple_token_embeds_0)

0.11.0 regression: spec-base engine for Qwen3.8-27B fails to build at maxInputLen 196608 (Myelin autotune requests 582.5 GiB) and OOM-kills the device at maxInputLen 98304–131072 — 0.10.1 builds the identical config cleanly

OpenAI-compatible server cannot build MTP spec-base engine for Qwen3.8-27B: `--speculative-config {"method":"mtp"}` fails in the Python builder (Myelin tactic assertion)

llm_build fails with Myelin codegen error for maxInputLen >= 229376 (and 131072 at default batch size) on Qwen3.8-27B / SM110

feat: Add image-text datasets

Decoding cannot be constrained: no per-step hook in the sampling path, and `logit_bias` cannot substitute for one

Direct export/build fails for Qwen3 VL FP8 fine-grained 2D block-FP8 checkpoints

tensorrt-edgellm-build --dense nvfp4-qdq SIGSEGVs in a libnvinfer compiler worker thread on Jetson Thor (CC 11.0) for Cosmos-Reason2-2B; --dense fp16 builds fine from the same NVFP4 checkpoint

llm_inference ignores temperature/top_p/top_k from the input JSON its own help tells you to set them in

Int4GroupwiseGemmPluginV2 fails silently when ENABLE_CUTE_DSL omits int4_fp16_gemm: "Failed to enqueue status -1" with no diagnostic

ONNX-less direct builder cannot resolve decoder tensors for multimodal cosmos3_edge, and renaming them yields a numerically wrong engine