Skip to content

Seven llama_sampler_init_* bindings admit keyword arguments that the ctypes function object silently drops #2371

Description

@GreenFuze

Summary

In llama_cpp/llama_cpp.py, seven bindings are declared as defs whose parameters are NOT
positional-only, while every other decorated binding in the file uses /:

line (main d736646) def
4553 def llama_sampler_init_dist(seed: int) -> llama_sampler_p
4560 def llama_sampler_init_top_k(k: int) -> llama_sampler_p
4570 def llama_sampler_init_top_p(p: float, min_keep: int) -> llama_sampler_p
4580 def llama_sampler_init_min_p(p: float, min_keep: int) -> llama_sampler_p
4590 def llama_sampler_init_typical(p: float, min_keep: int) -> llama_sampler_p
4596 def llama_sampler_init_temp(t: float) -> llama_sampler_p
4606 def llama_sampler_init_temp_ext(t: float, delta: float, exponent: float) -> llama_sampler_p

ctypes_function (llama_cpp/_ctypes_extensions.py:146-152) replaces the def with the
ctypes function object:

def decorator(f):
    if enabled:
        func = getattr(lib, name)
        func.argtypes = argtypes
        func.restype = restype
        functools.wraps(f)(func)
        return func

so the name the module exports IS getattr(lib, "llama_sampler_init_top_k"). A ctypes
function object without paramflags binds no argument by name, but functools.wraps and
the (f: F) -> F typing make inspect.signature, pyright and IDEs show the def's
signature — which invites keyword calls the object cannot honour.

What happens (CPython 3.12.10, ctypes)

>>> import ctypes; lib = ctypes.CDLL(None); f = lib.abs
>>> f.argtypes = [ctypes.c_int]; f.restype = ctypes.c_int
>>> f(-3, x=1)        # an extra keyword is DROPPED silently
3
>>> f(x=-3)           # all-keyword: refused
TypeError: this function takes at least 1 argument (0 given)

Applied to these bindings:

  • llama_sampler_init_top_k(k=40) — type-checks clean, raises TypeError at run time.
  • llama_sampler_init_top_p(0.9, min_keep=2) — type-checks clean, runs, and passes ONE
    positional to a two-parameter C function: min_keep is silently dropped and the C side
    reads whatever the register/stack held for it.

No caller inside the repository is affected today (_internals.py:705-732 and the server
pass positionals), so this is a latent API hazard rather than a current bug — but the
signature is public and the failure is invisible to every static check.

Fix

Make the seven defs positional-only, as the other bindings already are:

def llama_sampler_init_top_k(k: int, /) -> llama_sampler_p: ...

(Alternatively, build the functions through a CFUNCTYPE prototype with paramflags,
which does bind by name — but the one-character fix matches the file's convention.)

Found with SemLinker, a cross-language contract checker we are developing; happy to send a PR.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions