Issues

• On BigCodeBench-Hard (n=148), ~55 of the leaderboard's models are statistically tied with #1 — consider showing significance tiers

  bigcode-project/bigcodebench#121 · Filter by author ipezygj opened on Jul 18, 2026

• Test Qwen2.5-Coder-32b-instruct pass@10

  bigcode-project/bigcodebench#120 · Filter by author hyperplane2021-hub opened on Mar 19, 2026

• [Feat] Give estimates for exact cost/token count for hard

  bigcode-project/bigcodebench#118 · Filter by author GhostDog98 opened on Feb 22, 2026

• Seeding / Reproducibility

  bigcode-project/bigcodebench#115 · Filter by author TerryTong-Git opened on Sep 25, 2025

• 🤗 [REQUEST] - Qwen3-Next series

  bigcode-project/bigcodebench#114 · Filter by author BradKML opened on Sep 12, 2025

• 🐛 [TaskRemoval/TaskRepair] - <120> <Unexpected test cases>

  bigcode-project/bigcodebench#111 · Filter by author bajinsheng opened on Aug 8, 2025

• 🤗 [REQUEST] - GPT-OSS 20B and 120B

  bigcode-project/bigcodebench#110 · Filter by author BradKML opened on Aug 6, 2025

• Question: Why do sequential generation on OpenAIChatDecoder

  bigcode-project/bigcodebench#109 · Filter by author floatingsun opened on Aug 5, 2025
  1

• where is the file: audio.wav?

  bigcode-project/bigcodebench#108 · Filter by author xinghang5029 opened on Jul 17, 2025

• Is it normal for the ground truth accuracy not to be 100%?

  bigcode-project/bigcodebench#107 · Filter by author p81sunshine opened on Jul 14, 2025

• Gradio does not work properly

  bigcode-project/bigcodebench#106 · Filter by author p81sunshine opened on Jul 14, 2025
  1

• 🤗 [REQUEST] - Kimi K2 + Kimi-Dev

  bigcode-project/bigcodebench#105 · Filter by author BradKML opened on Jul 14, 2025

• TypeError: pass_k int not iterable in evaluate.py

  bigcode-project/bigcodebench#103 · Filter by author KedarnathKC opened on Jun 23, 2025
  2

• 🤗 [REQUEST] - Arsh-llm-0.7b

  bigcode-project/bigcodebench#102 · Filter by author arsh-team opened on Jun 12, 2025
  2

• 🤗 [REQUEST] - RedNote Dots.LLM1

  bigcode-project/bigcodebench#101 · Filter by author BradKML opened on Jun 8, 2025

• 🤗 [REQUEST] - DeepSeek 3.1 + DeepSeek-R1-0528 update

  bigcode-project/bigcodebench#100 · Filter by author BradKML opened on May 28, 2025

• 🤗 [REQUEST] - Devstral

  bigcode-project/bigcodebench#99 · Filter by author BradKML opened on May 28, 2025

• Add Qwen3

  bigcode-project/bigcodebench#98 · Filter by author tzurtutjuzrtzurtzurtz opened on May 25, 2025
  1

• 🤗 [REQUEST] - Arsh-llm

  bigcode-project/bigcodebench#97 · Filter by author arsh-team opened on May 22, 2025
  1

• Are there any standard instructions for the prompts?

  bigcode-project/bigcodebench#96 · Filter by author fzfz666 opened on May 16, 2025
  3

• What is the prompt format?

  bigcode-project/bigcodebench#91 · Filter by author ccocks opened on Apr 1, 2025
  9

• how to add new model?

  bigcode-project/bigcodebench#89 · Filter by author pengzhangzhi opened on Mar 24, 2025
  1

• 🤗 [REQUEST] - Qwen2.5-Coder-7b

  bigcode-project/bigcodebench#82 · Filter by author eightHundreds opened on Feb 14, 2025
  1

• [docs] Clarify risks of running evals locally.

  documentation
  bigcode-project/bigcodebench#41 · Filter by author hvaara opened on Sep 3, 2024
  

• [docs] Add working end-to-end example for a local model

  documentation
  bigcode-project/bigcodebench#40 · Filter by author hvaara opened on Sep 3, 2024
  1