Issues

• Saved model checkpoint in different precision ?

  bigcode-project/Megatron-LM#88 · Filter by author Ankush2k opened on May 9, 2024
  1

• IndexError: too many indices for tensor of dimension 2

  bigcode-project/Megatron-LM#87 · Filter by author YFeather opened on Jan 3, 2024

• LM Head FLOPs

  bigcode-project/Megatron-LM#78 · Filter by author Muennighoff opened on Sep 28, 2023
  2

• How to use my dataset to finetune

  bigcode-project/Megatron-LM#77 · Filter by author zxyscz opened on Aug 26, 2023

• How to generate several result with num_beams

  bigcode-project/Megatron-LM#75 · Filter by author Eggwardhan opened on Jul 27, 2023

• RuntimeError: Error building extension 'scaled_upper_triang_masked_softmax_cuda'

  bigcode-project/Megatron-LM#61 · Filter by author KOVVURISATYANARAYANAREDDY opened on Jun 14, 2023

• how to prepare the training data to train starcoder?

  bigcode-project/Megatron-LM#60 · Filter by author wwngh1233 opened on Jun 6, 2023
  2

• ValueError: Invalid attention arguments: AttnType.self_attn, None

  bigcode-project/Megatron-LM#59 · Filter by author chen-lee-li opened on Jun 6, 2023
  1

• Incomplete humaneval evaluation code

  bigcode-project/Megatron-LM#57 · Filter by author huybery opened on May 30, 2023

• Support interleaved pipeline schedules in checkpoint merging tools

  enhancement
  bigcode-project/Megatron-LM#45 · Filter by author RaymondLi0 opened on Mar 29, 2023

• Improve loading of the data-paths

  enhancement
  bigcode-project/Megatron-LM#38 · Filter by author RaymondLi0 opened on Mar 21, 2023

• OOM on preprocessing dataset with large number of documents

  bug
  bigcode-project/Megatron-LM#34 · Filter by author RaymondLi0 opened on Mar 10, 2023

• Lower throughput with UL2 training

  bigcode-project/Megatron-LM#28 · Filter by author RaymondLi0 opened on Feb 20, 2023

• Wandb init error

  bug
  bigcode-project/Megatron-LM#27 · Filter by author RaymondLi0 opened on Feb 17, 2023

• Create data composition

  bigcode-project/Megatron-LM#25 · Filter by author RaymondLi0 opened on Feb 7, 2023 · Train final model
  1

• Literature review on scaling laws

  help wanted
  bigcode-project/Megatron-LM#21 · Filter by author RaymondLi0 opened on Jan 26, 2023 · Scaling laws for HumanEval
  3

  

• Benchmarking Memory Consumption of Optimizers Adam v.s. Adan

  bigcode-project/Megatron-LM#20 · Filter by author SivilTaram opened on Jan 25, 2023

• Conversion of Huggingface bigcode/santacoder to Nvidia Triton Inference server

  bigcode-project/Megatron-LM#17 · Filter by author michaelfeil opened on Jan 24, 2023

• Timeout on creating the index mappings

  bug
  bigcode-project/Megatron-LM#15 · Filter by author RaymondLi0 opened on Jan 4, 2023

• Fine tuning and Pre training scripts

  bigcode-project/Megatron-LM#14 · Filter by author shaileshj2803 opened on Jan 3, 2023

• Selection of LLMs architecture

  bigcode-project/Megatron-LM#7 · Filter by author maoquan-ms opened on Sep 27, 2022