Skip to content

Issues: NVIDIA/Megatron-LM

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

205 Open 657 Closed
205 Open 657 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[BUG] Token routing probability all-gather precision in token_dispatcher causes differing results between EP ranks
#1421 opened Feb 20, 2025 by chotzen
[BUG] get_rotary_seq_len isn't always returning a float
#1420 opened Feb 20, 2025 by jasonchiu-codeium
[BUG] Dual meaning of max_position_embeddings, computing both embedding shape & yarn scaling base
#1418 opened Feb 19, 2025 by chotzen
[BUG] Checkpoint state dict remapping is not applied for MLA layers
#1417 opened Feb 19, 2025 by chotzen
[BUG] Sequence-parallel gather is attempted when sequence parallelism is disabled in MLA
#1416 opened Feb 19, 2025 by chotzen
[QUESTION] Why gpu utility varies so much?
#1415 opened Feb 18, 2025 by wplf
Why this lr_decay_steps is logged as total num of iterations
#1413 opened Feb 18, 2025 by wplf
[BUG] Multi-Head-Latent-Attention Error
#1412 opened Feb 18, 2025 by AlbertZhangHIT
[BUG] multi_latent_attention does not support apply_rope_fusion
#1411 opened Feb 18, 2025 by AlbertZhangHIT
[ENHANCEMENT] GRPO rlhf support
#1409 opened Feb 16, 2025 by thincal
[ENHANCEMENT] Distillation training supports PP
#1408 opened Feb 15, 2025 by JinXiaozhao
[BUG] NaNs when unfreezing vision encoder in the multi-modal example
#1407 opened Feb 14, 2025 by CoderPat
1
[QUESTION] Why not gather routing_map in sequence_load_balancing_loss_func function
#1406 opened Feb 14, 2025 by tendar
1
[QUESTION]Save format problem
#1405 opened Feb 14, 2025 by liuyifan123123
1
[ENHANCEMENT] Multi-token Prediction(MTP) support
#1404 opened Feb 14, 2025 by lk137095576
3
[QUESTION] Performance Impact of Using item() in total_num_tokens += num_tokens.item() in megatron/core/pipeline_parallel/schedules.py
#1403 opened Feb 13, 2025 by wan-nan
[QUESTION] does cuda support fp32 residual connection feature?
#1402 opened Feb 12, 2025 by Jianhong-Zhang
[QUESTION] plan to implement zero bubble pipeline or dual pipeline and MoE comm-comp overlapping
#1399 opened Feb 11, 2025 by SeunghyunSEO
[QUESTION] Does MLA in Megatron-Core support PackedSeqParams?
#1398 opened Feb 11, 2025 by lostkevin
1
[BUG]The unit test test_different_initialize_order_unconsistency in test_parallel_state.py allways fails
#1396 opened Feb 11, 2025 by heavyrain-lzy
[QUESTION]Missing code branch
#1395 opened Feb 11, 2025 by fxnie
[ENHANCEMENT] Nemo Megatron Retries Missing Index Files Without Exponential Backoff
#1386 opened Feb 7, 2025 by ankitaluthra1
[ENHANCEMENT] Sequential Deletion of Old Checkpoint Files Slows Down Checkpointing
#1385 opened Feb 7, 2025 by ankitaluthra1
[BUG] Multiple Nodes Attempt to Create Checkpoint Folder Simultaneously, Causing Errors
#1384 opened Feb 7, 2025 by ankitaluthra1
[BUG] Distributed Checkpoint Files Written in Random Order During Nemo Megatron Checkpointing
#1383 opened Feb 7, 2025 by ankitaluthra1
ProTip! Type g p on any issue or pull request to go back to the pull request listing page.