Skip to content

[Model] Support to train glm5#1227

Open
heavyrain-lzy wants to merge 4 commits into
flagos-ai:mainfrom
heavyrain-lzy:support_glm5
Open

[Model] Support to train glm5#1227
heavyrain-lzy wants to merge 4 commits into
flagos-ai:mainfrom
heavyrain-lzy:support_glm5

Conversation

@heavyrain-lzy

Copy link
Copy Markdown
Collaborator

PR Category

Train

PR Types

Others

PR Description

  • Add GLM5 training support with new example configs under examples/glm5/, including both a small demo config and a 744B/40B Active model config.
  • Add GLM5 checkpoint conversion support in tools/checkpoint/, but not verified enoughly.
  • Update Megatron GPT builder to preserve experimental attention layer specs for MTP and print model structure/basic parameter-memory information after model construction.
  • Fix checkpoint save flow so energy monitor pause/resume is only triggered when log_energy is enabled.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant