-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Support the LingBot-Video MoE refiner (two-stage high-resolution refinement) #1546
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Artiprocher
merged 43 commits into
modelscope:main
from
NancyFyong:draft/lingbot-video-moe-refiner
Aug 12, 2026
Merged
Changes from all commits
Commits
Show all changes
43 commits
Select commit
Hold shift + click to select a range
182ea3a
Add LingBot-Video (Dense-1.3B) support with attention-only LoRA SFT
NancyFyong 67bcd02
Add docs, prompt rewriting, LoRA validation, and low-VRAM example for…
NancyFyong 55c8357
Add LingBot-Video MoE-30B-A3B support
NancyFyong df7ea8e
Address PR #1539 review: unify scheduler, restore VAE, split rewriter
NancyFyong 5fbf50a
Fix low-VRAM fp32/bf16 dtype mismatch in LingBot-Video DiT
NancyFyong a9e7cc2
Fold normalize_caption into lingbot_video pipeline module
NancyFyong c62ea4f
Add normalize_caption to lingbot_video and fix importers
NancyFyong e83d7d6
Trim verbose comments in LingBot-Video code
NancyFyong 038bad0
Merge lingbot_sft into lingbot_moe: apply PR #1539 review fixes to th…
NancyFyong 3333443
Fix dtype mismatch feeding the fp32-pinned time_embedder
NancyFyong 4ecc892
Add t2v_example_4 structured caption
NancyFyong 0a107bb
Merge lingbot_sft into lingbot_moe (dtype fix + sample 4 prompt)
NancyFyong ea73e8b
Add TI2V (image-to-video) support to LingBot-Video pipeline
NancyFyong 670ec22
Add text-to-image (t2i) support and example to LingBot-Video
NancyFyong 8d8b7c7
Add ti2v/t2i low-VRAM inference + ti2v LoRA training examples
NancyFyong b2492dd
update vae&inner_func
mi804 a10c026
tmp commit for code refactor
mi804 94e8baa
low vram patch&refactor scripts
mi804 7caae83
remove fp32 cast to keep compatibility with low-vram-inference
mi804 5f604e2
add support for full train
mi804 cae44a0
style refactor
mi804 531ba56
pop changes for ti2v pipeline
mi804 4c9ee93
Align TI2V / T2I examples, LoRA + full training, docs and README with…
NancyFyong 5fa49f5
fix: manage Qwen3-VL vision tower in LingBotVideoTextEncoder VRAM map
NancyFyong 56334bf
style: group input_image under image-to-video in pipeline __call__
NancyFyong 7d0852e
refactor: move TI2V helpers into the units that use them
NancyFyong 91fcb94
Merge lingbot_sft into lingbot_moe: integrate TI2V/T2I + full trainin…
NancyFyong 5bc41d2
style: strip comments and align MoE-30B-A3B integration with repo con…
NancyFyong eed0d72
style: align MoE examples with the Dense inference examples
NancyFyong b4aec8c
feat: add TI2V / T2I MoE-30B-A3B examples and load captions from the …
NancyFyong 9476867
fix: restore every DiT parameter under low-VRAM offload for the MoE map
NancyFyong ce0e732
Merge upstream main (LingBot-Video Dense-1.3B, #1539) into the MoE br…
NancyFyong 823fae0
fix: point the MoE examples at the published Dense example-dataset di…
NancyFyong 7bd0ad8
refactor: drop the resolve_bulk_dtype helper from the MoE DiT
NancyFyong 44c506b
fix: cast the MoE router and expert weights inside the model under of…
NancyFyong c2e4975
feat: add LoRA and full-parameter training examples for MoE-30B-A3B
NancyFyong 3cfbbf1
style: align the MoE accelerate config with the 14B training config
NancyFyong e0ce129
feat: support the LingBot-Video MoE refiner for two-stage high-resolu…
NancyFyong cf6dd4d
Merge remote-tracking branch 'origin/main' into lingbot_refiner
NancyFyong a4387c9
style: drop the '# --- Stage N' decorative separators from refiner ex…
NancyFyong 0131131
review: address PR #1546 comments
NancyFyong 54a0b7c
support cpu offload
mi804 34cabc2
refine docs
mi804 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The changes to the scheduler also affect the existing Dense and MoE models. Please make sure that both models can still generate videos correctly after these changes.