Skip to content

feat: add FlashQLA GDN prefill backend - #1408

Merged
hiworldwzj merged 8 commits into
ModelTC:mainfrom
sufubao:support_flashqla
Aug 13, 2026
Merged

feat: add FlashQLA GDN prefill backend#1408
hiworldwzj merged 8 commits into
ModelTC:mainfrom
sufubao:support_flashqla

Conversation

@sufubao

@sufubao sufubao commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

为 GDN chunked prefill 增加 --gdn_prefill_backend flashqla。FlashQLA 仅在 SM90+ 且可导入时启用;首次调用失败会回退到 FLA,成功后运行时异常保持上抛。

Docker 镜像固定安装 flash-qla==0.1.1(依赖 tilelang==0.1.9)。

H200、Qwen3.5-0.8B、TP=1,prefill:

prompt / batch FLA FlashQLA
4K / 8 194.5k 234.4k
8K / 1 150.3k 191.7k
16K / 1 153.2k 200.9k

Dockerfile build check 通过。

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@sufubao
sufubao force-pushed the support_flashqla branch 2 times, most recently from 0ddbcd9 to 3314cd4 Compare July 30, 2026 10:50
@hiworldwzj
hiworldwzj merged commit fe9bdab into ModelTC:main Aug 13, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants