Conversation
Port the released SwiftVR ReAE and transformer into a causal single-device pipeline with stateful chunk processing and pinned output conversion. Add a FlashVSR-style H100 example, documentation, and focused model/session tests. The example warms full and tail shapes and sustains 1080p generation on one H100 without keeping the development validation scripts. Verified with 16 SwiftVR unit tests, Ruff check/format, diff checks, and an 81-frame 1080p H100 run at 34.78 generation FPS and 22.18 FPS including H.264 encoding.
Keep the generic TORCH_SDPA path on transpose views instead of forcing contiguous copies. Add a unit test that verifies the SDPA backend receives transpose views for BSND input.
Align the default offline path with the upstream causal benchmark and return PIL frames directly from pipeline sessions.\n\nAdd opt-in torch.compile, tf-kernel FP8Linear, selectable attention backends, and three-stage tensor-channel execution. Record H100 parity and acceleration measurements, including FP8 numerical validation.\n\nVerification: pytest tests/unit/models/test_swiftvr_transformer.py tests/unit/pipelines/swiftvr/test_example.py tests/unit/pipelines/swiftvr/test_streaming.py -q (22 passed); ruff check; ruff format --check; git diff --check.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Port the released SwiftVR ReAE and transformer into TeleFuser as a stateful
causal restoration pipeline.
The integration preserves the released checkpoint format, one-step DiT,
causal ReAE state, chunk/flush semantics, and supports direct PIL frame output.
It also adds optional torch.compile, tf-kernel FP8Linear, selectable attention
backends, and three-stage execution with WorkerTensorChannel latent handoff.
Motivation
SwiftVR was not previously available through TeleFuser. This change provides a
production-oriented single-GPU pipeline and FlashVSR-style H100 example while
keeping the default path at least as fast as the released implementation.
The default TeleFuser QHD core path reaches 32.42 FPS versus 31.05 FPS for a
local run of the released implementation under the same checkpoint and setup.
Type of Change
Changes Made
session isolation.
List[PIL.Image.Image]with pinned outputconversion.
FP8LinearW8A8 support.WorkerTensorChannel.FP8Linear tests.
Testing
Test commands:
Focused SwiftVR tests:
22 passed.Manual H100 validation:
34.78 FPS22.18 FPS32.42 FPStorch.compile:40.3-40.5 FPS35.8-36.1 FPStorch.compile+ FP8Linear:45.2-45.7 FPS39.87-40.11 FPSChecklist
ruff)ruff,ruff-format)pytest tests/) — full repository suite not runfeat(swiftvr): add optimized streaming executionRelated Issues
N/A
Additional Notes
The published SwiftVR QHD result is 31.32 FPS on one H100. A local run of the
official implementation measured 31.05 FPS under the same checkpoint.
The recommended acceleration configuration is:
SageAttention remains selectable, but was slower for this workload:
30.8-31.1 FPS eager and 38.1-38.4 FPS with compile.
The first compile shape incurs approximately 44 seconds of one-time compilation
overhead. The stage-parallel result is a pipeline throughput measurement and is
not directly comparable to single-GPU latency or memory.
GPU Architecture Support
No new CUDA kernel implementation is added in this PR. Existing SageAttention
and tf-kernel FP8Linear backends are used and validated on H100.
Performance Impact
The default path is faster than the released implementation at QHD.
31.32 FPS31.05 FPS32.42 FPS40.3-40.5 FPS45.2-45.7 FPSThe FP8 output comparison against BF16 measured
52.24 dB PSNR,0.00146 MAE,and
0.015625maximum absolute error on a QHD test request.