Runway cuts its 57B real-time video model down to 4 H100s

Real-time avatar video has zero margin for error. Drop a frame or stall for half a second, and the conversational illusion immediately breaks.

Runway published deep technical research detailing how it slashed the hardware requirements for Runway Characters—its 57B-parameter autoregressive video diffusion transformer—from eight Nvidia H100s per user down to just four, all while sustaining real-time 24fps generation.

Doing the math at write time

Standard model serving relies on batching many requests together to saturate GPUs. Interactive conversational characters cannot do that. Every call is a private, one-on-one session running at a batch size of one.

To hit 24 fps, the model must output a four-frame chunk of 704x1088 video every 166.7ms. On four H100s, Runway's original software took 240.9ms per diffusion step, well beyond the budget.

Profiling revealed that 29% of all GPU time was wasted simply re-processing old cache data across 240 attention calls per step. The engine repeatedly applied rotary position embeddings (RoPE), re-quantized weights to FP8, and shifted memory windows through copies.

Runway rearchitected the cache into a ring buffer and pushed RoPE and FP8 quantization to the exact moment a new frame enters the cache. That single design change collapsed per-step cache upkeep from 69.7ms to just 6.2ms.

Hiding communication and dodging bugs

With cache overhead cleared, communication became the next target. Runway fused its tensor-parallel all-reduce kernel directly into the FP8 matrix multiplication using techniques extended from ParallelKittens.

By reducing output tiles in the NVLink fabric while subsequent tiles were still computing, the team hid 54% to 68% of the communication time. That dragged the step down to 147.4ms.

Runway also moved the heavy 114ms latent video decoder onto its own GPU, overlapping decoding of the previous chunk with generation of the current one.

The paper is remarkably frank about production pitfalls. After an NCCL upgrade, dynamic prompt lengths triggered memory leaks because workers allocated fresh multicast buffers for every token length. Another bug—a single .item() call innocently added to track FP8 scale factors—stalled the CPU and inflated steps from 168.5ms to 278ms, wiping out weeks of progress.

Why it matters

For builders and creators, halving the GPU footprint per stream makes deploying frontier-grade interactive avatars economically viable. 57B video models previously required compute clusters that made real-time digital humans an expensive proof of concept rather than a sustainable business.

For the AI landscape, Runway is proving that the frontier of video is shifting from offline rendering to low-latency streaming at batch size one. The paper offers a masterclass in kernel engineering and systems orchestration that will force competitors to rethink their serving architectures.

Still, video is only part of the puzzle. In live tests, Runway found median turn latency stood at 2.13 seconds, with the upstream voice agent eating 1.42 seconds and the video pipeline taking 0.49 seconds. Given that natural human conversation pauses for around 208 milliseconds, we think true real-time conversational agents will remain constrained by voice intelligence long after video rendering has been solved.