Scalingcache: Extreme Acceleration of DiT through Difference Scaling and Dynamic Interval Caching

Lihui Gu¹^*^‡, Jingbin He²^‡, Lianghao Su², Kang He², Wenxiao Wang¹^†, Yuliang Liu²^†

¹Zhejiang University, ²Kuaishou Technology

* Work done during an internship at Kling AI Infra, Kuaishou Technology

‡ contributed equally to this work † Corresponding author

Paper Code arXiv

Demos

~2.5x Speedup on Wan2.1 14B Text-to-Video Task

Prompt: A person is air drumming.

Original

ScalingCache

Prompt: A sleek motorcycle shreds through neon-lit alleyways, bullets sparking off dumpsters.

Original

ScalingCache

Prompt: CG animation digital art, a sleek and advanced robot standing in the bustling center of Times Square.

Original

ScalingCache

Prompt: Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage.

Original

ScalingCache

Prompt: A cat wearing sunglasses and working as a lifeguard at a pool.

Original

ScalingCache

Abstract

Here is the abstract.

Method

Here is the method.

Text-to-Image Visual Quality Comparison

Comparison of generation results across different methods

Flux1.dev Sample 1

Flux1.dev-Original

1.0$\times$ speed

Taylor Sample 1

TaylorSeer($N=2,O=1$)

1.9$\times$ speed

ScalingCache Sample 1

ScalingCache($S_f=14$)

2.1$\times$ speed

ScalingCache Sample 1

ScalingCache($S_f=10$)

3.0$\times$ speed

Click image to view details

Flux1.dev Sample 2

Flux1.dev-Original

1.0$\times$ speed

Taylor Sample 2

TaylorSeer($N=2,O=1$)

1.9$\times$ speed

ScalingCache Sample 2

ScalingCache($S_f=14$)

2.1$\times$ speed

ScalingCache Sample 2

ScalingCache($S_f=10$)

3.0$\times$ speed

Click image to view details

Flux1.dev Sample 3

Flux1.dev-Original

1.0$\times$ speed

Taylor Sample 3

TaylorSeer($N=2,O=1$)

1.9$\times$ speed

ScalingCache Sample 3

ScalingCache($S_f=14$)

2.1$\times$ speed

ScalingCache Sample 3

ScalingCache($S_f=10$)

3.0$\times$ speed

Click image to view details

Flux1.dev Sample 4

Flux1.dev-Original

1.0$\times$ speed

Taylor Sample 4

TaylorSeer($N=2,O=1$)

1.9$\times$ speed

ScalingCache Sample 4

ScalingCache($S_f=14$)

2.1$\times$ speed

ScalingCache Sample 4

ScalingCache($S_f=10$)

3.0$\times$ speed

Click image to view details

Flux1.dev Sample 6

Flux1.dev-Original

1.0$\times$ speed

Taylor Sample 6

TaylorSeer($N=2,O=1$)

1.9$\times$ speed

ScalingCache Sample 6

ScalingCache($S_f=14$)

2.1$\times$ speed

ScalingCache Sample 6

ScalingCache($S_f=10$)

3.0$\times$ speed

Click image to view details

Flux1.dev Sample 7

Flux1.dev-Original

1.0$\times$ speed

Taylor Sample 7

TaylorSeer($N=2,O=1$)

1.9$\times$ speed

ScalingCache Sample 7

ScalingCache($S_f=14$)

2.1$\times$ speed

ScalingCache Sample 7

ScalingCache($S_f=10$)

3.0$\times$ speed

Click image to view details

Flux1.dev Sample 8

Flux1.dev-Original

1.0$\times$ speed

Taylor Sample 8

TaylorSeer($N=2,O=1$)

1.9$\times$ speed

ScalingCache Sample 8

ScalingCache($S_f=14$)

2.1$\times$ speed

ScalingCache Sample 8

ScalingCache($S_f=10$)

3.0$\times$ speed

Click image to view details

Flux1.dev Sample 9

Flux1.dev-Original

1.0$\times$ speed

Taylor Sample 9

TaylorSeer($N=2,O=1$)

1.9$\times$ speed

ScalingCache Sample 9

ScalingCache($S_f=14$)

2.1$\times$ speed

ScalingCache Sample 9

ScalingCache($S_f=10$)

3.0$\times$ speed

Click image to view details

Flux1.dev Sample 10

Flux1.dev-Original

1.0$\times$ speed

Taylor Sample 10

TaylorSeer($N=2,O=1$)

1.9$\times$ speed

ScalingCache Sample 10

ScalingCache($S_f=14$)

2.1$\times$ speed

ScalingCache Sample 10

ScalingCache($S_f=10$)

3.0$\times$ speed

Click image to view details

Awesome Related Works

TaylorSeer: From Reusing to Forecasting Accelerating Diffusion Models with TaylorSeers.
EasyCache: Training-Free Video Diffusion Acceleration via Runtime-Adaptive Caching.
TeaCache: Timestep Embedding Tells It's Time to Cache for Video Diffusion Model.

BibTeX

BibTex Code Here