RestoreVAR

Abstract

The use of latent diffusion models (LDMs) such as Stable Diffusion has significantly improved the perceptual quality of All-in-One image Restoration (AiOR) methods, while also enhancing their generalization capabilities. However, these LDM-based frameworks suffer from slow inference due to their iterative denoising process, rendering them impractical for time-sensitive applications. To address this, we propose RestoreVAR, a novel generative approach for AiOR that significantly outperforms LDM-based models in restoration performance while achieving over 10× faster inference. RestoreVAR leverages visual autoregressive modeling (VAR), a recently introduced approach which performs scale-space autoregression for image generation. VAR achieves comparable performance to that of state-of-the-art diffusion transformers with drastically reduced computational costs. To optimally exploit these advantages of VAR for AiOR, we propose architectural modifications and improvements, including intricately designed cross-attention mechanisms and a latent-space refinement module, tailored for the AiOR task. Extensive experiments show that RestoreVAR achieves state-of-the-art performance among generative AiOR methods, while also exhibiting strong generalization capabilities.

RestoreVAR, our proposed VAR-based scale-space generative AiOR model (a), significantly outperforms LDM-based methods as shown in (b). RestoreVAR also offers drastic reductions in computational complexity as shown in (c).

Proposed Approach

Illustration of RestoreVAR for training and inference. (a) Shows the training procedure for each component of RestoreVAR, and (b) shows the overall pipeline during inference.

Quantitative Comparisons

Quantitative comparisons of RestoreVAR with the state-of-the-art LDM-based generative AiOR approaches, and non-generative methods. RestoreVAR significantly outperforms generative methods on PSNR, SSIM and LPIPS scores. The best generative approach is indicated in bold.

Quantitative comparisons of generalization of RestoreVAR against state-of-the-art non-generative approaches on real-world images. The best result is indicated in bold.

Qualitative Results

Qualitative comparisons of RestoreVAR with LDM-based generative AiOR approaches. RestoreVAR achieves consistent restoration with enhanced preservation of fine-details.

Qualitative comparisons of RestoreVAR with non-generative methods on real degradations. RestoreVAR consistently achieves better results, demonstrating its generalization capability.

Computational Complexity

Comparison of the computational complexity of RestoreVAR with LDM-based AiOR approaches.

BibTeX

@misc{rajagopalan2025restorevarvisualautoregressivegeneration,
      title={RestoreVAR: Visual Autoregressive Generation for All-in-One Image Restoration}, 
      author={Sudarshan Rajagopalan and Kartik Narayan and Vishal M. Patel},
      year={2025},
      eprint={2505.18047},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2505.18047}, 
}