Memory-V2V

Memory-V2V: Memory-Augmented Video-to-Video Diffusion for Consistent Multi-Turn Editing (ECCV 2026)

Dohun Lee, Chun-Hao Paul Huang, Xuelin Chen, Jong Chul Ye, Duygu Ceylan, Hyeonho Jeong

[Paper] [Project Page] [Code]

This repository contains the checkpoint of Memory-V2V for multi-turn video novel view synthesis; the text-guided long-video editing model of the paper is not included. A source video is re-rendered along several novel camera trajectories, one after another, and the view for each trajectory is generated conditioned on the source video and on all previously generated views, so that the views stay consistent with each other. The model is built on ReCamMaster (Wan2.1-T2V-1.3B).

Files

File Size Description
memoryv2v.ckpt 3.0 GB Full state dict of the video DiT (1.50B parameters, bfloat16).

SHA-256 of memoryv2v.ckpt: b57b2618ff48e07744bbb9c0e0918b90344072a1fa84e77c7a2dcd25ed7f546b

The checkpoint is the Wan2.1-T2V-1.3B DiT with the ReCamMaster camera modules, the Memory-V2V condition patch embedders and the token downsamplers of adaptive token merging (ATM). The self-attention layers and these added modules are fine-tuned from ReCamMaster; all other weights are those of Wan2.1-T2V-1.3B. It contains only tensors and is loaded with torch.load(path, weights_only=True).

The text encoder (with its tokenizer) and the VAE are those of Wan2.1-T2V-1.3B, which has to be downloaded separately (about 17.6 GB). inference.py also loads the Wan2.1 DiT file to build the model; all of its weights are then replaced by this checkpoint.

Usage

Clone the code repository and set up the environment as described there (conda environment and the PyTorch build for your GPU). Then, from the root of the repository, download the weights and run inference:

# Memory-V2V checkpoint (this repository)
python -c "from huggingface_hub import hf_hub_download; hf_hub_download('DHCAI/Memory-V2V', 'memoryv2v.ckpt', local_dir='models/checkpoints')"
# Wan2.1-T2V-1.3B (text encoder, VAE and base DiT)
python -c "from huggingface_hub import snapshot_download; snapshot_download('Wan-AI/Wan2.1-T2V-1.3B', local_dir='models/Wan-AI/Wan2.1-T2V-1.3B')"

# Re-render the example videos, each along its own camera trajectories
bash inference.sh
# or every example video along camera trajectories 2 -> 3 -> 12
python inference.py --cam_type 2 3 12 --ckpt_path ./models/checkpoints/memoryv2v.ckpt

VideoFOV (the previous views most relevant to the target get the finer condition tokenizers) and adaptive token merging (ATM, faster) are on by default; --no-use_videofov and --no-use_atm turn them off.

The trajectories in --cam_type are generated in the given order: the view for cam_type[i] is conditioned on the source video and on the views of cam_type[0], ..., cam_type[i-1]. The model generates 81-frame videos at 832x480 (input videos are resized to 832x480 and their first 81 frames are used). Generating the three views above takes about 18 GB of GPU memory, and memory and time grow with every view. See the code repository for the list of camera trajectories, how to use your own videos, and training.

License

The checkpoint is released under the Apache-2.0 license, like Wan2.1-T2V-1.3B and ReCamMaster on which it is built.

Citation

@inproceedings{lee2026memoryv2v,
  title={Memory-V2V: Memory-Augmented Video-to-Video Diffusion for Consistent Multi-Turn Editing},
  author={Lee, Dohun and Huang, Chun-Hao Paul and Chen, Xuelin and Ye, Jong Chul and Ceylan, Duygu and Jeong, Hyeonho},
  booktitle={European Conference on Computer Vision (ECCV)},
  year={2026}
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for DHCAI/Memory-V2V

Finetuned
(1)
this model

Paper for DHCAI/Memory-V2V