Image-to-Image
Diffusers
ONNX
Safetensors
StableDiffusionXLInpaintPipeline
stable-diffusion-xl
inpainting
virtual try-on
Instructions to use ModelsLab/IDM-VTON with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use ModelsLab/IDM-VTON with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import AutoPipelineForInpainting from diffusers.utils import load_image # switch to "mps" for apple devices pipe = AutoPipelineForInpainting.from_pretrained("ModelsLab/IDM-VTON", dtype=torch.float16, device_map="cuda") img_url = "https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo.png" mask_url = "https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo_mask.png" image = load_image(img_url).resize((1024, 1024)) mask_image = load_image(mask_url).resize((1024, 1024)) prompt = "a tiger sitting on a park bench" generator = torch.Generator(device="cuda").manual_seed(0) image = pipe( prompt=prompt, image=image, mask_image=mask_image, guidance_scale=8.0, num_inference_steps=20, # steps between 15 and 30 work well for us strength=0.99, # make sure to use `strength` below 1.0 generator=generator, ).images[0] - Notebooks
- Google Colab
- Kaggle
|
Download README.md from ModelsLab/IDM-VTON: direct link, hf CLI and curl.
- Browser
- Download file 1.56 kB
-
https://huggingface.co/ModelsLab/IDM-VTON/resolve/main/README.md
- Command line
-
hf download hf://ModelsLab/IDM-VTON/README.md
-
curl -L -o README.md https://huggingface.co/ModelsLab/IDM-VTON/resolve/main/README.md
1.56 kB
| base_model: stable-diffusion-xl-1.0-inpainting-0.1 | |
| tags: | |
| - stable-diffusion-xl | |
| - inpainting | |
| - virtual try-on | |
| license: cc-by-nc-sa-4.0 | |
| # Check out more codes on our [github repository](https://github.com/yisol/IDM-VTON)! | |
| # IDM-VTON : Improving Diffusion Models for Authentic Virtual Try-on in the Wild | |
| This is an official implementation of paper 'Improving Diffusion Models for Authentic Virtual Try-on in the Wild' | |
| - [paper](https://arxiv.org/abs/2403.05139) | |
| - [project page](https://idm-vton.github.io/) | |
| 🤗 Try our huggingface [Demo](https://huggingface.co/spaces/yisol/IDM-VTON) | |
|  | |
|  | |
| ## TODO LIST | |
| - [x] demo model | |
| - [x] inference code | |
| - [ ] training code | |
| ## Acknowledgements | |
| For the demo, GPUs are supported from [zerogpu](https://huggingface.co/zero-gpu-explorers), and auto masking generation codes are based on [OOTDiffusion](https://github.com/levihsu/OOTDiffusion) and [DCI-VTON](https://github.com/bcmi/DCI-VTON-Virtual-Try-On). | |
| Parts of the code are based on [IP-Adapter](https://github.com/tencent-ailab/IP-Adapter). | |
| ## Citation | |
| ``` | |
| @article{choi2024improving, | |
| title={Improving Diffusion Models for Virtual Try-on}, | |
| author={Choi, Yisol and Kwak, Sangkyung and Lee, Kyungmin and Choi, Hyungwon and Shin, Jinwoo}, | |
| journal={arXiv preprint arXiv:2403.05139}, | |
| year={2024} | |
| } | |
| ``` | |
| ## License | |
| The codes and checkpoints in this repository are under the [CC BY-NC-SA 4.0 license](https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode). | |