Instructions to use infly/Infinity-Parser2-Pro with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use infly/Infinity-Parser2-Pro with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="infly/Infinity-Parser2-Pro")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("infly/Infinity-Parser2-Pro")
model = AutoModelForMultimodalLM.from_pretrained("infly/Infinity-Parser2-Pro", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use infly/Infinity-Parser2-Pro with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "infly/Infinity-Parser2-Pro"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "infly/Infinity-Parser2-Pro",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/infly/Infinity-Parser2-Pro

SGLang

How to use infly/Infinity-Parser2-Pro with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "infly/Infinity-Parser2-Pro" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "infly/Infinity-Parser2-Pro",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "infly/Infinity-Parser2-Pro" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "infly/Infinity-Parser2-Pro",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use infly/Infinity-Parser2-Pro with Docker Model Runner:
```
docker model run hf.co/infly/Infinity-Parser2-Pro
```

Infinity-Parser2-Pro / README.md

Commit History

Jul 20

08:46
Update README.md

zuminghuang

Verified signature
b27d470

Jul 10

02:24
Update README.md

zuminghuang

Verified signature
2f82e70

May 28

11:39
Update README.md

zuminghuang

Verified signature
1d070df

May 15

14:19
Update README.md

zuminghuang

Verified signature
c76f59c

May 13

13:20
Update README.md

KexuanRen

Verified signature
ffa38ae
13:15
Update README.md

KexuanRen

Verified signature
5d388f5
13:13
Update README.md

KexuanRen

Verified signature
78d52c4

May 12

03:13
Update README.md

zuminghuang

Verified signature
22f5687

May 11

12:07
Update README.md

zuminghuang

Verified signature
a08ba71
12:07
Update README.md

zuminghuang

Verified signature
9fd6419
12:01
Update README.md

zuminghuang

Verified signature
cdbbc6f

Apr 28

12:44
Update README.md

zuminghuang

Verified signature
72b3342

Apr 20

04:12
Update README.md

zuminghuang

Verified signature
0b82703

Apr 15

13:11
Update README.md

zuminghuang

Verified signature
afaf1a3
13:10
Update README.md

zuminghuang

Verified signature
fc763a6
06:36
Update README.md

zuminghuang

Verified signature
a9ac0e9
05:28
Update README.md

zuminghuang

Verified signature
cade246
05:24
Update README.md

zuminghuang

Verified signature
e35b4c7
03:28
Update README.md

zuminghuang

Verified signature
b641725
02:21
Update README.md

zuminghuang

Verified signature
10d15d5
02:16
Update README.md

zuminghuang

Verified signature
d89f3db
02:09
Update README.md

zuminghuang

Verified signature
4a56f82

Apr 14

14:59
Update README.md

zuminghuang

Verified signature
b62196d
14:43
Update README.md

zuminghuang

Verified signature
f5b1684
14:34
Update README.md

zuminghuang

Verified signature
43a9a56

Apr 11

04:42
Update README.md

zuminghuang

Verified signature
e13e7a8

Apr 9

14:42
Update README.md

zuminghuang

Verified signature
6a4987d
14:38
Update README.md

zuminghuang

Verified signature
0023127
10:17
Update README.md

zuminghuang

Verified signature
dc196c8
10:15
Update README.md

zuminghuang

Verified signature
09db769
10:01
Update README.md

zuminghuang

Verified signature
0ae7110
03:50
Update README.md

zuminghuang

Verified signature
a1f46dc
03:15
Update README.md

zuminghuang

Verified signature
12fa14f

Apr 8

12:48
Update README.md

zuminghuang

Verified signature
28db656
12:47
Update README.md

zuminghuang

Verified signature
38405e1
12:44
Update README.md

zuminghuang

Verified signature
91332be
12:28
initial commit

zuminghuang

Verified signature
106e84c

Commit History

Jul 20

Update README.md

Jul 10

Update README.md

May 28

Update README.md

May 15

Update README.md

May 13

Update README.md

Update README.md

Update README.md

May 12

Update README.md

May 11

Update README.md

Update README.md

Update README.md

Apr 28

Update README.md

Apr 20

Update README.md

Apr 15

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Apr 14

Update README.md

Update README.md

Update README.md

Apr 11

Update README.md

Apr 9

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Apr 8

Update README.md

Update README.md

Update README.md

initial commit