Visual Document Retrieval
Transformers
Safetensors
sentence-transformers
multilingual
qwen3_5
feature-extraction
text
image
multimodal-embedding
vidore
colbert
colqwen3_5
multilingual-embedding
multi-vector
custom_code
Instructions to use webAI-Official/webAI-ColVec1.1-8b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use webAI-Official/webAI-ColVec1.1-8b with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModel processor = AutoProcessor.from_pretrained("webAI-Official/webAI-ColVec1.1-8b", trust_remote_code=True) model = AutoModel.from_pretrained("webAI-Official/webAI-ColVec1.1-8b", trust_remote_code=True, device_map="auto") - sentence-transformers
How to use webAI-Official/webAI-ColVec1.1-8b with sentence-transformers:
from sentence_transformers import MultiVectorEncoder model = MultiVectorEncoder("webAI-Official/webAI-ColVec1.1-8b", trust_remote_code=True) queries = ["Which planet is known as the Red Planet?"] documents = [ "Venus is often called Earth's twin because of its similar size and proximity.", "Mars, known for its reddish appearance, is often referred to as the Red Planet.", "Jupiter, the largest planet in our solar system, has a prominent red spot.", ] query_embeddings = model.encode_query(queries) document_embeddings = model.encode_document(documents) similarities = model.similarity(query_embeddings, document_embeddings) print(similarities) - Notebooks
- Google Colab
- Kaggle
Update 8B model card
Browse files
README.md
CHANGED
|
@@ -38,25 +38,21 @@ L2-normalized multi-vector embeddings for late-interaction retrieval.
|
|
| 38 |
The model uses bidirectional attention in Qwen3.5's full-attention layers and
|
| 39 |
a learned 640-dimensional projection head. The unused language-model head has
|
| 40 |
been removed from the released checkpoint. Consequently, the released
|
| 41 |
-
embedding model contains 8
|
| 42 |
`webAI-ColVec1.1-8b` repository name, while its original backbone is
|
| 43 |
Qwen3.5-9B.
|
| 44 |
|
| 45 |
### Training data
|
| 46 |
|
| 47 |
-
We created filtered, balanced, and multilingual curated subsets from
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
| [VisRAG In-Domain](https://huggingface.co/datasets/openbmb/VisRAG-Ret-Train-In-domain-data) |
|
| 57 |
-
| [VisRAG Synthetic](https://huggingface.co/datasets/openbmb/VisRAG-Ret-Train-Synthetic-data) |
|
| 58 |
-
| [VDR Multilingual Train](https://huggingface.co/datasets/llamaindex/vdr-multilingual-train) |
|
| 59 |
-
| [Wiki-SS-NQ](https://huggingface.co/datasets/Tevatron/wiki-ss-nq) |
|
| 60 |
|
| 61 |
## 🛠️ Model specifications
|
| 62 |
|
|
@@ -84,10 +80,9 @@ separate 500,000-sample curated subset.
|
|
| 84 |
|
| 85 |
## 📊 Evaluation results
|
| 86 |
|
| 87 |
-
The table reports NDCG@10
|
| 88 |
-
|
| 89 |
-
|
| 90 |
-
the public average is the unweighted mean of the eight public task values.
|
| 91 |
|
| 92 |
The result artifacts record MTEB 2.18.5, Transformers 5.14.1, PyTorch 2.9.0
|
| 93 |
with CUDA 12.8, `bfloat16`, FlashAttention 2.8.3, and batch size 32. The release
|
|
@@ -100,8 +95,8 @@ Model encoding runs in `bfloat16`. Before MaxSim scoring, query and document
|
|
| 100 |
embeddings are moved to CPU and converted to `float32`. All reported ViDoRe
|
| 101 |
results use this FP32 scoring path.
|
| 102 |
|
| 103 |
-
| Model |
|
| 104 |
-
| :--- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
| 105 |
| **webAI-ColVec1.1-8b (this model)** | 80.14 | 70.09 | **71.89** | **55.10** | 68.49 | **57.41** | 67.88 | 51.29 | **65.29** |
|
| 106 |
| [VultronRetriever Prime](https://huggingface.co/vultr/VultronRetrieverPrime-Qwen3.5-8B) | 79.81 | **70.26** | 69.01 | 54.51 | 66.82 | **57.41** | **68.19** | **51.73** | 64.72 |
|
| 107 |
| [webAI-ColVec1-9b](https://huggingface.co/webAI-Official/webAI-ColVec1-9b) | **80.92** | 69.77 | 68.28 | 53.72 | **70.04** | 57.18 | 67.32 | 48.38 | 64.45 |
|
|
@@ -238,18 +233,18 @@ change retrieval scores.
|
|
| 238 |
|
| 239 |
### Strengths
|
| 240 |
|
| 241 |
-
-
|
| 242 |
-
|
| 243 |
-
-
|
| 244 |
-
-
|
| 245 |
-
-
|
| 246 |
-
|
|
|
|
| 247 |
|
| 248 |
### Limitations
|
| 249 |
|
| 250 |
-
-
|
| 251 |
-
|
| 252 |
-
SDPA.
|
| 253 |
|
| 254 |
## License
|
| 255 |
|
|
|
|
| 38 |
The model uses bidirectional attention in Qwen3.5's full-attention layers and
|
| 39 |
a learned 640-dimensional projection head. The unused language-model head has
|
| 40 |
been removed from the released checkpoint. Consequently, the released
|
| 41 |
+
embedding model contains ~8.4B parameters, reflected by the
|
| 42 |
`webAI-ColVec1.1-8b` repository name, while its original backbone is
|
| 43 |
Qwen3.5-9B.
|
| 44 |
|
| 45 |
### Training data
|
| 46 |
|
| 47 |
+
We created filtered, balanced, and multilingual curated subsets from six public
|
| 48 |
+
datasets: [ColPali Train Set](https://huggingface.co/datasets/vidore/colpali_train_set),
|
| 49 |
+
[Docmatix-IR](https://huggingface.co/datasets/Tevatron/docmatix-ir),
|
| 50 |
+
[VisRAG In-Domain](https://huggingface.co/datasets/openbmb/VisRAG-Ret-Train-In-domain-data),
|
| 51 |
+
[VisRAG Synthetic](https://huggingface.co/datasets/openbmb/VisRAG-Ret-Train-Synthetic-data),
|
| 52 |
+
[VDR Multilingual Train](https://huggingface.co/datasets/llamaindex/vdr-multilingual-train),
|
| 53 |
+
and [Wiki-SS-NQ](https://huggingface.co/datasets/Tevatron/wiki-ss-nq). This
|
| 54 |
+
Qwen3.5-9B-backbone model was trained on a 750,000-sample curated subset as well
|
| 55 |
+
as synthetically generated data.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 56 |
|
| 57 |
## 🛠️ Model specifications
|
| 58 |
|
|
|
|
| 80 |
|
| 81 |
## 📊 Evaluation results
|
| 82 |
|
| 83 |
+
The table reports NDCG@10 scores on the eight **public** ViDoRe V3 tasks as
|
| 84 |
+
percentages rather than values between 0 and 1 (for example, 0.80 is shown as
|
| 85 |
+
80.00). Each task value is the mean of its six language subsets; the public average is the unweighted mean of the eight public task values.
|
|
|
|
| 86 |
|
| 87 |
The result artifacts record MTEB 2.18.5, Transformers 5.14.1, PyTorch 2.9.0
|
| 88 |
with CUDA 12.8, `bfloat16`, FlashAttention 2.8.3, and batch size 32. The release
|
|
|
|
| 95 |
embeddings are moved to CPU and converted to `float32`. All reported ViDoRe
|
| 96 |
results use this FP32 scoring path.
|
| 97 |
|
| 98 |
+
| Model | Computer Science | Energy | FinanceEn | FinanceFr | HR | Industrial | Pharmaceuticals | Physics | **Avg. public** |
|
| 99 |
+
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
| 100 |
| **webAI-ColVec1.1-8b (this model)** | 80.14 | 70.09 | **71.89** | **55.10** | 68.49 | **57.41** | 67.88 | 51.29 | **65.29** |
|
| 101 |
| [VultronRetriever Prime](https://huggingface.co/vultr/VultronRetrieverPrime-Qwen3.5-8B) | 79.81 | **70.26** | 69.01 | 54.51 | 66.82 | **57.41** | **68.19** | **51.73** | 64.72 |
|
| 102 |
| [webAI-ColVec1-9b](https://huggingface.co/webAI-Official/webAI-ColVec1-9b) | **80.92** | 69.77 | 68.28 | 53.72 | **70.04** | 57.18 | 67.32 | 48.38 | 64.45 |
|
|
|
|
| 233 |
|
| 234 |
### Strengths
|
| 235 |
|
| 236 |
+
- **Performance:** State-of-the-art retrieval performance on the public ViDoRe
|
| 237 |
+
V3 tasks, with excellent multimodal document retrieval results.
|
| 238 |
+
- **Complex layouts:** Excellent handling of chart-rich PDFs and
|
| 239 |
+
domain-specific documents.
|
| 240 |
+
- **End-to-end retrieval:** OCR-free retrieval on unseen multimodal documents
|
| 241 |
+
without using an intermediate vision-language model to generate summaries.
|
| 242 |
+
- **Multilingualism:** Strong performance on non-English document inputs.
|
| 243 |
|
| 244 |
### Limitations
|
| 245 |
|
| 246 |
+
- **Storage Cost:** Still larger than single-vector baselines despite the
|
| 247 |
+
smaller token dimension.
|
|
|
|
| 248 |
|
| 249 |
## License
|
| 250 |
|