Restyle model card: Modular logo, diagram, structured sections

.gitattributes

@@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 models/labse-ig-ha-yo/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 tokenizer.json filter=lfs diff=lfs merge=lfs -text
+images/native_bird_diagram.jpg filter=lfs diff=lfs merge=lfs -text

README.md

@@ -19,112 +19,123 @@ tags:
 base_model: sentence-transformers/LaBSE
 ---
 
-# Native-Bird
-
-**Native-Bird** is a 768-dimensional multilingual sentence-embedding model fine-tuned from **LaBSE** for cross-lingual retrieval involving **Igbo, Hausa, Yoruba, and English**.
-
-Its primary use case is **native-language query → English document retrieval**. A user can describe what they are looking for in Igbo while the indexed document is written in English, and Native-Bird maps both into the same embedding space for semantic search.
-
-## What it is designed to do
-
-Example:
+<div align="center">
+  <img src="images/modular_logo.jpg" width="38%" alt="Modular" />
+</div>
+<hr>
+<div align="center" style="line-height: 1;">
+  <a href="https://huggingface.co/Modularcomputing" target="_blank"><img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Modular-ffc107?color=ffc107&logoColor=white"/></a>
+  <a href="https://huggingface.co/Modularcomputing/Native-Bird2" target="_blank"><img alt="Native-Bird2" src="https://img.shields.io/badge/Multimodal-Native--Bird2-536af5?color=536af5&logoColor=white"/></a>
+  <a href="https://github.com/Dox45" target="_blank"><img alt="GitHub" src="https://img.shields.io/badge/GitHub-Dox45-181717?logo=github&logoColor=white"/></a>
+  <br>
+  <img alt="Languages" src="https://img.shields.io/badge/Languages-Igbo%20%7C%20Hausa%20%7C%20Yoruba%20%7C%20English-1f6feb"/>
+  <img alt="Embedding dim" src="https://img.shields.io/badge/Embedding-768--d-2ea44f"/>
+  <img alt="Base" src="https://img.shields.io/badge/Base-LaBSE-6f42c1"/>
+</div>
+
+<p align="center">
+  <img src="images/native_bird_diagram.jpg" width="100%" alt="Native-Bird: an Igbo query and English documents in one shared semantic vector space" />
+</p>
+
+## Table of Contents
+
+1. [Introduction](#1-introduction)
+2. [Model Summary](#2-model-summary)
+3. [Model Downloads](#3-model-downloads)
+4. [Evaluation Results](#4-evaluation-results)
+5. [Quick Start](#5-quick-start)
+6. [Searching Real Document Collections](#6-searching-real-document-collections)
+7. [Limitations](#7-limitations)
+8. [Citation](#8-citation)
+9. [Contact](#9-contact)
+
+## 1. Introduction
+
+**Native-Bird** is a 768-dimensional multilingual sentence-embedding model fine-tuned from **LaBSE** for cross-lingual retrieval across **Igbo, Hausa, Yoruba and English**.
+
+Its primary use case is **native-language query → English document retrieval**: a user describes what they are looking for in Igbo, Hausa or Yoruba, the indexed documents are written in English, and Native-Bird maps both into the same embedding space for semantic search.
 
 > **Igbo query:** “Neuklọs ahụ bụ ahụ ihe abụọ mejupụtara ya - neutrọn na protọn.”
 >
 > **English document:** “The nucleus consists of two particles - neutrons and protons.”
 
-The model should assign a high cosine similarity to the matching English text even though the languages differ.
-
-## Model details
-
-- **Model name:** Native-Bird
-- **Architecture:** SentenceTransformer based on LaBSE
-- **Embedding dimension:** 768
-- **Maximum sequence length used during training:** 128 tokens
-- **Similarity:** cosine similarity
-- **Base model:** sentence-transformers/LaBSE
-- **Training objective:** Cached Multiple Negatives Ranking Loss (CachedMNRL)
-- **Languages:** Igbo, Hausa, Yoruba, English
-- **Training examples:** 76,150 paired examples
-- **Epochs:** 4
-- **Per-device batch size:** 256
-- **Learning rate:** 2e-5
-- **Warmup:** 10%
-- **Scheduler:** cosine
-- **Training precision:** BF16
-- **Negative sampling:** in-batch negatives with NO_DUPLICATES; CachedMNRL mini-batch size 128
-
-## Evaluation
-
-The retrieval evaluation uses an English corpus of **1,004 sentences** and **204 queries per language**. For each query, the correct English sentence is known. We report Recall@1, Recall@10, and Mean Reciprocal Rank (MRR).
-
-### Fine-tuned Native-Bird
+Native-Bird is the text backbone of [**Native-Bird2**](https://huggingface.co/Modularcomputing/Native-Bird2), which extends the same space to images.
 
-| Language | Query condition | R@1 | R@10 | MRR |
-|---|---|---:|---:|---:|
-| Igbo | Clean | 1.0000 | 1.0000 | 1.0000 |
-| Igbo | ASR-style noise | 1.0000 | 1.0000 | 1.0000 |
-| Hausa | Clean | 1.0000 | 1.0000 | 1.0000 |
-| Hausa | ASR-style noise | 1.0000 | 1.0000 | 1.0000 |
-| Yoruba | Clean | 0.9804 | 1.0000 | 0.9902 |
-| Yoruba | ASR-style noise | 0.9804 | 1.0000 | 0.9867 |
+## 2. Model Summary
 
-### Before fine-tuning: LaBSE baseline
+**Architecture**
 
-| Language | Query condition | R@1 | R@10 | MRR |
-|---|---|---:|---:|---:|
-| Igbo | Clean | 1.0000 | 1.0000 | 1.0000 |
-| Igbo | ASR-style noise | 0.9853 | 0.9951 | 0.9888 |
-| Hausa | Clean | 1.0000 | 1.0000 | 1.0000 |
-| Hausa | ASR-style noise | 0.9951 | 1.0000 | 0.9975 |
-| Yoruba | Clean | 0.9804 | 1.0000 | 0.9871 |
-| Yoruba | ASR-style noise | 0.9559 | 0.9951 | 0.9683 |
+| | |
+|---|---|
+| Architecture | SentenceTransformer (LaBSE encoder → pooling → dense → normalize) |
+| Base model | [`sentence-transformers/LaBSE`](https://huggingface.co/sentence-transformers/LaBSE) |
+| Embedding dimension | 768 |
+| Max sequence length (training) | 128 tokens |
+| Similarity | Cosine |
+| Languages | Igbo (`ig`), Hausa (`ha`), Yoruba (`yo`), English (`en`) |
 
-### Interpretation
+**Training**
 
-Native-Bird is **strong on the benchmark it was trained and evaluated against**, particularly for Igbo and Hausa. Fine-tuning improved noisy-query retrieval for Igbo and Hausa and slightly improved Yoruba.
+| | |
+|---|---|
+| Objective | Cached Multiple Negatives Ranking Loss (CachedMNRL) |
+| Training pairs | 76,150 native-language ↔ English pairs |
+| Epochs | 4 |
+| Per-device batch size | 256 (CachedMNRL mini-batch 128) |
+| Learning rate / schedule | 2e-5, cosine, 10% warmup |
+| Negatives | In-batch, NO_DUPLICATES sampling |
+| Precision | BF16 |
 
-The scores do **not** prove that Native-Bird will retrieve arbitrary long English documents from arbitrary Igbo descriptions with the same accuracy. The current evaluation is sentence-level. Real document search should therefore split documents into passages/chunks before embedding.
+## 3. Model Downloads
 
-## Direct Igbo → English retrieval checks
+<div align="center">
 
-The trained model was directly run against the English evaluation corpus. Examples:
+| **Model** | **Modality** | **Embedding dim** | **Download** |
+| :---: | :---: | :---: | :---: |
+| Native-Bird | Text (ig / ha / yo / en) | 768 | [🤗 Hugging Face](https://huggingface.co/Modularcomputing/Native-Bird) |
+| Native-Bird2 | Text + Image | 512 | [🤗 Hugging Face](https://huggingface.co/Modularcomputing/Native-Bird2) |
 
-**Igbo:** “Mgbanwe n'ọdịdị na-agbakwunye ọdịdị kejenetiki ọhụrụ, ma nhọpụta na-ewepụ ya n'ogwu ọdịdị nke a kọwapụtara.”
+</div>
 
-→ **Top English result:** “Mutation adds new genetic variation, and selection removes it from the pool of expressed variation.”
+## 4. Evaluation Results
 
-Cosine similarity: **0.7079**
+Cross-lingual retrieval against an English corpus of **1,004 sentences**, with **204 queries per language**. Each query has one known matching English sentence. Two query variants are tested: **clean** (original text) and **ASR-style noise** (diacritics removed, lowercased, punctuation stripped, whitespace normalized).
 
-**Igbo:** “Neuklọs ahụ bụ ahụ ihe abụọ mejupụtara ya - neutrọn na protọn.”
+<div align="center">
 
-→ **Top English result:** “The nucleus consists of two particles - neutrons and protons.”
+| **Language** | **Query** | **LaBSE R@1** | **Native-Bird R@1** | **LaBSE MRR** | **Native-Bird MRR** |
+| :--- | :--- | :---: | :---: | :---: | :---: |
+| Igbo | Clean | 1.0000 | **1.0000** | 1.0000 | **1.0000** |
+| Igbo | ASR-style noise | 0.9853 | **1.0000** | 0.9888 | **1.0000** |
+| Hausa | Clean | 1.0000 | **1.0000** | 1.0000 | **1.0000** |
+| Hausa | ASR-style noise | 0.9951 | **1.0000** | 0.9975 | **1.0000** |
+| Yoruba | Clean | 0.9804 | **0.9804** | 0.9871 | **0.9902** |
+| Yoruba | ASR-style noise | 0.9559 | **0.9804** | 0.9683 | **0.9867** |
 
-Cosine similarity: **0.7049**
+</div>
 
-**Igbo:** “Enwere ọtụtụ ihe mere ha ji kara web prọgzi mma: ha na-agbanwe ụzọ trafik ịntanetị niile, ọ bụghị naanị http.”
+Native-Bird reaches R@10 = 1.0000 in every setting (LaBSE: 0.9951 on noisy Igbo and Yoruba). Fine-tuning mainly improves **robustness to transcription-style noise**, which matters for voice and keyboard input without diacritics.
 
-→ **Top English result:** “They are superior to web proxies for several reasons: They re-route all Internet traffic, not only http.”
+**Direct Igbo → English retrieval examples**
 
-Cosine similarity: **0.6774**
+| Igbo query | Top English result | Cosine |
+| :--- | :--- | :---: |
+| Mgbanwe n'ọdịdị na-agbakwunye ọdịdị kejenetiki ọhụrụ, ma nhọpụta na-ewepụ ya n'ogwu ọdịdị nke a kọwapụtara. | Mutation adds new genetic variation, and selection removes it from the pool of expressed variation. | 0.7079 |
+| Neuklọs ahụ bụ ahụ ihe abụọ mejupụtara ya - neutrọn na protọn. | The nucleus consists of two particles - neutrons and protons. | 0.7049 |
+| Enwere ọtụtụ ihe mere ha ji kara web prọgzi mma: ha na-agbanwe ụzọ trafik ịntanetị niile, ọ bụghị naanị http. | They are superior to web proxies for several reasons: They re-route all Internet traffic, not only http. | 0.6774 |
 
-These are direct demonstrations of the intended behavior: the query is Igbo while the retrieved target is English.
+## 5. Quick Start
 
-## Installation
-
-~~~bash
+```bash
 pip install -U sentence-transformers torch
-~~~
-
-## Basic usage
+```
 
-~~~python
+```python
 from sentence_transformers import SentenceTransformer
 
 model = SentenceTransformer("Modularcomputing/Native-Bird")
 
 query = "Neuklọs ahụ bụ ahụ ihe abụọ mejupụtara ya - neutrọn na protọn."
-
 documents = [
     "The nucleus consists of two particles - neutrons and protons.",
     "The liver is an organ responsible for many metabolic functions.",
@@ -133,100 +144,49 @@ documents = [
 
 q = model.encode(query, normalize_embeddings=True)
 d = model.encode(documents, normalize_embeddings=True)
-
 scores = d @ q
 
 for i in scores.argsort()[::-1]:
-    print(float(scores[i]), documents[i])
-~~~
+    print(f"{scores[i]:.4f}  {documents[i]}")
+```
 
 The matching English sentence should rank first.
 
-## Testing the model
-
-A minimal test is:
-
-1. Load Native-Bird.
-2. Encode an Igbo query.
-3. Encode several English candidate documents.
-4. Compute cosine similarity.
-5. Sort candidates by similarity.
-6. Check whether the semantically matching English document is ranked first.
-
-For a real search system, replace the small list with a vector database or FAISS index.
-
-## Searching a real English document collection
+## 6. Searching Real Document Collections
 
-Do **not** embed a multi-page document as one vector. Split each document into passages, normally around 100–300 words with overlap, and store each passage embedding together with its parent document ID.
+Do **not** embed a multi-page document as one vector. Split each document into passages of roughly 100–300 words with overlap, and store each passage embedding with its parent document ID.
 
-At query time:
-
-1. Encode the user's Igbo query with Native-Bird.
-2. Search the English passage vectors using cosine similarity or an ANN index such as FAISS.
-3. Return the highest-scoring passages.
-4. Group or rerank passages by their parent document.
-
-Conceptually:
-
-~~~text
-Igbo query
-    ↓
-Native-Bird encoder
-    ↓
-768-dimensional query vector
-    ↓ cosine similarity
+```text
+Igbo / Hausa / Yoruba query
+        ↓  Native-Bird encoder
+768-d query vector
+        ↓  cosine similarity (FAISS / vector DB)
 English passage index
-    ↓
-Top-k English passages
-    ↓
-Parent documents
-~~~
-
-This is the architecture needed for the intended “describe it in Igbo, find the English document” product.
-
-## Important limitation
-
-Native-Bird is an **embedding/retrieval model**, not a translator and not a generative language model. It does not generate an English answer. It represents text as vectors so that semantically related text can be found across languages.
-
-The current model uses a 128-token training sequence length and was evaluated on sentence-level retrieval. For long documents, chunking is therefore important.
-
-A production deployment should be evaluated on the actual target domain. Nigerian government documents, university material, medical documents, legal documents, and technical documentation can have vocabulary and writing styles that differ substantially from the current benchmark.
-
-## Evaluation methodology
-
-The evaluation script embeds the 1,004-item English corpus, embeds each language's 204 queries, computes cosine similarities, and measures the rank of the known matching English sentence.
-
-Two query variants are evaluated:
-
-- **clean:** original native-language query
-- **ASR-style noise:** Unicode diacritics removed, lowercased, punctuation stripped, and whitespace normalized
-
-The benchmark is useful for measuring cross-lingual retrieval and robustness to transcription-style normalization, but should not be interpreted as a universal real-world retrieval score.
-
-## Intended applications
-
-- Igbo → English semantic document search
-- Hausa → English semantic document search
-- Yoruba → English semantic document search
-- Multilingual educational search
-- Native-language interfaces for English knowledge bases
-- Cross-lingual RAG retrieval
-- Search over Nigerian institutional documents
-
-## Training provenance
-
-Native-Bird was fine-tuned from LaBSE using paired native-language/English retrieval data prepared for this project. Training used 76,150 pairs and a contrastive retrieval objective.
+        ↓
+Top-k passages → grouped by parent document
+```
 
-The final training run completed successfully for four epochs, saved the model, and produced the evaluation results reported above.
+**Intended applications:** native-language search over English knowledge bases, cross-lingual RAG retrieval, multilingual educational search, and search over Nigerian institutional documents.
 
-## Model status
+## 7. Limitations
 
-**Status: research / early release.**
+- **Embedding model, not a translator or generator.** It represents text as vectors; it does not produce English answers.
+- **Sentence-level evaluation.** Trained at 128 tokens and evaluated on sentences, so long documents need chunking. Scores do not guarantee the same accuracy on arbitrary long documents.
+- **Domain shift.** Government, legal, medical and technical text can differ substantially from the benchmark; evaluate on your target domain before production use.
+- **Status: research / early release.** The next validation step is a human-annotated benchmark of real Nigerian documents and real Igbo, Hausa and Yoruba information needs.
 
-The core cross-lingual retrieval capability is demonstrated. The next meaningful validation step is a human-annotated benchmark of real Nigerian documents and real Igbo/Hausa/Yoruba information needs, especially for multi-paragraph documents.
+## 8. Citation
 
-## Citation
+```bibtex
+@misc{nativebird2026,
+  title  = {Native-Bird: Cross-lingual Native-Language Embeddings for English Document Retrieval},
+  author = {Akuegbo, Chima Emmanuel},
+  year   = {2026},
+  publisher = {Modular},
+  howpublished = {\url{https://huggingface.co/Modularcomputing/Native-Bird}}
+}
+```
 
-If you use Native-Bird in research or a project, please reference the model as:
+## 9. Contact
 
-**Native-Bird: Cross-lingual Native-Language Embeddings for English Document Retrieval.**
+Questions, issues or collaboration: open a discussion on the [Community tab](https://huggingface.co/Modularcomputing/Native-Bird/discussions).