Feature Extraction
Transformers
Safetensors
audio_embeddings
audio
custom_code
self-supervised-learning
audio-embeddings
best-rq-2
audioset
Instructions to use ltuncay/BEST-RQ-2.2-base-2M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ltuncay/BEST-RQ-2.2-base-2M with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="ltuncay/BEST-RQ-2.2-base-2M", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("ltuncay/BEST-RQ-2.2-base-2M", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
fix: default to single-phase extraction without overlap
Browse files- README.md +4 -2
- config.json +1 -1
- configuration_audio.py +1 -1
README.md
CHANGED
|
@@ -96,7 +96,7 @@ challenge's standardized ranking. These evaluations precede the Transformers exp
|
|
| 96 |
| Training precision | bfloat16 mixed precision |
|
| 97 |
| Training batch | 128 per GPU × 2 GPUs, DDP; effective batch 256 |
|
| 98 |
| Exported weight dtype | float32 |
|
| 99 |
-
| Extraction policy | `
|
| 100 |
|
| 101 |
## Load the model
|
| 102 |
|
|
@@ -129,7 +129,7 @@ Resample to `processor.sampling_rate` and downmix stereo before preprocessing.
|
|
| 129 |
The extractor performs padding only; spectrogram/convolution features are computed
|
| 130 |
inside the model. Pass the returned sample attention mask for variable durations.
|
| 131 |
Outputs include a frame attention mask and timestamps in milliseconds (-1 for padding).
|
| 132 |
-
`pooler_output` uses the saved HEAR extraction preset,
|
| 133 |
Frame features average frequency patches; they are not the raw frequency-time ViT grid.
|
| 134 |
|
| 135 |
## Fine-tuning
|
|
@@ -187,3 +187,5 @@ BEST-RQ-2.2 is a variant in the BEST-RQ-2 family. Cite the family paper when usi
|
|
| 187 |
|
| 188 |
The model weights and bundled implementation are released under the MIT license.
|
| 189 |
See `LICENSE` and `CODE_LICENSE`.
|
|
|
|
|
|
|
|
|
| 96 |
| Training precision | bfloat16 mixed precision |
|
| 97 |
| Training batch | 128 per GPU × 2 GPUs, DDP; effective batch 256 |
|
| 98 |
| Exported weight dtype | float32 |
|
| 99 |
+
| Extraction policy | `native_one_phase` |
|
| 100 |
|
| 101 |
## Load the model
|
| 102 |
|
|
|
|
| 129 |
The extractor performs padding only; spectrogram/convolution features are computed
|
| 130 |
inside the model. Pass the returned sample attention mask for variable durations.
|
| 131 |
Outputs include a frame attention mask and timestamps in milliseconds (-1 for padding).
|
| 132 |
+
`pooler_output` uses the saved HEAR extraction preset, using a single phase with no context-window overlap by default.
|
| 133 |
Frame features average frequency patches; they are not the raw frequency-time ViT grid.
|
| 134 |
|
| 135 |
## Fine-tuning
|
|
|
|
| 187 |
|
| 188 |
The model weights and bundled implementation are released under the MIT license.
|
| 189 |
See `LICENSE` and `CODE_LICENSE`.
|
| 190 |
+
|
| 191 |
+
Default inference uses `native_one_phase` (no context-window overlap and one encoding phase). Reported benchmark results retain their stated evaluation presets; to reproduce overlapping two-phase evaluation, set `model.config.extraction_preset = "overlap50_two_phase"`.
|
config.json
CHANGED
|
@@ -27,7 +27,7 @@
|
|
| 27 |
"residual_type": "standard",
|
| 28 |
"rope_mode": "1d"
|
| 29 |
},
|
| 30 |
-
"extraction_preset": "
|
| 31 |
"feature_encoder_kwargs": {},
|
| 32 |
"hidden_size": 768,
|
| 33 |
"model_target": "src.models.best_rq22_module.BestRQ22Module",
|
|
|
|
| 27 |
"residual_type": "standard",
|
| 28 |
"rope_mode": "1d"
|
| 29 |
},
|
| 30 |
+
"extraction_preset": "native_one_phase",
|
| 31 |
"feature_encoder_kwargs": {},
|
| 32 |
"hidden_size": 768,
|
| 33 |
"model_target": "src.models.best_rq22_module.BestRQ22Module",
|
configuration_audio.py
CHANGED
|
@@ -45,7 +45,7 @@ class AudioEmbeddingConfig(PretrainedConfig):
|
|
| 45 |
feature_encoder_kwargs: dict[str, Any] | None = None,
|
| 46 |
sampling_rate: int = 16000,
|
| 47 |
spectrogram_adjustment_mode: str = "pad",
|
| 48 |
-
extraction_preset: str = "
|
| 49 |
**kwargs: Any,
|
| 50 |
) -> None:
|
| 51 |
super().__init__(**kwargs)
|
|
|
|
| 45 |
feature_encoder_kwargs: dict[str, Any] | None = None,
|
| 46 |
sampling_rate: int = 16000,
|
| 47 |
spectrogram_adjustment_mode: str = "pad",
|
| 48 |
+
extraction_preset: str = "native_one_phase",
|
| 49 |
**kwargs: Any,
|
| 50 |
) -> None:
|
| 51 |
super().__init__(**kwargs)
|