Hugging Face's logo

qualcomm
/
Segment-Anything-Model-2

Image Segmentation
PyTorch
foundation
android
Model card Files
xet
 Community

Segment-Anything-Model-2: Optimized for Qualcomm Devices

SAM 2, the successor to Meta's Segment Anything Model (SAM), is a cutting-edge tool designed for comprehensive object segmentation in both images and videos. It excels in handling complex visual data through a unified, promptable model architecture that supports real-time processing and zero-shot generalization.

This is based on the implementation of Segment-Anything-Model-2 found here. This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the Qualcomm® AI Hub Models library to export with custom configurations. More details on model performance across various devices, can be found here.

Qualcomm AI Hub Models uses Qualcomm AI Hub Workbench to compile, profile, and evaluate this model. Sign up to run these models on a hosted Qualcomm® device.

Getting Started

There are two ways to deploy this model on your device:

Option 1: Download Pre-Exported Models

Below are pre-exported model assets ready for deployment.

Runtime Precision Chipset SDK Versions Download
ONNX w8a8 Universal QAIRT 2.45, ONNX Runtime 1.27.1 Download
QNN_DLC float Universal QAIRT 2.45 Download
QNN_DLC w8a8 Universal QAIRT 2.45 Download
TFLITE w8a8 Universal QAIRT 2.45 Download

For more device-specific assets and performance metrics, visit Segment-Anything-Model-2 on Qualcomm® AI Hub.

Option 2: Export with Custom Configurations

Use the Qualcomm® AI Hub Models Python library to compile and export the model with your own:

  • Custom weights (e.g., fine-tuned checkpoints)
  • Custom input shapes
  • Target device and runtime configurations

This option is ideal if you need to customize the model beyond the default configuration provided here.

See our repository for Segment-Anything-Model-2 on GitHub for usage instructions.

Model Details

Model Type: Model_use_case.semantic_segmentation

Model Stats:

  • Input resolution: 720p (720x1280)
  • Model checkpoint: sam2.1_hiera_t
  • Model size (decoder) (float): 23.7 MB
  • Model size (encoder) (float): 128 MB
  • Number of parameters (decoder): 6.22M
  • Number of parameters (encoder): 33.5M

Performance Summary

Model Runtime Precision Chipset Inference Time (ms) Peak Memory Range (MB) Primary Compute Unit
decoder ONNX float Snapdragon® X2 Elite 3.319 ms 16 - 16 MB NPU
decoder ONNX float Snapdragon® 8 Gen 3 Mobile 4.963 ms 17 - 273 MB NPU
decoder ONNX float Qualcomm® Dragonwing™ QCS8550 (Proxy) 6.748 ms 16 - 19 MB NPU
decoder ONNX float Snapdragon® 8 Elite Gen 5 Mobile 3.185 ms 2 - 225 MB NPU
decoder ONNX w8a8 Snapdragon® X2 Elite 1.335 ms 4 - 4 MB NPU
decoder ONNX w8a8 Snapdragon® X Elite 2.701 ms 4 - 4 MB NPU
decoder ONNX w8a8 Snapdragon® 8 Gen 3 Mobile 1.766 ms 4 - 156 MB NPU
decoder ONNX w8a8 Snapdragon® 8 Gen 1 Mobile 3.463 ms 4 - 161 MB NPU
decoder ONNX w8a8 Qualcomm® Dragonwing™ QCS6490 13.151 ms 4 - 7 MB NPU
decoder ONNX w8a8 Qualcomm® Dragonwing™ IQ-8275 2.796 ms 4 - 8 MB NPU
decoder ONNX w8a8 Qualcomm® Dragonwing™ QCS8550 (Proxy) 2.584 ms 2 - 8 MB NPU
decoder ONNX w8a8 Qualcomm® QCS8450 3.463 ms 4 - 161 MB NPU
decoder ONNX w8a8 Qualcomm® Dragonwing™ IQ-9075 3.125 ms 4 - 7 MB NPU
decoder ONNX w8a8 Qualcomm® Dragonwing™ IQ-X7181 2.701 ms 4 - 4 MB NPU
decoder ONNX w8a8 Qualcomm® Dragonwing™ Q-6690 16.993 ms 2 - 251 MB NPU
decoder ONNX w8a8 Qualcomm® Dragonwing™ Q-7790 3.354 ms 4 - 301 MB NPU
decoder ONNX w8a8 Qualcomm® Dragonwing™ Q-8750 1.424 ms 0 - 128 MB NPU
decoder ONNX w8a8 Snapdragon® 8 Elite Mobile 1.424 ms 0 - 128 MB NPU
decoder ONNX w8a8 Snapdragon® 8 Elite Gen 5 Mobile 1.245 ms 0 - 153 MB NPU
decoder ONNX w8a8 Snapdragon® 7 Gen 4 Mobile 3.354 ms 4 - 301 MB NPU
decoder QNN_DLC float Snapdragon® X2 Elite 3.111 ms 16 - 16 MB NPU
decoder QNN_DLC float Snapdragon® 8 Gen 3 Mobile 3.665 ms 3 - 213 MB NPU
decoder QNN_DLC float Snapdragon® 8 Gen 1 Mobile 9.985 ms 4 - 217 MB NPU
decoder QNN_DLC float Qualcomm® QCS8450 9.985 ms 4 - 217 MB NPU
decoder QNN_DLC float Qualcomm® Dragonwing™ Q-8750 2.939 ms 4 - 192 MB NPU
decoder QNN_DLC float Qualcomm® SA8295P 8.213 ms 0 - 195 MB NPU
decoder QNN_DLC float Snapdragon® 8 Elite Mobile 2.939 ms 4 - 192 MB NPU
decoder QNN_DLC float Snapdragon® 8 Elite Gen 5 Mobile 2.59 ms 4 - 192 MB NPU
decoder QNN_DLC w8a8 Snapdragon® X2 Elite 1.275 ms 4 - 4 MB NPU
decoder QNN_DLC w8a8 Snapdragon® X Elite 2.273 ms 4 - 4 MB NPU
decoder QNN_DLC w8a8 Snapdragon® 8 Gen 3 Mobile 1.286 ms 0 - 110 MB NPU
decoder QNN_DLC w8a8 Snapdragon® 8 Gen 1 Mobile 2.537 ms 0 - 109 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ QCS6490 9.95 ms 1 - 7 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ IQ-8275 2.154 ms 1 - 7 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ QCS8550 (Proxy) 1.968 ms 4 - 6 MB NPU
decoder QNN_DLC w8a8 Qualcomm® SA8775P 2.559 ms 4 - 89 MB NPU
decoder QNN_DLC w8a8 Qualcomm® SA8650P 2.559 ms 4 - 89 MB NPU
decoder QNN_DLC w8a8 Qualcomm® SA8255P 2.559 ms 4 - 89 MB NPU
decoder QNN_DLC w8a8 Qualcomm® QCS8450 2.537 ms 0 - 109 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ IQ-9075 2.43 ms 1 - 7 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ IQ-X7181 2.273 ms 4 - 4 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ Q-6690 10.058 ms 14 - 283 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ Q-7790 2.706 ms 4 - 217 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ Q-8750 1.025 ms 1 - 90 MB NPU
decoder QNN_DLC w8a8 Qualcomm® SA7255P 3.933 ms 4 - 89 MB NPU
decoder QNN_DLC w8a8 Qualcomm® SA8295P 2.973 ms 4 - 92 MB NPU
decoder QNN_DLC w8a8 Snapdragon® 8 Elite Mobile 1.025 ms 1 - 90 MB NPU
decoder QNN_DLC w8a8 Snapdragon® 8 Elite Gen 5 Mobile 0.918 ms 1 - 126 MB NPU
decoder QNN_DLC w8a8 Snapdragon® 7 Gen 4 Mobile 2.706 ms 4 - 217 MB NPU
decoder TFLITE float Snapdragon® 8 Gen 3 Mobile 3.742 ms 0 - 214 MB NPU
decoder TFLITE float Snapdragon® 8 Gen 1 Mobile 10.018 ms 0 - 219 MB NPU
decoder TFLITE float Qualcomm® Dragonwing™ QCS8550 (Proxy) 5.328 ms 0 - 8 MB NPU
decoder TFLITE float Qualcomm® QCS8450 10.018 ms 0 - 219 MB NPU
decoder TFLITE float Qualcomm® SA8295P 8.154 ms 0 - 196 MB NPU
decoder TFLITE float Snapdragon® 8 Elite Gen 5 Mobile 2.625 ms 0 - 185 MB NPU
decoder TFLITE w8a8 Snapdragon® 8 Gen 3 Mobile 5.796 ms 0 - 267 MB NPU
decoder TFLITE w8a8 Snapdragon® 8 Gen 1 Mobile 10.182 ms 0 - 245 MB NPU
decoder TFLITE w8a8 Qualcomm® Dragonwing™ QCS6490 35.497 ms 12 - 58 MB NPU
decoder TFLITE w8a8 Qualcomm® Dragonwing™ IQ-8275 7.778 ms 0 - 15 MB NPU
decoder TFLITE w8a8 Qualcomm® Dragonwing™ QCS8550 (Proxy) 7.68 ms 0 - 22 MB NPU
decoder TFLITE w8a8 Qualcomm® SA8775P 8.613 ms 0 - 240 MB NPU
decoder TFLITE w8a8 Qualcomm® SA8650P 8.613 ms 0 - 240 MB NPU
decoder TFLITE w8a8 Qualcomm® SA8255P 8.613 ms 0 - 240 MB NPU
decoder TFLITE w8a8 Qualcomm® QCS8450 10.182 ms 0 - 245 MB NPU
decoder TFLITE w8a8 Qualcomm® Dragonwing™ IQ-9075 8.354 ms 0 - 14 MB NPU
decoder TFLITE w8a8 Qualcomm® Dragonwing™ Q-6690 32.754 ms 14 - 212 MB NPU
decoder TFLITE w8a8 Qualcomm® Dragonwing™ Q-7790 14.243 ms 12 - 203 MB NPU
decoder TFLITE w8a8 Qualcomm® Dragonwing™ Q-8750 3.507 ms 0 - 239 MB NPU
decoder TFLITE w8a8 Qualcomm® SA7255P 13.509 ms 0 - 237 MB NPU
decoder TFLITE w8a8 Qualcomm® SA8295P 9.459 ms 0 - 237 MB NPU
decoder TFLITE w8a8 Snapdragon® 8 Elite Mobile 3.507 ms 0 - 239 MB NPU
decoder TFLITE w8a8 Snapdragon® 8 Elite Gen 5 Mobile 3.109 ms 0 - 255 MB NPU
decoder TFLITE w8a8 Snapdragon® 7 Gen 4 Mobile 14.243 ms 12 - 203 MB NPU
encoder ONNX float Snapdragon® X2 Elite 85.935 ms 70 - 70 MB NPU
encoder ONNX float Snapdragon® 8 Gen 3 Mobile 124.907 ms 56 - 2168 MB NPU
encoder ONNX float Qualcomm® Dragonwing™ QCS8550 (Proxy) 169.137 ms 8 - 95 MB NPU
encoder ONNX float Snapdragon® 8 Elite Gen 5 Mobile 78.877 ms 43 - 1714 MB NPU
encoder ONNX w8a8 Snapdragon® X2 Elite 35.498 ms 18 - 18 MB NPU
encoder ONNX w8a8 Snapdragon® X Elite 90.809 ms 51 - 51 MB NPU
encoder ONNX w8a8 Snapdragon® 8 Gen 3 Mobile 62.759 ms 15 - 1940 MB NPU
encoder ONNX w8a8 Snapdragon® 8 Gen 1 Mobile 123.723 ms 14 - 2402 MB NPU
encoder ONNX w8a8 Qualcomm® Dragonwing™ QCS6490 676.442 ms 6 - 11 MB NPU
encoder ONNX w8a8 Qualcomm® Dragonwing™ IQ-8275 77.629 ms 10 - 17 MB NPU
encoder ONNX w8a8 Qualcomm® Dragonwing™ QCS8550 (Proxy) 87.996 ms 0 - 57 MB NPU
encoder ONNX w8a8 Qualcomm® QCS8450 123.723 ms 14 - 2402 MB NPU
encoder ONNX w8a8 Qualcomm® Dragonwing™ IQ-9075 74.762 ms 11 - 17 MB NPU
encoder ONNX w8a8 Qualcomm® Dragonwing™ IQ-X7181 90.809 ms 51 - 51 MB NPU
encoder ONNX w8a8 Qualcomm® Dragonwing™ Q-6690 1462.489 ms 3 - 1932 MB NPU
encoder ONNX w8a8 Qualcomm® Dragonwing™ Q-7790 88.495 ms 6 - 2391 MB NPU
encoder ONNX w8a8 Qualcomm® Dragonwing™ Q-8750 39.562 ms 11 - 2123 MB NPU
encoder ONNX w8a8 Snapdragon® 8 Elite Mobile 39.562 ms 11 - 2123 MB NPU
encoder ONNX w8a8 Snapdragon® 8 Elite Gen 5 Mobile 34.125 ms 10 - 1676 MB NPU
encoder ONNX w8a8 Snapdragon® 7 Gen 4 Mobile 88.495 ms 6 - 2391 MB NPU
encoder QNN_DLC float Snapdragon® X2 Elite 85.736 ms 12 - 12 MB NPU
encoder QNN_DLC float Snapdragon® 8 Gen 3 Mobile 129.905 ms 12 - 2144 MB NPU
encoder QNN_DLC float Snapdragon® 8 Gen 1 Mobile 342.279 ms 13 - 1861 MB NPU
encoder QNN_DLC float Qualcomm® QCS8450 342.279 ms 13 - 1861 MB NPU
encoder QNN_DLC float Qualcomm® Dragonwing™ Q-8750 100.888 ms 0 - 2511 MB NPU
encoder QNN_DLC float Qualcomm® SA8295P 282.45 ms 0 - 1581 MB NPU
encoder QNN_DLC float Snapdragon® 8 Elite Mobile 100.888 ms 0 - 2511 MB NPU
encoder QNN_DLC float Snapdragon® 8 Elite Gen 5 Mobile 79.078 ms 12 - 1718 MB NPU
encoder QNN_DLC w8a8 Snapdragon® X2 Elite 36.3 ms 3 - 3 MB NPU
encoder QNN_DLC w8a8 Snapdragon® X Elite 83.129 ms 3 - 3 MB NPU
encoder QNN_DLC w8a8 Snapdragon® 8 Gen 3 Mobile 55.009 ms 5 - 1887 MB NPU
encoder QNN_DLC w8a8 Snapdragon® 8 Gen 1 Mobile 139.075 ms 4 - 1902 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ QCS6490 678.303 ms 5 - 15 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ IQ-8275 70.357 ms 0 - 10 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ QCS8550 (Proxy) 80.158 ms 3 - 5 MB NPU
encoder QNN_DLC w8a8 Qualcomm® SA8775P 81.826 ms 3 - 1581 MB NPU
encoder QNN_DLC w8a8 Qualcomm® SA8650P 81.826 ms 3 - 1581 MB NPU
encoder QNN_DLC w8a8 Qualcomm® SA8255P 81.826 ms 3 - 1581 MB NPU
encoder QNN_DLC w8a8 Qualcomm® QCS8450 139.075 ms 4 - 1902 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ IQ-9075 80.808 ms 3 - 13 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ IQ-X7181 83.129 ms 3 - 3 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ Q-6690 1301.506 ms 3 - 1919 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ Q-7790 95.866 ms 3 - 2367 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ Q-8750 39.716 ms 3 - 1630 MB NPU
encoder QNN_DLC w8a8 Qualcomm® SA7255P 136.353 ms 2 - 1579 MB NPU
encoder QNN_DLC w8a8 Qualcomm® SA8295P 95.755 ms 0 - 1592 MB NPU
encoder QNN_DLC w8a8 Snapdragon® 8 Elite Mobile 39.716 ms 3 - 1630 MB NPU
encoder QNN_DLC w8a8 Snapdragon® 8 Elite Gen 5 Mobile 35.174 ms 3 - 1663 MB NPU
encoder QNN_DLC w8a8 Snapdragon® 7 Gen 4 Mobile 95.866 ms 3 - 2367 MB NPU
encoder TFLITE float Snapdragon® 8 Gen 3 Mobile 129.727 ms 19 - 2227 MB NPU
encoder TFLITE float Snapdragon® 8 Gen 1 Mobile 349.017 ms 0 - 1938 MB NPU
encoder TFLITE float Qualcomm® Dragonwing™ QCS8550 (Proxy) 178.988 ms 20 - 25 MB NPU
encoder TFLITE float Qualcomm® QCS8450 349.017 ms 0 - 1938 MB NPU
encoder TFLITE float Qualcomm® SA8295P 282.672 ms 20 - 1665 MB NPU
encoder TFLITE float Snapdragon® 8 Elite Gen 5 Mobile 76.393 ms 20 - 1777 MB NPU
encoder TFLITE w8a8 Snapdragon® 8 Gen 3 Mobile 69.733 ms 5 - 2227 MB NPU
encoder TFLITE w8a8 Snapdragon® 8 Gen 1 Mobile 185.683 ms 5 - 2289 MB NPU
encoder TFLITE w8a8 Qualcomm® Dragonwing™ QCS6490 1728.682 ms 76 - 338 MB CPU
encoder TFLITE w8a8 Qualcomm® Dragonwing™ IQ-8275 87.519 ms 5 - 71 MB NPU
encoder TFLITE w8a8 Qualcomm® Dragonwing™ QCS8550 (Proxy) 100.89 ms 5 - 8 MB NPU
encoder TFLITE w8a8 Qualcomm® SA8775P 98.068 ms 6 - 1877 MB NPU
encoder TFLITE w8a8 Qualcomm® SA8650P 98.068 ms 6 - 1877 MB NPU
encoder TFLITE w8a8 Qualcomm® SA8255P 98.068 ms 6 - 1877 MB NPU
encoder TFLITE w8a8 Qualcomm® QCS8450 185.683 ms 5 - 2289 MB NPU
encoder TFLITE w8a8 Qualcomm® Dragonwing™ IQ-9075 96.126 ms 2 - 68 MB NPU
encoder TFLITE w8a8 Qualcomm® Dragonwing™ Q-6690 2334.138 ms 19 - 252 MB CPU
encoder TFLITE w8a8 Qualcomm® Dragonwing™ Q-7790 1138.193 ms 78 - 311 MB CPU
encoder TFLITE w8a8 Qualcomm® Dragonwing™ Q-8750 54.788 ms 4 - 1791 MB NPU
encoder TFLITE w8a8 Qualcomm® SA7255P 163.041 ms 5 - 1877 MB NPU
encoder TFLITE w8a8 Qualcomm® SA8295P 118.327 ms 5 - 1881 MB NPU
encoder TFLITE w8a8 Snapdragon® 8 Elite Mobile 54.788 ms 4 - 1791 MB NPU
encoder TFLITE w8a8 Snapdragon® 8 Elite Gen 5 Mobile 34.475 ms 4 - 1794 MB NPU
encoder TFLITE w8a8 Snapdragon® 7 Gen 4 Mobile 1138.193 ms 78 - 311 MB CPU

License

  • The license for the original implementation of Segment-Anything-Model-2 can be found here.

References

Community

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
Image Segmentation
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Paper for qualcomm/Segment-Anything-Model-2