by openai-community
⬇ 29.8M♥ 3.9K
These are 4-bit quantized weights for NVIDIA GPUs; load them with a library that supports AWQ, such as vLLM or Hugging Face Transformers.
Downloads are gated: accept the publisher's terms on Hugging Face first.
The model card on Hugging Face has the exact usage code.
Check the model card for its memory needs; cloud GPUs range from 16 GB to 80 GB and beyond.
Data from Hugging Face's public API, last checked October 24, 2026. Downloads are Hugging Face's count for the last 30 days.