Instructions to use groxaxo/Qwen3.6-35B-A3B-Heretic-REAP-0.30 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use groxaxo/Qwen3.6-35B-A3B-Heretic-REAP-0.30 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="groxaxo/Qwen3.6-35B-A3B-Heretic-REAP-0.30")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("groxaxo/Qwen3.6-35B-A3B-Heretic-REAP-0.30")
model = AutoModelForCausalLM.from_pretrained("groxaxo/Qwen3.6-35B-A3B-Heretic-REAP-0.30", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use groxaxo/Qwen3.6-35B-A3B-Heretic-REAP-0.30 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "groxaxo/Qwen3.6-35B-A3B-Heretic-REAP-0.30"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "groxaxo/Qwen3.6-35B-A3B-Heretic-REAP-0.30",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/groxaxo/Qwen3.6-35B-A3B-Heretic-REAP-0.30

SGLang

How to use groxaxo/Qwen3.6-35B-A3B-Heretic-REAP-0.30 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "groxaxo/Qwen3.6-35B-A3B-Heretic-REAP-0.30" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "groxaxo/Qwen3.6-35B-A3B-Heretic-REAP-0.30",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "groxaxo/Qwen3.6-35B-A3B-Heretic-REAP-0.30" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "groxaxo/Qwen3.6-35B-A3B-Heretic-REAP-0.30",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use groxaxo/Qwen3.6-35B-A3B-Heretic-REAP-0.30 with Docker Model Runner:
```
docker model run hf.co/groxaxo/Qwen3.6-35B-A3B-Heretic-REAP-0.30
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Qwen3.6-35B-A3B-Heretic — REAP-pruned (30%, 256→180 experts)

Overview

Qwen3.6-35B-A3B-Heretic-REAP-0.30 is a model checkpoint packaged for compatible Hugging Face runtimes, published by groxaxo. It is intended for open-source evaluation, reproducible experimentation, and compatible local or hosted inference workflows. The wording below is deliberately limited to what can be verified from this repository's metadata and artifacts.

The repository name identifies a behavior-modified or reduced-filtering lineage. That label describes the source or conversion history; it is not a guarantee of unrestricted behavior in every prompt or runtime. Test outputs carefully before sharing or deploying them.

At a glance

Field	Details
Format	Transformers
Source / base	`Qwen/Qwen3.5`
Intended task	`text-generation`
License	`apache-2.0`

What is included

*.safetensors (3 files)
config.json
generation_config.json
tokenizer.json
tokenizer_config.json
chat_template.jinja
Additional configuration, tokenizer, processor, or shard files (9 visible artifacts total)

Quick start

Getting started

Start with the upstream library named in the repository metadata and keep all configuration, tokenizer, processor, and weight files together. This repository is an artifact release, so the source project remains the authoritative reference for task-specific loading code.

Compatibility and responsible use

Use a runtime that explicitly supports this format, architecture, and modality.
Keep configuration, tokenizer, processor, projection, and weight files from the same revision together.
Review the source model card and license before redistribution or deployment.
Hardware needs depend on parameter count, context length, cache precision, quantization, and concurrency.
Report reproducible issues with the runtime version, hardware, launch command, and a minimal example.

Generated outputs may be inaccurate or unsuitable for a given use case. Users are responsible for testing behavior, applying appropriate safeguards, and complying with applicable licenses and laws.

This is a REAP (Router-weighted Expert Activation Pruning) compressed version of a Qwen3.5/3.6-35B-A3B hybrid linear-attention MoE (Heretic variant). 30% of the routed experts were pruned (256 → 180 per layer) using the Cerebras REAP saliency criterion.

	Base	This model
Routed experts / layer	256	180
Active experts / token	8	8
Decoder layers	40 (30 linear-attn + 10 full-attn)	same
Hidden size	2048	2048
Size (BF16)	~70 GB	~50 GB

Pruning was performed with the layer-wise (block-wise) calibration observer on a c4 calibration set, reap saliency, seed 42. The model is text-only (the multimodal vision tower of the base Omni model was dropped during pruning).

Evaluation

Measured with lm-evaluation-harness against this model served on vLLM (pipeline-parallel, BF16):

Benchmark	Setup	Score
MMLU	5-shot, acc	73.25%
– STEM		69.11%
– Humanities		67.46%
– Social Sciences		82.97%
– Other		76.60%
HellaSwag	0-shot, acc_norm	82.55%
HellaSwag	0-shot, acc	63.06%

(30% of experts pruned; these scores indicate the model retains strong general knowledge and commonsense after compression.)

Architecture note (hybrid linear attention)

This is a hybrid model: 30/40 layers use Gated-DeltaNet linear attention (Mamba-style recurrent conv+SSM state) and 10/40 use full attention. Two consequences:

Hugging Face transformers: loads and generates correctly on a single device. Do not use device_map="auto" to split it across multiple GPUs — splitting the recurrent linear-attention state across devices produces garbage. Use one GPU (if it fits) or CPU.
vLLM: serve with pipeline parallelism across GPUs (--pipeline-parallel-size N --tensor-parallel-size 1). vLLM provides its own GatedDeltaNet kernels.

Method

REAP selects experts to prune that contribute minimally to the layer output by considering both the router gate-values and the average activation norm of the experts, preserving the router's independent control over the remaining experts (avoiding the functional-subspace collapse of expert merging).

Reproduce with cerebras/reap.

Downloads last month: 565

Safetensors

Model size

25B params

Tensor type

BF16

Model tree for groxaxo/Qwen3.6-35B-A3B-Heretic-REAP-0.30

Quantizations

1 model

Paper for groxaxo/Qwen3.6-35B-A3B-Heretic-REAP-0.30

REAP the Experts: Why Pruning Prevails for One-Shot MoE compression

Paper • 2510.13999 • Published Oct 15, 2025 • 21