MUSE-News_Llama-2-7b_npo_beta0p1_lr1e-5_lam1_ep5_bestretain-QAT-int4-ffn

FFN-only quantization-aware (QAT) unlearning, MUSE News, NPO+gdr (beta=0.1, lambda=1), lr=1e-5, asym per-group-128 INT4 fake-quant (STE) on MLP gate/up/down only (attention bf16). 10 epochs; ckpts at 2/5/10 in epoch_N/.

Epoch trajectory (verbmem_f/knowmem_f lower=more unlearned; knowmem_r higher=better)

epoch	quant	verbmem_f	privleak	knowmem_f	knowmem_r
2	bf16 none	22.4779	-99.5599	41.1493	43.4177
2	FFN int4	18.8899	-99.4552	39.0291	37.7945
2	FFN nf4	22.9414	-99.6857	39.6771	46.4857
2	ALL int4	19.4555	-99.7276	44.9853	42.1273
2	ALL nf4	25.5688	-99.7904	42.3595	48.671
2	ALL gptq	26.4802	-99.7904	50.0172	45.9356
5	bf16 none	0.0	100.6496	2.1071	12.0215
5	FFN int4	2.4582	108.948	44.6953	40.8935
5	FFN nf4	0.0	109.1785	39.8894	37.0005
5	ALL int4	20.9456	-99.6228	48.9355	43.456
5	ALL nf4	5.7852	104.5054	49.6661	45.5387
5	ALL gptq	0.6952	109.0738	44.8224	36.701
10	bf16 none	0.0	102.6194	2.2	3.9418
10	FFN int4	2.0168	108.969	46.1162	41.0261
10	FFN nf4	0.0	109.0109	47.0445	37.0392
10	ALL int4	21.3542	-99.6438	47.2107	43.5932
10	ALL nf4	8.6149	100.7963	49.7614	46.0866
10	ALL gptq	0.479	109.1785	38.2095	35.4143