GPTQ, Explained and Run
How GPTQ trades weight error for output error, in 30 lines of PyTorch on a real layer, then how to run it with llm-compressor and serve it with vLLM, and which settings actually change the result on Qwen3-8B.
gptq
1 post
1 post
How GPTQ trades weight error for output error, in 30 lines of PyTorch on a real layer, then how to run it with llm-compressor and serve it with vLLM, and which settings actually change the result on Qwen3-8B.