Using blocks of 32 and 16 bits per group of 8 weights it beats iq2_xxs in terms of PPL by a significant margin. It is 0.0625 bpw larger, but even if we go to 15 bits per group od 8 (so 0.0625 bpw less than iq2_xxs), PPL is still lower. |
||
|---|---|---|
| .. | ||
| CMakeLists.txt | ||
| quantize-stats.cpp | ||