diff options
author | Georgi Gerganov <ggerganov@gmail.com> | 2023-04-19 20:10:08 +0300 |
---|---|---|
committer | GitHub <noreply@github.com> | 2023-04-19 20:10:08 +0300 |
commit | 884e7d7a2bfd7325b107442d6758983f5886ed3d (patch) | |
tree | 9b3bcda080b127f069092cfc04db151421746754 /README.md | |
parent | 7cd5c4a3e9106151d48f328bb3c94c298a211f18 (diff) |
ggml : use 8-bit precision for Q4_1 intermediate results (#1047)
* ggml : use 8-bit precision for Q4_1 intermediate results (ARM)
* ggml : optimize ggml_vec_dot_q4_1_q8_0() via vmalq_n_f32
56 ms/token with Q4_1 !
* ggml : AVX2 implementation of ggml_vec_dot_q4_1_q8_0 (#1051)
* gitignore : ignore ppl-*.txt files
---------
Co-authored-by: slaren <2141330+slaren@users.noreply.github.com>
Diffstat (limited to 'README.md')
0 files changed, 0 insertions, 0 deletions