diff options
| author | Georgi Gerganov <ggerganov@gmail.com> | 2023-04-25 23:40:51 +0300 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-04-25 23:40:51 +0300 |
| commit | 7a32fcb3b29f4db8aed8a85dc58eb958fb118153 (patch) | |
| tree | b363c851cead2b5d6efced33cc461e37d8ed6bf8 /ggml-cuda.h | |
| parent | dd0eabc049fb1efc631cab8eb0a646808d704e18 (diff) | |
ggml : add Q8_0 quantization format (rename the old one to Q8_1) (ARM NEON) (#1179)
* ggml : add Q8_0 quantization format (rename the old one to Q8_1)
* tests : fix test-quantize-fns
* ggml : finalize Q8_0 implementation
* ggml : use q4_0_q8_0 and q4_2_q8_0
* ggml : fix Q8_0 dot product bug (ARM)
* ggml : Q8_0 unroll x2
* ggml : fix bug - using wrong block type
* ggml : extend quantize_fns_t with "vec_dot_type"
* ggml : fix Q8_0 to use 255 values out of 256
* ggml : fix assert using wrong QK4_2 instead of QK4_3
Diffstat (limited to 'ggml-cuda.h')
| -rw-r--r-- | ggml-cuda.h | 1 |
1 files changed, 1 insertions, 0 deletions
diff --git a/ggml-cuda.h b/ggml-cuda.h index 370bbc7..4048ea4 100644 --- a/ggml-cuda.h +++ b/ggml-cuda.h @@ -35,6 +35,7 @@ void dequantize_row_q4_0_cuda(const void * vx, float * y, int k, cudaStream_t st void dequantize_row_q4_1_cuda(const void * vx, float * y, int k, cudaStream_t stream); void dequantize_row_q4_2_cuda(const void * vx, float * y, int k, cudaStream_t stream); void dequantize_row_q4_3_cuda(const void * vx, float * y, int k, cudaStream_t stream); +void dequantize_row_q8_0_cuda(const void * vx, float * y, int k, cudaStream_t stream); #ifdef __cplusplus } |