diff options
| author | Georgi Gerganov <ggerganov@gmail.com> | 2023-04-26 23:14:13 +0300 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-04-26 23:14:13 +0300 |
| commit | 574406dc7e350ddbffaeca33bf0392b7bfeb1436 (patch) | |
| tree | 03c50ad8b07a612b2169b0bba6b08bd20b11d83a /ggml-cuda.h | |
| parent | 87a6f846d3e929632c45916dd08f1e2a9c72d2a3 (diff) | |
ggml : add Q5_0 and Q5_1 quantization (#1187)
* ggml : add Q5_0 quantization (cuBLAS only)
* ggml : fix Q5_0 qh -> uint32_t
* ggml : fix q5_0 histogram stats
* ggml : q5_0 scalar dot product
* ggml : q5_0 ARM NEON dot
* ggml : q5_0 more efficient ARM NEON using uint64_t masks
* ggml : rename Q5_0 -> Q5_1
* ggml : adding Q5_0 mode
* quantize : add Q5_0 and Q5_1 to map
* ggml : AVX2 optimizations for Q5_0, Q5_1 (#1195)
---------
Co-authored-by: Stephan Walter <stephan@walter.name>
Diffstat (limited to 'ggml-cuda.h')
| -rw-r--r-- | ggml-cuda.h | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/ggml-cuda.h b/ggml-cuda.h index 4048ea4..ed9b441 100644 --- a/ggml-cuda.h +++ b/ggml-cuda.h @@ -35,6 +35,8 @@ void dequantize_row_q4_0_cuda(const void * vx, float * y, int k, cudaStream_t st void dequantize_row_q4_1_cuda(const void * vx, float * y, int k, cudaStream_t stream); void dequantize_row_q4_2_cuda(const void * vx, float * y, int k, cudaStream_t stream); void dequantize_row_q4_3_cuda(const void * vx, float * y, int k, cudaStream_t stream); +void dequantize_row_q5_0_cuda(const void * vx, float * y, int k, cudaStream_t stream); +void dequantize_row_q5_1_cuda(const void * vx, float * y, int k, cudaStream_t stream); void dequantize_row_q8_0_cuda(const void * vx, float * y, int k, cudaStream_t stream); #ifdef __cplusplus |