diff options
| author | Georgi Gerganov <ggerganov@gmail.com> | 2023-04-18 23:54:57 +0300 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-04-18 23:54:57 +0300 |
| commit | 77a73403ca8eaced2590559d0f9cebd2b3649d32 (patch) | |
| tree | 7b95e7565ce86b81d8dd620117564da901ce3ce7 /examples/chat.sh | |
| parent | 50a8a2af97cb92e53e7a3195aa201c3d87da5415 (diff) | |
ggml : add new Q4_2 quantization (ARM only) (#1046)
* ggml : Q4_2 ARM
* ggml : add ggml_is_quantized()
* llama : update llama_type_name() with Q4_2 entry
* ggml : speed-up q4_2
- 4 threads: ~100ms -> ~90ms
- 8 threads: ~55ms -> ~50ms
* ggml : optimize q4_2 using vmlaq_n_f32 + vmulq_n_f32
Diffstat (limited to 'examples/chat.sh')
0 files changed, 0 insertions, 0 deletions