diff options
| author | Georgi Gerganov <ggerganov@gmail.com> | 2023-03-24 23:17:37 +0200 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-03-24 23:17:37 +0200 |
| commit | 7a9b6c3a8bdc1cb75fefc826dfaa7331eb63695d (patch) | |
| tree | 339815189c912e9a759a0259613621f6a2adcbf4 /examples | |
| parent | 31572d966531f7d768eb773322016ab78eb6e835 (diff) | |
Reduce memory usage and allocate enough memory for largest context (#473)
* Reduce memory usage and allocate enough memory for large contexts
* Simpler scratch buffer usage
* Reenable BLAS for quantized mul_mat
* Fix number of layers in 30B and 65B
* Fix KV cache size for F32
Diffstat (limited to 'examples')
0 files changed, 0 insertions, 0 deletions