diff options
| author | Christian Falch <875252+chrfalch@users.noreply.github.com> | 2023-04-02 12:23:04 +0200 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-04-02 12:23:04 +0200 |
| commit | e986f94829bae0b9e66b326acbbba179931c84f1 (patch) | |
| tree | 2bfe56177c5a08f4cf46c8174925f61bd82992cc /.github | |
| parent | c0bb1d3ce21005ab21d686626ba87261a6e3a660 (diff) | |
Added api for getting/setting the kv_cache (#685)
The api provides access methods for retrieving the current memory buffer for the kv_cache and its token number.
It also contains a method for setting the kv_cache from a memory buffer.
This makes it possible to load/save history - maybe support --cache-prompt paramater as well?
Co-authored-by: Pavol Rusnak <pavol@rusnak.io>
Diffstat (limited to '.github')
0 files changed, 0 insertions, 0 deletions