diff options
| author | l3utterfly <gc.pthzfoldr@gmail.com> | 2023-08-04 19:29:52 +0800 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-08-04 13:29:52 +0200 |
| commit | 415e99fec27be5a2e4283f1937afd17eb33fbd66 (patch) | |
| tree | b16b4735a94fa5a5fc76cb4fef691fbe6c210489 /examples/server | |
| parent | ff966e7ca6af127c9405523cdb07ef8fa01bf6d6 (diff) | |
Stream save llama context data to file instead of allocating entire buffer upfront (#2488)
* added stream saving context data to file to avoid allocating unnecessary amounts of memory
* generalised copying state data to file or buffer
* added comments explaining how copy_state_data works
* fixed trailing whitespaces
* fixed save load state example
* updated save load state to use public function in llama.cpp
* - restored breakage of the llama_copy_state_data API
- moved new logic for copying llama state data to internal function
* fixed function declaration order
* restored save load state example
* fixed whitepace
* removed unused llama-util.h include
* Apply suggestions from code review
Co-authored-by: slaren <slarengh@gmail.com>
* Apply code review suggestions
Co-authored-by: slaren <slarengh@gmail.com>
---------
Co-authored-by: slaren <slarengh@gmail.com>
Diffstat (limited to 'examples/server')
0 files changed, 0 insertions, 0 deletions