diff options
| author | Erik Scholz <Green-Sky@users.noreply.github.com> | 2023-03-22 17:09:38 +0100 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-03-22 18:09:38 +0200 |
| commit | 56e659a0b271436e24813a801640d015e7b05328 (patch) | |
| tree | 599b4c53f4f81b5de59f477d331b28bd260fc78d /examples | |
| parent | 40ea807a972ec7b5a426f034ebfa593b5e7a06ed (diff) | |
fix perplexity after c-api refactor (#390)
* preallocate a buffer of fitting size for tokenization (utils.cpp)
* don't create a new std::string (especially here, where it's usually large)
Diffstat (limited to 'examples')
0 files changed, 0 insertions, 0 deletions