diff options
author | Erik Scholz <Green-Sky@users.noreply.github.com> | 2023-03-22 17:09:38 +0100 |
---|---|---|
committer | GitHub <noreply@github.com> | 2023-03-22 18:09:38 +0200 |
commit | 56e659a0b271436e24813a801640d015e7b05328 (patch) | |
tree | 599b4c53f4f81b5de59f477d331b28bd260fc78d /main.cpp | |
parent | 40ea807a972ec7b5a426f034ebfa593b5e7a06ed (diff) |
fix perplexity after c-api refactor (#390)
* preallocate a buffer of fitting size for tokenization (utils.cpp)
* don't create a new std::string (especially here, where it's usually large)
Diffstat (limited to 'main.cpp')
-rw-r--r-- | main.cpp | 2 |
1 files changed, 1 insertions, 1 deletions
@@ -85,7 +85,7 @@ void perplexity(llama_context * ctx, const gpt_params & params) { // Download: https://s3.amazonaws.com/research.metamind.io/wikitext/wikitext-2-raw-v1.zip?ref=salesforce-research // Run `./main --perplexity -m models/7B/ggml-model-q4_0.bin -f wiki.test.raw` // Output: `perplexity: 13.5106 [114/114]` - auto tokens = ::llama_tokenize(ctx, params.prompt.c_str(), true); + auto tokens = ::llama_tokenize(ctx, params.prompt, true); int count = 0; double nll = 0.0; |