diff options
| author | Georgi Gerganov <ggerganov@gmail.com> | 2023-03-24 23:17:58 +0200 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-03-24 23:17:58 +0200 |
| commit | 04c6f5ed6fafd63601fa06757877ed5ccf9d5991 (patch) | |
| tree | 1a730c214eae193ef234a4dd59cc543174f5bf2b /examples | |
| parent | 7a9b6c3a8bdc1cb75fefc826dfaa7331eb63695d (diff) | |
Immediately start processing the prompt before user input has been provided (#476)
Diffstat (limited to 'examples')
| -rwxr-xr-x | examples/chatLLaMa | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/examples/chatLLaMa b/examples/chatLLaMa index 97ababb..4265d7b 100755 --- a/examples/chatLLaMa +++ b/examples/chatLLaMa @@ -13,7 +13,7 @@ N_PREDICTS="${N_PREDICTS:-2048}" # Note: you can also override the generation options by specifying them on the command line: # For example, override the context size by doing: ./chatLLaMa --ctx_size 1024 -GEN_OPTIONS="${GEN_OPTIONS:---ctx_size 2048 --temp 0.7 --top_k 40 --top_p 0.5 --repeat_last_n 256 --repeat_penalty 1.17647}" +GEN_OPTIONS="${GEN_OPTIONS:---ctx_size 2048 --temp 0.7 --top_k 40 --top_p 0.5 --repeat_last_n 256 --batch_size 1024 --repeat_penalty 1.17647}" # shellcheck disable=SC2086 # Intended splitting of GEN_OPTIONS ./main $GEN_OPTIONS \ |