diff options
| author | Georgi Gerganov <ggerganov@gmail.com> | 2023-03-25 21:36:22 +0200 |
|---|---|---|
| committer | Georgi Gerganov <ggerganov@gmail.com> | 2023-03-25 21:36:22 +0200 |
| commit | e2d490dafd860eaaaf9aa8008ab790527d556daf (patch) | |
| tree | 927b92a97be1ed42ad9c0744fe74571ab606730f /examples/chat.sh | |
| parent | 03f7e335604b3d68f74995aa2ccb4955833ee423 (diff) | |
Inifinite generation via context swapping (#71)
Diffstat (limited to 'examples/chat.sh')
| -rwxr-xr-x | examples/chat.sh | 8 |
1 files changed, 7 insertions, 1 deletions
diff --git a/examples/chat.sh b/examples/chat.sh index 2b6a63e..97973d0 100755 --- a/examples/chat.sh +++ b/examples/chat.sh @@ -7,4 +7,10 @@ cd `dirname $0` cd .. -./main -m ./models/7B/ggml-model-q4_0.bin -b 128 -n 256 --repeat_penalty 1.0 --color -i -r "User:" -f prompts/chat-with-bob.txt +# Important: +# +# "--keep 48" is based on the contents of prompts/chat-with-bob.txt +# +./main -m ./models/7B/ggml-model-q4_0.bin -c 2048 -b 1024 -n 256 --keep 48 \ + --repeat_penalty 1.0 --color -i \ + -r "User:" -f prompts/chat-with-bob.txt |