diff options
| author | Evan Jones <evan.q.jones@gmail.com> | 2023-05-24 02:24:01 -0400 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-05-24 09:24:01 +0300 |
| commit | c31bbe934b9666af42f32ce12d32cae9160e5dc4 (patch) | |
| tree | 1724822c87be283cb5fdde0d01d1791d3998f0b0 | |
| parent | 1359b6aba55d5b0410f6adaa0aa2e49bbfd01d84 (diff) | |
readme : add docs for chat-persistent.sh (#1568)
* readme : add docs for chat-persistent.sh
* Update README.md
| -rw-r--r-- | README.md | 19 |
1 files changed, 19 insertions, 0 deletions
@@ -391,6 +391,25 @@ Note the use of `--color` to distinguish between user input and generated text.  +### Persistent Interaction + +The prompt, user inputs, and model generations can be saved and resumed across calls to `./main` by leveraging `--prompt-cache` and `--prompt-cache-all`. The `./examples/chat-persistent.sh` script demonstrates this with support for long-running, resumable chat sessions. To use this example, you must provide a file to cache the initial chat prompt and a directory to save the chat session, and may optionally provide the same variables as `chat-13B.sh`. The same prompt cache can be reused for new chat sessions. Note that both prompt cache and chat directory are tied to the initial prompt (`PROMPT_TEMPLATE`) and the model file. + +```bash +# Start a new chat +PROMPT_CACHE_FILE=chat.prompt.bin CHAT_SAVE_DIR=./chat/default ./examples/chat-persistent.sh + +# Resume that chat +PROMPT_CACHE_FILE=chat.prompt.bin CHAT_SAVE_DIR=./chat/default ./examples/chat-persistent.sh + +# Start a different chat with the same prompt/model +PROMPT_CACHE_FILE=chat.prompt.bin CHAT_SAVE_DIR=./chat/another ./examples/chat-persistent.sh + +# Different prompt cache for different prompt/model +PROMPT_TEMPLATE=./prompts/chat-with-bob.txt PROMPT_CACHE_FILE=bob.prompt.bin \ + CHAT_SAVE_DIR=./chat/bob ./examples/chat-persistent.sh +``` + ### Instruction mode with Alpaca 1. First, download the `ggml` Alpaca model into the `./models` folder |