aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorWeird Constructor <weirdconstructor@gmail.com>2023-07-28 10:44:43 +0200
committerGitHub <noreply@github.com>2023-07-28 11:44:43 +0300
commitd91f3f0c55663719ea03b76311e8c36ed55eb0e2 (patch)
tree4459662eb1ce985b4f267fd178719f4df3731b51
parent65cdf34bdc469fa86248e667a5880992684ef114 (diff)
readme : fix the description of the Tail free sampling (TFS) method (#2431)
-rw-r--r--examples/main/README.md4
1 files changed, 2 insertions, 2 deletions
diff --git a/examples/main/README.md b/examples/main/README.md
index 3753861..014112e 100644
--- a/examples/main/README.md
+++ b/examples/main/README.md
@@ -202,9 +202,9 @@ Example usage: `--top-p 0.95`
- `--tfs N`: Enable tail free sampling with parameter z (default: 1.0, 1.0 = disabled).
-Tail free sampling (TFS) is a text generation technique that aims to reduce the impact of less likely tokens, which may be less relevant, less coherent, or nonsensical, on the output. The method adjusts the logits (token probabilities) by raising them to the power of the parameter z. A higher value of z (e.g., 2.0) will further suppress less likely tokens from the tail of the distribution, while a value of 1.0 disables the effect of TFS. By setting the parameter z, you can control how much the probabilities of less likely tokens are reduced.
+Tail free sampling (TFS) is a text generation technique that aims to reduce the impact of less likely tokens, which may be less relevant, less coherent, or nonsensical, on the output. Similar to Top-P it tries to determine the bulk of the most likely tokens dynamically. But TFS filters out logits based on the second derivative of their probabilities. Adding tokens is stopped after the sum of the second derivatives reaches the parameter z. In short: TFS looks how quickly the probabilities of the tokens decrease and cuts off the tail of unlikely tokens using the parameter z. Typical values for z are in the range of 0.9 to 0.95. A value of 1.0 would include all tokens, and thus disables the effect of TFS.
-Example usage: `--tfs 2.0`
+Example usage: `--tfs 0.95`
### Locally Typical Sampling