Age | Commit message (Collapse) | Author | |
---|---|---|---|
2023-03-15 | Use `tokenizer.vocab_size()` instead of hardcoding 32000 in ↵ | Ronsor | |
convert-pth-to-ggml.py (#142) There are ways that special tokens or other new tokens could be added to the tokenizer; therefore it's probably best not to assume the vocabulary is only 32000 tokens. | |||
2023-03-13 | Fix UTF-8 handling (including colors) (#79) | Val Kharitonov | |
2023-03-12 | Revert "weights_only" arg - this causing more trouble than help | Georgi Gerganov | |
2023-03-12 | python/pytorch compat notes (#44) | Oleksandr Nikitin | |
2023-03-12 | use weights_only in conversion script (#32) | deepdiffuser | |
this restricts malicious weights from executing arbitrary code by restricting the unpickler to only loading tensors, primitive types, and dictionaries | |||
2023-03-11 | Support all LLaMA models + change Q4_0 quantization storage | Georgi Gerganov | |
2023-03-10 | Fix a bug in the rope calculation | Georgi Gerganov | |
2023-03-10 | Initial release | Georgi Gerganov | |