-
released this
2023-06-18 14:23:39 -04:00 | 79 commits to main since this releaseChanges
- Handle special tokens properly while tokenizing
- Handle incomplete UTF-8 multi-byte characters while generating text
- Increase buffer size in
tokenize()for BOS token - Update ggml and llama.cpp