63 releases 78 tags

RSS feed

• v0.21.0 1594142c7d

  Compare

  v0.21.0 - Improved streaming for koboldcpp Stable

  neph1 released this 2024-01-07 15:18:51 -05:00 | 305 commits to master since this release

  What's Changed

  • Update v0.21.0 by @neph1 in https://github.com/neph1/LlamaTale/pull/57

  I've begun some refactoring work for how prompting works to make it more OOP. This may not be so interesting from a user point of view. In any case, I like to do it in small steps to make testing less of a burden. The goal, besides making it more manageable, is to improve character consistency and memory.

  I also spent some time to fix streaming for kobold cpp. There was an experimental version prior to this, but it didn't work very well. This works better, but one quirk is that the initial location description will appear before the 'welcome' message. Consider it a prologue. The current implementation will also only work for prompts where the response is plain text (as opposed to json, which is more prevalent). The good news is that the descriptions of locations tend to be the longest, and they are just text.
  Hopefully this opens up to using larger models.

  It's off by default. To enable, set STREAM = True in backend_kobold_cpp.yaml

  Full Changelog: https://github.com/neph1/LlamaTale/compare/v0.20.1...v0.21.0