The current implementation will also only work for prompts where the response is plain text (as opposed to json, which is more common). The descriptions of locations tend to be the longest, and they are just text.

As of now, it's off by default. To enable, set STREAM = True in backend_kobold_cpp.yaml or backend_llama_cpp.yaml