4
Configuring llama.cpp server
Rickard Edén edited this page 2024-10-05 10:04:05 +02:00
Llama.cpp works works simply using the server:
- Start the llama.cpp server with the model and settings you would like to use with
server -m ... - Open
llm_config.yamlin the /tale folder and change the following fields:
URL: "http://127.0.0.1:8080"or wherever you server is running
or if you want to use OAI like interface(legacy):
- Start the llama.cpp server with the model and settings you would like to use with
server -m ... - Start an 'OAI-like' server interface, ie run
python examples/server/api_like_OAI.py - Open
llm_config.yamlin the /tale folder and change the following fields:
URL: "http://127.0.0.1:8081"or wherever you server is runningENDPOINT: "/v1/chat/completions"BACKEND: "openai"
The game will be using OPENAI_BODY if you want to change any message settings, not DEFAULT_BODY