Chat is very slow when using with the llama.cpp server

[lehuythangit]

Validations

• I believe this is a way to improve. I'll try to join the Continue Discord for questions
• I'm not able to find an open issue that requests the same enhancement

Problem

Chat is very slow when using with the llama.cpp server when increasing the message because missing cache_prompt = true when call the /completion API of llama.cpp, so llama.cpp process also all previous message history when prompt instead of using from cache.

Solution

Please help add more property cache_prompt = true when call the /completion API,
or add more configuration property into the config.json

Thanks

[lehuythangit]

i just workaround with modified the .vscode\extensions\continue.continue-0.8.55-win32-x64\out\extension.js