Adding Ollama details to Local LLM doc #14
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Updating with the version of this file from https://github.com/Azure-Samples/openai-chat-app-quickstart/blob/main/docs/local_ollama.md. It was missing Ollama details
pamelafox
reviewed
Sep 26, 2024
| @@ -4,16 +4,32 @@ You may want to save costs by developing against a local LLM server, such as | |||
| [llamafile](https://github.com/Mozilla-Ocho/llamafile/). Note that a local LLM | |||
| will generally be slower and not as sophisticated. | |||
|
|
|||
| Once you've got your local LLM running and serving an OpenAI-compatible endpoint, define `LOCAL_OPENAI_ENDPOINT` in your `.env` file. | |||
| Once you've got your local LLM running and serving an OpenAI-compatible endpoint, define `LOCAL_MODELS_ENDPOINT` in your `.env` file. | |||
There was a problem hiding this comment.
Hm, why is it LOCAL_MODELS?
There was a problem hiding this comment.
Looks like another difference between this and the openai-chat-app-quickstart sample. That one uses LOCAL_MODELS_ENDPOINT. I like that name more than LOCAL_OPENAI_ENDPOINT, but your call if you want to align the two repos. Happy to edit this PR either way.
There was a problem hiding this comment.
Ohh.. Hmm. I like LOCAL_OPENAI in terms of it being an openai-compatible endpoint, but LOCAL_MODELS also nice in that youre using local models. I'm fine with LOCAL_MODELS change, please modify rest of repo accordingly.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose
Does this introduce a breaking change?
Pull Request Type
What kind of change does this Pull Request introduce?