WebLLM

[sublimator]

Stumbled back on this repo, and was looking at the changes/issues, and then onwards to Web Speech stuff.

Something in the back of my head for a while is WebLLM. Basically adding a polyfill to the page in the MAIN context that adds an LLM api (accessible by webpages)

I really haven't thought about it much at all

1. Is it even worth doing?
2. What would it look like?
3. Requesting specific models? model families/classes ?

Shrug

[andrewnguonly]

Basically adding a polyfill to the page in the MAIN context that adds an LLM api (accessible by webpages)

Can you explain with a use case? Not sure if I'm following.

[sublimator]

Haha, I am not sure I am following, myself! Like I said, I haven't thought this through much! WebApps that want to use an LLM? All the advantages of web based apps. No need to install etc I mean, I guess you just configure an app to use a localhost openai compatible url and configure allow origins etc as an alternative. I guess it really just saves building that permissions UI in multiple places? Anything else? Shrug??