Using groq as a chat completion service in semantic kernel

[giannik]

What would be the recommended way in semantic kernel to consume groq api for chatCompletion ?
I found this article suggesting a httpClient with a custom delegate handler:
HttpClient with a custom delegate handler

but i also discovered Microsoft.Extensions.AI that recommends using an IChatClient :
https://devblogs.microsoft.com/dotnet/introducing-microsoft-extensions-ai-preview/

What would be the recommended way in SK. @RogerBarreto

[RogerBarret0]

@giannik Keep in mind that we don't have an official connector for groq, and normally for API's that follows the OpenAI Chat Completion Standard, for simple tasks using the OpenAI connector may work as expected.

Important

Semantic Kernel is very extensible and there's not a definitive answer, you can use below approaches

OpenAI Custom Endpoint

In the first example you don't necessarily need a httphandler, as the OpenAI SDK allows custom endpoints to be passed.

We also have some UnitTest's ensuring this, see:

semantic-kernel/dotnet/src/Connectors/Connectors.OpenAI.UnitTests/Services/OpenAIChatCompletionServiceTests.cs

Line 87 in ffac88a

public async Task ItUsesCustomEndpointsWhenProvidedDirectlyAsync(string endpointProvided, string expectedEndpoint)

Using a custom endpoint:

var chatCompletion = new OpenAIChatCompletionService(
    modelId: "any", 
    apiKey: "api-key", 
    endpoint: new Uri("https://api.groq.com/openai/v1"))

AI Abstractions

Using the example provided in the blog you can convert the IChatClient to IChatCompletionService easily like below:

using Microsoft.SemanticKernel;

// <ProviderChatClient> here can be any, OllamaChatClient, OpenAIChatClient, etc... for groq, OpenAI may work best.
var service =  new <ProviderChatClient>(...)
    .AsChatCompletionService();


// your SK code

Http Handler

Follow the approach mentioned in the link HttpClient with a custom delegate handler

[giannik]

@RogerBarret0 thanks