About Function Calling deserialization

[goranged-intc]

Hello team,

First, some context, our company group has been heavily using SK for the past year and a half. However, we are using it as a backend, we abstract multiple models through it and we expose chat completion, embedding and other services through APIs:

Client -> Inference API -> SK AI Backend -> Provider

Now, it works but now we are trying to enable FUnction Calling for clients using function calling:

Client (Using SK with function calling) -> Inference API -> SK AI Backend -> Provider.

And the challenge is that, since the SK in the client is serializing the tools and passing that in the payload, the SK in the backend is just receiving those tools, but we don't see a way to convert (in the backend) those tools (json) into plugin function with parameters since the functions are not in the context of the backend SK.

Do you know a way to handle the function calling passthrough in the backend?

Best,
Guillermo.

[moonbox3]

Hi @goranged-intc, thanks for your question. If I understand correctly, you're asking about how to handle the serialized tool as JSON in your backend like this:

{
    "type": "function",
    "function": {
        "name": "DateTimePlugin-GetData",
        "description": "Get a Data class.",
        "parameters": {
            "type": "object",
            "properties": {
                "data": {
                    "type": "object",
                    "properties": {"timestamp": {"type": "object"}},
                    "required": ["timestamp"],
                    "description": "The data.",
                }
            },
            "required": ["data"],
        },
    },
}

Is that accurate? The name of the tool is the full qualified name (plugin name - function name, in this example DateTimePlugin-GetData) so you'd be able to get the plugin/function in your backend if you have them registered on the kernel.

Is my understanding correct here, or am I way off?

[goranged-intc]

Hi moonbox3

Yes, the client using SK is sending that as part of the payload to our backend. Now, our backend (we also use SK in the backend) and we want to take those incoming tools (the methods are not located in the backend so we only have the incoming tools json), injected in the kernel and then use GetChatMessageContentsAsync to call Azure OpenAI (passing the tools).

So our backend acts like a pass through but we are using SK.

And since SK assumed the Methods are available and use plugin.GetFromType(Class) we are wondering if there is a workaround (aka plugins.GetFromJsonTool) or something similar.