New Feature: FunctionChoiceBehavior.Required Answer Generation

[nmoeller]

Feature for FunctionChoiceBehavior.Required()

i was trying out FunctionChoiceBehavior.Required() in a Project.

But it is only returning the FunctionCall Content for me.

I noticed that since the maximum_auto_invoke_attempts is set to 1, it will only give me a function call back, but never the generated Answer based on the Function Call from the Model. I am not sure if the else after the for loop is working for me, maybe i am doing something wrong.

I guess it's intended behavior, because we cannot make sure it's generating an answer in the second run ?

But probably it would be nice if we could add an option for to iterate a second time with FunctionChoiceBehavior.Required without tools in the Body to ensure a model answer from FunctionChoiceBehavior.Required(). Or maybe the option just that the developer can set the maximum_auto_invoke_attempts.

Please let me know what you think, if this could be helpful.

[nmoeller]

Ok looks like something is wrong on my site, when testing it, sorry for the confusion.

I've stored the settings in an object and then they were overwritten by SK with the settings.function_choice_behavior.auto_invoke_kernel_functions = False in the else case.

So the first time i get an answer and the second time i got only function calls.

[yanshinian]

it will only give me a function call back, but never the generated Answer based on the Function Call from the Model.

I encountered the same issue. How did you handle it afterward?

[nmoeller]

@yanshinian so if you have the FunctionCallContent Object you can invorke it either with kernel.invoke() or kernel.invoke_function_call().

You can find the coresponding parameters for the function here : https://github.com/microsoft/semantic-kernel/blob/main/python/semantic_kernel/kernel.py

These Functions will either give you a FunctionCallResult Object or adding the FunctionCallResult automatically to the Chat History.
Then you can call the Model again with the full history containing ChatMessage --> FunctionCallContent --> FunctionCallResult.

To call the Model again you can either use kernel.invoke() with a Function/Prompt containing the ChatHistory or using the get_chat_message_content in https://github.com/microsoft/semantic-kernel/blob/main/python/semantic_kernel/connectors/ai/chat_completion_client_base.py from the corresponding connector you are using.

[yanshinian]

Using the invoke method of ChatCompletionAgent is fine, but I have issues when using invoke_stream.

[yanshinian]

I debugged the code. When the value of maximum_auto_invoke_attempts corresponding to FunctionChoiceBehavior.Required() is 1, I used invoke_stream to execute. After the function call, the invoke loop stops, so no LLM invoke occurs.

The reason the invoke works fine is because of the following code:

# Do a final call, without function calling when the max has been reached.
self._reset_function_choice_settings(settings)
# After retrieving the recall results here, a final call to the large model is made
return await self._inner_get_chat_message_contents(chat_history, settings)

[yanshinian]

When FunctionChoiceBehavior.Required() is set, I call ChatCompletionAgent->invoke_stream and then call it again. This meets my needs.

Or do you have any other good suggestions? There are too few examples online.