v1.3.0

Enhancements

• Added ToolCallParsers -- these classes are wrappers around Kani Engines that parse raw text generated by a model, and return Kani-format tool calls. This is an easy way to enable tool calling on open-source models!

Example:

from kani.engines.huggingface import HuggingEngine
from kani.prompts.impl.mistral import MISTRAL_V3_PIPELINE
from kani.tool_parsers.mistral import MistralToolCallParser
model = HuggingEngine(model_id="mistralai/Mistral-Small-Instruct-2409", prompt_pipeline=MISTRAL_V3_PIPELINE)
engine = MistralToolCallParser(model)

• Added NaiveJSONToolCallParser (e.g., Llama 3)
• Added MistralToolCallParser
• Added DeepseekR1ToolCallParser

Bug Fixes et al.

• Fix compatibility issues with Pydantic 2.10
• Update documentation to better reflect supported HF models

v1.2.4

• Pin the Pydantic dependency to pydantic<2.10.0 as this version breaks JSON schema generation and MessagePart serialization

v1.2.3

• Fixes Anthropic tool calling being broken with anthropic-sdk>0.26.0
• Fixes an issue where Anthropic prompts were over-eagerly trimming prompts that did not start with a user message
• Added support for tool calling while streaming with Anthropic models

v1.2.2

• fix(mistral): ensure prompt and completion tokens are passed through in the MistralFunctionCallingAdapter when streaming
• fix(streaming): don't emit text in DummyStream if it is None
• feat: add standalone width formatters
• docs: gpt-3.5-turbo -> gpt-4o-mini defaults
• fix(streaming): potential line len miscount in format_stream

v1.2.1

• Fixes various issues in the MistralFunctionCallingAdapter wrapper engine for Mistral-Large and Mistral-Small function calling models.
• Fixes an issue in PromptPipeline.explain() where manual examples would not be explained.
• Fixes an issue in PromptPipeline.ensure_bound_function_calls() where passing an ID translator would mutate the ID of the underlying messages

v1.2.0

New Features

• Hugging Face: Models loaded through the HuggingEngine now use chat templates for conversational prompting and tool usage if available by default. This should make it much easier to get started with a Hugging Face model in Kani.
• Added the ability to supply a custom tokenizer to the OpenAIEngine (e.g., for using OpenAI-compatible APIs)\

Fixes/Improvements

• Fixed a missing dependency in the llama extra
• The HuggingEngine will now automatically set device_map="auto" if the accelerate library is installed

v1.1.1

• Fixes an issue where PromptPipeline.ensure_bound_function_calls() could still let unbound function calls through in cases of particularly long prompts with prefixing system prompts

v1.1.0

• Added max_function_rounds to Kani.full_round, Kani.full_round_str, and Kani.full_round_stream:

  The maximum number of function calling rounds to perform in this round. If this number is reached, the model is allowed to generate a final response without any functions defined.
  Default unlimited (continues until model's response does not contain a function call).

• Added __repr__ to engines
• Fixed an issue where Kani could underestimate the token usage for certain OpenAI models using parallel function calling

v1.0.2

• Add Kani.add_completion_to_history (useful for token counting, see #29)
• Add an example of an AIFunction definition to PromptPipeline.explain() when a function-related step is included
• Add id_translator arg to PromptPipeline.ensure_bound_function_calls()
• Ensure that OpenAIEngine and HuggingEngine streams yield a completion including prompt and completion token usage
• Various Mistral-7B Instruct v0.3 prompt fixes

v1.0.1

• OpenAI: Added support for GPT-4o