Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compatibility Issues with DeepSeek-R1 Model Response Format #200

Open
darielnoel opened this issue Jan 23, 2025 · 0 comments
Open

Compatibility Issues with DeepSeek-R1 Model Response Format #200

darielnoel opened this issue Jan 23, 2025 · 0 comments
Labels
core Fundamental operations and essential features of AgenticJS integrations Adding support for third-party tools, agents, or services library Issues and enhancements directly related to the codebase of the library component of the project. llms

Comments

@darielnoel
Copy link
Contributor

Describe the Bug

The DeepSeek-R1 model outputs a response format incompatible with KaibanJS's default prompt engineering setup, which expects JSON-formatted responses. This mismatch causes workflows using the DeepSeek-R1 model to fail during execution. The reasoning-based response structure from the model needs specific adjustments to the parsing logic or prompt tuning in KaibanJS.


Steps to Reproduce

  1. Configure an agent in KaibanJS to use the DeepSeek-R1 model:

    const reasoningAgent = new Agent({
      name: 'Reasoner',
      role: 'Analyze complex data patterns',
      llmConfig: {
        provider: 'deepseek',
        model: 'deepseek-r1',
      },
    });
  2. Execute a workflow with the agent.

Image
  1. Observe that the workflow fails due to the model's reasoning-based response format.

Current vs. Expected Behavior

  • Current Behavior: The agent fails to process the output from DeepSeek-R1 due to its non-JSON response format, leading to workflow interruptions.
  • Expected Behavior: The agent should successfully process the model's responses and complete the workflow without issues.

Additional Notes

  • DeepSeek-R1 provides a reasoning-based response format, as detailed in their API documentation: DeepSeek Reasoning Model Guide.
  • This format requires adjustments to KaibanJS's response handling or prompt engineering to accommodate the reasoning model's output style.
  • Reference issue in LangChain regarding similar compatibility concerns: LangChain Issue #29317.
  • Potential solutions include extending the prompt engineering to support reasoning-style models or adapting KaibanJS's JSON parsing logic.

@darielnoel darielnoel added bug Something isn't working core Fundamental operations and essential features of AgenticJS integrations Adding support for third-party tools, agents, or services library Issues and enhancements directly related to the codebase of the library component of the project. labels Jan 23, 2025
@darielnoel darielnoel changed the title [BUG] - Compatibility Issues with DeepSeek-R1 Model Response Format Compatibility Issues with DeepSeek-R1 Model Response Format Jan 23, 2025
@darielnoel darielnoel added llms and removed bug Something isn't working labels Jan 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Fundamental operations and essential features of AgenticJS integrations Adding support for third-party tools, agents, or services library Issues and enhancements directly related to the codebase of the library component of the project. llms
Projects
None yet
Development

No branches or pull requests

1 participant