Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mlflow docs #1338

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Mlflow docs #1338

wants to merge 3 commits into from

Conversation

djliden
Copy link

@djliden djliden commented Feb 6, 2025

This PR adds a quick guide in the integrations section of the docs on how to use MLflow tracing with Instructor via the client libraries.

Open to suggestions on whether this makes sense elsewhere!


Important

Adds documentation for MLflow tracing and model logging with Instructor, including examples for OpenAI and Anthropic models, and updates navigation.

  • Documentation:
    • Adds mlflow.md to docs/integrations/ for MLflow tracing and model logging with Instructor.
    • Includes examples for OpenAI and Anthropic models using mlflow.<provider>.autolog().
  • Navigation:
    • Updates mkdocs.yml to include MLflow in the integrations section.
    • Adds MLflow to docs/integrations/index.md under Observability.

This description was created by Ellipsis for e5825c2. It will automatically update as commits are pushed.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Changes requested. Reviewed everything up to e5825c2 in 52 seconds

More details
  • Looked at 146 lines of code in 3 files
  • Skipped 1 files when reviewing.
  • Skipped posting 5 drafted comments based on config settings.
1. docs/integrations/index.md:38
  • Draft comment:
    Ensure linking is consistent with other integrations. The MLflow link under Observability is clear!
  • Reason this comment was not posted:
    Confidence changes required: 0% <= threshold 85%
    None
2. docs/integrations/mlflow.md:34
  • Draft comment:
    Clarify reuse of the EmailSubject model in the Anthropic example to avoid confusion about its definition.
  • Reason this comment was not posted:
    Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 20% vs. threshold = 85%
    The comment has merit since readers might be confused about where EmailSubject comes from in the Anthropic example. However, the code already has a clear comment "Use the same EmailSubject model" and the examples are presented sequentially in the same document. The connection should be clear to readers following the document from top to bottom.
    The comment might be addressing a real readability concern - some readers might skip directly to the Anthropic example without reading the OpenAI example first.
    The document is structured logically with the OpenAI example introducing core concepts first. The existing comment "Use the same EmailSubject model" is sufficient to point readers back if needed.
    The comment should be deleted as the existing documentation is already clear enough with the inline comment about model reuse.
3. mkdocs.yml:228
  • Draft comment:
    New MLflow link is added to the navigation. Ensure it aligns with the latest documentation structure.
  • Reason this comment was not posted:
    Confidence changes required: 33% <= threshold 85%
    None
4. docs/integrations/index.md:37
  • Draft comment:
    Nice addition of the 'Observability' section. Consider adding a brief description to explain the benefits of MLflow integration.
  • Reason this comment was not posted:
    Confidence changes required: 33% <= threshold 85%
    None
5. mkdocs.yml:229
  • Draft comment:
    MLflow integration has been correctly added to the nav. Ensure future integration updates are consistently reflected here.
  • Reason this comment was not posted:
    Confidence changes required: 33% <= threshold 85%
    None

Workflow ID: wflow_nmQuaooCut4jUC3k


Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

### Anthropic Tracing Example

As noted above, MLflow autologging works with Instructor by tracing the client library calls. Here we'll adapt the above example to use an Anthropic model.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the Anthropic example, include installation instructions for the 'anthropic' package to mirror the OpenAI example.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant