Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prompting changes to better support smaller models. #5386

Merged
merged 15 commits into from
Feb 7, 2025
Merged

Conversation

afourney
Copy link
Member

@afourney afourney commented Feb 5, 2025

A series of changes to the python/packages/autogen-ext/src/autogen_ext/agents/web_surfer/_multimodal_web_surfer.py file have been made to better support smaller models.

This includes changes to the prompts, state descriptions, and ordering of messages.

Regression tasks with OpenAI models shows no change in GAIA scores, while scores for Llama are significantly improved.

Copy link

codecov bot commented Feb 6, 2025

Codecov Report

Attention: Patch coverage is 75.75758% with 8 lines in your changes missing coverage. Please review.

Project coverage is 78.08%. Comparing base (3b2bf82) to head (6dd3e7b).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
...en_ext/agents/web_surfer/_multimodal_web_surfer.py 75.75% 8 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5386      +/-   ##
==========================================
+ Coverage   77.93%   78.08%   +0.14%     
==========================================
  Files         158      158              
  Lines        9578     9576       -2     
==========================================
+ Hits         7465     7477      +12     
+ Misses       2113     2099      -14     
Flag Coverage Δ
unittests 78.08% <75.75%> (+0.14%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@afourney afourney marked this pull request as ready for review February 6, 2025 18:18
@afourney afourney enabled auto-merge (squash) February 7, 2025 00:44
@afourney afourney disabled auto-merge February 7, 2025 01:36
@afourney afourney merged commit 3c30d89 into main Feb 7, 2025
63 of 64 checks passed
@afourney afourney deleted the llama_surfer branch February 7, 2025 01:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants