Skip to content
@prometheus-eval

prometheus-eval

Codebase to inference and train foundation models specialized on evaluating other foundation models
README.md

We train language models specialized in evaluating other language models and optimize evaluation pipelines!

Repositories

Below are our key projects, with links to their repositories and related publications:

Repository Description Paper
prometheus-eval A repository for evaluating LLMs in generation tasks. Supports Prometheus 2, GPT-4, and others. Link
prometheus An Evaluator LM that is open-source, offers reproducible evaluation, and inexpensive to use. Link
prometheus-vision An Evaluator VLM that is open-source, offers reproducible evaluation, and inexpensive to use. Link

Popular repositories Loading

  1. prometheus-eval prometheus-eval Public

    Evaluate your LLM's response with Prometheus and GPT4 💯

    Python 795 49

  2. prometheus prometheus Public

    [ICLR 2024 & NeurIPS 2023 WS] An Evaluator LM that is open-source, offers reproducible evaluation, and inexpensive to use. Specifically designed for fine-grained evaluation on a customized score ru…

    Python 287 17

  3. prometheus-vision prometheus-vision Public

    [ACL 2024 Findings & ICLR 2024 WS] An Evaluator VLM that is open-source, offers reproducible evaluation, and inexpensive to use. Specifically designed for fine-grained evaluation on customized scor…

    Python 57 6

  4. .github .github Public

    Organization README for prometheus-eval

  5. prometheus-eval.github.io prometheus-eval.github.io Public

    Documentation and blogposts for Prometheus

    1

  6. leaderboard leaderboard Public

    BiGGen-Bench Leaderboard

    Python

Repositories

Showing 6 of 6 repositories
  • prometheus-vision Public

    [ACL 2024 Findings & ICLR 2024 WS] An Evaluator VLM that is open-source, offers reproducible evaluation, and inexpensive to use. Specifically designed for fine-grained evaluation on customized score rubric, Prometheus-Vision is a good alternative for human evaluation and GPT-4V evaluation.

    prometheus-eval/prometheus-vision’s past year of commit activity
    Python 57 Apache-2.0 6 2 0 Updated Sep 13, 2024
  • prometheus-eval Public

    Evaluate your LLM's response with Prometheus and GPT4 💯

    prometheus-eval/prometheus-eval’s past year of commit activity
    Python 795 Apache-2.0 49 4 0 Updated Sep 9, 2024
  • .github Public

    Organization README for prometheus-eval

    prometheus-eval/.github’s past year of commit activity
    0 0 0 0 Updated Jun 11, 2024
  • leaderboard Public

    BiGGen-Bench Leaderboard

    prometheus-eval/leaderboard’s past year of commit activity
    Python 0 0 0 0 Updated Jun 4, 2024
  • prometheus-eval.github.io Public

    Documentation and blogposts for Prometheus

    prometheus-eval/prometheus-eval.github.io’s past year of commit activity
    0 1 0 0 Updated May 1, 2024
  • prometheus Public

    [ICLR 2024 & NeurIPS 2023 WS] An Evaluator LM that is open-source, offers reproducible evaluation, and inexpensive to use. Specifically designed for fine-grained evaluation on a customized score rubric, Prometheus is a good alternative for human evaluation and GPT-4 evaluation.

    prometheus-eval/prometheus’s past year of commit activity
    Python 287 MIT 17 4 0 Updated Nov 11, 2023

Top languages

Python

Most used topics

Loading…