Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature request] Add ORPO finetuning #1

Open
s-kostyaev opened this issue Apr 15, 2024 · 2 comments
Open

[Feature request] Add ORPO finetuning #1

s-kostyaev opened this issue Apr 15, 2024 · 2 comments
Assignees
@armbues
Labels
enhancement New feature or request

Comments

@s-kostyaev
Copy link

Hi @armbues, thank you for this great project.

Please add ORPO finetuning to do SFT and DPO in one step https://arxiv.org/abs/2403.07691
@armbues armbues added the enhancement New feature or request label Apr 15, 2024
@armbues armbues self-assigned this Apr 15, 2024
@armbues
Copy link
Owner

armbues commented Apr 15, 2024

Great idea! I will add this to the roadmap.

@armbues
Copy link
Owner

armbues commented Apr 25, 2024

The official implementation of the "ORPO Trainer" can be found here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@armbues armbues

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@armbues @s-kostyaev