You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
After running the bench bot, we execute subweight to compare the fresh weight files with the base branch. Currently, this base branch is hardcoded as refs/remotes/origin/master for polkadot-sdk and refs/remotes/origin/main for polkadot-fellows.
This setup works fine when the bench bot is used on a pull request targeting master or main within the same repository. However, it has several downsides, as it can produce misleading results:
If the PR originates from a fork, the comparison is made against the fork’s master or main, which is incorrect.
If the PR’s target/base branch is different from master or main, issues arise. For example, if I have one PR (origin/pr1 → origin/master) and then another PR (origin/pr2 → origin/pr1), running the bench bot in the second PR should compare origin/pr2 with origin/pr1.
Problem
After running the bench bot, we execute subweight to compare the fresh weight files with the base branch. Currently, this base branch is hardcoded as
refs/remotes/origin/master
for polkadot-sdk andrefs/remotes/origin/main
for polkadot-fellows.This setup works fine when the bench bot is used on a pull request targeting
master
ormain
within the same repository. However, it has several downsides, as it can produce misleading results:master
ormain
, which is incorrect.master
ormain
, issues arise. For example, if I have one PR (origin/pr1
→origin/master
) and then another PR (origin/pr2
→origin/pr1
), running the bench bot in the second PR should compareorigin/pr2
withorigin/pr1
.Potential Solution
Instead of using hardcoded references to
refs/remotes/origin/master
(polkadot-sdk) andrefs/remotes/origin/main
(polkadot-fellows), we should dynamically determine the PR’s actual target/base branch.This can potentially be achieved using some
git remote add
scripting along with fetching PR data via the GitHub REST API, as referenced here.The text was updated successfully, but these errors were encountered: