Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled. #2581

Open
Kshitij-Katiyar opened this issue Nov 6, 2024 · 8 comments

Comments

@Kshitij-Katiyar
Copy link

My repo has codeQL go analysis enabled and was working fine until a few weeks ago.
no it fails with the following error
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.

Logs show no error everything seems to work fine except the below line, which I think is unrelated

2024-11-05T12:25:48.3087301Z 2024/11/05 12:25:48 Makefile found. 2024-11-05T12:25:48.3088372Z 2024/11/05 12:25:48 Trying build command make [] 2024-11-05T12:26:09.5990096Z : No names found, cannot describe anything.

@jketema
Copy link

jketema commented Nov 6, 2024

Hi @Kshitij-Katiyar,

It's very difficult to tell what is going on from the information you provided. Could you share a link to the repository on which this is happening?

@Kshitij-Katiyar
Copy link
Author

Kshitij-Katiyar commented Nov 6, 2024

@jketema Sure, https://github.com/mattermost/mattermost-plugin-jira
Let me know if you are able to look into the action logs

@jketema
Copy link

jketema commented Nov 6, 2024

Thanks. I have access to the logs. It looks like the Go analysis started failing when we rolled out CodeQL 2.19.2. I'll loop in the responsible engineering team.

@Kshitij-Katiyar
Copy link
Author

Thank you so much, @jketema. Let me know if you need any more help from us.
I'm hoping for a quick fix on this one.

@Kshitij-Katiyar
Copy link
Author

Hey @jketema, have you informed the engineering team responsible for this issue?

@jketema
Copy link

jketema commented Nov 7, 2024

Yes, they're looking into this.

@Kshitij-Katiyar
Copy link
Author

Yes, they're looking into this.

@jketema Thanks for the update

@mbg
Copy link
Member

mbg commented Nov 7, 2024

Hi @Kshitij-Katiyar 👋🏻

I am on the engineering team that supports Go for CodeQL. I have looked at your repo and what's going on in your CodeQL runs.

Fundamentally, there's no difference in what is happening in the successful runs and the ones that are now failing consistently, except that the latter crash when running /home/runner/work/mattermost-plugin-jira/mattermost-plugin-jira/bin/golangci-lint run ./.... That's part of your Makefile and our Go autobuilder invokes that (this hasn't changed).

What I can observe, however, is that bin/golangci-lint now appears to consume excessive amounts of memory (160GB+ during my test). The Actions runner doesn't have nearly that much memory, so it runs out of memory, crashes, and the workflow fails.

If I run a system-wide version of golangci-lint on your codebase (v1.55.0), it works fine. However, if I run the version of golangci-lint that your Makefile installs (v1.51.1) and then run that on your codebase, it exhibits the excessive memory usage. In your main CI, it seems like you may also be installing v1.55.2, which also works fine.

In any case, this suggests that there is an issue with v1.51.1 of golangci-lint when used on your codebase. I would suggest updating your Makefile to pin a newer version. That should resolve the issue with your CodeQL workflow.

Alternatively, if you don't want to pin a newer version of golangci-lint there or pinning the newer version doesn't help, you could also:

  1. Modify your CodeQL workflow to use a manual build command, instead of relying on the autobuilder, and then run a more specific build command that avoids running make check-style and golangci-lint.
  2. Change your Makefile so that check-style is not run by default.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants