Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Github Action workflows Hang #3535

Open
chenele opened this issue Oct 30, 2024 · 41 comments
Open

Github Action workflows Hang #3535

chenele opened this issue Oct 30, 2024 · 41 comments
Labels
bug Something isn't working

Comments

@chenele
Copy link

chenele commented Oct 30, 2024

Seems like there is an issue with Github Action Runners.
all our different actions looks 'wierd'.
The job output seems collapsed into a single Text output and seems ilke the job are never ends.

just created a simple dummy action to test it and it got reproduce.

name: Tester

on:
  push:

  
jobs:
  test:    
    runs-on: ubuntu-latest   
    
    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0

      - name: echo
        run: |
          echo Bla  

Output is endless while at the end seems like job cannot complete the cleanup process.

Post job cleanup.
/usr/bin/git version
git version 2.47.0
Temporarily overriding HOME='/home/runner/work/_temp/ab3e00c9-67e9-4e9a-83c1-740a432d20c6' before making global git config changes
Adding repository directory to the temporary git global config as a safe directory
/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
http.https://github.com/.extraheader
/usr/bin/git config --local --unset-all http.https://github.com/.extraheader
/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
Cleaning up orphan processes

Anyone else encountering such issues?
havent seen anything yet on Github-Status indicating the issue

@chenele chenele added the bug Something isn't working label Oct 30, 2024
@iairu
Copy link

iairu commented Oct 30, 2024

same issue here, tried renaming the workflow, canceling the workflow doesn't work either, seems github never recognizes the output properly and considers the job "queued" forever even though the text in logs shows it as complete and successful

@junha6316
Copy link

same issue here

@melon-ruet
Copy link

Facing the same issue with ubuntu-24.04 as runner os

@Mohammed-Keafeel
Copy link

Facing same issue even though the runners are hosted private. Either it is queues to pick runner or getting stuck at Cleaning up orphan processes.

@Kavindu-Jayasinghe
Copy link

Kavindu-Jayasinghe commented Oct 30, 2024

same issue here, also showing last successful build is still running.
Screenshot 2024-10-30 125232

@iairu
Copy link

iairu commented Oct 30, 2024

same issue here, tried renaming the workflow, canceling the workflow doesn't work either, seems github never recognizes the output properly and considers the job "queued" forever even though the text in logs shows it as complete and successful

renaming the job doesn't work either, left image is how it looks, right image is how it should look:

image
image

here is my setup, still broken, haven't tried different OS from ubuntu-latest:

on:
  push:
    branches:
      - "*"

jobs:
  deploy_job:
    runs-on: ubuntu-latest
    name: build-deploy
    steps:
    ...

@newdeal123
Copy link

same issue here :(

@chenele
Copy link
Author

chenele commented Oct 30, 2024

Tested it both from Organization account and private account, same result.
also, seems like both jobs that worked yesterday and the ones that are failing today are using the SAME Ubuntu image version and OS.

Image: ubuntu-22.04
Version: 20241015.1.0
Image Provisioner 2.0.384.1

so it doesnt seems to be related to any update on the runner version
Jobs are able to execute the flow, its just that they are failing to cleanup the process.

also, cancle does not work.

@SoftradixAD
Copy link

My Deploymemt Is working fine and its done i can verify it on my server but still action showing queued.

@tracle-tw-joe
Copy link

same issue here :(

@jocelyn-sm
Copy link

Same

1 similar comment
@zzunny97
Copy link

Same

@neilscallywag
Copy link

neilscallywag commented Oct 30, 2024

Facing the same issue. Getting status 500'd on force cancellation
image

@niranx
Copy link

niranx commented Oct 30, 2024

same here, stuck on Cleaning up orphan processes - even for self-hosted runners.

@ramadhansyah
Copy link

same

@dongthg7
Copy link

Same

1 similar comment
@beuself
Copy link

beuself commented Oct 30, 2024

Same

@wouter-leistra
Copy link

+1

@jeypc
Copy link

jeypc commented Oct 30, 2024

Same

@KevinArnaudLille
Copy link

Also here

@iairu
Copy link

iairu commented Oct 30, 2024

tried different runs-on:

  • ubuntu-latest stuck
  • ubuntu-22.04 stuck
  • ubuntu-20.04 stuck

i'm out of options, seems the issue is platform independent, workflow independent, job name independent, branch and repository independent

@yeongsheng-tan
Copy link

same.

@procinger
Copy link

Same issue. Self Hosted K8S Runner.

I see that the pods start and work, but the feedback in the actions (web ui) is missing.
It looks like all runners are stuck

@iairu
Copy link

iairu commented Oct 30, 2024

canceling the workflow actually does cancel it in logs if its still in progress however it doesn't affect the UI still showing as "queued"; canceling after Post job cleanup. has no effect on logs.

@young-min-choo
Copy link

Getting the same issue as well
image
image

@Klassikcat
Copy link

getting same isssue here. stuck after cleaning up orphan processes

@yuseok-kim-edushare
Copy link

@iairu
Copy link

iairu commented Oct 30, 2024

for anyone waiting:

  • it seems to be a UI issue, not a problem of actually getting the workflow to finish internally, simply ignore the UI "queued" state for now and read the logs
  • if you know how yours logs should end and your logs end without errors you can consider the run a success and proceed with your day

@ayussh-verma
Copy link

An incident has been filed: https://www.githubstatus.com/incidents/9yk1fbk0qjjc

@FonsBiemans
Copy link

Same issue, stuck after cleaning up orphan processes

@bh-niladri-mondal
Copy link

same issue

@RubenFr
Copy link

RubenFr commented Oct 30, 2024

Same issue!

@ezmartech
Copy link

Same issue

image

@jusvit
Copy link

jusvit commented Oct 30, 2024

Same issue

@ezmartech
Copy link

Started working but taking too much time

@RubenFr
Copy link

RubenFr commented Oct 30, 2024

Started working but taking too much time

It took it 24 minutes to understand that the job was cancelled and stop it

@shelwinjue
Copy link

same issue

{
"message": "Failed to cancel workflow run",
"documentation_url": "https://docs.github.com/rest/actions/workflow-runs#force-cancel-a-workflow-run",
"status": "500"
}

@snoa93
Copy link

snoa93 commented Oct 30, 2024

We are noticing the same issue at my company (using Github Enterprise).

It also takes a while for runs to stop after you press the cancel button

@ItsManikantaGopi
Copy link

Same issue.

@j4tmr
Copy link

j4tmr commented Oct 30, 2024

seems ok now

@chenele
Copy link
Author

chenele commented Oct 30, 2024

Yep, back to normal from our end

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests