Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

训练几次后报错RuntimeError:CUDA error #86

Open
LQ1024 opened this issue Nov 4, 2024 · 0 comments
Open

训练几次后报错RuntimeError:CUDA error #86

LQ1024 opened this issue Nov 4, 2024 · 0 comments

Comments

@LQ1024
Copy link

LQ1024 commented Nov 4, 2024

报错信息:
RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling cublasLtMatmul with transpose_mat1 1 transpose_mat2
现象:训练过程中,当模型包含2层DCNv4时,训练5次后报错;当模型只包含1层DCNv4时,训练10次后报错。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant