You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have searched the PaddleOCR Docs and found no similar bug report.
I have searched the PaddleOCR Issues and found no similar bug report.
I have searched the PaddleOCR Discussions and found no similar bug report.
🐛 Bug (问题描述)
Traceback (most recent call last):
File "/data/PaddleOCR/PaddleOCR-2.8.1/tools/train.py", line 292, in
main(config, device, logger, vdl_writer, seed)
File "/data/PaddleOCR/PaddleOCR-2.8.1/tools/train.py", line 221, in main
lwf_program.train(
File "/data/PaddleOCR/PaddleOCR-2.8.1/tools/LwF_program.py", line 357, in train
loss = loss_class(preds, batch)
File "/home/server/anaconda3/envs/paddle/lib/python3.10/site-packages/paddle/nn/layer/layers.py", line 1429, in call
return self.forward(*inputs, **kwargs)
File "/data/PaddleOCR/PaddleOCR-2.8.1/ppocr/losses/rec_multi_loss.py", line 53, in forward
loss_func(predicts["ctc"], [batch[:2] + batch[3:], predicts["preds_distill"]["ctc"]])["loss"]
File "/home/server/anaconda3/envs/paddle/lib/python3.10/site-packages/paddle/nn/layer/layers.py", line 1429, in call
return self.forward(*inputs, **kwargs)
File "/data/PaddleOCR/PaddleOCR-2.8.1/ppocr/losses/rec_ctc_loss.py", line 63, in forward
ctc_loss = self.loss_func.forward(predicts, batch[0])
File "/data/PaddleOCR/PaddleOCR-2.8.1/ppocr/losses/rec_ctc_loss.py", line 92, in forward
loss = self.loss_func(predicts, labels, preds_lengths, label_lengths)
File "/home/server/anaconda3/envs/paddle/lib/python3.10/site-packages/paddle/nn/layer/layers.py", line 1429, in call
return self.forward(*inputs, **kwargs)
File "/home/server/anaconda3/envs/paddle/lib/python3.10/site-packages/paddle/nn/layer/loss.py", line 1261, in forward
return paddle.nn.functional.ctc_loss(
File "/home/server/anaconda3/envs/paddle/lib/python3.10/site-packages/paddle/nn/functional/loss.py", line 1940, in ctc_loss
loss_out = warpctc(
File "/home/server/anaconda3/envs/paddle/lib/python3.10/site-packages/paddle/nn/functional/loss.py", line 1901, in warpctc
loss_out = _C_ops.warpctc(
MemoryError: (ResourceExhausted) Fail to alloc memory of 559140344358520 size, error code is 12.
[Hint: Expected error == 0, but received error:12 != 0:0.] (at ../paddle/fluid/memory/allocation/cpu_allocator.cc:50)
🔎 Search before asking
🐛 Bug (问题描述)
Traceback (most recent call last):
File "/data/PaddleOCR/PaddleOCR-2.8.1/tools/train.py", line 292, in
main(config, device, logger, vdl_writer, seed)
File "/data/PaddleOCR/PaddleOCR-2.8.1/tools/train.py", line 221, in main
lwf_program.train(
File "/data/PaddleOCR/PaddleOCR-2.8.1/tools/LwF_program.py", line 357, in train
loss = loss_class(preds, batch)
File "/home/server/anaconda3/envs/paddle/lib/python3.10/site-packages/paddle/nn/layer/layers.py", line 1429, in call
return self.forward(*inputs, **kwargs)
File "/data/PaddleOCR/PaddleOCR-2.8.1/ppocr/losses/rec_multi_loss.py", line 53, in forward
loss_func(predicts["ctc"], [batch[:2] + batch[3:], predicts["preds_distill"]["ctc"]])["loss"]
File "/home/server/anaconda3/envs/paddle/lib/python3.10/site-packages/paddle/nn/layer/layers.py", line 1429, in call
return self.forward(*inputs, **kwargs)
File "/data/PaddleOCR/PaddleOCR-2.8.1/ppocr/losses/rec_ctc_loss.py", line 63, in forward
ctc_loss = self.loss_func.forward(predicts, batch[0])
File "/data/PaddleOCR/PaddleOCR-2.8.1/ppocr/losses/rec_ctc_loss.py", line 92, in forward
loss = self.loss_func(predicts, labels, preds_lengths, label_lengths)
File "/home/server/anaconda3/envs/paddle/lib/python3.10/site-packages/paddle/nn/layer/layers.py", line 1429, in call
return self.forward(*inputs, **kwargs)
File "/home/server/anaconda3/envs/paddle/lib/python3.10/site-packages/paddle/nn/layer/loss.py", line 1261, in forward
return paddle.nn.functional.ctc_loss(
File "/home/server/anaconda3/envs/paddle/lib/python3.10/site-packages/paddle/nn/functional/loss.py", line 1940, in ctc_loss
loss_out = warpctc(
File "/home/server/anaconda3/envs/paddle/lib/python3.10/site-packages/paddle/nn/functional/loss.py", line 1901, in warpctc
loss_out = _C_ops.warpctc(
MemoryError: (ResourceExhausted) Fail to alloc memory of 559140344358520 size, error code is 12.
[Hint: Expected error == 0, but received error:12 != 0:0.] (at ../paddle/fluid/memory/allocation/cpu_allocator.cc:50)
🏃♂️ Environment (运行环境)
Linux CentOS 3090 x4 paddleocr 2.8.1
🌰 Minimal Reproducible Example (最小可复现问题的Demo)
The text was updated successfully, but these errors were encountered: