Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

STFPM异常检测模型,检测结果不佳且与评估指标不一致 #3303

Open
bittergourd1224 opened this issue Feb 12, 2025 · 2 comments
Assignees

Comments

@bittergourd1224
Copy link

教程 https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/cv_modules/anomaly_detection.html 提供的模型以及训练集,都属于MVTec_AD中的grid这类数据,这个测试下来没有问题

又下载了MVTec_AD中的metal_nut这类数据,重新训练后,评估的指标很高:

[2025/02/12 14:06:43] INFO: Start evaluating (total_samples: 93, total_iters: 93)...
93/93 [==============================] - 1s 16ms/step - batch_cost: 0.0155 - reader cost: 4.9394e-04
[2025/02/12 14:06:44] INFO: [EVAL] #Images: 93 mIoU: 0.9999 Acc: 1.0000 Kappa: 0.0000 Dice: 0.9999
[2025/02/12 14:06:44] INFO: [EVAL] Class IoU: 
[0.9999]
[2025/02/12 14:06:44] INFO: [EVAL] Class Precision: 
[1.]
[2025/02/12 14:06:44] INFO: [EVAL] Class Recall: 
[0.9999]

但是在验证集上推理,很多都没有识别出异常。下面分别是验证集的原图,GroundTruth和推理结果:

Image

Image

Image

请求帮忙复现一下看看目前STFPM模型在metal_nut数据上的训练和推理效果,是不是与我的结果一致

@Sunting78
Copy link
Collaborator

您好,可以根据这个教程看一下 https://github.com/PaddlePaddle/PaddleX/blob/release/3.0-beta2/docs/practical_tutorials/anomaly_detection_tutorial.md 比如训练轮次的这些参数可能需要调整,另外由于无监督训练过程中只有正常图像,如果指标达到很高,目前也只能说明在正常图像上表现效果比较好。参考上面的教程调整下训练,如果依然有问题,请您发下具体的执行命令,以及整理的metal_nut训练数据集给我。谢谢

@Sunting78 Sunting78 assigned Sunting78 and unassigned Bobholamovic Feb 13, 2025
@bittergourd1224
Copy link
Author

@Sunting78
修改迭代次数从10000到5000,学习率从0.4到0.1,重新训练以后,效果确实好了很多

Image

和GroundTruth比较接近了,谢谢解答

但是在评估指标上我还是有疑问。
首先指标显示是在93张验证集上计算的,这些验证集都是异常图片并且有GroundTruth,所以指标越高应该意味着越符合GroundTruth。
但这次训练下来的指标如下,还不如之前训的效果不佳的模型

[2025/02/13 11:56:05] INFO: [EVAL] #Images: 93 mIoU: 0.9883 Acc: 1.0000 Kappa: 0.0000 Dice: 0.9941
[2025/02/13 11:56:05] INFO: [EVAL] Class IoU:
[0.9883]
[2025/02/13 11:56:05] INFO: [EVAL] Class Precision:
[1.]
[2025/02/13 11:56:05] INFO: [EVAL] Class Recall:
[0.9883]

怀疑是不是代码里的指标计算有问题?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants