Skip to content

直接运行代码进行测试无法得到较好的效果 #43

Open

Description

@hukexin326

作者您好!我自己运行得到的效果很差,比如说我选择了inception model,得到的效果如下所示。我想请问是我进行测试的图片数量太少导致这样的结果还是一些其他原因。
*Transfer to mixer-b
Accuracy on benign examples: 70.0%
Accuracy on adversarial examples: 30.0%

*Transfer to mixer-l
Accuracy on benign examples: 70.0%
Accuracy on adversarial examples: 10.0%

*Transfer to tf2torch_adv_inception_v3
Accuracy on benign examples: 70.0%
Accuracy on adversarial examples: 10.0%

*Transfer to tf2torch_ens3_adv_inc_v3
Accuracy on benign examples: 90.0%
Accuracy on adversarial examples: 0.0%

*Transfer to tf2torch_ens4_adv_inc_v3
Accuracy on benign examples: 70.0%
Accuracy on adversarial examples: 10.0%

*Transfer to tf2torch_ens_adv_inc_res_v2
Accuracy on benign examples: 100.0%
Accuracy on adversarial examples: 20.0%
clean_accuracy: 100.0 90.0 80.0 100.0 100.0 90.0 100.0 90.0 100.0 70.0 70.0 70.0 90.0 70.0 100.0
adv_accuracy: 30.0 30.0 30.0 10.0 50.0 30.0 40.0 40.0 30.0 30.0 10.0 10.0 0.0 10.0 20.0

*fid: 287.84357605437066

期待您的答复,祝您生活愉快!

Activity

  1. WindVChen commented on Apr 3, 2025

    @WindVChen
    Owner

    哈喽 @hukexin326 ,

    “效果很差”具体是指哪方面呢?

    因为对比我之前在inception model上的测试结果 #8 (comment) , 你上面所附的攻击指标看起来更好些(accuracy on adv samples越低越好)

    测试的数据集是自己的数据集嘛?

  2. hukexin326 commented on Apr 3, 2025

    @hukexin326
    Author

    感谢回复!
    效果比较差具体是指我的fid比较高,在多种模型上测试结果都在200-300左右,肉眼观察对抗样本也能发现与原图之间有比较明显的不同,而原文中的fid都在60左右,对抗样本也会更不可察觉。

  3. WindVChen commented on Apr 3, 2025

    @WindVChen
    Owner

    可以尝试调整参数来平衡攻击性能和不可查觉性哈~

    比如增大--self_attn_loss_weight,--start_step,或减少--iterations,来缩小对抗样本与干净样本的区别(不过也同时会降低攻击性能)

  4. hukexin326 commented on Apr 5, 2025

    @hukexin326
    Author

    感谢回复!
    我发现fid高是因为fid这个指标在比较多的样本上会比较准,我在100张图片上跑出来的fid在200左右,在1000张图片上跑出来的fid和论文中的就一样了。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions