论文标题
通过不可察觉的后门触发，在持续学习者中形成错误的记忆
False Memory Formation in Continual Learners Through Imperceptible Backdoor Trigger
论文作者
论文摘要
在此简介中，我们表明，依次学习给持续的（增量）学习模型的新信息引入了新的安全风险：智能对手可以在培训期间向模型引入少量错误信息，以导致在测试时间内故意忘记特定的任务或类别，从而在该任务中创建“错误的内存”。我们通过使用MNIST的持续学习基准变体以及更具挑战性的SVHN和CIFAR 10数据集来证明这种对手可以通过将“后门”攻击样本注入常用的生成重播和基于正规化的持续学习方法来控制模型的能力。也许最具破坏性的是，我们表明这种脆弱性非常敏捷且异常有效：我们的攻击模型中的后门模式可以被人眼无法察觉，可以在任何时间点提供，甚至可以将单个可能无关的任务的培训数据添加到训练数据中，并且只需在单一任务的总培训数据集中就可以达到1 \％。
In this brief, we show that sequentially learning new information presented to a continual (incremental) learning model introduces new security risks: an intelligent adversary can introduce small amount of misinformation to the model during training to cause deliberate forgetting of a specific task or class at test time, thus creating "false memory" about that task. We demonstrate such an adversary's ability to assume control of the model by injecting "backdoor" attack samples to commonly used generative replay and regularization based continual learning approaches using continual learning benchmark variants of MNIST, as well as the more challenging SVHN and CIFAR 10 datasets. Perhaps most damaging, we show this vulnerability to be very acute and exceptionally effective: the backdoor pattern in our attack model can be imperceptible to human eye, can be provided at any point in time, can be added into the training data of even a single possibly unrelated task and can be achieved with as few as just 1\% of total training dataset of a single task.
