Trustworthy Autonomy in Cyber-Physical systems

Machine learning models, mainly those utilized in computer vision using convolutional neural networks (CNNs), can currently train on extensive datasets and images. Despite their capabilities, they are still vulnerable to special images called adversarial patches that can manipulate their output. Adversarial patches can penetrate various defense models, such as NutNet and NAPGuard. This thesis aims to attack a representative machine learning vision model, YOLOv2. Such adversarial patch attacks could target YOLOv2, and a defense model, NutNet, that is designed to protect victim models against such attacks. Our study was conducted by creating adversarial patches and exploring their impact based on where the patch was located on the image, the number of patches, real-world execution scenarios, and the size of training dataset, which consists of pictures of people in various environments. Patch creation involved training on different-sized dataset for different numbers of epochs. We also used a patch from other work to launch attacks. The attack methods were also varied, for digital attacks, patches were placed directly on the images. For real-world attacks, the adversarial patches were printed and physically applied on a person. We assumed two situations throughout the evaluations, with and without adversarial patch defenses enabled. This approach led to the effectiveness of adversarial patches being evaluated under various conditions. The result showed that patches could attack the object detector with high effectiveness, reaching 75% when NutNet was deactivated. Without the defense model activated the attack success rate reached up to 49.6%.