A query-efficient black-box attack that jointly optimizes patch location, texture, and size under strict query budgets now exposes clear trade-offs between suppression success and visual footprint across YOLOv5, Faster R-CNN, and YOLOS.
Three Shortcomings of Prior Work
Existing adversarial patch attacks usually fix either location or size, ignore query budgets, or conflate robustness against random transformations (EOT) with actual plain-view suppression. This work closes all three gaps at once: it treats the patch as a learnable variable that grows only when progress stalls, couples a Contextual Thompson-Sampling placer with NES-style gradient-free pixel updates, and separates EOT auditing from the primary plain-image suppression metric.
What the Attack Actually Does
The method, unnamed in the preprint but clearly described, operates in a strict black-box setting: the attacker only sees class scores, no gradients. A lightweight Thompson-Sampling module proposes where to place the patch; NES pixel updates optimize the texture; and a budget-adaptive growth rule expands the patch area when the query budget allows and suppression plateaus. Optional appearance and printability weights let the attacker dial between suppression strength and how obvious the patch looks.
Results That Matter
On YOLOv5 and Faster R-CNN, the attack achieves strong suppression using compact patches. On the transformer-based YOLOS, suppression is substantial but weaker, consistent with transformers being less vulnerable to local texture perturbations. The authors report clear query-footprint trade-offs: more queries buy smaller patches, and smaller patches produce better suppression per pixel. Fixed-size and heuristic baselines are outperformed at every budget level.
Physical Transfer and the Plain-Image Test
A print-capture pilot shows the patches survive deployment on real objects under unseen viewpoints, confirming the attack transfers from simulation to physical scenes. By reporting a strict plain-image suppression test first and auditing EOT separately, the evaluation avoids conflating robustness tricks with genuine vulnerability. This gives practitioners a reliable baseline for how easily their detector can be nullified.
If your object detector runs in a security or autonomous system, these budget-aware patches are a practical red-teaming tool you can use today to find out exactly how many queries and how visible a patch an attacker actually needs to blind you.
Source: Budget-Aware Adaptive Adversarial Patches for Black-Box Object Detection
Domain: arxiv.org
Comments load interactively on the live page.