Batch Fuzzing for DNNs Hits 40x Throughput with Adaptive Perturbation Scaling

40x higher throughput and 4x more violations under the same time budget: that's what a new tensor-based batch fuzzing framework delivers over sequential DNN fuzzing baselines.

Why One-Input-at-a-Time Fuzzing Falls Short

Coverage-guided sequential fuzzers for deep neural networks inherited the one-input-per-iteration design from traditional software fuzzing. They apply a uniform perturbation budget (a fixed epsilon) across every input dimension, ignoring the fact that real-world input spaces have heterogeneous feature scales. That uniform budget either under-explores sensitive dimensions or over-perturbs insensitive ones, limiting both precision and throughput.

The authors wrap input constraints and output property checks directly into the network as non-trainable layers. The result is a single network that processes B specification instances in one batched forward pass, instead of B sequential passes. That tensor-level batching is what drives the throughput multiplier.

Perturbation Scaling That Respects the Input Space

Instead of a fixed global epsilon, the framework derives mutation step sizes from specification-defined feasible ranges (the gap between lower and upper bounds) using a shared scale factor. Two variants exist: isotropic (global scalar) and anisotropic (per-dimension step sizes). Both keep perturbations consistent with the underlying constraint structure, so the fuzzer explores input spaces with wildly different feature scales more effectively.

This matters most for safety-critical applications like autonomous driving and medical diagnosis, where inputs can span orders of magnitude in scale and different dimensions encode fundamentally different physical quantities.

Benchmarks: 4X More Violations, 40X Faster

Evaluation covered three benchmarks: TrafficSigns, Cifar100, and TinyImageNet, across six networks and over 400 specifications. The tensor-based batch fuzzer achieved up to 40x higher input throughput and discovered 4x more violations (counterexamples to specification properties) than the sequential baseline under identical time budgets.

Those numbers come from real specification-guided fuzzing, not synthetic toy problems. The framework generalizes across datasets and network architectures, suggesting the bottleneck in DNN reliability testing has shifted from algorithmic coverage to raw input generation speed.

The next step is integrating this batch approach with more sophisticated coverage metrics and adaptive specification synthesis, moving from fuzzing predefined properties to automatically discovering meaningful input constraints on the fly.

Source: Tensor-Based Batch Fuzzing with Adaptive Perturbation Scaling for Deep Neural Networks
Domain: arxiv.org