χsao GPU Optimizer Finds All Modes at 34x-39x Speedup

100% mode recovery on all 42 functions of the Simon Fraser University optimization benchmark suite across dimensions 2, 4, 8, 16, 32, and 64, where every CPU-based baseline collapses at d ≥ 8 on the hardest multimodal functions. Plus up to 34x speedup over basin-hopping on functions where all methods succeed (Michalewicz d=64) and up to 39x on unimodal functions (Rotated Hyper-Ellipsoid d=64). All of it derivative-free. That's the claim from the χsao paper.

Why Existing Optimizers Hit a Wall

Multimodal black-box functions are everywhere: hyperparameter tuning, molecular docking, sensor placement, Bayesian inverse problems. Classical tools like basin-hopping, CMA-ES, and multistart gradient descent navigate from peak to peak, but they're fundamentally sequential. GPUs idle while the CPU churns through one candidate at a time. At dimension 8 or higher, those methods miss modes entirely. The authors of χsao identified the bottleneck: serial population generation and no mechanism to preserve discovered modes without halting exploration.

The Core Idea: Freeze Peaks, Keep Searching

χsao (Convergence-Halt-Invert-Stick-And-Oscillate) runs the entire sample batch simultaneously on the GPU. Its structural move is asymmetric: samples that converge to a true peak get frozen ("stuck") and preserved exactly, while the rest keep exploring via momentum-based anti-convergence and stochastically smoothed gradients. This convergence-anticonvergence oscillation cycle deliberately escapes local traps. Two adaptive reseeding strategies, Repulse Monkey and Golden Rooster, maintain population diversity throughout.

Performance Under Pressure

On all 42 functions of the SFU suite, the optimizer hits perfect mode recovery. Because gradients are computed via finite differences - no analytic gradients - the reported speedups represent a derivative-free worst case. Even under substantial objective noise (σ_noise up to 1.0), mode detection stays at 100%. The authors packaged the algorithm as a standalone open-source Python package on PyPI, so you can drop it into any black-box optimization workflow that needs GPU parallelism.

This isn't a tweak to an existing method. It's a new paradigm for how we think about exploiting GPU hardware for optimization: freeze what works, oscillate what doesn't, and let the massive parallel throughput do the rest. Expect to see this pattern applied to other population-based metaheuristics.

Source: \chisao{}: A GPU-Native Parallel Optimizer for Multimodal Black-Box Functions via Convergence-Anticonvergence Oscillation
Domain: arxiv.org