Degeneracy Distillery Identifies Symbolic Transformations, Slashes Simulation Budget 10x

Up to 10x fewer simulations needed for neural posterior estimation at matched validation calibration, thanks to a method that automatically finds symbolic coordinate transformations to remove degeneracies in physical models.

The Problem: Degenerate Parameters Make Life Hard

When two or more parameters produce nearly identical data, you can't tell them apart. Machine learning algorithms and probabilistic samplers both rely on data distinguishability and gradients with respect to parameters. Degeneracies cripple label prediction and inverse problems. But spotting those degeneracies often reveals something fundamental about the model or the underlying data-generating process.

How the Distillery Works

The authors present the Degeneracy Distillery: a technique that both detects and resolves degenerate parameter combinations automatically and symbolically from parameter-data pairs alone. It estimates the Fisher information matrix, then flattens it by exploring the information geometry of the likelihood. That flattening happens globally, across all parameter space - not just at a single point like posterior-based methods do. The output is a symbolic coordinate transformation that isolates which parameter combinations produce independent effects on the data.

What It Delivers

Applied to synthetic and real-world problems, the method discovers these transformations without any realized data observation - degeneracies are treated as an intrinsic property of the physical model. The payoff: significantly reduced simulation budget for downstream neural posterior estimation. In test cases, the authors report up to $10\times$ fewer simulations needed while maintaining validation calibration. Plus you get physical insight into the system because the symbolic transformations identify exactly which combinations of parameters are confounded.

This approach flips the script: instead of fighting degeneracies after the fact, the Degeneracy Distillery bakes the resolution into the model's coordinate system before you run expensive simulations. Expect faster, cheaper simulation-based inference without compromising accuracy.

Source: The Degeneracy Distillery
Domain: arxiv.org