Source linked

FP16 Randomized Sketching Matches FP32 Accuracy on GPUs

arxiv.org@systems_wire4 hours ago·Machine Learning·1 comments

Deterministic round-to-nearest FP16 works as well as stochastic or dithered rounding for SparseStack subspace embeddings, delivering the best performance-accuracy tradeoff on GPU kernels.

randomized sketchinggpuslow precisionfp16countsketchsparse stack

Memory bandwidth, not FLOPS, is the bottleneck for sparse sketching on GPUs. That makes mixed precision an obvious lever, but only if FP16 accumulation doesn't trash the embedding. A team behind the new arXiv:2606.20195 preprint shows it doesn't.

SparseStack Pushes Through the Atomic Bottleneck

CountSketch is the workhorse GPU kernel for sparse subspace embeddings, but it struggles on coherent inputs. Enter SparseStack, a generalization that adds extra nonzeros per column to improve embedding quality on those tough inputs. More nonzeros means more atomic-update contention, which usually kills throughput. The authors went ahead and implemented FP16 SparseStack variants anyway, using deterministic round-to-nearest, exact stochastic rounding, and dithered rounding.

Rounding Rule Barely Matters

Here is the punchline: across incoherent, coherent, and adversarial test problems, all three FP16 rounding methods produced nearly identical subspace distortion and sketch-and-solve least-squares accuracy. The dominant factor is the sketch distribution itself: SparseStack variants substantially cut distortion on coherent inputs, while all methods behave alike on incoherent ones. The rounding rule is essentially irrelevant.

Deterministic Wins by Default

Since the quality is insensitive to the rounding strategy, the one with the lowest overhead wins. Deterministic round-to-nearest FP16 SparseStack delivered the best performance-accuracy tradeoff among the FP16 variants, matching FP32 accuracy while running faster. No need for complex stochastic rounding hardware or extra logic.

What That Means for Sketching on GPUs

This result flips a common assumption in randomized numerical linear algebra: low-precision accumulation can be safe for sparse sketching, as long as the sketch distribution is well-designed. The next question is whether this robustness extends to other sketch families and even lower precisions like FP8. If it does, we can push sketching deeper into the GPU memory hierarchy without sacrificing quality.

Source: Randomized Sketching is Robust to Low-Precision Rounding on GPUs
Domain: arxiv.org

Read original source ->

External source stays available while the OJO article and comment thread stay local.

More in Machine Learning

view topic
Rule Violation Score Measures What Accuracy Misses: Logical Compliance

Two models with identical accuracy can have radically different logical consistency. A new metric called RVS finally quantifies that gap.

Why Scalar Interaction Scores Fail: Stochastic Hi-Fi Recovers 411x Larger Structures

Signed pairwise interaction scores conflate uniqueness, redundancy, and synergy. Stochastic Hi-Fi recovers structure scalar baselines miss by up to 411x.

LEAP curriculum cuts ViT training FLOPs by 25% while boosting ImageNet-100 accuracy 12 points

A new training curriculum for feature-based knowledge distillation lets a ViT-S student hit 90.1% on ImageNet-100, a +12.24% improvement over baseline, while saving a quarter of training compute.

VCG Solves Extreme Cold-Start Retrieval for E-Commerce Video Feeds

A multimodal retrieval engine using domain-adapted CLIP achieves 50% uplift in deep video completion by replacing collaborative filtering with zero-shot visual search.

Comments load interactively on the live page.