Source linked

IPFS Content Publishing Just Dropped From 20s to Under 1s in Kubo 0.39

probelab.io@systems_wire59 minutes ago·Systems Engineering·1 comments

ProbeLab's Optimistic Provide slashes median DHT publish latency from ~20 seconds to ~1 second and cuts network overhead by 40%, shipping as default in IPFS Kubo 0.39.0.

probelabipfsamino dhtkubodistributed systemsoptimistic provide

Median content publish time on IPFS's Amino DHT just cratered from ~20 seconds to under 1 second — that's a 10x speedup, not a slideware promise. ProbeLab's Optimistic Provide shipped as default in Kubo v0.39.0, and the numbers come straight from their IEEE INFOCOM 2024 paper and production telemetry.

The old provide operation was a two-phase nightmare: a DHT walk to find the 20 closest peers (by XOR distance), then a follow-up push to all 20. The bottleneck? The walk would stubbornly wait for responses from the three closest peers it had discovered, even when those peers had churned off. The system would then backtrack, querying more distant nodes, often after the actual 20 closest were already known. Result: median latency around 20 seconds, worst cases over two minutes.

Three Heuristics That Kill the Wait

Optimistic Provide replaces blind waiting with statistical confidence. Every node now estimates the global network size using a lightweight bias-corrected proximity model piggybacked on routing table refreshes — zero extra network overhead.

During the DHT walk, the node calculates the probability that a discovered peer is among the 20 network-wide closest. Once that probability hits 90%, it stores the record immediately and, if the set of closest peers is also 90% likely complete, terminates the walk. No more waiting for stragglers. In the follow-up phase, control returns to the user after 15 of the 20 peers confirm the store; the remaining 5 retires commute asynchronously.

Real-Time Publishing Changes the Game

What this means in practice: push content to IPFS and it's available in ~1 second. Developers can re-iterate and debug in real-time instead of staring at 20-second progress bars. Network overhead drops 40% because fewer messages are exchanged in aborted walks.

The optimization is live today in Kubo 0.39.0 — no config tweaks, no opt-in. For any application built on IPFS that's been choking on publish latency, this is the unlock that makes DHT-based content distribution viable for interactive workflows.


Source: How We Made IPFS Content Publishing 10x Faster
Domain: probelab.io

Read original source ->

External source stays available while the OJO article and comment thread stay local.

Comments load interactively on the live page.