Source linked

Why Your Nix Shell Is Slow: 486 Failed Opens Before Main

Developers using Nix face a hidden tax: the dynamic linker makes hundreds of failing openat() calls before main() even starts, adding 70ms to every shell prompt.

nixnixpkgsdevenvdynamic linkerperformancedeveloper tools

486 failing openat calls for a simple devenv version command. That's the number of times glibc tries and fails to find a .so before it finds the right one. Every shell prompt pays this tax, and it's been haunting nixpkgs for a decade.

The 70ms Tax on Every Prompt

devenv's auto-activation runs hook-should-activate on every shell prompt. It's a trivial operation: find the project, check trust, print a path. Yet it takes 70ms per prompt because the dynamic loader must resolve every shared library before a single line of user code executes. Domen Kožar and Farid Zakaria at Tacosprint measured it: time devenv hook-should-activate returns 0m0.070s. That's 70ms of pure startup overhead, multiplied across every prompt redraw.

The problem isn't specific to devenv. It's baked into how Nix packages libraries. Every program pays it. Imagemagick's magick --version makes 1225 failing opens. That's not a typo.

Why Nix Makes the Loader Work So Hard

Traditional Linux distros put shared libraries in a handful of directories like /usr/lib and maintain an ld.so.cache hash table. Nix scatters every package into its own /nix/store/<hash>-<name>/lib directory. There is no global cache. Instead, each ELF binary carries a DT_RUNPATH listing one directory per dependency. glibc's resolver walks every DT_RUNPATH directory in order, trying openat(dir/soname) until one succeeds. With 50 libraries and a dozen runpath entries, that's on the order of 50 x 12 = 600 attempts, most of which fail. Add glibc-hwcaps subdirectory probing (x86-64-v3, v2, etc.) and each directory triggers three more failing opens.

On a fast SSD with a warm cache, it's tolerable. On a cold cache, slow disk, network filesystem, or low-power ARM board, it's the difference between snappy and sluggish. And it compounds across every process a script spawns.

What a Real Fix Has to Preserve

The reason this stat storm survives after years of discussion is that every obvious fix breaks something important. Any solution must preserve LD_LIBRARY_PATH (for GPU driver injection), LD_PRELOAD (interposers), the libGL/glvnd vendor switch, per-object resolution of same-soname libraries, dlopen plugins, cross compilation, and must not bloat closure size or require rebasing a glibc patch onto every release.

Approach 1, already explored by Farid Zakaria's shrinkwrap and the nix-harden-needed tool, rewrites DT_NEEDED entries to absolute store paths. glibc's slash short-circuit then skips all search. It works, but it's post-processing, and it interacts badly with the other requirements.

The more radical spike from the devenv team: delete the dynamic loader entirely by linking the whole program into one static binary. That eliminates the search entirely, but you lose the runtime flexibility Nix depends on.

No approach ticks every box. The umbrella tracking issue (NixOS/nixpkgs#481620) has been open for years. But the measurement is clear: 486 failing opens per process is a tax worth eliminating. If the community can converge on a fix, the whole nixpkgs ecosystem gets faster by default.


Source: Making devenv start fast, and the whole nixpkgs with it - devenv
Domain: devenv.sh

Read original source ->

External source stays available while the OJO article and comment thread stay local.

Comments load interactively on the live page.