Why Dense Retrievers Beat BM25 Against RAG Poisoning

432 is the number of RAG configurations one team tested to find out what actually makes these systems vulnerable to poisoning. Not a single component, but the interaction of retriever choice, retrieval depth, database composition, and generator model.

The study, a full factorial experiment covering datasets, retrievers, chunking strategies, and generator models, isolates the strongest factors. Retriever architecture tops the list: dense and graph-based retrievers consistently resist poisoning better than BM25. Retrieval depth runs second — the more documents you pull, the more likely an adversarial passage sneaks in.

Retriever Architecture Dominates Poisoning Exposure

BM25, the classic sparse retriever, is the most vulnerable. Dense retrievers (like contriever or similar embedding-based models) and graph-based retrievers show markedly lower poisoning success at the retrieval stage. The gap is large enough that switching from BM25 to a dense retriever is the single highest-impact defensive move a RAG system can make. Generator choice and database composition become decisive only after the retriever lets a poisoned passage through.

Replication Amplifies, Clean Sources Mitigate

Replicating poisoned content across multiple databases in the knowledge store dramatically amplifies adversarial influence. The paper calls it a force multiplier for the attacker. Conversely, adding clean, relevant sources to the database reduces the odds that a poisoned passage gets selected. This isn't a theoretical concern — the effect shows up across all tested configurations.

Depth and Dataset Matter More Than You Think

Retrieval depth — the number of passages pulled per query — linearly increases exposure. At higher depths, even robust retrievers start retrieving adversarial content. Dataset choice also significantly affects poisoning exposure, suggesting that some domains are inherently harder to defend. Chunking strategy and generator model have smaller but non-negligible effects on downstream attack success.

The takeaway: hardening RAG against poisoning means starting with the retriever, then constraining retrieval depth, then curating the database. The 432-configuration sweep gives engineers a ranked list of levers to pull, not just a cautionary tale.

Source: Influence Factors on RAG Poisoning
Domain: arxiv.org