RAG Poisoning Fails After Reranking - Here's Why and How to Fix It

Corpus poisoning attacks against RAG pipelines that score well in retrieval-stage relevance regularly fail after reranking — dropping attack success rates substantially despite appearing dangerous early on.

A new paper on arXiv (2606.11265) from researchers revisiting corpus poisoning under realistic multi-stage retrieval pipelines nails the core failure mode: retrieval granularity mismatch. Document-level adversarial signals crafted for dense retrieval get fragmented during chunking, while rerankers favor locally coherent, answer-bearing passages over globally optimized semantic similarity. In other words, the poison looks good to the retriever but falls apart when the reranker actually reads the chunks.

The Real Pipeline Breaks the Easy Attacks

Existing corpus poisoning studies mostly evaluate against simplified retrieval setups — no chunking, no reranker. This paper shows that’s a dangerous simplification. When you add real-world components like document chunking with varying chunk sizes, standard attacks become highly sensitive. Same for reranking strategies: different rerankers react differently to adversarial passages. The result is that most published poisoning methods are not reproducible in production RAG systems.

To quantify this, the authors tested multiple retrievers and rerankers on standard RAG benchmarks. Their core finding: attack success rates crater when chunking and reranking are introduced, even if the adversarial passages were highly relevant to the retriever.

CRCP Bakes Pipeline Consistency Into the Poison

The proposed fix is Chunk-aware and Rerank-Consistent Poisoning (CRCP). Instead of optimizing only retrieval relevance, CRCP explicitly models chunking transformations during optimization to generate locally self-contained adversarial passages that stay effective under varying chunk sizes. It also enforces reranker consistency — the adversarial passage must look coherent and directly answer-bearing to the reranker, not just semantically close to the query.

CRCP jointly optimizes three objectives: retrieval relevance, reranker consistency, and chunk-boundary robustness. The result is a poisoning framework that achieves substantially higher attack success rates across realistic multi-stage pipelines compared to existing methods. The paper reports strong robustness across different chunking configurations and reranker architectures.

What This Means for RAG Security

This work highlights an important realism gap in current RAG security evaluation. Poisoning in modern RAG systems is not a retrieval-only problem — it’s a multi-stage retrieval consistency problem. Security researchers and practitioners building guardrails for RAG need to test attacks under the full pipeline, not just the retriever. And defenders need to watch for adversarial passages that look benign to a reranker but carry malicious payloads after chunking.

Expect future poisoning research to stop ignoring chunking and reranking. The CRCP approach sets a new baseline for what a realistic attack looks like — and that baseline is harder to defend against than the old toy setups.

Source: When Poison Fails After Retrieval: Revisiting Corpus Poisoning under Chunking and Reranking Pipelines
Domain: arxiv.org