72x Smaller Model Beats Qwen-Embedding-8B on Temporal Retrieval by 12%

72.7x fewer parameters and still 15% better on temporal queries. That's the headline from TPOUR, a new unsupervised dense retriever that solves a problem most retriever papers ignore: time.

Standard contrastive learning treats documents as static. Ask "Who is the president in 2019?" and a model trained on documents from 2018-2025 might pull up a 2024 article if the semantic similarity is high enough. TPOUR fixes that without any supervised training or explicit timestamps.

Preference Learning in the Time Dimension

The core trick is TRPO (Temporal Retrieval Preference Optimization). It reinterprets contrastive preference learning along a temporal axis. For each query, TPOUR learns to rank documents that are temporally closer to the query's implicit time period above those that are distant, all from unlabeled document sequences.

TRPO doesn't need timestamped supervision. It uses the natural document order in a corpus as a weak signal, then refines via pairwise preference loss between temporally aligned and misaligned passages. The authors claim it generalizes to unseen time periods through interpolation in a learned time embedding.

Small Model, Big Benchmarks

The paper evaluates TPOUR on T-IR, a temporal information retrieval benchmark. Against Qwen-Embedding-8B, a model with roughly 8 billion parameters, TPOUR built on the small Contriever architecture (likely around 110M parameters) delivers +4.04 nDCG@5 on explicit temporal queries (a 12.15% relative gain) and +4.98 nDCG@5 on implicit queries (a 15.21% relative gain).

That's a 72.7x parameter efficiency advantage over Qwen-Embedding-8B. Beats both unsupervised and supervised baselines, including models that were trained on timestamped data.

What This Unlocks

Temporal retrieval is a blind spot for most dense retrievers. TPOUR shows you don't need timestamps to fix it-just a clever preference loss that leverages document order. Code is already up on GitHub at github.com/agwaBom/TPOUR. Expect this approach to show up in production search pipelines that need to separate 2020 news from 2024 news without a separate date filter.

Source: Temporal Preference Optimization for Unsupervised Retrieval
Domain: arxiv.org