Weak-to-Strong Generalization for Better Retrieval and Reranking
Teams train models on weak, noisy signals every day and accidentally teach them to copy mistakes. A confidence-aware approach that lets a strong reranker disagree when it knows better.
Weak-to-strong generalization sounds like a research idea, but it solves a very practical problem: teams train models on weak, noisy data every day, and accidentally teach them to copy mistakes. This talk shows a different way to think about it. Instead of treating weak signals as ground truth, treat them as hints, then train a stronger model in a way that lets it disagree when it knows better. The focus is a use case every data team understands: retrieval and reranking in RAG systems. Most teams already use signals like BM25, click logs, or small rerankers, which are cheap and fast but not very accurate. The usual approach trains bigger models to imitate them, which inevitably caps performance. The talk presents a simple pipeline where these weak signals supervise a stronger reranker, but with a confidence-aware loss that lets the model go beyond the labels, so it fixes errors instead of copying them.
What this covers:
- A simple explanation of weak-to-strong generalization and why normal fine-tuning falls short
- Why retrieval systems are a perfect fit for this idea
- A concrete setup using BM25, a small reranker, and a larger cross-encoder
- What works, what breaks, and how to debug it
- A clear decision guide for when to use this versus distillation or human labels
The main takeaway: perfect labels are not required to get better models, the training just has to be designed so strong models can use what they already know. Code and an evaluation framework are included so you can try it yourself.