rbpbound

confirmedupstream bugPython · ranxInference & servingJuly 24, 2026

the question

Does ranx Rank-Biased Precision treat relevance as binary and stay within [0,1] as its docstring defines?

what came out

ranx Rank-Biased Precision multiplies each document by its raw graded relevance while its docstring defines r_i in {0,1}. On graded qrels the value inflates by roughly the mean grade (2.56x) and can exceed 1 (2.21 measured), breaking the [0,1] bound; binarizing the grades recovers the documented value.

method & receipts

Result: confirmed
Reproducible — one script re-runs the whole thing from scratch.
Tested — a correctness/benchmark suite ships alongside the code.

→ read the code and re-run it

github.com/v-code01/rbpbound