Skip to main content
Vansh Verma
← Lab

maskbreak

confirmedupstream bugPython · transformersInference & servingJuly 24, 2026

the question

Does transformers' apply_chat_template return a correct assistant loss mask under left truncation?

what came out

transformers' apply_chat_template with return_assistant_tokens_mask returns an all-zero mask under left truncation: the loop breaks at the first dropped early span before marking the surviving final assistant turn, so it trains with zero loss. Reproduces on 5.12.1 with a Qwen2.5 template. Fix: continue, and guard end_token is not None.

method & receipts

  • Result: confirmed
  • Reproducible — one script re-runs the whole thing from scratch.
  • Tested — a correctness/benchmark suite ships alongside the code.

→ read the code and re-run it

github.com/v-code01/maskbreak