Skip to main content
Vansh Verma
← Lab

oftfold

confirmedupstream bugPython · PEFT / PyTorchModel behavior & reasoningJuly 25, 2026

the question

Does PEFT's OFT Conv2d adapter act as an identity at initialization?

what came out

PEFT's OFT Conv2d forward folds rotated patches with F.fold, the adjoint not the inverse, scaling each pixel by its patch-coverage. A zero-init identity-rotation adapter already shifts a 3x3 conv output by 7.089, and merge_and_unload disagrees with the forward by 6.410. The 1x1 conv and Linear controls are exact.

method & receipts

  • Result: confirmed
  • Reproducible — one script re-runs the whole thing from scratch.
  • Tested — a correctness/benchmark suite ships alongside the code.

→ read the code and re-run it

github.com/v-code01/oftfold