← Lab
gatehorizon
confirmedshipPython · PyTorch bfloat16Models, training & reasoningJuly 25, 2026
the question
Can a bfloat16-stored decay gate represent slow-forgetting memory half-lives?
what came out
In bfloat16 near 1 the gate spacing is 2^-8, so any target decay below 2^-9 snaps to exactly 1.0 (never forgets) and the longest finite half-life is about 177 steps (177.1 exactly; ln2*2^8 = 177.4 to first order). Longer finite half-lives are unreachable, and targets between 177 and 355 collapse to it. Absent in fp32.
method & receipts
- Result: confirmed
- Reproducible: one script re-runs the whole thing from scratch.
- Tested: a correctness/benchmark suite ships alongside the code.
→ read the code and re-run it
github.com/v-code01/gatehorizon