← Lab
geluapprox
null resultC++ · CPUSystems & data structuresJuly 7, 2026
the question
Are the three standard GELU forms interchangeable?
what came out
No. The sigmoid approximation is ~10x faster than exact-erf but 43x less faithful than the tanh approximation (2% vs 0.05% activation error), so tanh is the speed/accuracy sweet spot. The error is provably symmetric in the |x|~2-3 elbow. C++, pre-registered, code-reviewed.
method & receipts
- Result: null result — an honest negative, reported as it came out.
- Pre-registered — the prediction was written down and committed before the run.
- Reproducible — one script re-runs the whole thing from scratch.
- Independently reviewed — an adversarial review pass, kept in the repo.
- Machine-readable claims — every reported number, in a checkable file.
- Tested — a correctness/benchmark suite ships alongside the code.
→ read the code and re-run it
github.com/v-code01/geluapprox