arXiv:2607.03664v1 Announce Type: new Abstract: The Gaussian Error Linear Unit is usually motivated as the expected output of an input-dependent stochastic Bernoulli gate. This work gives a complementary interpretation based on the Gaussian complementary first-order loss function: GELU is the signal-transmission term of the expected surplus of a hard linear gate with a Gaussian random threshold. This view separates loss accounting from forward signal transmission and generalises to a threshold-transmission family that includes ReLU, GELU, SiLU/Swish, and hard swish as special cases. The unifor
Source: arXiv cs.LG — read the full report at the original publisher.