Yep. Well, one could use shorter chains and then clamp the signal to 0 or 1 then feed it to another short chain, eliminating noise to some extent. Brain does something like that basically. If you have sigmoid functions along the chain you sort of digitalize the signal and get rid of the noise.
You’d lose some performance that way though, unless you wanted to do away with function composition. (That would probably be a bad idea.) It’s not entirely clear (to me) that the cost of making this work in a real-life environment wouldn’t nullify the performance gains.
Especially given that we already do a fair bit of approximation (eg floating-point calculations).
Yep. Well, one could use shorter chains and then clamp the signal to 0 or 1 then feed it to another short chain, eliminating noise to some extent. Brain does something like that basically. If you have sigmoid functions along the chain you sort of digitalize the signal and get rid of the noise.
That would make things worse, not better. Clamping destroys the noise by adding more, in an amount that ‘happens’ to move the value to an integer...
You’d lose some performance that way though, unless you wanted to do away with function composition. (That would probably be a bad idea.) It’s not entirely clear (to me) that the cost of making this work in a real-life environment wouldn’t nullify the performance gains.
Especially given that we already do a fair bit of approximation (eg floating-point calculations).