The exotic activation function almost feels like cheating? Like I want the model the model to discover these useful structures, then try to understand them. But trying to do everything at once may be too hard.
Incredibility minor, but changing from onchange to oninput and dropping the animation will make the slider feel much slicker.
The optimization section of Learning Transformer Programs might work with your task/model
You’ve probably seen David Ha’s work, but something like https://es-clip.github.io/ could be a good starting point for dropping backprop.
The exotic activation function almost feels like cheating? Like I want the model the model to discover these useful structures, then try to understand them. But trying to do everything at once may be too hard.
Incredibility minor, but changing from
onchangetooninputand dropping the animation will make the slider feel much slicker.