Hey David, I really like your paper, hadn’t seen it til now. Sorry for not doing a thorough literature review and catching it!
Super cool paper too, exciting to see. Seems like there’s a good amount of overlap in what motivated our approaches, too, though your rationale seems more detailed/rigorous/sophisticated—I’ll have to read it more thoroughly and try to absorb the generators of that process.
Then it looks like my contribution here was just making the threshold have a parameter per-feature and defining some pseudoderivatives so that threshold parameters could be learned (though I was framing it as an ‘inhibitory bias’ at the time, I now like the threshold framing)
I’ll add a citation to your paper shortly (probably this evening, though possibly tomorrow)
Wild that you did that already, ten years ago. super cool.
This activation function was introduced in one of my papers from 10 years ago ;)
See Figure 2 of https://arxiv.org/abs/1402.3337
Hey David, I really like your paper, hadn’t seen it til now. Sorry for not doing a thorough literature review and catching it!
Super cool paper too, exciting to see. Seems like there’s a good amount of overlap in what motivated our approaches, too, though your rationale seems more detailed/rigorous/sophisticated—I’ll have to read it more thoroughly and try to absorb the generators of that process.
Then it looks like my contribution here was just making the threshold have a parameter per-feature and defining some pseudoderivatives so that threshold parameters could be learned (though I was framing it as an ‘inhibitory bias’ at the time, I now like the threshold framing)
I’ll add a citation to your paper shortly (probably this evening, though possibly tomorrow)
Wild that you did that already, ten years ago. super cool.
Thanks for commenting and letting me know! :)