Hi, thanks for sharing and experimentally trying out the theory in the previous post! Super cool.
Do you have the code for this up anywhere?
I’m also a little confused by the training procedure. Are you just instantiating a random vector and then doing GD with regards to the loss function you defined? Do the charts show the loss averaged over many random vectors (and splotch function variants)?
Thanks. I initially tried putting the code in a comment on this post, but it ended up being deleted as spam. It’s now up on github: https://github.com/DaemonicSigil/tessellating-hills It isn’t particularly readable, for which I apologize.
The initial vector has all components set to 0, and the charts show the evolution of these components over time. This is just for a particular run, there isn’t any averaging. x0 gets its own chart, since it changes much more than the other components. If you want to know how the loss varies with time, you can just flip figure 1 upside down to get a pretty good proxy, since the splotch functions are of secondary importance compared to the -x0 term.
Hi, thanks for sharing and experimentally trying out the theory in the previous post! Super cool.
Do you have the code for this up anywhere?
I’m also a little confused by the training procedure. Are you just instantiating a random vector and then doing GD with regards to the loss function you defined? Do the charts show the loss averaged over many random vectors (and splotch function variants)?
Thanks. I initially tried putting the code in a comment on this post, but it ended up being deleted as spam. It’s now up on github: https://github.com/DaemonicSigil/tessellating-hills It isn’t particularly readable, for which I apologize.
The initial vector has all components set to 0, and the charts show the evolution of these components over time. This is just for a particular run, there isn’t any averaging. x0 gets its own chart, since it changes much more than the other components. If you want to know how the loss varies with time, you can just flip figure 1 upside down to get a pretty good proxy, since the splotch functions are of secondary importance compared to the -x0 term.
Oops, sorry for that. I restored your original comment.