The weak LRH I would say is now well supported by considerable empirical evidence.
a couple of people have shrug reacted to this sentence. . I think the theory I had in mind as being ‘well supported by empirical evidence’ was something like ‘you can often find examples of networks representing stuff with a linear direction pretty uncontroversially’. I think this is probably still a fair statement, though it’s a bit vague, and you could argue that it’s a bit of a leap from this (which is like >0 things are represented as linear directions) to how I phrased the weak LRH in the post.
while I think this post has mostly aged quite well, I think, looking back, I was hedging to try to avoid writing a post entitled ‘here’s why SAEs are doomed’.
a couple of people have shrug reacted to this sentence. . I think the theory I had in mind as being ‘well supported by empirical evidence’ was something like ‘you can often find examples of networks representing stuff with a linear direction pretty uncontroversially’. I think this is probably still a fair statement, though it’s a bit vague, and you could argue that it’s a bit of a leap from this (which is like >0 things are represented as linear directions) to how I phrased the weak LRH in the post.
while I think this post has mostly aged quite well, I think, looking back, I was hedging to try to avoid writing a post entitled ‘here’s why SAEs are doomed’.