The title is still wildly overconfident. There are no ASIs in the set of deep learning programs? If someone believes that, they don’t understand deep learning—deep learning could be said to be the biology of software engineering.
Personally, I believe the strong inverse of this: there are absolutely no intelligent systems which are not a special case of deep learning, because deep learning is the anti-spotlight-effect of ai: connectionism is the perspective that we don’t need to constrain types of connections, because statistical understanding will be able to generate an understanding of structural connectivity. Which means that even if the structural connectivity has enormous amounts of learned complexity, even if you already had another system that had more useful priors than mere connectionism, it still counts as connectionism! the point here is not merely that everything is a special case of everything else or something vacuous, but that the additional constraints, while useful, would need to import the core basis of deep learning: adjustment of large networks of interaction as the key basis for intelligence. it is the biology of software engineering because it allows us to talk about the shape of any program in terms of its manifold of behavior.
yes, there’s a lot more an ai could use to shape its manifold of behavior than mere gradient descent. but as long as you have a need to do bayesian inference, you’ll have something that looks like gradient descent in networks.
I mean, I agree I think the claim is wrong. But I think I’m my preferred LessWrong culture people straightforwardly voice their contrarian positions, and get, like weakly upvoted, and the maybe strong disagree downvoted.
I like this post in part because it’s short and clearly states a claim in a way that isn’t very weaselable, which most contrarian-take posts fail to do. (It was at zero karma when I found it)
I do think the karma system is fundamentally flawed; I’d suggest considering what sort of multi-currency prediction market system might be better. what outcomes are being predicted? which currencies should trade easily with others? are there nonlinearities between the currencies? are there costs to do some things? is there a way to say “I think this should get this much total”, rather than merely “I want to offset the current amount by this much”? which actions can be taken back? etc etc etc. It isn’t a trivial project to figure out the details, I imagine there’s research from mechanism design I could pull out of a hat with the right search engine. But I’d still spend a bet token “this post will turn out to have been not only wrong, but represents an overconfident style of discourse that doesn’t express sufficient scientific hedging”.
Of course, I get downvoted for similar reasons fairly often, so, like, you know, maybe what we really need is non-anonymous votes or something so people know when downvotes are by goofballs like me, establish reputation as thoughtful voters, etc. What if you could only vote by writing a review, and then the vote was generated by a sentiment analyzer? What if you had to select a phrase highlight in order to vote on a post?
edit: none of these are meant to be definitely good ideas and I think I’ll be confident tomorrow whether they’re all definitely bad ideas or not
The title is still wildly overconfident. There are no ASIs in the set of deep learning programs? If someone believes that, they don’t understand deep learning—deep learning could be said to be the biology of software engineering.
Personally, I believe the strong inverse of this: there are absolutely no intelligent systems which are not a special case of deep learning, because deep learning is the anti-spotlight-effect of ai: connectionism is the perspective that we don’t need to constrain types of connections, because statistical understanding will be able to generate an understanding of structural connectivity. Which means that even if the structural connectivity has enormous amounts of learned complexity, even if you already had another system that had more useful priors than mere connectionism, it still counts as connectionism! the point here is not merely that everything is a special case of everything else or something vacuous, but that the additional constraints, while useful, would need to import the core basis of deep learning: adjustment of large networks of interaction as the key basis for intelligence. it is the biology of software engineering because it allows us to talk about the shape of any program in terms of its manifold of behavior.
yes, there’s a lot more an ai could use to shape its manifold of behavior than mere gradient descent. but as long as you have a need to do bayesian inference, you’ll have something that looks like gradient descent in networks.
edit: not necessarily backprop, though.
I mean, I agree I think the claim is wrong. But I think I’m my preferred LessWrong culture people straightforwardly voice their contrarian positions, and get, like weakly upvoted, and the maybe strong disagree downvoted.
I like this post in part because it’s short and clearly states a claim in a way that isn’t very weaselable, which most contrarian-take posts fail to do. (It was at zero karma when I found it)
I do think the karma system is fundamentally flawed; I’d suggest considering what sort of multi-currency prediction market system might be better. what outcomes are being predicted? which currencies should trade easily with others? are there nonlinearities between the currencies? are there costs to do some things? is there a way to say “I think this should get this much total”, rather than merely “I want to offset the current amount by this much”? which actions can be taken back? etc etc etc. It isn’t a trivial project to figure out the details, I imagine there’s research from mechanism design I could pull out of a hat with the right search engine. But I’d still spend a bet token “this post will turn out to have been not only wrong, but represents an overconfident style of discourse that doesn’t express sufficient scientific hedging”.
Of course, I get downvoted for similar reasons fairly often, so, like, you know, maybe what we really need is non-anonymous votes or something so people know when downvotes are by goofballs like me, establish reputation as thoughtful voters, etc. What if you could only vote by writing a review, and then the vote was generated by a sentiment analyzer? What if you had to select a phrase highlight in order to vote on a post?
edit: none of these are meant to be definitely good ideas and I think I’ll be confident tomorrow whether they’re all definitely bad ideas or not