DeepMind Gemini Safety lead; Foundation board member
Dave Orr
Karma: 1,637
Pradyumna: You a reasonable person: the city should encourage carpooling to reduce congestion
Bengaluru’s Transport Department (a very stable genius): Taxi drivers complained and so we will ban carpooling
It’s not really that Bangalore banned carpooling, they required licenses for ridesharing apps. Maybe that’s a de facto ban of those apps, but that’s a far cry from banning carpooling in general.
Partly this will be because in fact current ML systems are not analogous to future AGI in some ways—probably if you tell the AGI that A is B, it will also know that B is A.
One oddity of LLMs is that we don’t have a good way to tell the model that A is B in a way that it can remember. Prompts are not persistent, and as this paper shows, fine tuning doesn’t do a good job of getting a fact into the model without doing a bunch of paraphrasing. Pretraining presumably works in a similar way.
This is weird! And I think helps make sense of some of the problems we see with current language models.
45->55% is a 22% relative gain, while 90->100% is only an 11% gain.
On the other hand, 45->55% is a reduction in error by 18%, while 90->100% is a 100% reduction in errors.
Which framing is best depends on the use case. Preferring one naively over the other is definitely an error. :)
I think the argument against LeCun is simple: while it may be true that AIs won’t necessarily have a dominance instinct the way that people do, they could try to dominate for other reasons: namely that such dominance is an instrumental goal towards whatever its objective is. And in fact that is a significant risk, and can’t be discounted by pointing out that they may not have a natural instinct towards dominance.
I just think that to an economist, models and survey results are different things, and he’s not asking for the latter.
I think that Tyler is thinking more of an economic type model that looks at the incentives of various actors and uses that to understand what might go wrong and why. I predict that he would look at this model and say, “misaligned AI can cause catastrophes” is the hand-wavy bit that he would like to see an actual model of.
I’m not an economist (is IANAE a known initialization yet?), but it would probably include things like the AI labs, the AIs, and potentially regulators or hackers/thieves, try to understand and model their incentives and behaviors, and see what comes out of that. It’s less about subjective probabilities from experts and more about trying to understand the forces acting on the players and how they respond to them.
So… when can we get the optimal guide, if this isn’t it? :)
In general to solve an NP complete problem like 3-SAT, you have to spend compute or storage to solve it.
Suppose you solve one 3-SAT problem. If you don’t write down the solution and steps along the way, then you have no way to get the benefit of the work for the next problem. But if you do store the results of the intermediate steps, then you need to store data that’s also polynomial in size.
In practice often you can do much better than that because the problems you’re solving may share certain data or characteristics that lead to shortcuts, but in the general case you have to pay the cost every time you need to solve an NP complete problem.
If one person estimates the odds at a billion to one, and the other at even, you should clearly bet the middle. You can easily construct bets that offer each of them a very good deal by their lights and guarantee you a win. This won’t maximize your EV but seems pretty great if you agree with Nick.
Anthropic reportedly got a $4B valuation on negligible revenue. Cohere is reportedly asking for a $6B valuation on maybe a few $M in revenue.
AI startups are getting pretty absurd valuations based on I’m not sure what, but I don’t think it’s ARR.
I’m not sure multiple of revenue is meaningful right now. Nobody is investing in OAI because of their current business. Also there are tons of investments at infinite multiples once you realize that many companies get investments with no revenue.
I mean, computers aren’t technically continuous and neither are neural networks, but if your time step is small enough they are continuous-ish. It’s interesting that that’s enough.
I agree music would be a good application for this approach.
I think this is real, in the sense that they got the results they are reporting and this is a meaningful advance. Too early to say if this will scale to real world problems but it seems super promising, and I would hope and expect that Waymo and competitors are seriously investigating this, or will be soon.
Having said that, it’s totally unclear how you might apply this to LLMs, the AI du jour. One of the main innovations in liquid networks is that they are continuous rather than discrete, which is good for very high bandwidth exercises like vision. Our eyes are technically discrete in that retina cells fire discretely, but I think the best interpretation of them at scale is much more like a continuous system. Similar to hearing, the AI analog being speech recognition.
But language is not really like that. Words are mostly discrete—mostly you want to process things at the token level (~= words) or sometimes wordpieces or even letters, but it’s not that sensible to think of text as being continuous. So it’s not obvious how to apply liquid NNs to text understanding/generation.
Research opportunity!
But it’ll be a while, if ever, before continuous networks work for language.
Usually “any” means each person in the specific class individually. So perhaps not groups of people working together, but a much higher bar than a randomly sampled person.
But note that Richard doesn’t think that “the specific ‘expert’ threshold will make much difference”, so probably the exact definition of “any” doesn’t matter very much for his thoughts here.
Similar risk to Christiano, which might be medium by less wrong standards but is extremely high compared to the general public.
High risk tolerance (used to play poker for a living, comfortable with somewhat risky sports like climbing or scuba diving). Very low neuroticism, medium conscientiousness. I spend a reasonable amount of time putting probabilities on things, decently calibrated. Very calm in emergency situations.
I’m a product manager exec mostly working on applications of language AI. Previously an ml research engineer.
I don’t actually follow—how does change blindness in people relate to how much stuff you have to design?
Suppose you were running a simulation, and it had some problems around object permanence, or colors not being quite constant (colors are surprisingly complicated to calculate since some of them depend on quantum effects), or other weird problems. What might you do to help that?
One answer might be to make the intelligences you are simulating ignore the types of errors that your system makes. And it turns out that we are blind to many changes around us!
Or conversely, if you are simulating an intelligence that happens to have change blindness, then you worry a lot less about fidelity in the areas that people mostly miss or ignore anyway.
The point is this: reality seems flawless because your brain assumes it is, and ignores cases where it isn’t. Even when the changes are large, like a completely different person taking over halfway through a conversation, or numerous continuity errors in movies that almost all bounce right off of us. So I don’t think that you can take amazing glitch free continuity as evidence that we’re not in a simulation, since we may not see the bugs.
One thing that I think is missing (maybe just beyond the scope of this post) is thinking about newcomers with a positive frame: how do we help them get up to speed, be welcomed, and become useful contributors?
You could imagine periodic open posts, for instance, where we invite 101-style questions, post your objection to AI risks, etc where more experienced folks could answer those kind of things without cluttering up the main site. Possibly multiple more specific such threads if there’s enough interest.
Then you can tell people who try to post level 1-3 stuff that they should go to those threads instead, and help make sure they get attention.
I’m sure there are other ideas as well—the main point is that we should think of both positive as well as negative actions to take in response to an influx of newbies.
Let me suggest a different direction.
The risk is that a niche candidate will make the idea too associated with them, which will let everyone else off the hook—it’s easy to dismiss a weirdo talking about weird stuff.
A better direction might be to find a second tier candidate that wants to differentiate themselves, and help them with good snappy talking points that sound good in a debate. I think that’s both higher impact and has a much smaller chance of pushing things in the wrong direction accidentally.
This seems sensible, but I remember thinking something very similar about Full Tilt, and then they turned out to be doing a bunch of shady shit that was very not in their best interest. I think there’s a significant chance that fraudsters gonna fraud even when they really shouldn’t, and Tether in particular has such a ridiculous background that it just seems very possible that they will take unnecessary risks, lend money when they shouldn’t, etc, just because people do what they’ve been doing all too often.