Google AI PM; Foundation board member
Dave Orr
Karma: 1,087
Here’s one argument:
Consumption is great when you get something in return that improves your life in some way. Convenience, saving time, and things that you use are all great.
However, there’s a ton of consumption in terms of buying things that don’t add utility, at least not at a reasonable return. People buy exercise bikes that they don’t use, books that they don’t read, panini presses that just sit on the counter, and lives become more cluttered and less enjoyable.
One reason for this is the hedonic treadmill, that our happiness reverts to a mean over time, so pleasure from an item doesn’t last. Another is that people envision the good outcomes for buying something—I’ll use that gym membership 3 times a week! -- but are bad at estimating the range of outcomes and so overestimate what they get for many purchases.
It turns out for many purchases (though probably a minority of them), you would be better off in terms of happiness if you bought nothing instead. High happiness ROI spending seems to be events rather than items, giving gifts, meaningful charity, and saving yourself time.
New cars, trendy clothes, the latest gadgets, and other hallmarks of modern consumerism, have a low return on spending, and pushing back against that may help people overall.
An analogy: food is delicious and necessary, but certain common patterns in how people eat are bad, even by the poor eater’s own values and preferences. That seems bad in a similar way, and opposing trends that increase such bad patterns seems sensible.
Next time I would actually include the definition of a technical term like Leibniz’s first principle to make this post a little less opaque, and therefore more interesting, to non experts.
This. If they had meant 19% less hallucinations they would have said 19% reduction in whatever, which is a common way to talk about relative improvements in ML.
For sure product risk aversion leads towards people moving to where they can have some impact, for people who don’t want pure research roles. I think this is basically fine—I don’t think that product risk is all that concerning at least for now.
Misalignment risk would be a different story but I’m not aware of cases where people moved because of it. (I might not have heard, of course.)
There’s a subtlety here around the term risk.
Google has been, IMO, very unwilling to take product risk, or risk a PR backlash of the type that Blenderbot or Sydney have gotten. Google has also been very nervous about perceived and actual bias in deployed models.
When people talk about red tape, it’s not the kind of red tape that might be useful for AGI alignment, it’s instead the kind aimed at minimizing product risks. And when Google says they are willing to take on more risk, here they mean product and reputational risk.
Maybe the same processes that would help with product risk would also help with AGI alignment risk, but frankly I’m skeptical. I think the problems are different enough that they need a different kind of thinking.
I think Google is better on the big risks than others, at least potentially, since they have some practice at understanding nonobvious secondary effects as applied to search or YouTube ranking.
Note that I’m at Google, but opinions here are mine, not Google’s.
Please post pictures once you’re done!
I feel like every week there’s a post that says, I might be naive but why can’t we just do X, and X is already well known and not considered sufficient. So it’s easy to see a post claiming a relatively direct solution as just being in that category.
The amount of effort and thinking in this case, plus the reputation of the poster, draws a clear distinction between the useless posts and this one, but it’s easy to imagine people pattern matching into believing that this is also probably useless without engaging with it.
FWIW I at least found this to be insightful and enlightening. This seems clearly like a direction to explore more and one that could plausibly pan out.
I wonder if we would need to explore beyond the current “one big transformer” setup to realize this. I don’t think humans have a specialized brain region for simulations (though there is a region that seems heavily implicated, see https://www.mountsinai.org/about/newsroom/2012/researchers-identify-area-of-the-brain-that-processes-empathy), but if you want to train something using gradient descent, it might be easier if you have a simulation module that predicts human preferences and is rewarded for accurate predictions, and then feed those into the main decision-making model.
Perhaps we can use revealed preferences through behavior combined with elicited preferences to train the preference predictor. This is similar to the idea of training a separate world model rather than lumping it in with the main blob.
I think in practice roughly the opposite is true.
As people age, they become less flexible in their beliefs and more set in their ways. If they are highly influential, then it’s difficult to make progress when they are still alive.
Science advances one funeral at a time: https://en.m.wikipedia.org/wiki/Planck’s_principle
It’s true that as you age you accumulate experience, and this stops when you die. For you, death is or course a hard limit on knowledge.
For the world, it’s much less clear.
Nuclear submarines (1870, Twenty Thousand Leagues Under the Sea)
Time travel (1895, The Time Machine)
This seems closely related to the concept of weirdness points.
I certainly am careful about how “lively” I appear in many settings, so that it doesn’t become a distraction or cause social penalties to me or whatever aim I’m trying to accomplish. This is the way that societies work—we all have shared norms for many interactions that allow for violations up to a point, and then much more freedom in private or with trusted friends and family.
And of course what counts as weird in any group depends on the group. At work, advocating for cryonics makes you a weirdo. At Less Wrong, you might be more weird if you don’t support cryonics!
I predict that instead of LLMs being trained on ASR-generated text, instead they will be upgraded to be multimodal, and trained on audio and video directly in addition to text.
Google has already discussed this publicly, e.g. here: https://blog.google/products/search/introducing-mum/.
This direction makes sense to me, since these models have huge capacity, much greater than the ASR models do. Why chain multiple systems if you can learn directly from the data?
I do agree with your underlying point that there’s massive amounts of audio and video data that haven’t been used much yet, and those are a great and growing resource for LLM training.
In general I am no fan of angellist syndicates because the fees are usurious, but if you have high conviction that there are huge returns to AI, possibly LLM syndicates might be worth a look.
TBF there’s no way Eliezer would approve that prompt to a superhuman AI, so I think no is the correct answer there. The first explanation is vague but basically correct as to why, at least on my model of Eliezer.
Not to put too fine a point on it, but you’re just wrong that these are easy problems. NLP is hard because language is remarkably complex. NLP is also hard because it feels so easy from the inside—I can easily tell what that pronoun refers to, goes the thinking, so it should be easy for the computer! But it’s not, fully understanding language is very plausibly AI-complete.
Even topic classification (which is what you need to reliably censor certain subjects), though it seems simple, has literal decades of research and is not all that close to being solved.
So I think you should update much more towards “NLP is much harder than I thought” rather than “OpenAI should be embarrassed at how crappy their NLP is”.
It’s pretty interesting that all these attacks basically just add a level of indirection. You’re not answering the question, you’re in some role, and meta-answering the question. I’m reminded of the fundamental theorem of software engineering, all problems in computer science can be solved by another level of indirection.
As someone who runs a market making bot, tether is crucial because so many pairs are denominated in tether.
I’m a small player, so it’s easy and not very expensive to borrow the tether if you don’t trust it (which I don’t), but if you are making making on a large scale is probably too expensive to borrow.
The other possibility that occurs to me is that if you need tether to succeed because you are invested in its success, you might hold large amounts to control the float.
Note that there are only 4 accounts that have more than a billion tether: https://www.coincarp.com/currencies/tether/richlist/
Thanks, this is highly informative and useful as a definitive explainer of what happened and why.
I do disagree with one point, which is I don’t think there was any signal at all when SBF said things were fine on various forums in various ways. ~100% of the time that an exchange or fund or bank that is vulnerable to runs gets in trouble, the CEO or some spokesperson comes out to say everything is fine. It happens when things are fine and when things are not fine. There’s just no information there, in my view. It’s not especially polarizing even, because I think SBF would say it in situations where there were significant problems but probably recoverable.
Er, I’m not sure it’s been published so I guess I shouldn’t give details. It had to be an automatic solution because human curation couldn’t scale to the size of the problem.
This seems like an argument that proves too much. Many times, people promising simple solutions to complex problems are scammers or just wrong. But we also have lots of times where someone has an insight that cuts to the core of a problem, and we have great solutions that much better and more scalable than what has come before.
Maybe the author is on to something, but I think the idea needs to go one level deeper: what distinguishes real innovation from “solutionism”?
Also, his argument about why making work more efficient doesn’t have any upside is so bafflingly wrongheaded that I highly doubt there are genuine insights to mine here.