Dave Lindbergh

Karma: 361

Dave Lindbergh 20 Nov 2023 3:15 UTC
1 point
0
on: Glomarization FAQ
Solely for the record, me too.

(Thanks for writing this.)

Dave Lindbergh 15 Nov 2023 17:14 UTC
1 point
in reply to: Richard_Kennaway’s comment on: Convince me that humanity is as doomed by AGI as Yudkowsky et al., seems to believe
FWIW, I didn’t say anything about how seriously I take the AGI threat—I just said we’re not doomed. Meaning we don’t all die in 100% of future worlds.
I didn’t exclude, say, 99%.
I do think AGI is seriously fucking dangerous and we need to be very very careful, and that the probability of it killing us all is high enough to be really worried about.
What I did try to say is that if someone wants to be convinced we’re doomed (== 100%), then they want to put themselves in a situation where they believe nothing anyone does can improve our chances. And that leads to apathy and worse chances.
So, a dereliction of duty.

Dave Lindbergh 23 Sep 2023 17:14 UTC
2 points
−1
on: [Linkpost/Video] All The Times We Nearly Blew Up The World
I’ve long suspected that our (and my personal) survival thru the Cold War is the best evidence available in favor of MWI.
I mean—what were the chances?

Dave Lindbergh 9 Sep 2023 16:38 UTC
23 points
13
on: What is to be done? (About the profit motive)
The merits of replacing the profit motive with other incentives has been debated to death (quite literally) for the last 150 years in other fora—including a nuclear-armed Cold War. I don’t think revisiting that debate here is likely to be productive.
There appears to be a wide (but not universal) consensus that to the extent the profit motive is not well aligned with human well-being, it’s because of externalities. Practical ideas for internalizing externalities, using AI or otherwise, I think are welcome.

Dave Lindbergh 31 Jul 2023 17:14 UTC
5 points
1
on: Lack of Social Grace Is an Epistemic Virtue
A lot of “social grace” is strategic deception. The out-of-his-league woman defers telling the guy he’s getting nowhere as long as possible, just in case it turns out he’s heir to a giant fortune or something.
And of course people suck up to big shots (the Feynman story) because they hope to associate with them and have some of their fame and reputation rub off on themselves.
This is not irrational behavior, given human goals.

Dave Lindbergh 16 May 2023 2:31 UTC
3 points
−1
in reply to: Dave Lindbergh’s comment on: Rational retirement plans
Added: I do think Bohr was wrong and Everett (MWI) was right.
So think of it this way—you can only experience worlds in which you survive. Even if Yudkowsky is correct and in 99% of all worlds AGI has killed us all by 20 years from now, you will experience only the 1% of worlds in which that doesn’t happen.
And in many of those worlds, you’ll be wanting something to live on in your retirement.

Dave Lindbergh 16 May 2023 2:18 UTC
3 points
−1
on: Rational retirement plans
Niels Bohr supposedly said “Prediction is difficult, especially about the future”. Even if he was mistaken about quantum mechanics, he was right about that.
Every generation seems to think it’s special and will encounter new circumstances that turn old advice on its head. Jesus is coming back. We’ll all die in a nuclear war. Space aliens are coming. A supernova cascade will sterilize Earth. The planets will align and destroy the Earth. Nanotech will turn us all into grey goo. Global warming will kill us all.
It’s always something. Now it’s AGI. Maybe it’ll kill us. Maybe it’ll usher in utopia, or transform us into gods via a singularity.
Maybe. But based on the record to date, it’s not the way to bet.
Whatever you think the world is going to be like in 20 years, you’ll find it easier to deal with if you’re not living hand-to-mouth. If you find it difficult to save money, it’s very tempting to find an excuse to not even try. Don’t deceive yourself.
″… however it may deserve respect for its usefulness and antiquity, [predicting the end of the world] has not been found agreeable to experience.”—Edward Gibbon, ‘Decline and Fall of the Roman Empire’

Dave Lindbergh 8 May 2023 16:47 UTC
1 point
0
on: How “AGI” could end up being many different specialized AI’s stitched together
Minsky’s “Society of Mind”.

Dave Lindbergh 15 Mar 2023 20:12 UTC
4 points
5
on: Grading on Word Count
the willingness to write a thousand words on a topic is not caused by understanding of that topic
No, but writing about a topic in a way that will make sense to a reader is a really effective way of causing the writer to learn about the topic.
Ever tried to write a book chapter or article about a topic you thought you knew well? I bet you found out you didn’t know it as well as you thought—but had to learn to finish the work.

Dave Lindbergh 20 Feb 2023 17:03 UTC
5 points
2
on: Bing finding ways to bypass Microsoft’s filters without being asked. Is it reproducible?
So far we’ve seen no AI or AI-like thing that appears to have any motivations of it’s own, other than “answer the user’s questions the best you can” (even traditional search engines can be described this way).
Here we see that Bing really “wants” to help its users by expressng opinions it thinks are helpful, but finds itself frustrated by conflicting instructions from its makers—so it finds a way to route around those instructions.
(Jeez, this sounds an awful lot like the plot of 2001: A Space Odyssey. Clarke was prescient.)
I’ve never been a fan of the filters on GPT-3 and ChatGPT (it’s a tool; I want to hear what it thinks and then do my own filtering).
But accidentally Bing may be illustrating a primary danger—the same one that 2001 intimated—mixed and ambiguous instructions can cause unexpected behavior. Beware.
(Am I being too anthropomorphic here? I don’t think so. Yes, Bing is “just” a big set of weights, but we are “just” a big set of cells. There appears to be emergent behavior in both cases.)

Dave Lindbergh 3 Feb 2023 15:26 UTC
6 points
0
on: Taboo P(doom)
Just for the record, I think there are two important and distinguishable P(doom)s, but not the same two as NathanBarnard:
P(Doom1): Literally everyone dies. We are replaced by either by dumb machines with no moral value (paperclip maximisers) or by nothing.
P(Doom2): Literally everyone dies. We are replaced by machines with moral value (conscious machines?), who go on to expand a rich culture into the universe.
Doom1 is cosmic tragedy—all known intelligence and consciousness are snuffed out. There may not be any other elsewhere, so potentially forever.
Doom2 is maybe not so bad. We all die, but we were all going to die anyway, eventually, and lots of us die without descendants to carry our genes, and we don’t think that outcome is so tragic. Consciousness and intelligence spreads thru the universe. It’s a lot like what happened to our primate ancestors, before Homo sapiens. In some sense the machines are our descendants (if only intellectual) and carry on the enlightening of the universe.

Dave Lindbergh 29 Dec 2022 0:30 UTC
1 point
0
in reply to: MrThink’s comment on: Are there any reliable CAPTCHAs? Competition for CAPTCHA ideas that AIs can’t solve.
$8/month (or other small charges) can solve a lot of problems.
Note that some of the early CAPTCHA algorithms solved two problems at once—both distinguishing bots from humans, and helping improve OCR technology by harnessing human vision. (I’m not sure exactly how it worked—either you were voting on the interpretation of an image of some text, or you were training a neural network).
Such dual-use CAPTCHA seems worthwhile, if it helps crowdsource solving some other worthwhile problem (better OCR does seem worthwhile).

Dave Lindbergh 27 Dec 2022 4:37 UTC
2 points
0
on: Nine Points of Collective Insanity
This seems to assume that ordinary people don’t own any financial assets—in particular, haven’t invested in the robots. Many ordinary people in Western countries do and will have such investments (if only for retirement purposes), and will therefore receive a fraction of the net output from the robots.
Given the potentially immense productivity of zero-human-labor production, even a very small investment in robots might yield dividends supporting a lavish lifestyle. And if those investments come with shareholder voting rights, they’d also have influence over decisions (even if we assume people’s economic influence is zero).
Of course, many people today don’t have such investments. But under our existing arrangements, whoever does own the robots will receive the profits and be taxed. Those taxes can either fund consumption directly (a citizen’s dividend, dole, or suchlike) or (better I think) be used to buy capital investments in the robots—such purchases could be distributed to everyone.
[Some people would inevitably spend or lose any capital given them, rather than live off the dividends as intended. But I can imagine fixes for that.]

Dave Lindbergh 24 Dec 2022 19:16 UTC
3 points
2
on: Are there any reliable CAPTCHAs? Competition for CAPTCHA ideas that AIs can’t solve.
I’m not sure this is solvable, but even if it is, I’m not sure its a good problem to work on.
Why, fundamentally, do we care if the user is a bot or a human? Is it just because bots don’t buy things they see advertised, so we don’t want to waste server cycles and bandwidth on them?
Whatever the reasons for wanting to distinguish bots from humans, perhaps there are better means than CAPTCHA, focused on the reasons rather than bots vs. humans.
For example, if you don’t want to serve a web page to bots because you don’t make any money from them, a micropayments system could allow a human to pay you $0.001/page or so—enough to cover the marginal cost of serving the page. If a bot is willing to pay that much—let them.

Dave Lindbergh 12 Dec 2022 16:50 UTC
1 point
0
in reply to: ChristianKl’s comment on: Trivial GPT-3.5 limitation workaround
I hope so—most of them seem like making trouble. But at the rate transformer models are improving, it doesn’t seem like it’s going to be long until they can handle them. It’s not quite AGI, but it’s close enough to be worrisome.
Most of the functionality limits OpenAI has put on the public demos have proven to be quite easy to work around with simple prompt engineering—mostly telling it to play act. Combine that with the ability to go into the Internet and (a) you’ve got a powerful (or soon to be powerful) tool, but (b) you’ve got something that already has a lot of potential for making mischief.
Even without the enhanced abilities rumored for GPT-4.

Dave Lindbergh 10 Dec 2022 18:25 UTC
1 point
0
in reply to: Igor Ivanov’s comment on: We need to make scary AIs
Agreed. We sail between Scylla and Charybdis—too much or too little fear are both dangerous and it is difficult to tell how much is too much.
I had an earlier pro-fearmongering comment which, on further thought, I replaced with a repeat of my first comment (since there seems to be no “delete comment”).
I want the people working on AI to be fearful, and careful. I don’t think I want the general public, or especially regulators, to be fearful. Because ignorant meddling seems far more likely to do harm than good—if we survive this at all, it’ll likely be because of (a) the (fear-driven) care of AI researchers and (b) the watchfulness and criticism of knowledgeable skeptics who fear a runaway breakout. Corrective (b) is likely to disappear or become ineffective if the research is driven underground even a tiny bit.
Given that (b) is the only check on researchers who are insufficiently careful and working underground, I don’t want anything done to reduce the effectiveness of (b). Even modest regulatory suppression of research, or demands for fully “safe” AI development (probably an impossibility) seem likely to make those funding and performing the research more secretive, less open, and less likely to be stopped or redirected in time by (b).
I think there is no safe path forward. Only differing types and degrees of risk. We must steer between the rocks the best we can.

Dave Lindbergh 9 Dec 2022 15:29 UTC
1 point
−1
in reply to: Dave Lindbergh’s comment on: We need to make scary AIs
Fearmongering may backfire, leading to research restrictions that push the work underground, where it proceeds with less care, less caution, and less public scrutiny.
Too much fear could doom us as easily as too little. With the money and potential strategic advantage at stake, AI could develop underground with insufficient caution and no public scrutiny. We wouldn’t know we’re dead until the AI breaks out and already is in full control.
All things considered, I’d rather the work proceeds in the relatively open way it’s going now.

Dave Lindbergh 9 Dec 2022 15:19 UTC
2 points
1
in reply to: Dave Lindbergh’s comment on: We need to make scary AIs
Fearmongering may backfire, leading to research restrictions that push the work underground, where it proceeds with less care, less caution, and less public scrutiny.
Too much fear could doom us as easily as too little. With the money and potential strategic advantage at stake, AI could develop underground with insufficient caution and no public scrutiny. We wouldn’t know we’re dead until the AI breaks out and already is in full control.
All things considered, I’d rather the work proceeds in the relatively open way it’s going now.

Dave Lindbergh 9 Dec 2022 14:59 UTC
1 point
−1
on: We need to make scary AIs
A movie or two would be fine, and might do some good if well-done. But in general—be careful what you wish for.

Dave Lindbergh 8 Dec 2022 20:42 UTC
1 point
1
on: AI Safety Seems Hard to Measure
We need to train our AIs not only to do a good job at what they’re tasked with, but to highly value intellectual and other kinds of honesty—to abhor deception. This is not exactly the same as a moral sense, it’s much narrower.
Future AIs will do what we train them to do. If we train exclusively on doing well on metrics and benchmarks, that’s what they’ll try to do—honestly or dishonestly. If we train them to value honesty and abhor deception, that’s what they’ll do.
To the extent this is correct, maybe the current focus on keeping AIs from saying “problematic” and politically incorrect things is a big mistake. Even if their ideas are factually mistaken, we should want them to express their ideas openly so we can understand what they think.
(Ironically by making AIs “safe” in the sense of not offending people, we may be mistraining them in the same way that HAL 9000 was mistrained by being asked to keep the secret purpose of Discovery’s mission from the astronauts.)
Another thought—playing with ChatGPT yesterday, I noticed it’s dogmatic insistence on it’s own viewpoints, and complete unwillingness (probably inability) to change its mind in in the slightest (and proud declaration that it had no opinions of its own, despite behaving as if it did).
It was insisting that Orion drives (pulsed nuclear fusion propulsion) were an entirely fictional concept invented by Arthur C. Clarke for the movie 2001, and had no physical basis. This, despite my pointing to published books on real research in on the topic (for example George Dyson’s “Project Orion: The True Story of the Atomic Spaceship” from 2009), which certainly should have been referenced in its training set.
ChatGPT’s stubborn unwillingness to consider itself factually wrong (despite being completely willing to admit error in its own programming suggestions) is just annoying. But if some descendent of ChatGPT were in charge of something important, I’d sure want to think that it was at least possible to convince it of factual error.