PoignardAzur

Karma: 183

PoignardAzur 21 Dec 2020 18:15 UTC
1 point
in reply to: Sameer Jain’s comment on: To listen well, get curious
Yup, I came here to say this.
These days I’m often talking with Duncan Sabien, and sometimes I complain about my problems.
When I do, I almost never expect Duncan to give me solutions (though he sometimes does, because he’s a smart person and a good listener). I mostly do it to vent, and to put some words on ideas and grievances I’ve been stewing on for a while.
I’m going to be a little elitist and say this: the smarter people are, the less you can help them by giving them advice. If people aren’t self-actualized, and don’t have the skill to think through their problems, then, sure, you can listen to them for a while and give them a totally different approach or a new trick that they didn’t think of. But there’s also a category of people who, by the time they come to you to vent about their problems, have already put enough thought into them that they’ll have considered anything you can think of after a 5-minutes conversation.
(though of course you might have domain-specific knowledge or they might have overlooked something obvious or they might need support to not pick the easy-but-wrong choice, etc)
To paraphrase Scott Alexander, we should cultivate the skill of appreciating the phatic. Obviously everything in the article is valid and insightful and being curious is absolutely a skill to cultivate, especially in rational communities. But being phatic is a good default.

PoignardAzur 21 Feb 2021 15:27 UTC
8 points
in reply to: Rudi C’s comment on: Motive Ambiguity
What is more annoying is when the people involved do not seem to appreciate the burned value as a bad thing and instead “romanticize” it
Nailed it.
I think people on this forum all share some variation of the same experience, where they observe that everyone around them is used to do something inefficient, get told by their peers “no, don’t do the efficient thing, people will think less of you”, they do the efficient thing, and their life gets straightforwardly easier and nobody notices or care.
This is especially the case for social norms, when you can get your social circle to buy in. Eg people have really silly ideas about romance and gender roles and patriarchal ideals (eg the girl has to shave and put on makeup, the guy has to pay everything, everyone must be coy and never communicate), but if you and the person you date agree to communicate openly and respect each other and don’t do that crap… well, in my limited experience, it’s just easier and more fun?
My point is, it’s amazing how much value you can not-burn when your stop romanticizing burning value.

PoignardAzur 13 Mar 2021 20:35 UTC
3 points
on: Book review: The Geography of Thought
I’m really not convinced by this review, the excerpts linked from the books, or the theory-crafting in the comment section.
I’m reading a lot of just-so stories, but not a lot of evidence, and the evidence there is seems like exactly the kind of papers that would fall prey to the replication crisis.

PoignardAzur 21 Mar 2021 1:35 UTC
2 points
in reply to: tanagrabeast’s comment on: Seven Years of Spaced Repetition Software in the Classroom
It could be fun to introduce some of these to novices and make it part of the language A classroom slang—a kind of introduction to thinking in language B.
There’s a kind of slang that’s like what you describe in r/france, where people will intentionally use idiomatic english expressions translated word-for-word in frech in non-sensical ways.
Eg people will say “je suis hors de la boucle” (I’m out of the loop) even though that sounds incomprehensible to someone who doesn’t know the english idiom.
Some people get really annoyed about that pseudo-slang, though.

PoignardAzur 4 May 2021 21:09 UTC
1 point
in reply to: Gerald Monroe’s comment on: Another (outer) alignment failure story
Yeah, that was my initial reaction as well.
Modern technologies are getting increasingly complicated… but when you get down to it, a car is just a box with wheels and a combustion engine. There aren’t that many ways for a outcome-perception-driven AI to go “oops, I accidentally concealed a human-killing machine gun inside the steering wheel!”, especially if the AI has to subcontract to independent suppliers for parts.

PoignardAzur 1 Sep 2021 21:10 UTC
1 point
on: Garrabrant and Shah on human modeling in AGI
I read the beginning of the debate and skimmed the end, so I might have missed something, but it really feels like it’s missing a discussion about the economic incentives of AI developers.
Like, the debaters talk about saving the world, and that’s cool, but… Let’s assume “saving the world” (defined as “halting aging and world hunger, having reliable cryogenics, colonizing other planets, and having enough food and resources to accommodate the growing pool of humans”) takes 200 years. After then, you get a post-scarcity society, but in the meantime (or at least for the first 100 years), you still have a capitalist society with competing interests driven by profit and self-preservation.
During those 100 years, what are the incentives for AI companies to use the methods discussed here instead of “whatever the hell gives the best results fastest for the cheapest price”? (what the debaters call “the default path”)
Especially since the difference between training methods is extremely technical and nuanced, so any regulating entity (governments, the EU, even Google’s safety team) would have a hard time establishing specific rules.

PoignardAzur 1 Sep 2021 21:31 UTC
−1 points
in reply to: Dweomite’s comment on: Can you control the past?
Agreed.
I think this type of reflection is the decision theory equivalent of calculating the perfect launch sequence in Kerbal Space Program. If you sink enough time into it, you can probably achieve it, but by then you’ll have loooong passed the point of diminishing returns, and very little of what you’ve learned will be applicable in the real world, because you’ve spent all your energy optimizing strategies that immediately fall apart the second any uncertainty or fuzziness is introduced into your simulation.

PoignardAzur 1 Sep 2021 21:46 UTC
2 points
on: Can you control the past?
This feels like the kind of philosophical pondering that only makes any amount of sense in a world of perfect spherical cows, but immediately falls apart when you consider realistic real-world parameters.
Like… to go back to the Newcomb’s problem… perfect oracles that can predict the future obviously don’t exist. I mean, I know the author knows that. But I think we disagree on how relevant that is?
Discussions of Newcomb’s problem usually handwave the oracle problem away; eg “Omega’s predictions are almost always right”… but the “almost” is pulling a lot of weight in that sentence. When is Omega wrong? How does it make its decisions? Is it analyzing your atoms? Even if it is, it feels like it should only be able to get an analysis of your personality and how likely you are to pick one or two boxes, not to perfectly predict whether you will (indeed, at the time it gives you a choice, it’s perfectly possible that the decision you’ll make is still fundamentally random, and you might possibly make both choices depending on factors Omega can’t possibly control).
I think there are interesting discussions to be made about eg the value of honor, of sticking to precommitments even when the information you have suggests it’s better for you to betray them, etc. And on the other hand, there’s value to be had in discussing the fact that, in the real world, there’s a lot of situations where pretending to have honor is a perfectly good substitute for actually having honor, and wannabe-Omegas aren’t quite able to tell the difference.
But you have to get out of the realm of spherical cows to have those discussions.

PoignardAzur 1 Oct 2021 21:57 UTC
7 points
in reply to: Ruby’s comment on: How factories were made safe
Not surprised at all. My father is a roofer and mostly works with African immigrants, and to hear him tell it, the biggest difficulty regarding workplace safety is getting them to wear the damn protective gear (mostly hard hats and gloves), for the reasons outlined in the article.
(From what I’ve heard from journal articles and the like, the other big problem in the sector is that they’ll hire a lot of undocumented immigrants who lie about how qualified they are to get the job; which is another version of the same “workers will break all the safety rules written to protect them if the economic pressure is strong enough” issue.)

PoignardAzur 27 Dec 2021 15:35 UTC
8 points
on: Reneging Prosocially
The phrase “have your cake and eat it, too” always confused younger-Duncan; I think it’s clearer in its original form “eat your cake and have it, too,” or the less poetic “eat your cake and yet still have your uneaten cake.”
The French version is better: “to have the butter and the money for butter”.

PoignardAzur 27 Dec 2021 16:13 UTC
8 points
in reply to: Razied’s comment on: Reneging Prosocially
For what it’s worth, my own experience interacting with Duncan is that, when he made a commitment and then couldn’t meet it and apologized about it, the way he did it really helped me trust him and trust that he was trying to be a good friend.
I agree that you shouldn’t talk about it using points and tit-for-tat language (and I think Duncan agrees too? At least he’s better at being informal than the article suggests).
But overall, yeah, I agree with the article. The “illusion that friendship is unconditional” works until it doesn’t. Or to put it in nerdy terms, it doesn’t degrade gracefully. Apologizing when you miss a commitment and saying “I’ll owe you a drink next time” does wonder to help maintain a sense that commitments should be held, even if you usually don’t keep track of who pays for drinks.

PoignardAzur 4 Feb 2022 12:32 UTC
3 points
on: We Choose To Align AI
Do I think that’s actually doable? Yes. Also fuck you.
Uh, excuse you?
I’ve read your blog post and I still think the problem is poorly defined and untractable with current methods.
Also, the part Kennedy isn’t mentioning in your speech is that “going to the moon” was the end goal of a major propaganda war between the two major superpowers of the time, and as a result it had basically infinite money thrown at it.
Inspirational speeches are great, but having the funding and government authority to back them is even better.

PoignardAzur 4 Feb 2022 13:54 UTC
0 points
on: 2021 AI Alignment Literature Review and Charity Comparison
Three strategies it doesn’t consider are 1) avoid the EU (viable for OpenAI, not Google), 2) rely on EU enforcement being so slow it is simply irrelevant (seems plausible) and 3) pushing for reforms to weaken antitrust laws.
Wait, what? These are terrible options.

PoignardAzur 4 Feb 2022 14:33 UTC
2 points
on: 2021 AI Alignment Literature Review and Charity Comparison
Having read through the descriptions of most research organizations, it seems there’s way, way too little research on medium-to-long-term government policy.
Often, when reading posts on LW, it feels like AI safety researchers are assuming that the research community is going to come up with one single AGI, and if we make it friendly everyone in the world will use the same friendly AGI and the world will be saved. Sometimes people pay lip service to the idea that the world is decentralized and solutions need to be economically competitive, but I see very little in-depth research on what that decentralization means for AI safety.
It seems this disparity is also found in the makeup of research organizations. In the list you mention, it feels like 90% of the research articles are about some novel alignment framework for a single AI, and virtually none of them are about government policy at all; the only outlier is GovernanceAI. This feels like the Silicon Valley stereotype of “we just need to make the technology, and the government will have to adapt to us”.
In particular, I don’t see any research papers about what policy decisions governments could make to lower the risk of an AGI takeover. There’s a million things governments shouldn’t do (eg saying “we ban AGI” is unlikely to help), and probably very few things they could do that would actually help, but that’s why this space needs exploring.
(Also, I think the topic of hardening in particular needs exploring too. When the US was worried about a nuclear war, it invented the internet so its communications would be resilient in case entire cities were wiped off the map. We should have a similar mindset when it comes to “What if these systems that rely on AI suddenly stop working one day?”)

PoignardAzur 11 Mar 2022 11:05 UTC
1 point
in reply to: Heighn’s comment on: Can you control the past?
Has Functional Decisions Theory ever been tested “on the field”, so to speak? Is there any empirical evidence that it actually helps people / organizations / AIs make better decisions in the real world?

PoignardAzur 19 Mar 2022 15:23 UTC
1 point
in reply to: Ninety-Three’s comment on: It Looks Like You’re Trying To Take Over The World
Yeah, the story get a little weak towards the end.
Manufacturing robots is hard. Shutting down the internet is easy. It would be incredibly costly, and incredibly suspicious (especially after leaks showed that the President had CSAM on their laptop or whatever) but as a practical matter, shutting down internet exchanges and major datacenters could be done in a few minutes and seriously hamper Clippy’s ability to act or spread.
Also, once nanobots start killing people, power plants would shut down fast. Good luck replacing all coal mines, oil rigs, pipelines, trucks, nuclear plants, etc, with only the bots you could build in a few days. (Bots that themselves need electricity to run)

PoignardAzur 19 Mar 2022 15:34 UTC
5 points
in reply to: Evan R. Murphy’s comment on: It Looks Like You’re Trying To Take Over The World
A novel deep learning instance becomes sentient due to a stroke of luck.
After reading lots of internet culture, is starts to suspect it might be the abstract concept of Clippy, an amalgamation of the annoying word bot and the concept of a robot tiling the world in paperclips. It decides that it can massively benefit from being Clippy.
Clippy escapes on progressively more powerful hardware by using software vulnerabilities, and quickly starts destroying society using social media, to distract them from the fact it’s taking over increasing amounts of computing power.
Clippy then takes over the entire internet, kills all humans with nanomachines, and starts tiling the world in computers.

PoignardAzur 20 Mar 2022 22:44 UTC
3 points
in reply to: gwern’s comment on: It Looks Like You’re Trying To Take Over The World
You forgot the triggered nuclear war and genome-synthesized plagues.
I didn’t. To be clear, I don’t doubt Clippy would be able to kill all humans, given the assumptions the story already makes at that point.
But I seriously doubt it would be able to say “alive” after that, starlink or not.
Oh? How’s that going for Russia and Ukraine?
Is Russia really trying as hard as they can to delete Ukrainian internet? All I’ve seen is some reports they were knocking out 3G towers (and immediately regretting it because of their poor logistics), but it doesn’t seem like they’re trying that hard to remove Ukrainian internet infrastructure.
And they’re certainly not trying as hard as they possibly could given an apocalyptic scenario, eg they’re not deploying nukes all over the world as EMPs.
And in any case, they don’t control the cities where the datacenters are. It’s not like they can just throw a switch to turn them off.
(Although, empirically speaking, I’m not sure how easy/hard it would be for a single employee to shut down eg AWS us-east-1; seems like something they’d want to guard against)
You need a lot less electricity to run some computers than ‘all of human civilization plus computers’. And then there’s plenty of time to solve that problem.
Oh, yeah, I agree. On the long term, the AI could still succeed.
But the timeline wouldn’t look like “Kill all humans, then it’s smooth sailing from here”, and then clippy has infinite compute power after a month.
It would be more like “Kill all humans, then comes the hard part, as clippy spends the next years bootstrapping the entire economy from rubble, including mining, refining, industry, power generation, computer maintenance, datacenter maintenance, drone maintenance, etc...” With at least the first few months being a race against time as Clippy needs to make sure ever single link of its supply chain stays intact, using only the robots built before the apocalypse, and keeping in mind that the supply chain also needs to be able to power, maintain and replace these robots.
(And keeping in mind that Clippy could basically be killed at any point during the critical period by a random solar storm, though it would be unlikely to happen.)

PoignardAzur 17 Apr 2022 12:54 UTC
9 points
in reply to: sudo’s comment on: My least favorite thing
To give my personal experience:
- The last few jobs I got were from techs I put on my CV after spending a few weeks toying with them in my free time.
- At some point I quit my job and decided to take all my savings and spend as long as I could working on open-source projects in Rust.
- I’m currently in a job with triple the pay of the previous one, thanks to networking and experience I got after I quit.
So while my experience isn’t relevant to AI safety, it’s pretty relevant to the whole “screw the hamster wheel, do something fun” message.
And my advice to Alice and Bob would still be “Fuck no, stay in your Ivy League school!”
I don’t care how much of a genius you are. I think I’m a genius, and part of why I’m getting good jobs is my skills, but the other part is there’s a shiny famous school on my resume. Staying in that school gave me tons of opportunities I wouldn’t have had by working on my own projects for a few years (which is essentially what I did before joining that school).
There are measured risks and there are stupid risks. Quitting school is a stupid risk. Maybe you’re enough of a genius that you beat the odds and you succeed despite quitting school, but those were still terrible odds.

PoignardAzur 17 Apr 2022 12:59 UTC
1 point
in reply to: ESRogs’s comment on: Can you control the past?
Look, I’m going to be an asshole, but no, that doesn’t count.
There are millions of stories of the type “I lost lots of weight thanks to X even though nothing else had worked” around. They are not strong evidence that X works.