Amalthea

Karma: 873

Amalthea 14 Sep 2026 3:26 UTC
1 point
0
in reply to: alkjash’s comment on: Brand New AI Solves a Millennium Prize
You might indeed have different norms and it depends a bit on the circumstances. If you say already knew how to do it (in the sense of having developed the idea somewhat) before talking to your colleague, and you just start working it out more to see if you’re correct given the information from them it seems fine.
I do think it’s in pretty bad taste if you didn’t know beforehand you were able to do it, and you only tried hard after it became clear that it’s probably kind of doable due to their work (and you know say that you’re more overall capable/have more spare resources than your colleague, so you fell confident you can beat them). I agree that Bubeck’s behavior might have been even worse, but hard to know for sure and less squarely on OpenAI as such.

Amalthea 13 Sep 2026 23:03 UTC
3 points
0
in reply to: alkjash’s comment on: Brand New AI Solves a Millennium Prize
They were broadly gesturing at taking the general norms of the math community seriously and signaling deference and respect. One specific norm I’m thinking of is of not racing towards something just because someone else’s ongoing work has indicated a solution is within reach (admittedly not an absolute norm especially with certain very competitive people, but doing this is at least bad taste).
My main issue is that they were simply dishonest by playing up their respect for the community (which we now know was at best not deep-rooted). There was always going to be some friction with the math community just because the AI is getting better and mathematicians were going to feel the effects of this in some way—so I’m not sure there’d been a way to handle this without upsetting the math community in some way.
E.g. I don’t have any problems with Anthropic’s actions towards the math community in particular. (I personally find the employees using internal models to prove stuff obnoxious, but am overall less sure how to evaluate this)

Amalthea 13 Sep 2026 21:29 UTC
3 points
3
in reply to: alkjash’s comment on: Brand New AI Solves a Millennium Prize
I also rather they’d care about alignment, but I don’t think upholding some norm gives you some moral clearance to break it in cases where you really want to—and if that happens you were probably not really caring about the norm so much!
E.g. one alternative would have been to straightforwardly tell mathematicians that they won’t hold back publishing AI results, and understand that this may upset the community. This would’ve still upset the community, but in a more “honest” way.

Amalthea 13 Sep 2026 17:06 UTC
−5 points
−4
in reply to: alkjash’s comment on: Brand New AI Solves a Millennium Prize
If OpenAI hadn’t pretended they cared about norms, this would arguably be less bad on their part. (And mathematicians would have reasonably be upset earlier)

Amalthea 11 Sep 2026 18:39 UTC
5 points
2
in reply to: MichaelDickens’s comment on: strawberry calm’s Shortform
I think you can reasonably not want to be associated with Elmore, without that being an issue of her integrity. (I haven’t seen any good argument that her integrity is in question?)

Amalthea 11 Sep 2026 12:08 UTC
1 point
0
in reply to: Cleo Nardo’s comment on: strawberry calm’s Shortform
I’m not sure what Coxon’s reasoning was, but the effect on the public might be due to something like this:
If you’re working on safety, it’s already likely you’re kind of concerned, and if you quit it’s not clear whether that’s because the situation is bad or because you’re too scrupulous.
If you’re working on capabilities, then instead you’re most likely not the type to worry so much about the situation, so if you ring the alarm, it must really be pretty serious.
Note that this is kind of dumb, but I’d guess this models the situation pretty well, and I think this effect is kind of predictable if you’re sufficiently attuned.

Amalthea 9 Sep 2026 15:51 UTC
1 point
0
in reply to: MichaelLowe’s comment on: anaguma’s Shortform
It’s about who contributed significantly to the final solution in terms of developing the necessary mathematical ideas (and I believe it’s unclear what to do if an AI did all the intellectual labor).
This is not something you can hand out. Clearly there’s enough ambiguity so that different reasonable arrangements can benefit different people, so it’s still possible to be more or less respectful when you go about creating outputs, but generosity is really the wrong frame. (In some cases, like if you’re an advisor, and you invite your student to help on your work it could apply)

Amalthea 9 Sep 2026 5:27 UTC
4 points
1
in reply to: alkjash’s comment on: anaguma’s Shortform
If the pressure tactics described by Buckmaster here are true, I think this is a really bad showing of OpenAI and Bubeck should reasonably face consequences. It doesn’t seem like either party is super trustworthy though, so it’s unclear whom to believe.

Amalthea 9 Sep 2026 5:23 UTC
1 point
0
in reply to: David Matolcsi’s comment on: anaguma’s Shortform
I don’t think generous is the right word here. It’s not like they did anything particularly impressive themselves. Bringing a bigger excavator to dig for the gold that everyone suspects is there is hardly an achievement.

Amalthea 25 Aug 2026 23:58 UTC
1 point
0
in reply to: kman’s comment on: Wei Dai’s Shortform
Is there anything you’d like Eliezer to say? He obviously doesn’t feel happy about it.

Amalthea 25 Aug 2026 15:11 UTC
9 points
11
in reply to: Mo Putera’s comment on: Mo Putera’s Shortform
His secret sauce is presumably being an Anthropic employee with infinite Claude access.

Amalthea 24 Aug 2026 14:57 UTC
1 point
0
in reply to: sarahconstantin’s comment on: sarahconstantin’s Shortform
I think this kind of thing would’ve been more appropriate in the 2000s or so, when the issue at stake was somewhat less that the powerful people are just going to shove their preferences down everyone’s throat, norms be damned, and there was still more civility going on.

Amalthea 24 Aug 2026 14:48 UTC
3 points
2
in reply to: sarahconstantin’s comment on: sarahconstantin’s Shortform
What’s missing in the discourse is capitalists making the case that it’s none of your business whether you like it or not.
I think this is really poorly phrased, and could reasonably be read as tone-deaf and outright contemptuous.

Amalthea 24 Aug 2026 5:28 UTC
2 points
0
in reply to: Linch’s comment on: leogao’s Shortform
I wonder if that’s upstream of why Claude likes the “secret third thing” so much.

Amalthea 14 Aug 2026 16:10 UTC
1 point
0
in reply to: sarahconstantin’s comment on: What just happened? A retrospective of AI alignment
A lot of people I respect don’t believe in “godlike” AI at all, so I am confused there.
Can you name 5? Who also have clearly thought about the topic in a serious way? (E.g. not Tyler Cowen)
Or is the issue with “godlike”, but they’d be fine with “wildly superhuman”.

Amalthea 4 Aug 2026 3:17 UTC
1 point
0
in reply to: testingthewaters’s comment on: sam’s Shortform
It seemed to me that OpenAI have been somewhat intentionally fostering this (starting with providing early access to famous mathematicians and shopping any results around on twitter back when the models weren’t good enough to one shot significant stuff)

Amalthea 21 Jul 2026 20:24 UTC
2 points
−3
in reply to: Raemon’s comment on: ryan_greenblatt’s Shortform
My filter here is that Cowen isn’t trying to be serious based on his past communications and outright dismissal of certain positions. If you take 10 random economists, I’d expect that maybe ⁹⁄₁₀ or so would be happy to properly engage on hypothetical scenarios.
It’s also a simple issue of sampling from the most extreme end of a distribution (extremely opinionated + high visibility) rather than from the distribution itself.

Amalthea 21 Jul 2026 17:56 UTC
12 points
10
in reply to: ryan_greenblatt’s comment on: ryan_greenblatt’s Shortform
Please don’t overly index on Tyler Cowen. He’s been clearly not trying very hard to be correct on things in this area. There’s probably better examples of economists who have bad takes and are more serious about it.

Amalthea 21 Jun 2026 17:46 UTC
1 point
0
in reply to: Thomas Kwa’s comment on: Aaron_Scher’s Shortform
I don’t see how “nontrivial to enforce a pause” follows (beyond it requiring some amount of international coordination at some point which is unrelated to your point).
It was my impression that the continued progress in e.g. Moore’s law requires a comparable steady increase in investment and R&D spending, so if the government does a large scale intervention on the inputs, this could effectively halt progress in many ways that matter. (Some forms of algorithmic progress seem indeed hard to halt without aggressive enforcement)

Amalthea 16 Jun 2026 17:04 UTC
1 point
0
in reply to: jacquesthibs’s comment on: Joseph Miller’s Shortform
Last time I mentored (for SPAR), I had to turn down a lot of folks who were more accomplished and likely to succeed (e.g. professors) so that I could accept those I felt would have more long-term potential to contribute to safety.
Wouldn’t professors be more likely to only apply if they genuinely care about safety, while younger people (overall) might be still figuring things out and happy for any opportunity to advance their career?