Erez Abrams

Karma: 8

Erez Abrams 18 Sep 2026 17:19 UTC
3 points
0
in reply to: Richard_Ngo’s comment on: ricraz’s Shortform

But we don’t in fact have AGI

This may be a hot take, but Astra meets my bar for AGI. I was surprised to find this to be the case, but so far I haven’t found anything in which it doesn’t perform better than my median friend. Of course it’s quite possible I’ve simply failed to try some large category of tasks in which it sucks, but take that for what you will.

and the world would look very different if we did.

As much as I disagree with his long-term conclusions, Arvind Narayanan’s work (AI as normal technology, etc.) should at least make you suspicious about whether you can use “the world hasn’t changed much from 1 year ago” as strong evidence for “we have not developed AGI in the past year”.

Erez Abrams 10 Sep 2026 4:48 UTC
1 point
0
in reply to: momom2’s comment on: Pausing AI ASAP is preferable to agreeing to pause at some future time
You say it’s clear that in practice we can’t do “specified future”, but this is not at all clear to me. In particular, it seems clear that in practice we can’t do “pause right now”, since as far as I know there’s currently no magic red button labeled “nondestructively pause specifically all training jobs” that the CEOs can ceremoniously press together at the drop of a hat; pausing training would require some internal time and effort, so something like “pause at the end of September” seems to me possible whereas “pause immediately” seems impossible. (If we then weaken “agree now to pause right now” to “agree now to pause as soon as possible”, it seems like everyone has incentive to half-ass the pausing effort, and it would be hard to tell who is acting in good faith; the easy resolution is “agree now to pause within two weeks”, which is “pause at specified future” again.)

With regards to pausing in the future: of course there is no clear Schelling point a priori, but the nice thing about Shelling points is that it’s easy to manufacture sufficiently-common knowledge and therefore easy to generate a coordination point (which can then become Schelling by virtue of its salience); if Anthropic were to say “if OpenAI commits to pausing all frontier training runs by the end of September 2026, we also commit to pausing all training runs by the end of September 2026”, then voilà, a Schelling point is born.

Erez Abrams 9 Sep 2026 6:44 UTC
3 points
−1
on: Pausing AI ASAP is preferable to agreeing to pause at some future time
I found this post initially confusing because it seems like you’re contrasting “agree right now to pause right now” against “agree right now to pause at some unspecified future time”, where from the title I expected the post to contrast “agree to pause right now” against “agree to pause at some particular future time”. In particular, the first point in particular seems like an argument against “agree right now to pause right now” and in favor of “agree right now to pause at some particular future time”. It would be helpful if you clarified your stance between all three scenarios (pause now, pause at specified future, pause at unspecified future) (either as a reply here or, ideally, by editing your post), because right now it’s still unclear to me which of these arguments were intended to apply towards each scenario.

Erez Abrams 11 Aug 2026 14:47 UTC
1 point
0
on: Prospective methods and mechanisms of motive reinforcement in LLMs
This post is absolutely phenomenal and I am genuinely baffled that there’s not an abundance of comments here. I’ll be sharing this with anyone who’ll listen...

Erez Abrams 10 Aug 2026 8:20 UTC
3 points
0
in reply to: pleuviri’s comment on: Imprecise beliefs: a tiny introduction
FWIW, it’s not really clear to me how to map the IR/higher-language analogy into this situation, but when I try to it feels more like belief functionals are the higher-level language and PDGs are a particular lower-level implementation thereof (and from this perspective it doubly makes sense why the map isn’t injective, since there are a wide variety of ways to compile a given higher-level program into a given lower-level representation); I don’t really see how it could be the case that PDGs are less burdensome than belief functionals given that if you want to specify your belief functional with a PDG, you’re always free to do so, and given that you’re also free to do so with simpler techniques (e.g. all of the items under ”...to express beliefs” in the original post, which are IMO much simpler to express with belief functionals that aren’t PDGs).
Regardless, I do find your argument at the end (“in my view [...] had a flaw)”) quite compelling; I think we’re on the same page about belief-revision being an extremely important aspect of belief-modeling, and if there’s in fact not a clear way to do this sort of credit assignment for belief functionals then it does make them significantly less attractive than PDGs for understanding belief dynamics.

Erez Abrams 4 Aug 2026 6:09 UTC
3 points
2
in reply to: pleuviri’s comment on: Imprecise beliefs: a tiny introduction
I’d like to add another point here regarding burden. The fact that davidad didn’t make this point is surprising to me, so I may be misunderstanding something here—please let me know if so!
I want to push back on your point that “the burden of selecting a lower semi-continuous function on distributions (over which variables?) is WAY higher, since there are far more choices to make, and less standard guidance on how to make them”—to do so I will draw an analogy. Consider a problem which asks the user for a function satisfying some properties on some space (e.g. a holomorphic function on the upper half-plane): in my view, if I were to instead ask the user to give me a function on just some subspace (e.g. a holomorphic function on the upper half-disk), this is less burden, not more. But of course the latter function space is larger^[1] than the original function space . By increasing the freedom of choice, the burden is decreased, not increased. I view this as analogous to the case of belief functionals vs PDGs; asking for a belief functional is less burden than asking for a PDG, since any PDG yields a belief functional and thus the burden of a expressing a PDG is an upper bound on the burden of expressing a belief functional :)
This of course does not discount the utility of PDGs. As you note, their clearly interpretable presentation makes them a very natural^[2] way to construct belief functionals, and since they are a more structured type there are probably theorems that will be easier to prove about PDGs than about belief functionals (especially since there should be things that are provable about PDGs but are not true or even expressible for general belief functionals). Furthermore, since PDGs carry more structure than belief functionals, it is more clear how to describe/understand the dynamics of the former (such as LIR) as opposed to the latter (is there a natural way to do something like LIR in this context?). But it does seem to me that belief functionals are in some sense more natural / more fundamental than PDGs (on account of being a broader/less-structured/less-artificial type), so I hope that the two can inform each other: PDGs are fertile ground for interesting belief dynamics, and the theoretical constraints of working at the belief functional level might help prune those dynamics down to the most essential ones!
1. ^
  In the sense that all functions on are also functions on by inclusion/restriction, but not all functions on can be extended to all of
2. ^
  I’m not sure about “the natural way” though, especially given that it looks like the map from PDGs to belief functionals is non-injective!