Andrii Vasylenko

Karma: 127

Andrii Vasylenko 5 Sep 2026 7:28 UTC
1 point
0
in reply to: cousin_it’s comment on: Cat-Belling Problems
What a bottleneck is, is a way to reduce the amorphous difficulty of a problem to something specific; that to solve the problem you have to figure out how to overcome this particular obstacle, or that by the laws of the universe a solution must take this particular shape. It’s not a luxury you always have, with hard problems, but it sure does help.

Andrii Vasylenko 11 Aug 2026 8:21 UTC
10 points
2
on: Redux: (∃ Stochastic Natural Latent) Implies (∃ Deterministic Natural Latent)
I will not be providing the proof in prose in this post, as it is not suitable for even impolite human company, but it sure does compile and comes out the other side with a machine-certified proof of what sure looks to be an even stronger correctly-expressed statement than the one I was originally aiming for.
It seems plausible to me that the the thing the AI proved turns out not to be the thing that you intended to prove. In my experience, this is absolutely the sort of thing that can happen, even when the proof is relatively simple and comprehensible.

Andrii Vasylenko 10 Aug 2026 21:19 UTC
4 points
1
in reply to: jimrandomh’s comment on: Jimrandomh’s Shortform Posts
I don’t think “amount of time a human is looking at the reasoning logs” is the sort of thing that could be usefully worked into an international agreement. It’s very easily gameable, for one.

Andrii Vasylenko 12 Jul 2026 6:55 UTC
4 points
0
in reply to: Eva Lu’s comment on: evalu’s Shortform
Not enough though, I think. “smart people are attracted into unproductive directions because they stick to what’s memetic, avoid conflicting with allies, and are too panicked to think carefully” does describe many people in the EA-sphere. More is possible.

Andrii Vasylenko 12 Jul 2026 6:46 UTC
1 point
−1
in reply to: elifland’s comment on: elifland’s Shortform
I think Plan A is significantly worse than Plan S.
I am uncertain that even the best versions of the sorts of control/superficial alignment techniques portrayed in the plan will be sufficient to make sure nothing catastrophic happens, when “Top-Expert-Dominating AI” is on the table.
And it does not seem to me that every time, across years and dozens of companies, the best possible versions of said alignment techniques will be implemented. It seems very plausible that something of the flavor of Anthropic and OpenAI training against the CoT will happen, except much more dangerous, since “Top-Expert-Dominating AI”.
I do not think that scaling to what Plan A portrays, will speed up the time to “alignment is solved” sufficiently to outweigh the risk; in my experience, the sorts of things current AIs are good at, or are on track to become good at, are not the limiting factor in alignment research, especially the sorts of exceptional alignment research that bring us substantially closer to “alignment is solved”.
I agree that deal breakdown is a big problem, one that people should be working on post-pause. I do not think getting closer to the edge of existentially dangerous capabilities helps with that.^[1]
1. ^
  If anything, it might make a deal weaker, since it goes past a natural Schelling fence.

Andrii Vasylenko 5 Jul 2026 1:25 UTC
3 points
4
in reply to: J Bostock’s comment on: Jemist’s Shortform
We do not currently know the laws that apply to superintelligent systems. It took centuries to discover e.g. the laws of motion and gravity. And it is substantially harder in the case of superintelligence, since we cannot run experiments on superintelligences.

Andrii Vasylenko 3 Jul 2026 21:56 UTC
2 points
0
in reply to: Linch’s comment on: Linch’s Shortform
I think “extend the amount of useful inference you can do on question X” is the wrong frame. To me, thinking more about a topic usually feels like working through different-but-related subquestions.

Andrii Vasylenko 23 Jun 2026 22:16 UTC
1 point
0
in reply to: Steven Byrnes’s comment on: steve2152′s Shortform
I think LLMs are (and will always be) worse than humans at the kind of research involved in inventing a new AI paradigm. I think they’ll be helpful in the same way that PyTorch and GitHub and arXiv are helpful, but not involved in the way that’s similar to massively multiplying the quantity or quality of intellectual labor working on the problem.
I basically agree with that description, as applied to the LLMs of 2026. However, I think that there’s a smooth-ish path in design space from here to actually-dangerous AGI. It seems plausible to me that the AI companies will be able to follow that path to its destination over the course of several years.

Andrii Vasylenko 20 Jun 2026 4:19 UTC
1 point
0
in reply to: Dean Valentine’s comment on: lc’s Shortform
What do you mean, specifically, by “the enlightenment”?

Andrii Vasylenko 19 Jun 2026 22:42 UTC
1 point
0
on: The one-week sprint
I’ve had benefit in doing this sort of thing over the course of a day.

Andrii Vasylenko 19 Jun 2026 22:01 UTC
5 points
0
in reply to: Morpheus’s comment on: Morpheus’s Shortform
See also

Andrii Vasylenko 18 Jun 2026 6:50 UTC
1 point
0
in reply to: Caleb Biddulph’s comment on: CBiddulph’s Shortform
I think a lot of conceptual progress needs to happen before we’re remotely close to the kind of implementation details that current AIs can meaningfully help with. So I don’t think making experiments easier to run would substantially advance alignment.

Andrii Vasylenko 17 Jun 2026 3:50 UTC
6 points
1
on: Scaling Hypothesis #2: Are Humans Just More Over-Parameterized?
and by giving true generalization, provide a sturdy foundation for AI safety in the form of useful NNs which are aligned & safe for the right reasons.
I doubt this. High capabilities are at least somewhat an attractor basin, which makes them possible to target using tools like GD. There is no corresponding attractor at the particular utility function we want the AI to have, so I think there would be a lot of gotchas with trying to learn it using GD.

Andrii Vasylenko 16 Jun 2026 3:40 UTC
1 point
0
in reply to: Gretta Duleba’s comment on: Gretta Duleba’s Shortform
Huh, I always got the feeling from MIRI that “alignment is really hard, we have to do a pause to have any chance of succeeding”. I’m kind of surprised to see that MIRI is investing effort into doing technical alignment research.

Andrii Vasylenko 16 Jun 2026 3:29 UTC
4 points
0
in reply to: Gretta Duleba’s comment on: Gretta Duleba’s Shortform
What prompted you to pivot? If I recall correctly you were working on comms for MIRI before this.

Andrii Vasylenko 14 Jun 2026 20:02 UTC
1 point
0
on: The Hidden Structures of Problems
5. Lead to Gold
A problem is so hard that humans aren’t even close to being smart enough or technologically advanced enough to solve it. We toil away pointlessly at trying to solve it.
I think it would help to decompose this one. Knowing that a problem is hard doesn’t help much; knowing why a problem is hard does sometimes make it easier to solve.

Andrii Vasylenko 14 Jun 2026 18:41 UTC
7 points
3
in reply to: Jan Betley’s comment on: Jan Betley’s Shortform
Even if there is some warning, I don’t expect that it will lead to that kind of costly, extreme action. And I think it is possible to shield electronics against EMPs, so I don’t think it would be effective at stopping an AGI.

Andrii Vasylenko 13 Jun 2026 3:11 UTC
1 point
0
in reply to: RogerDearnaley’s comment on: Sympathy for both sides of the egregious misalignment debate
So if LLMs scale to superintelligence, this reduces to aligning an LLM, or if they don’t, attach an LLM to whatever we end up building, so it knows what human values are (preferably with Bayesian Learning so it can Value Learn more detail), and attach an explicit goal slot so that we can explicitly make it care. AIXI with an LLM as a subroutine.
In the limit of superintelligent optimization, the things that look the best to an LLM grader are not generally the things that we value.

Andrii Vasylenko 5 Jun 2026 2:46 UTC
3 points
0
in reply to: Eye You’s comment on: Eye You’s Shortform
Some Fermi estimates:
- On the lower end, 4 words/second times 16 bits/word times 4x10^8 seconds adds up to to ~3x10^10.
- On the higher end, assuming 10^7 bits/second (about the retina-to-brain bandwidth), it adds up to ~4x10^15.
However, I think the brain is much less data- and compute-efficient than an optimal AGI algorithm would be. So I don’t think it is a good predictor of how much data future AI algorithms will require.

Andrii Vasylenko 4 Jun 2026 19:38 UTC
4 points
0
in reply to: Leon Lang’s comment on: Cole Wyeth’s Shortform
I agree. I was making a more narrow claim that post quality is one of the things that correlate pretty well with karma.