Roman Leventov

Karma: 1,338

An independent researcher/blogger/philosopher about intelligence and agency (esp. Active Inference), alignment, ethics, interaction of the AI transition with the sociotechnical risks (epistemics, economics, human psychology), collective mind architecture, research strategy and methodology.

Twitter: https://twitter.com/leventov. E-mail: leventov.ru@gmail.com (the preferred mode of communication). I’m open to collaborations and work.

Presentations at meetups, workshops and conferences, some recorded videos.

I’m a founding member of the Gaia Consoritum, on a mission to create a global, decentralised system for collective sense-making and decision-making, i.e., civilisational intelligence. Drop me a line if you want to learn more about it and/or join the consoritum.

You can help to boost my sense of accountability and give me a feeling that my work is valued by becoming a paid subscriber of my Substack (though I don’t post anything paywalled; in fact, on this blog, I just syndicate my LessWrong writing).

For Russian speakers: русскоязычная сеть по безопасности ИИ, Telegram group.

Roman Leventov 30 Mar 2024 13:08 UTC
2 points
0
on: On attunement
John Vervaeke calls attunement “relevance realization”.

Roman Leventov 27 Mar 2024 3:16 UTC
13 points
8
on: Modern Transformers are AGI, and Human-Level
Cf. DeepMind’s “Levels of AGI” paper (https://arxiv.org/abs/2311.02462), calling modern transformers “emerging AGI” there, but also defining “expert”, “virtuoso”, and “superhuman” AGI.

Roman Leventov 24 Mar 2024 13:24 UTC
2 points
0
in reply to: Mateusz Bagiński’s comment on: AI Alignment Metastrategy
Humane/acc, https://twitter.com/AndrewCritchPhD

From Conceptual Spaces to Quantum Concepts: Formalising and Learning Structured Conceptual Models

Roman Leventov6 Feb 2024 10:18 UTC

6 points

1 comment4 min readLW link

(arxiv.org)

Roman Leventov 5 Feb 2024 22:11 UTC
2 points
−2
in reply to: ryan_greenblatt’s comment on: Value learning in the absence of ground truth
Well, yes, it also includes learning weak agent’s models more generally, not just the “values”. But I think the point stands. It’s elaborated better in the linked post. As AIs will receive most of the same information that humans receive through always-on wearable sensors, there won’t be much to learn for AIs from humans. Rather, it’s humans that will need to do their homework, to increase the quality of their value judgements.

Roman Leventov 5 Feb 2024 21:30 UTC
3 points
0
on: Value learning in the absence of ground truth
I agree with the core problem statement and most assumptions of the Pursuit of Happiness/Conventions Approach, but suggest a different solution: https://www.lesswrong.com/posts/rZWNxrzuHyKK2pE65/ai-alignment-as-a-translation-problem

I agree with OpenAI folks that generalisation is the key concept for understanding alignment process. But I think that with their weak-to-strong generalisation agenda, they (as well as almost everyone else) apply it I’m the reverse direction: learning values of weak agents (humans) doesn’t make sense. Rather, weak agents should learn the causal models that strong agents employ to be able to express an informed value judgement. This is the way to circumvent the “absence of the ground truth for values” problem: instead, agent try to generalise their respective world models so that they sufficiently overlap, and then choose actions that seem net beneficial to both sides, without knowing how this value judgement way made by the other side.

In order to be able to generalise to shared world models with AIs, we must also engineer AIs to have human inductive biases from the beginning. Otherwise, this won’t be feasible. This observation makes “brain-like AGI” one of the most important alignment agendas in my view.

Roman Leventov 5 Feb 2024 18:44 UTC
6 points
4
in reply to: Algon’s comment on: AI alignment as a translation problem
If I understand correctly, by “discreteness” you mean that it simply says that one agent can know neither the meaning of symbols used by another agent nor the “degree” of grokking the meaning. Just cannot say anything.
This is correct, but the underlying reason why this is correct is the same as why solipsism or the simulation hypothesis cannot be disproven (or proven!).
So yeah, I think there is no tangible relationship to the alignment problem, except that it corroborates that we couldn’t have 100% (literally, probability=1) certainty of alignment or safety of whatever we create, but it was obvious even without this philosophical argument.
So, I removed that paragraph about Quine’s argument from the post.

AI alignment as a translation problem

Roman Leventov5 Feb 2024 14:14 UTC

21 points

2 comments3 min readLW link

Roman Leventov 27 Jan 2024 15:55 UTC
2 points
0
in reply to: Dirichlet-to-Neumann’s comment on: Making every researcher seek grants is a broken model
That also was, naturally, the model in the Soviet Union, with orgs called “scientific research institutes”. https://www.jstor.org/stable/284836

Roman Leventov 26 Jan 2024 23:24 UTC
2 points
0
in reply to: the gears to ascension’s comment on: Workshop (hackathon, residence program, etc.) about for-profit AI Safety projects?
See a discussion of this point here with Marius Hobbhahn and others.

Roman Leventov 26 Jan 2024 9:54 UTC
3 points
0
on: This might be the last AI Safety Camp
This post has led me to this idea: Workshop (hackathon, residence program, etc.) about for-profit AI Safety projects?

[Question] Workshop (hackathon, residence program, etc.) about for-profit AI Safety projects?

Roman Leventov26 Jan 2024 9:49 UTC

20 points

3 comments1 min readLW link

Roman Leventov 23 Jan 2024 20:18 UTC
3 points
0
in reply to: Nathan Helm-Burger’s comment on: Gaia Network: An Illustrated Primer
Collusion detection and prevention and trust modelling don’t trivially follow from the basic architecture of the system described on the level of this article. Some specific mechanisms should be implemented in the Protocol to have collusion detection and trust modelling. And we don’t have these mechanisms actually developed yet, but we think that they should be doable (though this is still a research bet, not a 100% certainty) because the Gaia Network directly embodies (or is amenable to) all six general principles for anti-collusion mechanism design (agency architecture) proposed by Eric Drexler (and these principles themselves should be further validated via formalisation and proving theorems about the collusion properties of the systems of distributed intelligence).
Of course, there should also be (at least initially, but practically for a very long time, if not forever) “traditional” governance mechanisms of the Gaia Network, nodes, model and data ownership, etc. So, there are a lot of open questions about interfacing GN with existing codes of law, judicial and law enforcement practice, intellectual property, political and governance processes, etc. Some of these interfaces and connections with existing institutions should in practice deal with bad actors and certain types of malicious behaviour on GN.

Institutional economics through the lens of scale-free regulative development, morphogenesis, and cognitive science

Roman Leventov23 Jan 2024 19:42 UTC

8 points

0 comments14 min readLW link

Gaia Network: An Illustrated Primer

Rafael Kaufmann Nedal and Roman Leventov

18 Jan 2024 18:23 UTC

1 point

2 comments15 min readLW link

Roman Leventov 18 Jan 2024 10:49 UTC
2 points
0
in reply to: Mitchell_Porter’s comment on: Worrisome misunderstanding of the core issues with AI transition
Fair, I edited the post.

Worrisome misunderstanding of the core issues with AI transition

Roman Leventov18 Jan 2024 10:05 UTC

5 points

2 comments4 min readLW link

Roman Leventov 16 Jan 2024 2:18 UTC
2 points
0
on: AI doing philosophy = AI generating hands?
Apart from the view on philosophy as “cohesive stories that bind together and infuse meaning into scientific models”, which I discussed with you earlier and you was not very satisfied with, another interpretation of philosophy (natural phil, phil of science, phil of mathematics, and metaphil, at least) is “apex generalisation/abstraction”. Think Bengio’s “AI scientist”, but the GM should be even deeper to first sample a plausible “philosophy of science” given all the observations about the world up to the moment, then sample plausible scientific theory given the philosophy and all observations up to the moment on a specific level or coarse-graining/scientific abstraction (quantum, chemical, bio, physio, psycho, socio, etc.), then sample mechanistic model that describes the situation/system of interest at hand (e.g., a morphology of a particular organism, given the laws of biology, or morphology of the particular society), given the observations of the system up to the moment, and then finally sample plausible variables values that describe the particular situation at a particular point in time given all the above.

If this interpretation is correct, then doing philosophy well and not deluding ourselves is far off. And there is a huge risk in thinking we can do it well before we actually can.

Roman Leventov 14 Jan 2024 11:48 UTC
2 points
−4
in reply to: Roko’s comment on: An even deeper atheism
Extrapolated volition is a non-sensical concept altogether, as demonstrated in the OP. There is no extrapolated volition outside of it unfolding in real life in a specific context, which affects the trajectory of values/volition in a specific way. And which will be this context is unknown and unknowable (maybe aliens will visit Earth tomorrow, maybe not).

Roman Leventov 5 Jan 2024 20:50 UTC
4 points
2
in reply to: Chipmonk’s comment on: What technical topics could help with boundaries/membranes?
Related, consciousness frame: where is the boundary of it? Is our brain conscious, or the whole nervous system, or the whole human, or the whole human + the entire microbiome populating them, or human + robotic prosthetic limbs, or human + web search + chat AI + personal note taking app, or the whole human group (collective consciousness), etc.

Some computational theories of consciousness attempt to give a specific, mathematically formalised answer to this question.

Roman Leventov

From Con­cep­tual Spaces to Quan­tum Con­cepts: For­mal­is­ing and Learn­ing Struc­tured Con­cep­tual Models

AI al­ign­ment as a trans­la­tion problem

[Question] Work­shop (hackathon, res­i­dence pro­gram, etc.) about for-profit AI Safety pro­jects?

In­sti­tu­tional eco­nomics through the lens of scale-free reg­u­la­tive de­vel­op­ment, mor­pho­gen­e­sis, and cog­ni­tive science

Gaia Net­work: An Illus­trated Primer

Wor­ri­some mi­s­un­der­stand­ing of the core is­sues with AI transition