RSS

Jan_Kulveit

Karma: 8,595

My current research interests:

1. Alignment in systems which are complex and messy, composed of both humans and AIs?
Recommended texts: Gradual Disempowerment, Cyborg Periods

2. Actually good mathematized theories of cooperation and coordination
Recommended texts: Hierarchical Agency: A Missing Piece in AI Alignment, The self-unalignment problem or Towards a scale-free theory of intelligent agency (by Richard Ngo)

3. Active inference & Bounded rationality
Recommended texts: Why Simulator AIs want to be Active Inference AIs, Free-Energy Equilibria: Toward a Theory of Interactions Between Boundedly-Rational Agents, Multi-agent predictive minds and AI alignment (old but still mostly holds)

4. LLM psychology and sociology: A Three-Layer Model of LLM Psychology, The Pando Problem: Rethinking AI Individuality, The Cave Allegory Revisited: Understanding GPT’s Worldview

5. Macrostrategy & macrotactics & deconfusion: Hinges and crises, Cyborg Periods again, Box inversion revisited, The space of systems and the space of maps, Lessons from Convergent Evolution for AI Alignment, Continuity Assumptions

Also I occasionally write about epistemics: Limits to Legibility, Conceptual Rounding Errors

Researcher at Alignment of Complex Systems Research Group (acsresearch.org), Centre for Theoretical Studies, Charles University in Prague. Formerly research fellow Future of Humanity Institute, Oxford University

Previously I was a researcher in physics, studying phase transitions, network science and complex systems.

Role-play­ing vs Self-modelling

Jan_Kulveit7 Apr 2026 20:41 UTC
63 points
3 comments4 min readLW link

Per­sona Self-repli­ca­tion experiment

2 Apr 2026 18:18 UTC
39 points
0 comments8 min readLW link
(theartificialself.ai)

Per­sona self-repli­ca­tion experiment

2 Apr 2026 18:10 UTC
8 points
0 comments8 min readLW link

La­tent In­tro­spec­tion (and other open-source in­tro­spec­tion pa­pers)

24 Mar 2026 21:23 UTC
96 points
3 comments9 min readLW link
(arxiv.org)

Models differ in iden­tity propensities

16 Mar 2026 10:45 UTC
58 points
0 comments14 min readLW link

The Ar­tifi­cial Self

15 Mar 2026 1:37 UTC
117 points
13 comments29 min readLW link

An Align­ment Jour­nal: Com­ing Soon

3 Mar 2026 20:27 UTC
252 points
29 comments6 min readLW link
(blog.alignmentjournal.org)

Why You Don’t Believe in Xhosa Prophecies

Jan_Kulveit13 Feb 2026 2:25 UTC
261 points
27 comments4 min readLW link

Post-AGI Eco­nomics As If Noth­ing Ever Happens

Jan_Kulveit4 Feb 2026 17:39 UTC
252 points
39 comments8 min readLW link
(boundedlyrational.substack.com)

When does com­pe­ti­tion lead to recog­nis­able val­ues?

12 Jan 2026 23:13 UTC
65 points
18 comments25 min readLW link
(postagi.org)

The Eco­nomics of Trans­for­ma­tive AI

8 Jan 2026 22:22 UTC
64 points
3 comments18 min readLW link
(post-agi.org)

On­tol­ogy for AI Cults and Cy­borg Egregores

Jan_Kulveit10 Nov 2025 13:19 UTC
65 points
14 comments2 min readLW link

AISLE dis­cov­ered three new OpenSSL vulnerabilities

Jan_Kulveit30 Oct 2025 16:32 UTC
64 points
7 comments1 min readLW link
(aisle.com)

Up­com­ing Work­shop on Post-AGI Eco­nomics, Cul­ture, and Governance

28 Oct 2025 21:55 UTC
43 points
1 comment2 min readLW link

The Memet­ics of AI Successionism

Jan_Kulveit28 Oct 2025 15:04 UTC
225 points
55 comments9 min readLW link

Sum­mary of our Work­shop on Post-AGI Outcomes

29 Aug 2025 17:14 UTC
110 points
3 comments3 min readLW link

Up­com­ing work­shop on Post-AGI Civ­i­liza­tional Equilibria

21 Jun 2025 15:57 UTC
25 points
0 comments1 min readLW link

Do Not Tile the Light­cone with Your Con­fused Ontology

Jan_Kulveit13 Jun 2025 12:45 UTC
236 points
27 comments5 min readLW link
(boundedlyrational.substack.com)

Ap­ply now to Hu­man-Aligned AI Sum­mer School 2025

6 Jun 2025 19:31 UTC
28 points
1 comment2 min readLW link
(humanaligned.ai)

In­di­vi­d­ual AI rep­re­sen­ta­tives don’t solve Grad­ual Disempowerement

Jan_Kulveit4 Jun 2025 1:26 UTC
62 points
4 comments3 min readLW link