beren(Beren Millidge)

RLHF does not ap­pear to differ­en­tially cause mode-collapse

Against ubiquitous al­ign­ment taxes

Ad­den­dum: ba­sic facts about lan­guage mod­els dur­ing training

Ba­sic facts about lan­guage mod­els dur­ing training

Val­ida­tor mod­els: A sim­ple ap­proach to de­tect­ing goodharting

Em­pa­thy as a nat­u­ral con­se­quence of learnt re­ward models

AGI will have learnt util­ity functions

Gra­di­ent hack­ing is ex­tremely difficult

Scal­ing laws vs in­di­vi­d­ual differences

Ba­sic Facts about Lan­guage Model Internals

An ML in­ter­pre­ta­tion of Shard Theory

The ul­ti­mate limits of al­ign­ment will de­ter­mine the shape of the long term future

Ev­i­dence on re­cur­sive self-im­prove­ment from cur­rent ML

Hu­man sex­u­al­ity as an in­ter­est­ing case study of alignment

[In­terim re­search re­port] Tak­ing fea­tures out of su­per­po­si­tion with sparse autoencoders

De­con­fus­ing Direct vs Amor­tised Optimization

The Sin­gu­lar Value De­com­po­si­tions of Trans­former Weight Ma­tri­ces are Highly Interpretable

Cur­rent themes in mechanis­tic in­ter­pretabil­ity research

In­ter­pret­ing Neu­ral Net­works through the Poly­tope Lens