beren(Beren Millidge)

Karma: 1,975

Interested in many things. I have a personal blog at https://www.beren.io/

The Singular Value Decompositions of Transformer Weight Matrices are Highly Interpretable

beren and Sid Black

28 Nov 2022 12:54 UTC

183 points

31 comments31 min readLW link

Gradient hacking is extremely difficult

beren24 Jan 2023 15:45 UTC

146 points

19 comments5 min readLW link

Basic Facts about Language Model Internals

beren and Eric Winsor

4 Jan 2023 13:01 UTC

122 points

18 comments9 min readLW link

Deep learning models might be secretly (almost) linear

beren24 Apr 2023 18:43 UTC

98 points

20 comments4 min readLW link

Deconfusing Direct vs Amortised Optimization

beren2 Dec 2022 11:30 UTC

92 points

14 comments10 min readLW link

Basic facts about language models during training

beren21 Feb 2023 11:46 UTC

85 points

14 comments18 min readLW link

Scaffolded LLMs as natural language computers

beren12 Apr 2023 10:47 UTC

78 points

9 comments11 min readLW link

The surprising parameter efficiency of vision models

beren8 Apr 2023 19:44 UTC

73 points

28 comments4 min readLW link

The Computational Anatomy of Human Values

beren6 Apr 2023 10:33 UTC

63 points

30 comments30 min readLW link

Against ubiquitous alignment taxes

beren6 Mar 2023 19:50 UTC

56 points

10 comments2 min readLW link

The case for removing alignment and ML research from the training dataset

beren30 May 2023 20:54 UTC

46 points

8 comments5 min readLW link

Scaling laws vs individual differences

beren10 Jan 2023 13:22 UTC

42 points

21 comments7 min readLW link

Empathy as a natural consequence of learnt reward models

beren4 Feb 2023 15:35 UTC

38 points

27 comments13 min readLW link

Human sexuality as an interesting case study of alignment

beren30 Dec 2022 13:37 UTC

38 points

26 comments3 min readLW link

An ML interpretation of Shard Theory

beren3 Jan 2023 20:30 UTC

38 points

5 comments4 min readLW link

The ultimate limits of alignment will determine the shape of the long term future

beren2 Jan 2023 12:47 UTC

34 points

2 comments6 min readLW link

Orthogonality is expensive

beren3 Apr 2023 10:20 UTC

33 points

8 comments3 min readLW link

AGI will have learnt utility functions

beren25 Jan 2023 19:42 UTC

33 points

3 comments13 min readLW link

Evidence on recursive self-improvement from current ML

beren30 Dec 2022 20:53 UTC

31 points

12 comments6 min readLW link

Addendum: basic facts about language models during training

beren6 Mar 2023 19:24 UTC

22 points

2 comments5 min readLW link