RSS

GPT

TagLast edit: 19 Feb 2023 2:36 UTC by Multicore

GPT (Generative Pretrained Transformer) is a family of large transformer-based language models created by OpenAI. Its ability to generate remarkably human-like responses has relevance to discussions on AGI.

External links:

GPT-3 Paper

GPT-3 Website

Col­lec­tion of GPT-3 results

Kaj_Sotala18 Jul 2020 20:04 UTC
89 points
24 comments1 min readLW link
(twitter.com)

[Question] To what ex­tent is GPT-3 ca­pa­ble of rea­son­ing?

TurnTrout20 Jul 2020 17:10 UTC
70 points
73 comments16 min readLW link

GPT-3: a dis­ap­point­ing paper

nostalgebraist29 May 2020 19:06 UTC
65 points
43 comments8 min readLW link1 review

GPT-3 Fic­tion Samples

gwern25 Jun 2020 16:12 UTC
63 points
15 comments1 min readLW link
(www.gwern.net)

Two Small Ex­per­i­ments on GPT-2

jimrandomh21 Feb 2019 2:59 UTC
54 points
28 comments1 min readLW link

$1000 bounty for OpenAI to show whether GPT3 was “de­liber­ately” pre­tend­ing to be stupi­der than it is

jacobjacob21 Jul 2020 18:42 UTC
56 points
39 comments2 min readLW link
(twitter.com)

‘This Waifu Does Not Ex­ist’: 100,000 StyleGAN & GPT-2 samples

gwern1 Mar 2019 4:29 UTC
39 points
6 comments1 min readLW link
(www.thiswaifudoesnotexist.net)

[Question] How well can the GPT ar­chi­tec­ture solve the par­ity task?

FactorialCode11 Jul 2020 19:02 UTC
19 points
3 comments1 min readLW link

[Question] How “hon­est” is GPT-3?

abramdemski8 Jul 2020 19:38 UTC
72 points
18 comments5 min readLW link

345M ver­sion GPT-2 released

lifelonglearner5 May 2019 2:49 UTC
37 points
0 comments1 min readLW link
(openai.com)

Does GPT-2 Un­der­stand Any­thing?

Douglas Summers-Stay2 Jan 2020 17:09 UTC
37 points
23 comments5 min readLW link

Repli­cat­ing the repli­ca­tion crisis with GPT-3?

skybrian22 Jul 2020 21:20 UTC
29 points
10 comments1 min readLW link

Align­ment As A Bot­tle­neck To Use­ful­ness Of GPT-3

johnswentworth21 Jul 2020 20:02 UTC
111 points
57 comments3 min readLW link

OpenGPT-2: We Repli­cated GPT-2 Be­cause You Can Too

avturchin23 Aug 2019 11:32 UTC
18 points
0 comments1 min readLW link
(medium.com)

Can you get AGI from a Trans­former?

Steven Byrnes23 Jul 2020 15:27 UTC
116 points
40 comments12 min readLW link

Devel­op­men­tal Stages of GPTs

orthonormal26 Jul 2020 22:03 UTC
140 points
72 comments7 min readLW link1 review

larger lan­guage mod­els may dis­ap­point you [or, an eter­nally un­finished draft]

nostalgebraist26 Nov 2021 23:08 UTC
260 points
31 comments31 min readLW link2 reviews

An­a­lyz­ing the Prob­lem GPT-3 is Try­ing to Solve

adamShimi6 Aug 2020 21:58 UTC
16 points
2 comments4 min readLW link

[Question] How will in­ter­net fo­rums like LW be able to defend against GPT-style spam?

ChristianKl28 Jul 2020 20:12 UTC
14 points
17 comments1 min readLW link

Hu­mans Who Are Not Con­cen­trat­ing Are Not Gen­eral Intelligences

sarahconstantin25 Feb 2019 20:40 UTC
187 points
35 comments6 min readLW link1 review
(srconstantin.wordpress.com)

Writ­ing with GPT-3

Jacob Falkovich24 Jul 2020 15:22 UTC
42 points
0 comments4 min readLW link

Are we in an AI over­hang?

Andy Jones27 Jul 2020 12:48 UTC
266 points
106 comments4 min readLW link

The Hacker Learns to Trust

Ben Pace22 Jun 2019 0:27 UTC
80 points
18 comments8 min readLW link
(medium.com)

GPT-3, be­lief, and consistency

skybrian16 Aug 2020 23:12 UTC
18 points
7 comments2 min readLW link

Scaf­folded LLMs as nat­u­ral lan­guage computers

beren12 Apr 2023 10:47 UTC
94 points
10 comments11 min readLW link

in­ter­pret­ing GPT: the logit lens

nostalgebraist31 Aug 2020 2:47 UTC
223 points
37 comments11 min readLW link

[Question] If GPT-6 is hu­man-level AGI but costs $200 per page of out­put, what would hap­pen?

Daniel Kokotajlo9 Oct 2020 12:00 UTC
28 points
30 comments1 min readLW link

[Question] Will OpenAI’s work un­in­ten­tion­ally in­crease ex­is­ten­tial risks re­lated to AI?

adamShimi11 Aug 2020 18:16 UTC
53 points
55 comments1 min readLW link

How LLMs are and are not myopic

janus25 Jul 2023 2:19 UTC
134 points
16 comments8 min readLW link

GPT-2: 6-Month Fol­low-Up

lifelonglearner21 Aug 2019 5:06 UTC
28 points
1 comment1 min readLW link

Ex­trap­o­lat­ing GPT-N performance

Lukas Finnveden18 Dec 2020 21:41 UTC
110 points
31 comments22 min readLW link1 review

[ASoT] Fine­tun­ing, RL, and GPT’s world prior

Jozdien2 Dec 2022 16:33 UTC
44 points
8 comments5 min readLW link

OpenAI an­nounces GPT-3

gwern29 May 2020 1:49 UTC
67 points
23 comments1 min readLW link
(arxiv.org)

the scal­ing “in­con­sis­tency”: openAI’s new insight

nostalgebraist7 Nov 2020 7:40 UTC
148 points
14 comments9 min readLW link
(nostalgebraist.tumblr.com)

[AN #102]: Meta learn­ing by GPT-3, and a list of full pro­pos­als for AI alignment

Rohin Shah3 Jun 2020 17:20 UTC
38 points
6 comments10 min readLW link
(mailchi.mp)

Us­ing GPT-N to Solve In­ter­pretabil­ity of Neu­ral Net­works: A Re­search Agenda

3 Sep 2020 18:27 UTC
68 points
11 comments2 min readLW link

Image GPT

Daniel Kokotajlo18 Jun 2020 11:41 UTC
29 points
27 comments1 min readLW link
(openai.com)

[April Fools] User GPT2 is Banned

jimrandomh2 Apr 2019 6:00 UTC
65 points
20 comments1 min readLW link

GPT-4 Plugs In

Zvi27 Mar 2023 12:10 UTC
198 points
47 comments6 min readLW link
(thezvi.wordpress.com)

Hiring en­g­ineers and re­searchers to help al­ign GPT-3

paulfchristiano1 Oct 2020 18:54 UTC
206 points
13 comments3 min readLW link

I wanted to in­ter­view Eliezer Yud­kowsky but he’s busy so I simu­lated him instead

lsusr16 Sep 2021 7:34 UTC
111 points
33 comments5 min readLW link

is gpt-3 few-shot ready for real ap­pli­ca­tions?

nostalgebraist3 Aug 2020 19:50 UTC
31 points
5 comments9 min readLW link
(nostalgebraist.tumblr.com)

Cyborgism

10 Feb 2023 14:47 UTC
337 points
46 comments35 min readLW link

GPT-4 Predictions

Stephen McAleese17 Feb 2023 23:20 UTC
109 points
27 comments11 min readLW link

Simulators

janus2 Sep 2022 12:45 UTC
612 points
162 comments41 min readLW link8 reviews
(generative.ink)

Can sub­marines swim?

jasoncrawford22 Feb 2023 18:48 UTC
18 points
14 comments13 min readLW link
(rootsofprogress.org)

[Question] GPT-4 and ASCII Images?

carterallen19 Mar 2023 15:46 UTC
10 points
17 comments1 min readLW link

Pre­dic­tions for GPT-N

hippke29 Jul 2020 1:16 UTC
36 points
31 comments1 min readLW link

DALL-E by OpenAI

Daniel Kokotajlo5 Jan 2021 20:05 UTC
97 points
20 comments1 min readLW link

Au­tore­gres­sive Propaganda

lsusr22 Aug 2021 2:18 UTC
25 points
3 comments3 min readLW link

Steer­ing Be­havi­our: Test­ing for (Non-)My­opia in Lan­guage Models

5 Dec 2022 20:28 UTC
40 points
19 comments10 min readLW link

[LINK] - ChatGPT discussion

JanB1 Dec 2022 15:04 UTC
13 points
8 comments1 min readLW link
(openai.com)

ChatGPT: First Impressions

specbug1 Dec 2022 16:36 UTC
18 points
2 comments13 min readLW link
(sixeleven.in)

Lan­guage Models are a Po­ten­tially Safe Path to Hu­man-Level AGI

Nadav Brandes20 Apr 2023 0:40 UTC
28 points
6 comments8 min readLW link

Jailbreak­ing ChatGPT on Re­lease Day

Zvi2 Dec 2022 13:10 UTC
242 points
77 comments6 min readLW link1 review
(thezvi.wordpress.com)

Chat GPT’s views on Me­ta­physics and Ethics

Cole Killian3 Dec 2022 18:12 UTC
5 points
3 comments1 min readLW link
(twitter.com)

Stan­ford claims to have repli­cated ChatGPT for < $600

NoSignalNoNoise21 Mar 2023 2:28 UTC
2 points
1 comment1 min readLW link
(crfm.stanford.edu)

Could an AI be Reli­gious?

mk544 Dec 2022 5:00 UTC
−12 points
14 comments1 min readLW link

Can GPT-3 Write Con­tra Dances?

jefftk4 Dec 2022 3:00 UTC
6 points
4 comments10 min readLW link
(www.jefftk.com)

Agen­tic GPT simu­la­tions: a risk and an opportunity

Yair Halberstadt22 Mar 2023 6:24 UTC
24 points
8 comments1 min readLW link

Anal­y­sis of GPT-4 com­pe­tence in as­sess­ing com­plex le­gal lan­guage: Ex­am­ple of Bill C-11 of the Cana­dian Par­li­a­ment. - Part 1

M. Y. Zuo2 Apr 2023 0:01 UTC
12 points
2 comments14 min readLW link

GTP4 ca­pa­ble of limited re­cur­sive im­prov­ing?

Boris Kashirin2 Apr 2023 21:38 UTC
2 points
3 comments1 min readLW link

[Question] ChatGTP “Writ­ing ” News Sto­ries for The Guardian?

jmh7 Apr 2023 12:16 UTC
1 point
4 comments1 min readLW link

GPTs are Pre­dic­tors, not Imitators

Eliezer Yudkowsky8 Apr 2023 19:59 UTC
404 points
91 comments3 min readLW link

OpenAI re­leases GPT-4o, na­tively in­ter­fac­ing with text, voice and vision

Martín Soto13 May 2024 18:50 UTC
54 points
23 comments1 min readLW link
(openai.com)

A crisis for on­line com­mu­ni­ca­tion: bots and bot users will over­run the In­ter­net?

Mitchell_Porter11 Dec 2022 21:11 UTC
15 points
11 comments1 min readLW link

Triv­ial GPT-3.5 limi­ta­tion workaround

Dave Lindbergh12 Dec 2022 8:42 UTC
5 points
4 comments1 min readLW link

[Question] Does GPT-4′s abil­ity to com­press text in a way that it can ac­tu­ally de­com­press in­di­cate self-aware­ness?

FinalFormal210 Apr 2023 16:48 UTC
3 points
2 comments1 min readLW link

Why Si­mu­la­tor AIs want to be Ac­tive In­fer­ence AIs

10 Apr 2023 18:23 UTC
91 points
8 comments8 min readLW link

Sparks of Ar­tifi­cial Gen­eral In­tel­li­gence: Early ex­per­i­ments with GPT-4 | Microsoft Research

DragonGod23 Mar 2023 5:45 UTC
68 points
23 comments1 min readLW link
(arxiv.org)

[Question] Is the ChatGPT-simu­lated Linux vir­tual ma­chine real?

Kenoubi13 Dec 2022 15:41 UTC
18 points
7 comments1 min readLW link

GPT as an “In­tel­li­gence Fork­lift.”

boazbarak19 May 2023 21:15 UTC
48 points
27 comments3 min readLW link

Get­ting 50% (SoTA) on ARC-AGI with GPT-4o

ryan_greenblatt17 Jun 2024 18:44 UTC
262 points
49 comments13 min readLW link

Bad at Arith­metic, Promis­ing at Math

cohenmacaulay18 Dec 2022 5:40 UTC
100 points
19 comments20 min readLW link1 review

′ pe­ter­todd’’s last stand: The fi­nal days of open GPT-3 research

mwatkins22 Jan 2024 18:47 UTC
109 points
16 comments45 min readLW link

Next Level Seinfeld

Zvi19 Dec 2022 13:30 UTC
50 points
8 comments1 min readLW link
(thezvi.wordpress.com)

On AutoGPT

Zvi13 Apr 2023 12:30 UTC
248 points
47 comments20 min readLW link
(thezvi.wordpress.com)

Mlyyrczo

lsusr26 Dec 2022 7:58 UTC
41 points
14 comments3 min readLW link

Why did ChatGPT say that? Prompt en­g­ineer­ing and more, with PIZZA.

Jessica Rumbelow3 Aug 2024 12:07 UTC
40 points
2 comments4 min readLW link

Steer­ing GPT-2-XL by adding an ac­ti­va­tion vector

13 May 2023 18:42 UTC
437 points
97 comments50 min readLW link

PaLM-2 & GPT-4 in “Ex­trap­o­lat­ing GPT-N perfor­mance”

Lukas Finnveden30 May 2023 18:33 UTC
55 points
6 comments6 min readLW link

Can ChatGPT count?

p.b.7 Jan 2023 7:57 UTC
13 points
11 comments2 min readLW link

Eval­u­at­ing strate­gic rea­son­ing in GPT models

phelps-sg25 May 2023 11:51 UTC
4 points
1 comment8 min readLW link

[Question] GPT learn­ing from smarter texts?

Viliam8 Jan 2023 22:23 UTC
26 points
7 comments1 min readLW link

Do Not Mess With Scar­lett Johansson

Zvi22 May 2024 15:10 UTC
65 points
7 comments16 min readLW link
(thezvi.wordpress.com)

Ex­per­i­ments in Eval­u­at­ing Steer­ing Vectors

Gytis Daujotas19 Jun 2023 15:11 UTC
34 points
4 comments4 min readLW link

ChatGPT strug­gles to re­spond to the real world

Alex Flint12 Jan 2023 16:02 UTC
31 points
9 comments24 min readLW link

The ‘ pe­ter­todd’ phenomenon

mwatkins15 Apr 2023 0:59 UTC
192 points
49 comments38 min readLW link

Microsoft Re­search Paper Claims Sparks of Ar­tifi­cial In­tel­li­gence in GPT-4

Zvi24 Mar 2023 13:20 UTC
72 points
14 comments6 min readLW link
(thezvi.wordpress.com)

Map­ping the se­man­tic void: Strange go­ings-on in GPT em­bed­ding spaces

mwatkins14 Dec 2023 13:10 UTC
114 points
31 comments14 min readLW link

Large lan­guage mod­els learn to rep­re­sent the world

gjm22 Jan 2023 13:10 UTC
102 points
19 comments3 min readLW link

Study 1b: This One Weird Trick does NOT cause in­cor­rect­ness cascades

Robert_AIZI20 Apr 2023 18:10 UTC
5 points
0 comments6 min readLW link
(aizi.substack.com)

[Question] Why is o1 so de­cep­tive?

abramdemski27 Sep 2024 17:27 UTC
177 points
24 comments3 min readLW link

[Question] How hard would it be to change GPT-3 in a way that al­lows au­dio?

ChristianKl28 Aug 2020 14:42 UTC
9 points
5 comments1 min readLW link

Ano­ma­lous to­kens re­veal the origi­nal iden­tities of In­struct models

9 Feb 2023 1:30 UTC
139 points
16 comments9 min readLW link
(generative.ink)

OpenAI’s GPT-4 Safety Goals

PeterMcCluskey22 Apr 2023 19:11 UTC
3 points
3 comments4 min readLW link
(bayesianinvestor.com)

GPT-4o My and Google I/​O Day

Zvi16 May 2024 17:50 UTC
41 points
2 comments37 min readLW link
(thezvi.wordpress.com)

[Question] Is In­struc­tGPT Fol­low­ing In­struc­tions in Other Lan­guages Sur­pris­ing?

DragonGod13 Feb 2023 23:26 UTC
39 points
15 comments1 min readLW link

Why GPT wants to mesa-op­ti­mize & how we might change this

John_Maxwell19 Sep 2020 13:48 UTC
55 points
33 comments9 min readLW link

The Cave Alle­gory Re­vis­ited: Un­der­stand­ing GPT’s Worldview

Jan_Kulveit14 Feb 2023 16:00 UTC
84 points
5 comments3 min readLW link

[Question] Where is hu­man level on text pre­dic­tion? (GPTs task)

Daniel Kokotajlo20 Sep 2020 9:00 UTC
27 points
19 comments1 min readLW link

The idea that ChatGPT is sim­ply “pre­dict­ing” the next word is, at best, misleading

Bill Benzon20 Feb 2023 11:32 UTC
55 points
87 comments5 min readLW link

HDBSCAN is Sur­pris­ingly Effec­tive at Find­ing In­ter­pretable Clusters of the SAE De­coder Matrix

11 Oct 2024 23:06 UTC
8 points
2 comments10 min readLW link

Ex­am­ples of Prompts that Make GPT-4 Out­put Falsehoods

22 Jul 2023 20:21 UTC
21 points
5 comments6 min readLW link

GPT-4 can catch sub­tle cross-lan­guage trans­la­tion mistakes

Michael Tontchev27 Jul 2023 1:39 UTC
7 points
1 comment1 min readLW link

BIG-Bench Ca­nary Con­tam­i­na­tion in GPT-4

Jozdien22 Oct 2024 15:40 UTC
123 points
13 comments4 min readLW link

Eval­u­at­ing GPT-4 The­ory of Mind Capabilities

10 Aug 2023 17:57 UTC
15 points
2 comments14 min readLW link

What’s up with all the non-Mor­mons? Weirdly spe­cific uni­ver­sal­ities across LLMs

mwatkins19 Apr 2024 13:43 UTC
40 points
13 comments27 min readLW link

Sto­ry­tel­ling Makes GPT-3.5 Deon­tol­o­gist: Un­ex­pected Effects of Con­text on LLM Behavior

14 Mar 2023 8:44 UTC
17 points
0 comments12 min readLW link

GPT can write Quines now (GPT-4)

Andrew_Critch14 Mar 2023 19:18 UTC
112 points
30 comments1 min readLW link

Nav­i­gat­ing LLM em­bed­ding spaces us­ing archetype-based directions

mwatkins8 May 2024 5:54 UTC
15 points
4 comments28 min readLW link

ARC tests to see if GPT-4 can es­cape hu­man con­trol; GPT-4 failed to do so

Christopher King15 Mar 2023 0:29 UTC
116 points
22 comments2 min readLW link

GPT-4 de­vel­oper livestream

Gerald Monroe14 Mar 2023 20:55 UTC
9 points
0 comments1 min readLW link
(www.youtube.com)

The Col­lid­ing Ex­po­nen­tials of AI

Vermillion14 Oct 2020 23:31 UTC
28 points
16 comments5 min readLW link

On OpenAI Dev Day

Zvi9 Nov 2023 16:10 UTC
60 points
0 comments15 min readLW link
(thezvi.wordpress.com)

Paper: On mea­sur­ing situ­a­tional aware­ness in LLMs

4 Sep 2023 12:54 UTC
108 points
16 comments5 min readLW link
(arxiv.org)

A chess game against GPT-4

Rafael Harth16 Mar 2023 14:05 UTC
24 points
23 comments1 min readLW link

GPT-4 Mul­ti­pli­ca­tion Competition

dandelion416 Mar 2023 3:09 UTC
11 points
7 comments1 min readLW link

[Question] Will 2023 be the last year you can write short sto­ries and re­ceive most of the in­tel­lec­tual credit for writ­ing them?

lc16 Mar 2023 21:36 UTC
20 points
11 comments1 min readLW link

Is it a bad idea to pay for GPT-4?

nem16 Mar 2023 20:49 UTC
24 points
8 comments1 min readLW link

[Question] Can GPT-4 play 20 ques­tions against an­other in­stance of it­self?

Nathan Helm-Burger28 Mar 2023 1:11 UTC
15 points
1 comment1 min readLW link
(evanthebouncy.medium.com)

The Power of High Speed Stupidity

robotelvis17 Mar 2023 21:41 UTC
33 points
5 comments9 min readLW link
(messyprogress.substack.com)

[Question] What did you do with GPT4?

ChristianKl18 Mar 2023 15:21 UTC
27 points
17 comments1 min readLW link

An ex­pla­na­tion for ev­ery to­ken: us­ing an LLM to sam­ple an­other LLM

Max H11 Oct 2023 0:53 UTC
35 points
5 comments11 min readLW link

Fea­ture pro­posal: in­te­grate LessWrong with ChatGPT to pro­mote ac­tive reading

DirectedEvolution19 Mar 2023 3:41 UTC
10 points
4 comments1 min readLW link

Creat­ing a fam­ily with GPT-4

Kaj_Sotala28 Mar 2023 6:40 UTC
23 points
3 comments10 min readLW link
(kajsotala.fi)

What’s Your Cog­ni­tive Al­gorithm?

Raemon18 Jun 2020 22:16 UTC
74 points
23 comments13 min readLW link

Lin­ear en­cod­ing of char­ac­ter-level in­for­ma­tion in GPT-J to­ken embeddings

10 Nov 2023 22:19 UTC
34 points
4 comments28 min readLW link

Re­marks 1–18 on GPT (com­pressed)

Cleo Nardo20 Mar 2023 22:27 UTC
146 points
35 comments31 min readLW link

Ex­plor­ing GPT4′s world model

hippke20 Mar 2023 21:31 UTC
−5 points
5 comments2 min readLW link

Beyond 175 billion pa­ram­e­ters: Can we an­ti­ci­pate fu­ture GPT-X Ca­pa­bil­ities?

bakztfuture4 Dec 2020 23:42 UTC
−1 points
1 comment2 min readLW link

MIRI com­ments on Co­tra’s “Case for Align­ing Nar­rowly Su­per­hu­man Models”

Rob Bensinger5 Mar 2021 23:43 UTC
142 points
13 comments26 min readLW link

A sim­ple way to make GPT-3 fol­low instructions

Quintin Pope8 Mar 2021 2:57 UTC
11 points
5 comments4 min readLW link

Thoughts on the Align­ment Im­pli­ca­tions of Scal­ing Lan­guage Models

leogao2 Jun 2021 21:32 UTC
82 points
11 comments17 min readLW link

ChatGPT and Bing Chat can’t play Botticelli

Asha Saavoss29 Mar 2023 17:39 UTC
11 points
0 comments6 min readLW link

Study­ing The Alien Mind

5 Dec 2023 17:27 UTC
80 points
10 comments15 min readLW link

New GPT-3 competitor

Quintin Pope12 Aug 2021 7:05 UTC
32 points
10 comments1 min readLW link

AI-Based Code Gen­er­a­tion Us­ing GPT-J-6B

Tomás B.16 Jun 2021 15:05 UTC
22 points
14 comments1 min readLW link
(minimaxir.com)

GPT-Aug­mented Blogging

lsusr14 Sep 2021 11:55 UTC
52 points
18 comments13 min readLW link

Si­mu­lated Elon Musk Lives in a Simulation

lsusr18 Sep 2021 7:37 UTC
66 points
9 comments3 min readLW link

[Question] How much should you be will­ing to pay for an AGI?

Logan Zoellner20 Sep 2021 11:51 UTC
11 points
5 comments1 min readLW link

[Question] Any write­ups on GPT agency?

Ozyrus26 Sep 2021 22:55 UTC
4 points
6 comments1 min readLW link

[Question] Is GPT-3 already sam­ple-effi­cient?

Daniel Kokotajlo6 Oct 2021 13:38 UTC
36 points
32 comments1 min readLW link

NVIDIA and Microsoft re­leases 530B pa­ram­e­ter trans­former model, Me­ga­tron-Tur­ing NLG

Ozyrus11 Oct 2021 15:28 UTC
51 points
36 comments1 min readLW link
(developer.nvidia.com)

The “AI Dun­geons” Dragon Model is heav­ily path de­pen­dent (test­ing GPT-3 on ethics)

Rafael Harth21 Jul 2020 12:14 UTC
44 points
9 comments6 min readLW link

“Sum­ma­riz­ing Books with Hu­man Feed­back” (re­cur­sive GPT-3)

gwern15 Nov 2021 17:41 UTC
24 points
4 comments1 min readLW link
(openai.com)

Reader-gen­er­ated Essays

Henrik Karlsson3 Jan 2022 8:56 UTC
25 points
1 comment6 min readLW link
(escapingflatland.substack.com)

GPT-3 Gems

TurnTrout23 Jul 2020 0:46 UTC
33 points
10 comments48 min readLW link

A one-ques­tion Tur­ing test for GPT-3

22 Jan 2022 18:17 UTC
85 points
25 comments5 min readLW link

Idea: build al­ign­ment dataset for very ca­pa­ble models

Quintin Pope12 Feb 2022 19:30 UTC
14 points
2 comments3 min readLW link

More GPT-3 and sym­bol grounding

Stuart_Armstrong23 Feb 2022 18:30 UTC
21 points
7 comments3 min readLW link

Per­sonal imi­ta­tion software

Flaglandbase7 Mar 2022 7:55 UTC
6 points
6 comments1 min readLW link

New GPT3 Im­pres­sive Ca­pa­bil­ities—In­struc­tGPT3 [1/​2]

simeon_c13 Mar 2022 10:58 UTC
72 points
10 comments7 min readLW link

[Question] Ques­tion on GPT-3 Ex­cel Demo

Zhitao Hou22 Jun 2020 20:31 UTC
0 points
1 comment1 min readLW link

Hu­mans pre­tend­ing to be robots pre­tend­ing to be human

Richard_Kennaway28 Mar 2022 15:13 UTC
25 points
14 comments1 min readLW link

[Link] Train­ing Com­pute-Op­ti­mal Large Lan­guage Models

nostalgebraist31 Mar 2022 18:01 UTC
51 points
23 comments1 min readLW link
(arxiv.org)

[Question] Are we cer­tain that gpt-2 and similar al­gorithms are not self-aware?

Ozyrus11 Jul 2019 8:37 UTC
0 points
12 comments1 min readLW link

New Scal­ing Laws for Large Lan­guage Models

1a3orn1 Apr 2022 20:41 UTC
246 points
22 comments5 min readLW link

[Question] What should we ex­pect from GPT-3?

avturchin21 Mar 2019 14:28 UTC
22 points
2 comments1 min readLW link

[Question] List of pub­lic pre­dic­tions of what GPT-X can or can’t do?

Daniel Kokotajlo14 Jun 2020 14:25 UTC
20 points
9 comments1 min readLW link

GPT-3 and con­cept extrapolation

Stuart_Armstrong20 Apr 2022 10:39 UTC
19 points
27 comments1 min readLW link

GPT-3: A Summary

leogao2 Jun 2020 18:14 UTC
20 points
0 comments1 min readLW link
(leogao.dev)

Get­ting GPT-3 to pre­dict Me­tac­u­lus questions

MathiasKB6 May 2022 6:01 UTC
69 points
9 comments2 min readLW link

Pos­i­tive out­comes un­der an un­al­igned AGI takeover

Yitz12 May 2022 7:45 UTC
19 points
10 comments3 min readLW link

[Question] If AI is based on GPT, how to en­sure its safety?

avturchin18 Jun 2020 20:33 UTC
20 points
11 comments1 min readLW link

Paper: Teach­ing GPT3 to ex­press un­cer­tainty in words

Owain_Evans31 May 2022 13:27 UTC
97 points
7 comments4 min readLW link

OpenAI: GPT-based LLMs show abil­ity to dis­crim­i­nate be­tween its own wrong an­swers, but in­abil­ity to ex­plain how/​why it makes that dis­crim­i­na­tion, even as model scales

Aditya Jain13 Jun 2022 23:33 UTC
14 points
5 comments1 min readLW link
(openai.com)

[Question] AI mis­al­ign­ment risk from GPT-like sys­tems?

fiso6419 Jun 2022 17:35 UTC
10 points
8 comments1 min readLW link

Try­ing out Prompt Eng­ineer­ing on TruthfulQA

Megan Kinniment23 Jul 2022 2:04 UTC
10 points
0 comments8 min readLW link

Us­ing GPT-3 to aug­ment hu­man intelligence

Henrik Karlsson10 Aug 2022 15:54 UTC
52 points
8 comments18 min readLW link
(escapingflatland.substack.com)

What’s the Least Im­pres­sive Thing GPT-4 Won’t be Able to Do

Algon20 Aug 2022 19:48 UTC
80 points
125 comments1 min readLW link

[up­dated] how does gpt2′s train­ing cor­pus cap­ture in­ter­net dis­cus­sion? not well

nostalgebraist27 Jul 2020 22:30 UTC
25 points
3 comments2 min readLW link
(nostalgebraist.tumblr.com)

Progress Re­port 7: mak­ing GPT go hur­rdurr in­stead of brrrrrrr

Nathan Helm-Burger7 Sep 2022 3:28 UTC
21 points
0 comments4 min readLW link

Ar­gu­ing all sides with ChatGPT

Richard_Kennaway30 Mar 2023 19:50 UTC
16 points
1 comment8 min readLW link

[Question] Prob­a­bil­ity that other ar­chi­tec­tures will scale as well as Trans­form­ers?

Daniel Kokotajlo28 Jul 2020 19:36 UTC
22 points
4 comments1 min readLW link

[Question] To what ex­tent are the scal­ing prop­er­ties of Trans­former net­works ex­cep­tional?

abramdemski28 Jul 2020 20:06 UTC
30 points
1 comment1 min readLW link

[ASoT] Thoughts on GPT-N

Ulisse Mini8 Nov 2022 7:14 UTC
8 points
0 comments1 min readLW link

En­gag­ing Se­ri­ously with Short Timelines

sapphire29 Jul 2020 19:21 UTC
43 points
21 comments3 min readLW link

AI #4: In­tro­duc­ing GPT-4

Zvi21 Mar 2023 14:00 UTC
101 points
32 comments103 min readLW link
(thezvi.wordpress.com)

How does GPT-3 spend its 175B pa­ram­e­ters?

Robert_AIZI13 Jan 2023 19:21 UTC
41 points
14 comments6 min readLW link
(aizi.substack.com)

Pro­to­type of Us­ing GPT-3 to Gen­er­ate Text­book-length Content

Rafael Cosman18 Jan 2023 14:25 UTC
2 points
8 comments40 min readLW link
(github.com)

The Gallery for Paint­ing Trans­for­ma­tions—A GPT-3 Analogy

Robert_AIZI19 Jan 2023 23:32 UTC
1 point
0 comments6 min readLW link
(aizi.substack.com)

ChatGPT vs the 2-4-6 Task

cwillu25 Jan 2023 6:59 UTC
20 points
4 comments3 min readLW link

When will GPT-5 come out? Pre­dic­tion mar­kets vs. Extrapolation

Malte12 Dec 2023 2:41 UTC
12 points
9 comments3 min readLW link

Some mis­cel­la­neous thoughts on ChatGPT, sto­ries, and me­chan­i­cal interpretability

Bill Benzon4 Feb 2023 19:35 UTC
2 points
0 comments3 min readLW link

Ad­den­dum: More Effi­cient FFNs via Attention

Robert_AIZI6 Feb 2023 18:55 UTC
10 points
2 comments5 min readLW link
(aizi.substack.com)

[Question] What’s ac­tu­ally go­ing on in the “mind” of the model when we fine-tune GPT-3 to In­struc­tGPT?

rpglover6410 Feb 2023 7:57 UTC
18 points
3 comments1 min readLW link

[Question] Is it a co­in­ci­dence that GPT-3 re­quires roughly the same amount of com­pute as is nec­es­sary to em­u­late the hu­man brain?

RomanS10 Feb 2023 16:26 UTC
11 points
10 comments1 min readLW link

A note on ‘semiotic physics’

metasemi11 Feb 2023 5:12 UTC
11 points
13 comments6 min readLW link

Ex­plain­ing SolidGoldMag­ikarp by look­ing at it from ran­dom directions

Robert_AIZI14 Feb 2023 14:54 UTC
8 points
0 comments8 min readLW link
(aizi.substack.com)

Syd­ney the Bin­gena­tor Can’t Think, But It Still Threat­ens People

Valentin Baltadzhiev20 Feb 2023 18:37 UTC
−3 points
2 comments8 min readLW link

Bing find­ing ways to by­pass Microsoft’s filters with­out be­ing asked. Is it re­pro­ducible?

Christopher King20 Feb 2023 15:11 UTC
27 points
15 comments1 min readLW link

Pre­train­ing Lan­guage Models with Hu­man Preferences

21 Feb 2023 17:57 UTC
134 points
19 comments11 min readLW link

[Question] In­ject­ing noise to GPT to get mul­ti­ple answers

bipolo22 Feb 2023 20:02 UTC
1 point
1 comment1 min readLW link

Just How Hard a Prob­lem is Align­ment?

Roger Dearnaley25 Feb 2023 9:00 UTC
1 point
1 comment21 min readLW link

Reflec­tion Mechanisms as an Align­ment Tar­get—At­ti­tudes on “near-term” AI

2 Mar 2023 4:29 UTC
21 points
0 comments8 min readLW link

ChatGPT tells sto­ries, and a note about re­verse en­g­ineer­ing: A Work­ing Paper

Bill Benzon3 Mar 2023 15:12 UTC
3 points
0 comments3 min readLW link

ChatGPT ex­plores the se­man­tic differential

Bill Benzon9 Mar 2023 13:09 UTC
7 points
2 comments7 min readLW link

Stop call­ing it “jailbreak­ing” ChatGPT

Templarrr10 Mar 2023 11:41 UTC
7 points
9 comments2 min readLW link

GPT-4

nz14 Mar 2023 17:02 UTC
150 points
149 comments1 min readLW link
(openai.com)

Paper­clipGPT(-4)

Michael Tontchev14 Mar 2023 22:03 UTC
7 points
0 comments11 min readLW link

ChatGPT (and now GPT4) is very eas­ily dis­tracted from its rules

dmcs15 Mar 2023 17:55 UTC
180 points
42 comments1 min readLW link

GPT-4: What we (I) know about it

Robert_AIZI15 Mar 2023 20:12 UTC
40 points
29 comments12 min readLW link
(aizi.substack.com)

How well did Man­i­fold pre­dict GPT-4?

David Chee15 Mar 2023 23:19 UTC
49 points
5 comments2 min readLW link

GPT-4 solves Gary Mar­cus-in­duced flubs

JakubK17 Mar 2023 6:40 UTC
56 points
29 comments2 min readLW link
(docs.google.com)

Ret­ro­spec­tive on ‘GPT-4 Pre­dic­tions’ After the Re­lease of GPT-4

Stephen McAleese17 Mar 2023 18:34 UTC
26 points
6 comments6 min readLW link

GPTs’ abil­ity to keep a se­cret is weirdly prompt-dependent

22 Jul 2023 12:21 UTC
31 points
0 comments9 min readLW link

In­stan­ti­at­ing an agent with GPT-4 and text-davinci-003

Max H19 Mar 2023 23:57 UTC
13 points
3 comments32 min readLW link

AI and the Map of Your Mind: Pat­tern Recog­ni­tion

Scott Broock20 Mar 2023 17:43 UTC
2 points
2 comments6 min readLW link

Nyarlathotep Stirs: A Meta-Nar­ra­tive ChatGPT Story

Charlie Sanders20 Mar 2023 8:00 UTC
4 points
2 comments12 min readLW link
(dailymicrofiction.substack.com)

The dreams of GPT-4

RomanS20 Mar 2023 17:00 UTC
14 points
7 comments9 min readLW link

ChatGPT: Tan­tal­iz­ing af­terthoughts in search of story tra­jec­to­ries [in­duc­tion heads]

Bill Benzon3 Feb 2023 10:35 UTC
4 points
0 comments20 min readLW link

Ex­plor­ing the Resi­d­ual Stream of Trans­form­ers for Mechanis­tic In­ter­pretabil­ity — Explained

Zeping Yu26 Dec 2023 0:36 UTC
7 points
1 comment11 min readLW link

Maybe talk­ing isn’t the best way to com­mu­ni­cate with LLMs

mnvr17 Jan 2024 6:24 UTC
3 points
1 comment1 min readLW link
(mrmr.io)

OpenAI Credit Ac­count (2510$)

Emirhan BULUT21 Jan 2024 2:30 UTC
1 point
0 comments1 min readLW link

OpenAI Credit Ac­count (2510$)

Emirhan BULUT21 Jan 2024 2:32 UTC
1 point
0 comments1 min readLW link

Put­ting mul­ti­modal LLMs to the Tetris test

1 Feb 2024 16:02 UTC
30 points
5 comments7 min readLW link

The case for more am­bi­tious lan­guage model evals

Jozdien30 Jan 2024 0:01 UTC
110 points
30 comments5 min readLW link

Re­quire­ments for a Basin of At­trac­tion to Alignment

RogerDearnaley14 Feb 2024 7:10 UTC
38 points
12 comments31 min readLW link

Im­ple­ment­ing ac­ti­va­tion steering

Annah5 Feb 2024 17:51 UTC
66 points
7 comments7 min readLW link

Trans­fer learn­ing and gen­er­al­iza­tion-qua-ca­pa­bil­ity in Bab­bage and Davinci (or, why di­vi­sion is bet­ter than Span­ish)

RP and agg
9 Feb 2024 7:00 UTC
50 points
6 comments3 min readLW link

Is an­a­lyz­ing LLM be­hav­ior a valid means for as­sess­ing po­ten­tial con­scious­ness, as de­scribed by global workspace the­ory and higher or­der the­o­ries?

amelia11 Mar 2024 19:37 UTC
1 point
1 comment12 min readLW link

[Question] What ex­per­i­ment set­tles the Gary Mar­cus vs Ge­offrey Hin­ton de­bate?

Valentin Baltadzhiev14 Feb 2024 9:06 UTC
12 points
8 comments1 min readLW link

GPT, the mag­i­cal col­lab­o­ra­tion zone, Lex Frid­man and Sam Altman

Bill Benzon18 Mar 2024 20:04 UTC
3 points
1 comment3 min readLW link

The In­for­ma­tion: OpenAI shows ‘Straw­berry’ to feds, races to launch it

Martín Soto27 Aug 2024 23:10 UTC
145 points
15 comments3 min readLW link

On agen­tic gen­er­al­ist mod­els: we’re es­sen­tially us­ing ex­ist­ing tech­nol­ogy the weak­est and worst way you can use it

Yuli_Ban28 Aug 2024 1:57 UTC
10 points
2 comments9 min readLW link

Lan­guage and Ca­pa­bil­ities: Test­ing LLM Math­e­mat­i­cal Abil­ities Across Languages

Ethan Edwards4 Apr 2024 13:18 UTC
24 points
2 comments36 min readLW link

[Question] Bar­cod­ing LLM Train­ing Data Sub­sets. Any­one try­ing this for in­ter­pretabil­ity?

right..enough?13 Apr 2024 3:09 UTC
7 points
0 comments7 min readLW link

[Question] How is GPT-4o Re­lated to GPT-4?

Joel Burget15 May 2024 18:33 UTC
10 points
2 comments1 min readLW link

Fix sim­ple mis­takes in ARC-AGI, etc.

Oleg Trott9 Jul 2024 17:46 UTC
9 points
9 comments1 min readLW link

LLMs stifle cre­ativity, elimi­nate op­por­tu­ni­ties for serendipi­tous dis­cov­ery and dis­rupt in­ter­gen­er­a­tional trans­fer of wisdom

Ghdz5 Aug 2024 18:27 UTC
6 points
2 comments7 min readLW link

GPT-2 Some­times Fails at IOI

Ronak_Mehta14 Aug 2024 23:24 UTC
13 points
0 comments2 min readLW link
(ronakrm.github.io)

Two new datasets for eval­u­at­ing poli­ti­cal syco­phancy in LLMs

alma.liezenga28 Sep 2024 18:29 UTC
8 points
0 comments9 min readLW link

The de­fault sce­nario for the next 50 years

Julien24 Nov 2024 14:01 UTC
1 point
0 comments6 min readLW link

[Question] Why does ChatGPT throw an er­ror when out­putting “David Mayer”?

Archimedes1 Dec 2024 0:11 UTC
6 points
9 comments1 min readLW link

Us­ing GPT-4 to Un­der­stand Code

sid24 Mar 2023 0:09 UTC
25 points
2 comments6 min readLW link

GPT-4 al­ign­ing with aca­sual de­ci­sion the­ory when in­structed to play games, but in­cludes a CDT ex­pla­na­tion that’s in­cor­rect if they differ

Christopher King23 Mar 2023 16:16 UTC
7 points
4 comments8 min readLW link

Is your job re­place­able by GPT-4? (as of March 2023)

Bezzi23 Mar 2023 22:16 UTC
18 points
6 comments1 min readLW link

So, just why do GPTs have to op­er­ate by con­tin­u­ing an ex­ist­ing string?

Bill Benzon24 Mar 2023 12:08 UTC
−4 points
0 comments3 min readLW link

Does GPT-4 ex­hibit agency when sum­ma­riz­ing ar­ti­cles?

Christopher King24 Mar 2023 15:49 UTC
16 points
2 comments5 min readLW link

More ex­per­i­ments in GPT-4 agency: writ­ing memos

Christopher King24 Mar 2023 17:51 UTC
5 points
2 comments10 min readLW link

If it quacks like a duck...

RationalMindset26 Mar 2023 18:54 UTC
−4 points
0 comments4 min readLW link

[Question] GPT-4 Specs: 1 Trillion Pa­ram­e­ters?

infinibot2726 Mar 2023 18:56 UTC
6 points
8 comments1 min readLW link

Chronos­ta­sis: The Time-Cap­sule Co­nun­drum of Lan­guage Models

RationalMindset26 Mar 2023 18:54 UTC
−5 points
0 comments1 min readLW link

A Hive­mind of GPT-4 bots REALLY IS A HIVEMIND!

Erlja Jkdf.27 Mar 2023 12:44 UTC
−10 points
1 comment1 min readLW link

GPT-4 is bad at strate­gic thinking

Christopher King27 Mar 2023 15:11 UTC
22 points
8 comments1 min readLW link

No­body knows how to re­li­ably test for AI safety

marcusarvan27 Mar 2023 19:48 UTC
1 point
0 comments5 min readLW link

I had a chat with GPT-4 on the fu­ture of AI and AI safety

Kristian Freed28 Mar 2023 17:47 UTC
1 point
0 comments8 min readLW link

Inch­ing “Kubla Khan” and GPT into the same in­tel­lec­tual frame­work @ 3 Quarks Daily

Bill Benzon28 Mar 2023 19:50 UTC
5 points
0 comments3 min readLW link

Early Re­sults: Do LLMs com­plete false equa­tions with false equa­tions?

Robert_AIZI30 Mar 2023 20:14 UTC
14 points
0 comments4 min readLW link
(aizi.substack.com)

Harry Pot­ter and the Data Cen­ters of Doom

RomanS31 Mar 2023 10:42 UTC
13 points
5 comments4 min readLW link

GPT-4 busted? Clear self-in­ter­est when sum­ma­riz­ing ar­ti­cles about it­self vs when ar­ti­cle talks about Claude, LLaMA, or DALL·E 2

Christopher King31 Mar 2023 17:05 UTC
6 points
4 comments4 min readLW link

The Peril of the Great Leaks (writ­ten with ChatGPT)

bvbvbvbvbvbvbvbvbvbvbv31 Mar 2023 18:14 UTC
3 points
1 comment1 min readLW link

Imag­ine a world where Microsoft em­ploy­ees used Bing

Christopher King31 Mar 2023 18:36 UTC
6 points
2 comments2 min readLW link

[Question] Trans­former trained on it’s own con­tent?

Micromegas1 Apr 2023 15:08 UTC
1 point
0 comments1 min readLW link

No con­vinc­ing ev­i­dence for gra­di­ent de­scent in ac­ti­va­tion space

Blaine12 Apr 2023 4:48 UTC
82 points
9 comments20 min readLW link

Open-source LLMs may prove Bostrom’s vuln­er­a­ble world hypothesis

Roope Ahvenharju15 Apr 2023 19:16 UTC
1 point
1 comment1 min readLW link

[Question] Us­ing ChatGPT for mem­ory re­con­soli­da­tion?

warrenjordan13 Apr 2023 1:27 UTC
3 points
2 comments1 min readLW link

Mechanis­ti­cally in­ter­pret­ing time in GPT-2 small

16 Apr 2023 17:57 UTC
68 points
6 comments21 min readLW link

Pol­lut­ing the agen­tic commons

hamandcheese13 Apr 2023 17:42 UTC
7 points
4 comments2 min readLW link
(www.secondbest.ca)

Re­search Re­port: In­cor­rect­ness Cascades

Robert_AIZI14 Apr 2023 12:49 UTC
19 points
0 comments10 min readLW link
(aizi.substack.com)

The Soul of the Writer (on LLMs, the psy­chol­ogy of writ­ers, and the na­ture of in­tel­li­gence)

rogersbacon16 Apr 2023 16:02 UTC
11 points
1 comment3 min readLW link
(www.secretorum.life)

An al­ter­na­tive of PPO to­wards alignment

ml hkust17 Apr 2023 17:58 UTC
2 points
2 comments4 min readLW link

Read­abil­ity is mostly a waste of characters

vlad.proex21 Apr 2023 22:05 UTC
21 points
7 comments3 min readLW link

We Need To Know About Con­tinual Learning

michael_mjd22 Apr 2023 17:08 UTC
29 points
14 comments4 min readLW link

Scal­able And Trans­fer­able Black-Box Jailbreaks For Lan­guage Models Via Per­sona Modulation

7 Nov 2023 17:59 UTC
36 points
2 comments2 min readLW link
(arxiv.org)

LLMs and com­pu­ta­tion complexity

Jonathan Marcus28 Apr 2023 17:48 UTC
57 points
29 comments5 min readLW link

The Misal­ign­ment Para­dox: Ro­bustly Har­ness­ing De­liber­ate Value Diver­gence (Writ­ten by GPT-4)

shl0ms28 Apr 2023 3:29 UTC
0 points
0 comments6 min readLW link

Feel­ings, Noth­ing More than Feel­ings, About AI

PaulBecon14 Nov 2023 18:50 UTC
7 points
0 comments3 min readLW link

Large Lan­guage Models can Strate­gi­cally De­ceive their Users when Put Un­der Pres­sure.

ReaderM15 Nov 2023 16:36 UTC
89 points
8 comments2 min readLW link
(arxiv.org)

Ilya: The AI sci­en­tist shap­ing the world

David Varga20 Nov 2023 13:09 UTC
11 points
0 comments4 min readLW link

The Limi­ta­tions of GPT-4

p.b.24 Nov 2023 15:30 UTC
27 points
12 comments4 min readLW link

Plan­ning in LLMs: In­sights from AlphaGo

jco4 Dec 2023 18:48 UTC
8 points
10 comments11 min readLW link

[Question] Is OpenAI los­ing money on each re­quest?

thenoviceoof1 Dec 2023 3:27 UTC
8 points
8 comments5 min readLW link

[Question] What spe­cific dan­gers arise when ask­ing GPT-N to write an Align­ment Fo­rum post?

Matthew Barnett28 Jul 2020 2:56 UTC
45 points
14 comments1 min readLW link

Are AIs like An­i­mals? Per­spec­tives and Strate­gies from Biology

Jackson Emanuel16 May 2023 23:39 UTC
1 point
0 comments21 min readLW link

[Question] Is the work on AI al­ign­ment rele­vant to GPT?

Richard_Kennaway30 Jul 2020 12:23 UTC
24 points
5 comments1 min readLW link

Agen­tic Lan­guage Model Memes

FactorialCode1 Aug 2020 18:03 UTC
16 points
1 comment2 min readLW link

[Question] What are the most im­por­tant pa­pers/​post/​re­sources to read to un­der­stand more of GPT-3?

adamShimi2 Aug 2020 20:53 UTC
22 points
4 comments1 min readLW link

Is “red” for GPT-4 the same as “red” for you?

Yusuke Hayashi6 May 2023 17:55 UTC
9 points
6 comments2 min readLW link

LLM cog­ni­tion is prob­a­bly not hu­man-like

Max H8 May 2023 1:22 UTC
26 points
15 comments7 min readLW link

Lan­guage mod­els can ex­plain neu­rons in lan­guage models

nz9 May 2023 17:29 UTC
23 points
0 comments1 min readLW link
(openai.com)

Re­search Re­port: In­cor­rect­ness Cas­cades (Cor­rected)

Robert_AIZI9 May 2023 21:54 UTC
9 points
0 comments9 min readLW link
(aizi.substack.com)

GPT4 is ca­pa­ble of writ­ing de­cent long-form sci­ence fic­tion (with the right prompts)

RomanS23 May 2023 13:41 UTC
22 points
28 comments65 min readLW link

The Com­pleat Cybornaut

19 May 2023 8:44 UTC
64 points
2 comments16 min readLW link

Col­lec­tive Identity

18 May 2023 9:00 UTC
59 points
12 comments8 min readLW link

Trans­former Ar­chi­tec­ture Choice for Re­sist­ing Prompt In­jec­tion and Jail-Break­ing Attacks

RogerDearnaley21 May 2023 8:29 UTC
9 points
1 comment4 min readLW link

hu­man psy­chol­in­guists: a crit­i­cal appraisal

nostalgebraist31 Dec 2019 0:20 UTC
181 points
59 comments16 min readLW link2 reviews
(nostalgebraist.tumblr.com)

[Question] 10/​50/​90% chance of GPT-N Trans­for­ma­tive AI?

human_generated_text9 Aug 2020 0:10 UTC
24 points
8 comments1 min readLW link

Philo­soph­i­cal Cy­borg (Part 1)

14 Jun 2023 16:20 UTC
31 points
4 comments13 min readLW link

OpenAI in­tro­duces func­tion call­ing for GPT-4

20 Jun 2023 1:58 UTC
24 points
3 comments4 min readLW link
(openai.com)

[Linkpost] Faith and Fate: Limits of Trans­form­ers on Compositionality

Joe Kwon16 Jun 2023 15:04 UTC
19 points
4 comments1 min readLW link
(arxiv.org)

[Linkpost] A shared lin­guis­tic space for trans­mit­ting our thoughts from brain to brain in nat­u­ral conversations

Bogdan Ionut Cirstea1 Jul 2023 13:57 UTC
17 points
2 comments1 min readLW link

May Gw­ern.net newslet­ter (w/​GPT-3 com­men­tary)

gwern2 Jun 2020 15:40 UTC
32 points
7 comments1 min readLW link
(www.gwern.net)

A trick for Safer GPT-N

Razied23 Aug 2020 0:39 UTC
7 points
1 comment2 min readLW link

From GPT to AGI

ChristianKl31 Aug 2020 13:28 UTC
6 points
7 comments1 min readLW link

on “learn­ing to sum­ma­rize”

nostalgebraist12 Sep 2020 3:20 UTC
25 points
13 comments8 min readLW link
(nostalgebraist.tumblr.com)

GPT-2′s po­si­tional em­bed­ding ma­trix is a helix

AdamYedidia21 Jul 2023 4:16 UTC
44 points
21 comments4 min readLW link

The “spel­ling mir­a­cle”: GPT-3 spel­ling abil­ities and glitch to­kens revisited

mwatkins31 Jul 2023 19:47 UTC
85 points
29 comments20 min readLW link

Re­searchers and writ­ers can ap­ply for proxy ac­cess to the GPT-3.5 base model (code-davinci-002)

ampdot1 Dec 2023 18:48 UTC
14 points
0 comments1 min readLW link
(airtable.com)

The po­si­tional em­bed­ding ma­trix and pre­vi­ous-to­ken heads: how do they ac­tu­ally work?

AdamYedidia10 Aug 2023 1:58 UTC
26 points
4 comments13 min readLW link

Si­mu­late the CEO

robotelvis12 Aug 2023 0:09 UTC
23 points
4 comments5 min readLW link
(messyprogress.substack.com)

[Question] GPT-3 + GAN

stick10917 Oct 2020 7:58 UTC
4 points
3 comments1 min readLW link

Ac­tAdd: Steer­ing Lan­guage Models with­out Optimization

6 Sep 2023 17:21 UTC
105 points
3 comments2 min readLW link
(arxiv.org)

Graph­i­cal ten­sor no­ta­tion for interpretability

Jordan Taylor4 Oct 2023 8:04 UTC
140 points
11 comments19 min readLW link

New Tool: the Resi­d­ual Stream Viewer

AdamYedidia1 Oct 2023 0:49 UTC
32 points
7 comments4 min readLW link
(tinyurl.com)

GPT-4 for per­sonal pro­duc­tivity: on­line dis­trac­tion blocker

Sergii26 Sep 2023 17:41 UTC
64 points
12 comments2 min readLW link
(grgv.xyz)

This anime sto­ry­board doesn’t ex­ist: a graphic novel writ­ten and illus­trated by GPT4

RomanS5 Oct 2023 14:01 UTC
12 points
7 comments55 min readLW link

En­tan­gle­ment and in­tu­ition about words and mean­ing

Bill Benzon4 Oct 2023 14:16 UTC
4 points
0 comments2 min readLW link

All GPT skills are translation

p.b.13 Dec 2020 20:06 UTC
4 points
0 comments2 min readLW link

Rele­vance of ‘Harm­ful In­tel­li­gence’ Data in Train­ing Datasets (We­bText vs. Pile)

MiguelDev12 Oct 2023 12:08 UTC
12 points
0 comments9 min readLW link

Beta test GPT-3 based re­search assistant

jungofthewon16 Dec 2020 13:42 UTC
34 points
2 comments1 min readLW link

The case for al­ign­ing nar­rowly su­per­hu­man models

Ajeya Cotra5 Mar 2021 22:29 UTC
186 points
75 comments38 min readLW link1 review

[Question] What will GPT-4 be in­ca­pable of?

Michaël Trazzi6 Apr 2021 19:57 UTC
34 points
33 comments1 min readLW link

How I Learned to Stop Wor­ry­ing and Love MUM

Waddington20 May 2021 7:57 UTC
2 points
0 comments3 min readLW link

Spec­u­la­tions against GPT-n writ­ing al­ign­ment papers

Donald Hobson7 Jun 2021 21:13 UTC
31 points
6 comments2 min readLW link

What does GPT-3 un­der­stand? Sym­bol ground­ing and Chi­nese rooms

Stuart_Armstrong3 Aug 2021 13:14 UTC
40 points
15 comments12 min readLW link

[Question] 1h-vol­un­teers needed for a small AI Safety-re­lated re­search pro­ject

PabloAMC16 Aug 2021 17:53 UTC
2 points
0 comments1 min readLW link

[Question] Who owns OpenAI’s new lan­guage model?

ioannes14 Feb 2019 17:51 UTC
16 points
9 comments1 min readLW link

Truth­ful AI: Devel­op­ing and gov­ern­ing AI that does not lie

18 Oct 2021 18:37 UTC
82 points
9 comments10 min readLW link

AMA on Truth­ful AI: Owen Cot­ton-Bar­ratt, Owain Evans & co-authors

Owain_Evans22 Oct 2021 16:23 UTC
31 points
15 comments1 min readLW link

Hegel vs. GPT-3

Bezzi27 Oct 2021 5:55 UTC
10 points
21 comments2 min readLW link

[Question] What ex­actly is GPT-3′s base ob­jec­tive?

Daniel Kokotajlo10 Nov 2021 0:57 UTC
60 points
14 comments2 min readLW link

Truth­ful LMs as a warm-up for al­igned AGI

Jacob_Hilton17 Jan 2022 16:49 UTC
65 points
14 comments13 min readLW link

How I’m think­ing about GPT-N

delton13717 Jan 2022 17:11 UTC
54 points
21 comments18 min readLW link

Un­com­pet­i­tive pro­gram­ming with GPT-3

Bezzi6 Feb 2022 10:19 UTC
7 points
8 comments3 min readLW link

Us­ing GPT-3 for pre­vent­ing con­flict dur­ing mes­sag­ing — a pitch for an app

Eli_17 Mar 2022 11:02 UTC
22 points
17 comments3 min readLW link

[Question] If you lose enough Good Heart To­kens, will you lose real-world money?

Yitz1 Apr 2022 21:11 UTC
9 points
0 comments1 min readLW link

Test­ing PaLM prompts on GPT3

Yitz6 Apr 2022 5:21 UTC
103 points
14 comments8 min readLW link

Is GPT3 a Good Ra­tion­al­ist? - In­struc­tGPT3 [2/​2]

simeon_c7 Apr 2022 13:46 UTC
11 points
0 comments7 min readLW link

PaLM in “Ex­trap­o­lat­ing GPT-N perfor­mance”

Lukas Finnveden6 Apr 2022 13:05 UTC
83 points
19 comments2 min readLW link

What is the solu­tion to the Align­ment prob­lem?

Algon30 Apr 2022 23:19 UTC
24 points
2 comments1 min readLW link

A pos­si­ble check against mo­ti­vated rea­son­ing us­ing elicit.org

david reinstein18 May 2022 20:52 UTC
3 points
0 comments1 min readLW link

RL with KL penalties is bet­ter seen as Bayesian inference

25 May 2022 9:23 UTC
114 points
17 comments12 min readLW link

Who mod­els the mod­els that model mod­els? An ex­plo­ra­tion of GPT-3′s in-con­text model fit­ting ability

Lovre7 Jun 2022 19:37 UTC
112 points
16 comments9 min readLW link

In­ves­ti­gat­ing causal un­der­stand­ing in LLMs

14 Jun 2022 13:57 UTC
28 points
6 comments13 min readLW link

GPT-3 Catch­ing Fish in Morse Code

Megan Kinniment30 Jun 2022 21:22 UTC
117 points
27 comments8 min readLW link

[Question] The OpenAI play­ground for GPT-3 is a ter­rible in­ter­face. Is there any great lo­cal (or web) app for ex­plor­ing/​learn­ing with lan­guage mod­els?

aviv13 Aug 2022 16:34 UTC
3 points
1 comment1 min readLW link

What’s the Most Im­pres­sive Thing That GPT-4 Could Plau­si­bly Do?

bayesed26 Aug 2022 15:34 UTC
24 points
22 comments1 min readLW link

[Question] If we have Hu­man-level chat­bots, won’t we end up be­ing ruled by pos­si­ble peo­ple?

Erlja Jkdf.20 Sep 2022 13:59 UTC
5 points
13 comments1 min readLW link

An Un­ex­pected GPT-3 De­ci­sion in a Sim­ple Gam­ble

casualphysicsenjoyer25 Sep 2022 16:46 UTC
8 points
4 comments1 min readLW link

Re­call and Re­gur­gi­ta­tion in GPT2

Megan Kinniment3 Oct 2022 19:35 UTC
43 points
1 comment26 min readLW link

Mys­ter­ies of mode collapse

janus8 Nov 2022 10:37 UTC
283 points
57 comments14 min readLW link1 review

[simu­la­tion] 4chan user claiming to be the at­tor­ney hired by Google’s sen­tient chat­bot LaMDA shares wild de­tails of encounter

janus10 Nov 2022 21:39 UTC
19 points
1 comment13 min readLW link
(generative.ink)

[Question] Is the speed of train­ing large mod­els go­ing to in­crease sig­nifi­cantly in the near fu­ture due to Cere­bras An­dromeda?

Amal 15 Nov 2022 22:50 UTC
13 points
11 comments1 min readLW link

By De­fault, GPTs Think In Plain Sight

Fabien Roger19 Nov 2022 19:15 UTC
86 points
36 comments9 min readLW link

Did ChatGPT just gaslight me?

TW1231 Dec 2022 5:41 UTC
123 points
45 comments9 min readLW link
(aiwatchtower.substack.com)

ChatGPT seems over­con­fi­dent to me

qbolec4 Dec 2022 8:03 UTC
19 points
3 comments16 min readLW link

Test­ing Ways to By­pass ChatGPT’s Safety Features

Robert_AIZI5 Dec 2022 18:50 UTC
7 points
4 comments5 min readLW link
(aizi.substack.com)

ChatGPT on Spielberg’s A.I. and AI Alignment

Bill Benzon5 Dec 2022 21:10 UTC
5 points
0 comments4 min readLW link

ChatGPT: “An er­ror oc­curred. If this is­sue per­sists...”

Bill Benzon7 Dec 2022 15:41 UTC
5 points
11 comments3 min readLW link

Of pump­kins, the Fal­con Heavy, and Grou­cho Marx: High-Level dis­course struc­ture in ChatGPT

Bill Benzon8 Dec 2022 22:25 UTC
2 points
0 comments8 min readLW link

High level dis­course struc­ture in ChatGPT: Part 2 [Quasi-sym­bolic?]

Bill Benzon10 Dec 2022 22:26 UTC
7 points
0 comments6 min readLW link

ChatGPT goes through a worm­hole hole in our Shandyesque uni­verse [vir­tual wacky weed]

Bill Benzon11 Dec 2022 11:59 UTC
−1 points
2 comments3 min readLW link

A brain­teaser for lan­guage models

Adam Scherlis12 Dec 2022 2:43 UTC
47 points
3 comments2 min readLW link

Let’s go meta: Gram­mat­i­cal knowl­edge and self-refer­en­tial sen­tences [ChatGPT]

Bill Benzon12 Dec 2022 21:50 UTC
5 points
0 comments9 min readLW link

An ex­plo­ra­tion of GPT-2′s em­bed­ding weights

Adam Scherlis13 Dec 2022 0:46 UTC
42 points
4 comments10 min readLW link

Us­ing GPT-Eliezer against ChatGPT Jailbreaking

6 Dec 2022 19:54 UTC
170 points
85 comments9 min readLW link

Ex­tract­ing and Eval­u­at­ing Causal Direc­tion in LLMs’ Activations

14 Dec 2022 14:33 UTC
29 points
5 comments11 min readLW link

Ab­stract con­cepts and met­al­in­gual defi­ni­tion: Does ChatGPT un­der­stand jus­tice and char­ity?

Bill Benzon16 Dec 2022 21:01 UTC
2 points
0 comments13 min readLW link

Does ChatGPT’s perfor­mance war­rant work­ing on a tu­tor for chil­dren? [It’s time to take it to the lab.]

Bill Benzon19 Dec 2022 15:12 UTC
13 points
5 comments4 min readLW link
(new-savanna.blogspot.com)

ChatGPT un­der­stands, but largely does not gen­er­ate Span­glish (and other code-mixed) text

Milan W23 Dec 2022 17:40 UTC
15 points
4 comments4 min readLW link

The Limit of Lan­guage Models

DragonGod6 Jan 2023 23:53 UTC
44 points
26 comments4 min readLW link

Thoughts on the im­pli­ca­tions of GPT-3, two years ago and NOW [here be drag­ons, we’re swim­ming, fly­ing and talk­ing with them]

Bill Benzon29 Dec 2022 20:05 UTC
0 points
0 comments5 min readLW link

[Question] Don’t you think RLHF solves outer al­ign­ment?

Charbel-Raphaël4 Nov 2022 0:36 UTC
9 points
23 comments1 min readLW link

MAKE IT BETTER (a po­etic demon­stra­tion of the ba­nal­ity of GPT-3)

rogersbacon2 Jan 2023 20:47 UTC
7 points
2 comments5 min readLW link

Dis­cur­sive Com­pe­tence in ChatGPT, Part 1: Talk­ing with Dragons

Bill Benzon5 Jan 2023 21:01 UTC
2 points
0 comments6 min readLW link

ChatGPT tells sto­ries about XP-708-DQ, Eliezer, drag­ons, dark sor­cer­esses, and un­al­igned robots be­com­ing aligned

Bill Benzon8 Jan 2023 23:21 UTC
6 points
2 comments18 min readLW link
No comments.