RSS

GPT

TagLast edit: 25 Mar 2022 20:19 UTC by A_donor

GPT (Generative Pretrained Transformer) is a family of large transformer-based language models created by OpenAI. Its ability to generate remarkably human-like responses has relevance to discussions on AGI.

Col­lec­tion of GPT-3 results

Kaj_Sotala18 Jul 2020 20:04 UTC
88 points
24 comments1 min readLW link
(twitter.com)

[Question] To what ex­tent is GPT-3 ca­pa­ble of rea­son­ing?

TurnTrout20 Jul 2020 17:10 UTC
70 points
74 comments16 min readLW link

GPT-3: a dis­ap­point­ing paper

nostalgebraist29 May 2020 19:06 UTC
67 points
44 comments8 min readLW link1 review

Two Small Ex­per­i­ments on GPT-2

jimrandomh21 Feb 2019 2:59 UTC
54 points
28 comments1 min readLW link

GPT-3 Fic­tion Samples

gwern25 Jun 2020 16:12 UTC
62 points
18 comments1 min readLW link
(www.gwern.net)

$1000 bounty for OpenAI to show whether GPT3 was “de­liber­ately” pre­tend­ing to be stupi­der than it is

jacobjacob21 Jul 2020 18:42 UTC
54 points
40 comments2 min readLW link
(twitter.com)

Does GPT-2 Un­der­stand Any­thing?

Douglas Summers-Stay2 Jan 2020 17:09 UTC
37 points
23 comments5 min readLW link

‘This Waifu Does Not Ex­ist’: 100,000 StyleGAN & GPT-2 samples

gwern1 Mar 2019 4:29 UTC
39 points
6 comments1 min readLW link
(www.thiswaifudoesnotexist.net)

345M ver­sion GPT-2 released

lifelonglearner5 May 2019 2:49 UTC
37 points
0 comments1 min readLW link
(openai.com)

[Question] How “hon­est” is GPT-3?

abramdemski8 Jul 2020 19:38 UTC
72 points
18 comments5 min readLW link

[Question] How well can the GPT ar­chi­tec­ture solve the par­ity task?

FactorialCode11 Jul 2020 19:02 UTC
19 points
3 comments1 min readLW link

Align­ment As A Bot­tle­neck To Use­ful­ness Of GPT-3

johnswentworth21 Jul 2020 20:02 UTC
108 points
57 comments3 min readLW link

Repli­cat­ing the repli­ca­tion crisis with GPT-3?

skybrian22 Jul 2020 21:20 UTC
29 points
10 comments1 min readLW link

Can you get AGI from a Trans­former?

Steven Byrnes23 Jul 2020 15:27 UTC
106 points
39 comments12 min readLW link

OpenGPT-2: We Repli­cated GPT-2 Be­cause You Can Too

avturchin23 Aug 2019 11:32 UTC
18 points
0 comments1 min readLW link
(medium.com)

Devel­op­men­tal Stages of GPTs

orthonormal26 Jul 2020 22:03 UTC
140 points
74 comments7 min readLW link1 review

larger lan­guage mod­els may dis­ap­point you [or, an eter­nally un­finished draft]

nostalgebraist26 Nov 2021 23:08 UTC
221 points
28 comments31 min readLW link

Hu­mans Who Are Not Con­cen­trat­ing Are Not Gen­eral Intelligences

sarahconstantin25 Feb 2019 20:40 UTC
172 points
35 comments6 min readLW link1 review
(srconstantin.wordpress.com)

Writ­ing with GPT-3

Jacob Falkovich24 Jul 2020 15:22 UTC
42 points
0 comments4 min readLW link

Are we in an AI over­hang?

Andy Jones27 Jul 2020 12:48 UTC
253 points
106 comments4 min readLW link

[Question] How will in­ter­net fo­rums like LW be able to defend against GPT-style spam?

ChristianKl28 Jul 2020 20:12 UTC
14 points
18 comments1 min readLW link

An­a­lyz­ing the Prob­lem GPT-3 is Try­ing to Solve

adamShimi6 Aug 2020 21:58 UTC
16 points
2 comments4 min readLW link

The Hacker Learns to Trust

Ben Pace22 Jun 2019 0:27 UTC
80 points
18 comments8 min readLW link
(medium.com)

You Can Prob­a­bly Am­plify GPT3 Directly

Zachary Robertson26 Jul 2020 21:58 UTC
34 points
14 comments6 min readLW link

GPT-3, be­lief, and consistency

skybrian16 Aug 2020 23:12 UTC
18 points
7 comments2 min readLW link

[April Fools] User GPT2 is Banned

jimrandomh2 Apr 2019 6:00 UTC
63 points
20 comments1 min readLW link

GPT-2: 6-Month Fol­low-Up

lifelonglearner21 Aug 2019 5:06 UTC
28 points
1 comment1 min readLW link

Image GPT

Daniel Kokotajlo18 Jun 2020 11:41 UTC
29 points
27 comments1 min readLW link
(openai.com)

[AN #102]: Meta learn­ing by GPT-3, and a list of full pro­pos­als for AI alignment

Rohin Shah3 Jun 2020 17:20 UTC
38 points
6 comments10 min readLW link
(mailchi.mp)

OpenAI an­nounces GPT-3

gwern29 May 2020 1:49 UTC
67 points
23 comments1 min readLW link
(arxiv.org)

[Question] Will OpenAI’s work un­in­ten­tion­ally in­crease ex­is­ten­tial risks re­lated to AI?

adamShimi11 Aug 2020 18:16 UTC
50 points
56 comments1 min readLW link

in­ter­pret­ing GPT: the logit lens

nostalgebraist31 Aug 2020 2:47 UTC
137 points
32 comments11 min readLW link

Us­ing GPT-N to Solve In­ter­pretabil­ity of Neu­ral Net­works: A Re­search Agenda

3 Sep 2020 18:27 UTC
64 points
12 comments2 min readLW link

Hiring en­g­ineers and re­searchers to help al­ign GPT-3

paulfchristiano1 Oct 2020 18:54 UTC
206 points
14 comments3 min readLW link

[Question] If GPT-6 is hu­man-level AGI but costs $200 per page of out­put, what would hap­pen?

Daniel Kokotajlo9 Oct 2020 12:00 UTC
28 points
30 comments1 min readLW link

the scal­ing “in­con­sis­tency”: openAI’s new insight

nostalgebraist7 Nov 2020 7:40 UTC
146 points
12 comments9 min readLW link
(nostalgebraist.tumblr.com)

Ex­trap­o­lat­ing GPT-N performance

Lanrian18 Dec 2020 21:41 UTC
102 points
31 comments25 min readLW link1 review

Pre­dic­tions for GPT-N

hippke29 Jul 2020 1:16 UTC
36 points
31 comments1 min readLW link

is gpt-3 few-shot ready for real ap­pli­ca­tions?

nostalgebraist3 Aug 2020 19:50 UTC
31 points
5 comments9 min readLW link
(nostalgebraist.tumblr.com)

DALL-E by OpenAI

Daniel Kokotajlo5 Jan 2021 20:05 UTC
97 points
22 comments1 min readLW link

Au­tore­gres­sive Propaganda

lsusr22 Aug 2021 2:18 UTC
25 points
3 comments3 min readLW link

What’s Your Cog­ni­tive Al­gorithm?

Raemon18 Jun 2020 22:16 UTC
69 points
23 comments13 min readLW link

The “AI Dun­geons” Dragon Model is heav­ily path de­pen­dent (test­ing GPT-3 on ethics)

Rafael Harth21 Jul 2020 12:14 UTC
44 points
9 comments6 min readLW link

GPT-3 Gems

TurnTrout23 Jul 2020 0:46 UTC
33 points
10 comments48 min readLW link

[Question] Ques­tion on GPT-3 Ex­cel Demo

Zhitao Hou22 Jun 2020 20:31 UTC
0 points
2 comments1 min readLW link

[Question] Are we cer­tain that gpt-2 and similar al­gorithms are not self-aware?

Ozyrus11 Jul 2019 8:37 UTC
0 points
9 comments1 min readLW link

[Question] What should we ex­pect from GPT-3?

avturchin21 Mar 2019 14:28 UTC
22 points
2 comments1 min readLW link

[Question] List of pub­lic pre­dic­tions of what GPT-X can or can’t do?

Daniel Kokotajlo14 Jun 2020 14:25 UTC
20 points
9 comments1 min readLW link

GPT-3: A Summary

leogao2 Jun 2020 18:14 UTC
19 points
0 comments1 min readLW link
(leogao.dev)

[Question] If AI is based on GPT, how to en­sure its safety?

avturchin18 Jun 2020 20:33 UTC
20 points
11 comments1 min readLW link

[up­dated] how does gpt2′s train­ing cor­pus cap­ture in­ter­net dis­cus­sion? not well

nostalgebraist27 Jul 2020 22:30 UTC
25 points
3 comments2 min readLW link
(nostalgebraist.tumblr.com)

[Question] Prob­a­bil­ity that other ar­chi­tec­tures will scale as well as Trans­form­ers?

Daniel Kokotajlo28 Jul 2020 19:36 UTC
22 points
4 comments1 min readLW link

[Question] To what ex­tent are the scal­ing prop­er­ties of Trans­former net­works ex­cep­tional?

abramdemski28 Jul 2020 20:06 UTC
30 points
1 comment1 min readLW link

En­gag­ing Se­ri­ously with Short Timelines

sapphire29 Jul 2020 19:21 UTC
43 points
23 comments3 min readLW link

Suffi­ciently Ad­vanced Lan­guage Models Can Do Re­in­force­ment Learning

Zachary Robertson2 Aug 2020 15:32 UTC
21 points
7 comments7 min readLW link

[Question] How hard would it be to change GPT-3 in a way that al­lows au­dio?

ChristianKl28 Aug 2020 14:42 UTC
8 points
5 comments1 min readLW link

Why GPT wants to mesa-op­ti­mize & how we might change this

John_Maxwell19 Sep 2020 13:48 UTC
55 points
32 comments9 min readLW link

[Question] Where is hu­man level on text pre­dic­tion? (GPTs task)

Daniel Kokotajlo20 Sep 2020 9:00 UTC
27 points
19 comments1 min readLW link

The Col­lid­ing Ex­po­nen­tials of AI

VermillionStuka14 Oct 2020 23:31 UTC
27 points
16 comments5 min readLW link

Beyond 175 billion pa­ram­e­ters: Can we an­ti­ci­pate fu­ture GPT-X Ca­pa­bil­ities?

bakztfuture4 Dec 2020 23:42 UTC
−1 points
1 comment2 min readLW link

MIRI com­ments on Co­tra’s “Case for Align­ing Nar­rowly Su­per­hu­man Models”

Rob Bensinger5 Mar 2021 23:43 UTC
134 points
13 comments26 min readLW link

A sim­ple way to make GPT-3 fol­low instructions

Quintin Pope8 Mar 2021 2:57 UTC
10 points
5 comments4 min readLW link

Thoughts on the Align­ment Im­pli­ca­tions of Scal­ing Lan­guage Models

leogao2 Jun 2021 21:32 UTC
79 points
11 comments17 min readLW link

New GPT-3 competitor

Quintin Pope12 Aug 2021 7:05 UTC
32 points
10 comments1 min readLW link

AI-Based Code Gen­er­a­tion Us­ing GPT-J-6B

Tomás B.16 Jun 2021 15:05 UTC
21 points
15 comments1 min readLW link
(minimaxir.com)

GPT-Aug­mented Blogging

lsusr14 Sep 2021 11:55 UTC
52 points
18 comments13 min readLW link

I wanted to in­ter­view Eliezer Yud­kowsky but he’s busy so I simu­lated him instead

lsusr16 Sep 2021 7:34 UTC
109 points
33 comments5 min readLW link

Si­mu­lated Elon Musk Lives in a Simulation

lsusr18 Sep 2021 7:37 UTC
60 points
9 comments3 min readLW link

[Question] How much should you be will­ing to pay for an AGI?

Logan Zoellner20 Sep 2021 11:51 UTC
11 points
5 comments1 min readLW link

[Question] Any write­ups on GPT agency?

Ozyrus26 Sep 2021 22:55 UTC
4 points
6 comments1 min readLW link

[Question] Is GPT-3 already sam­ple-effi­cient?

Daniel Kokotajlo6 Oct 2021 13:38 UTC
36 points
32 comments1 min readLW link

NVIDIA and Microsoft re­leases 530B pa­ram­e­ter trans­former model, Me­ga­tron-Tur­ing NLG

Ozyrus11 Oct 2021 15:28 UTC
51 points
36 comments1 min readLW link
(developer.nvidia.com)

“Sum­ma­riz­ing Books with Hu­man Feed­back” (re­cur­sive GPT-3)

gwern15 Nov 2021 17:41 UTC
24 points
4 comments1 min readLW link
(openai.com)

Reader-gen­er­ated Essays

Henrik Karlsson3 Jan 2022 8:56 UTC
16 points
0 comments6 min readLW link
(escapingflatland.substack.com)

A one-ques­tion Tur­ing test for GPT-3

22 Jan 2022 18:17 UTC
84 points
23 comments5 min readLW link

Idea: build al­ign­ment dataset for very ca­pa­ble models

Quintin Pope12 Feb 2022 19:30 UTC
8 points
2 comments3 min readLW link

More GPT-3 and sym­bol grounding

Stuart_Armstrong23 Feb 2022 18:30 UTC
21 points
7 comments3 min readLW link

Per­sonal imi­ta­tion software

Flaglandbase7 Mar 2022 7:55 UTC
6 points
6 comments1 min readLW link

New GPT3 Im­pres­sive Ca­pa­bil­ities—In­struc­tGPT3 [1/​2]

WayZ13 Mar 2022 10:58 UTC
71 points
10 comments7 min readLW link

Hu­mans pre­tend­ing to be robots pre­tend­ing to be human

Richard_Kennaway28 Mar 2022 15:13 UTC
27 points
15 comments1 min readLW link

[Link] Train­ing Com­pute-Op­ti­mal Large Lan­guage Models

nostalgebraist31 Mar 2022 18:01 UTC
50 points
23 comments1 min readLW link
(arxiv.org)

New Scal­ing Laws for Large Lan­guage Models

1a3orn1 Apr 2022 20:41 UTC
205 points
20 comments5 min readLW link

GPT-3 and con­cept extrapolation

Stuart_Armstrong20 Apr 2022 10:39 UTC
19 points
28 comments1 min readLW link

Pos­i­tive out­comes un­der an un­al­igned AGI takeover

Yitz12 May 2022 7:45 UTC
19 points
12 comments3 min readLW link

Paper: Teach­ing GPT3 to ex­press un­cer­tainty in words

Owain_Evans31 May 2022 13:27 UTC
95 points
7 comments4 min readLW link

OpenAI: GPT-based LLMs show abil­ity to dis­crim­i­nate be­tween its own wrong an­swers, but in­abil­ity to ex­plain how/​why it makes that dis­crim­i­na­tion, even as model scales

Aditya Jain13 Jun 2022 23:33 UTC
14 points
5 comments1 min readLW link
(openai.com)

[Question] AI mis­al­ign­ment risk from GPT-like sys­tems?

fiso19 Jun 2022 17:35 UTC
10 points
8 comments1 min readLW link

[Question] What spe­cific dan­gers arise when ask­ing GPT-N to write an Align­ment Fo­rum post?

Matthew Barnett28 Jul 2020 2:56 UTC
43 points
14 comments1 min readLW link

Struc­tured Tasks for Lan­guage Models

Zachary Robertson29 Jul 2020 14:17 UTC
5 points
0 comments1 min readLW link

[Question] Is the work on AI al­ign­ment rele­vant to GPT?

Richard_Kennaway30 Jul 2020 12:23 UTC
12 points
5 comments1 min readLW link

Agen­tic Lan­guage Model Memes

FactorialCode1 Aug 2020 18:03 UTC
16 points
1 comment2 min readLW link

[Question] What are the most im­por­tant pa­pers/​post/​re­sources to read to un­der­stand more of GPT-3?

adamShimi2 Aug 2020 20:53 UTC
22 points
4 comments1 min readLW link

hu­man psy­chol­in­guists: a crit­i­cal appraisal

nostalgebraist31 Dec 2019 0:20 UTC
167 points
59 comments16 min readLW link2 reviews
(nostalgebraist.tumblr.com)

[Question] 10/​50/​90% chance of GPT-N Trans­for­ma­tive AI?

human_generated_text9 Aug 2020 0:10 UTC
24 points
8 comments1 min readLW link

May Gw­ern.net newslet­ter (w/​GPT-3 com­men­tary)

gwern2 Jun 2020 15:40 UTC
32 points
7 comments1 min readLW link
(www.gwern.net)

A trick for Safer GPT-N

Razied23 Aug 2020 0:39 UTC
7 points
1 comment2 min readLW link

From GPT to AGI

ChristianKl31 Aug 2020 13:28 UTC
6 points
7 comments1 min readLW link

on “learn­ing to sum­ma­rize”

nostalgebraist12 Sep 2020 3:20 UTC
25 points
13 comments8 min readLW link
(nostalgebraist.tumblr.com)

[Question] GPT-3 + GAN

stick10917 Oct 2020 7:58 UTC
4 points
4 comments1 min readLW link

All GPT skills are translation

p.b.13 Dec 2020 20:06 UTC
4 points
0 comments2 min readLW link

Beta test GPT-3 based re­search assistant

jungofthewon16 Dec 2020 13:42 UTC
34 points
2 comments1 min readLW link

The case for al­ign­ing nar­rowly su­per­hu­man models

Ajeya Cotra5 Mar 2021 22:29 UTC
181 points
74 comments38 min readLW link

[Question] What will GPT-4 be in­ca­pable of?

Michaël Trazzi6 Apr 2021 19:57 UTC
34 points
32 comments1 min readLW link

How I Learned to Stop Wor­ry­ing and Love MUM

Waddington20 May 2021 7:57 UTC
2 points
0 comments3 min readLW link

Spec­u­la­tions against GPT-n writ­ing al­ign­ment papers

Donald Hobson7 Jun 2021 21:13 UTC
31 points
6 comments2 min readLW link

What does GPT-3 un­der­stand? Sym­bol ground­ing and Chi­nese rooms

Stuart_Armstrong3 Aug 2021 13:14 UTC
39 points
15 comments12 min readLW link

[Question] 1h-vol­un­teers needed for a small AI Safety-re­lated re­search pro­ject

PabloAMC16 Aug 2021 17:53 UTC
2 points
0 comments1 min readLW link

[Question] Who owns OpenAI’s new lan­guage model?

ioannes14 Feb 2019 17:51 UTC
16 points
9 comments1 min readLW link

Truth­ful AI: Devel­op­ing and gov­ern­ing AI that does not lie

18 Oct 2021 18:37 UTC
81 points
9 comments10 min readLW link

AMA on Truth­ful AI: Owen Cot­ton-Bar­ratt, Owain Evans & co-authors

Owain_Evans22 Oct 2021 16:23 UTC
31 points
15 comments1 min readLW link

Hegel vs. GPT-3

Bezzi27 Oct 2021 5:55 UTC
9 points
21 comments2 min readLW link

[Question] What ex­actly is GPT-3′s base ob­jec­tive?

Daniel Kokotajlo10 Nov 2021 0:57 UTC
50 points
15 comments2 min readLW link

Truth­ful LMs as a warm-up for al­igned AGI

Jacob_Hilton17 Jan 2022 16:49 UTC
64 points
14 comments13 min readLW link

How I’m think­ing about GPT-N

delton13717 Jan 2022 17:11 UTC
44 points
21 comments18 min readLW link

Un­com­pet­i­tive pro­gram­ming with GPT-3

Bezzi6 Feb 2022 10:19 UTC
7 points
8 comments3 min readLW link

Us­ing GPT-3 for pre­vent­ing con­flict dur­ing mes­sag­ing — a pitch for an app

Eli_17 Mar 2022 11:02 UTC
19 points
16 comments3 min readLW link

[Question] If you lose enough Good Heart To­kens, will you lose real-world money?

Yitz1 Apr 2022 21:11 UTC
9 points
0 comments1 min readLW link

Test­ing PaLM prompts on GPT3

Yitz6 Apr 2022 5:21 UTC
101 points
15 comments8 min readLW link

Is GPT3 a Good Ra­tion­al­ist? - In­struc­tGPT3 [2/​2]

WayZ7 Apr 2022 13:46 UTC
10 points
0 comments7 min readLW link

PaLM in “Ex­trap­o­lat­ing GPT-N perfor­mance”

Lanrian6 Apr 2022 13:05 UTC
76 points
19 comments2 min readLW link

What is the solu­tion to the Align­ment prob­lem?

30 Apr 2022 23:19 UTC
24 points
2 comments1 min readLW link

Get­ting GPT-3 to pre­dict Me­tac­u­lus questions

MathiasKB6 May 2022 6:01 UTC
67 points
8 comments2 min readLW link

A pos­si­ble check against mo­ti­vated rea­son­ing us­ing elicit.org

david reinstein18 May 2022 20:52 UTC
4 points
0 comments1 min readLW link

RL with KL penalties is bet­ter seen as Bayesian inference

25 May 2022 9:23 UTC
67 points
12 comments12 min readLW link

Who mod­els the mod­els that model mod­els? An ex­plo­ra­tion of GPT-3′s in-con­text model fit­ting ability

Lovre7 Jun 2022 19:37 UTC
104 points
11 comments9 min readLW link

In­ves­ti­gat­ing causal un­der­stand­ing in LLMs

14 Jun 2022 13:57 UTC
23 points
2 comments13 min readLW link

Con­tra Hofs­tadter on GPT-3 Nonsense

rictic15 Jun 2022 21:53 UTC
199 points
17 comments2 min readLW link
No comments.