RSS

Ma­chine Learn­ing (ML)

TagLast edit: 30 Apr 2023 1:48 UTC by keshavchan

Machine Learning refers to the general field of study that deals with automated statistical learning and pattern detection by non-biological systems. It can be seen as a sub-domain of artificial intelligence that specifically deals with modeling and prediction through the knowledge extracted from training data. As a multi-disciplinary area, it has borrowed concepts and ideas from other areas like pure mathematics and cognitive science.

Understanding different machine learning algorithms

The most widely used distinction is between unsupervised (e.g. k-means clustering, principal component analysis) vs supervised (e.g. Support Vector Machines, logistic regression) methods. The first approach identifies interesting patterns (e.g. clusters and latent dimensions) in unlabeled training data, whereas the second takes labeled training data and tries to predict the label for unlabeled data points from the same distribution.

Another important distinction relates to the bias/​variance tradeoff—some machine learning methods are capable of recognizing more complex patterns, but the tradeoff is that these methods can overfit and generalize poorly if there’s noise in the training data—especially if there’s not much training data available.

There are also subfields of machine learning devoted to operating on specific kinds of data. For example, Hidden Markov Models and recurrent neural networks operate on time series data. Convolutional neural networks are commonly applied to image data.

Applications

The use of machine learning has been widespread since its formal definition in the 50’s. The ability to make predictions based on data has been extensively used in areas such as analysis of financial markets, natural language processing and even brain-computer interfaces. Amazon’s product suggestion system makes use of training data in the form of past customer purchases in order to predict what customers might want to buy in the future.

In addition to its practical usefulness, machine learning has also offered insight into human cognitive organization. It seems likely machine learning will play an important role in the development of artificial general intelligence.

Further Reading & References

See Also

Towards White Box Deep Learning

Maciej Satkiewicz27 Mar 2024 18:20 UTC
15 points
3 comments1 min readLW link
(arxiv.org)

“Deep Learn­ing” Is Func­tion Approximation

Zack_M_Davis21 Mar 2024 17:50 UTC
87 points
28 comments10 min readLW link
(zackmdavis.net)

User-in­cli­na­tion-guess­ing al­gorithms: reg­is­ter­ing a goal

ProgramCrafter20 Mar 2024 15:55 UTC
2 points
0 comments2 min readLW link

Mechanism for fea­ture learn­ing in neu­ral net­works and back­prop­a­ga­tion-free ma­chine learn­ing models

Matt Goldenberg19 Mar 2024 14:55 UTC
8 points
1 comment1 min readLW link
(www.science.org)

De­con­fus­ing In-Con­text Learning

Arjun Panickssery25 Feb 2024 9:48 UTC
37 points
1 comment2 min readLW link

ChatGPT re­fuses to ac­cept a challenge where it would get shot be­tween the eyes [game the­ory]

Bill Benzon20 Feb 2024 16:55 UTC
4 points
6 comments4 min readLW link

And All the Shog­goths Merely Players

Zack_M_Davis10 Feb 2024 19:56 UTC
138 points
56 comments12 min readLW link

Trans­fer learn­ing and gen­er­al­iza­tion-qua-ca­pa­bil­ity in Bab­bage and Davinci (or, why di­vi­sion is bet­ter than Span­ish)

RP and agg
9 Feb 2024 7:00 UTC
50 points
6 comments3 min readLW link

“Gen­langs” and Zipf’s Law: Do lan­guages gen­er­ated by ChatGPT statis­ti­cally look hu­man?

Justin-Diamond31 Jan 2024 18:30 UTC
2 points
2 comments1 min readLW link
(arxiv.org)

Pro­ces­sor clock speeds are not how fast AIs think

Ege Erdil29 Jan 2024 14:39 UTC
128 points
55 comments2 min readLW link

Krueger Lab AI Safety In­tern­ship 2024

Joey Bream24 Jan 2024 19:17 UTC
3 points
0 comments1 min readLW link

Pre­dict­ing AGI by the Tur­ing Test

Yuxi_Liu22 Jan 2024 4:22 UTC
21 points
2 comments10 min readLW link
(yuxi-liu-wired.github.io)

The Per­cep­tron Controversy

Yuxi_Liu10 Jan 2024 23:07 UTC
64 points
18 comments1 min readLW link
(yuxi-liu-wired.github.io)

Strik­ing Im­pli­ca­tions for Learn­ing The­ory, In­ter­pretabil­ity — and Safety?

RogerDearnaley5 Jan 2024 8:46 UTC
35 points
4 comments2 min readLW link

[Question] Ter­minol­ogy: <some­thing>-ware for ML?

Oliver Sourbut3 Jan 2024 11:42 UTC
17 points
27 comments1 min readLW link

AIOS

samhealy31 Dec 2023 13:23 UTC
−3 points
5 comments6 min readLW link

Does ChatGPT know what a tragedy is?

Bill Benzon31 Dec 2023 7:10 UTC
2 points
4 comments5 min readLW link

The fu­ture of Hu­mans: Oper­a­tors of AI

François-Joseph Lacroix30 Dec 2023 23:46 UTC
1 point
0 comments1 min readLW link
(medium.com)

Ex­plor­ing the Resi­d­ual Stream of Trans­form­ers for Mechanis­tic In­ter­pretabil­ity — Explained

Zeping Yu26 Dec 2023 0:36 UTC
7 points
1 comment11 min readLW link

AI’s im­pact on biol­ogy re­search: Part I, today

octopocta23 Dec 2023 16:29 UTC
31 points
6 comments2 min readLW link

Re­view Re­port of David­son on Take­off Speeds (2023)

Trent Kannegieter22 Dec 2023 18:48 UTC
32 points
11 comments38 min readLW link

[Paper] Tra­jec­to­ries through se­man­tic spaces in schizophre­nia and the re­la­tion­ship to rip­ple bursts

bvbvbvbvbvbvbvbvbvbvbv15 Dec 2023 13:37 UTC
3 points
0 comments1 min readLW link
(www.pnas.org)

Con­cep­tual co­her­ence for con­crete cat­e­gories in hu­mans and LLMs

Bill Benzon9 Dec 2023 23:49 UTC
13 points
1 comment2 min readLW link

On pos­si­ble cross-fer­til­iza­tion be­tween AI and neu­ro­science [Creativity]

Bill Benzon27 Nov 2023 16:50 UTC
15 points
22 comments7 min readLW link

Epoch is hiring an ML Distributed Sys­tems Se­nior Researcher

24 Nov 2023 22:33 UTC
2 points
0 comments4 min readLW link
(careers.rethinkpriorities.org)

Pos­si­ble OpenAI’s Q* break­through and Deep­Mind’s AlphaGo-type sys­tems plus LLMs

Burny23 Nov 2023 3:16 UTC
37 points
25 comments2 min readLW link

A Girar­dian in­ter­pre­ta­tion of the Alt­man af­fair, it’s on my to-do list

Bill Benzon20 Nov 2023 12:21 UTC
2 points
0 comments1 min readLW link

Cheap Model → Big Model design

Maxwell Peterson19 Nov 2023 22:50 UTC
15 points
2 comments7 min readLW link

My Crit­i­cism of Sin­gu­lar Learn­ing Theory

Joar Skalse19 Nov 2023 15:19 UTC
77 points
56 comments12 min readLW link

A di­alec­ti­cal view of the his­tory of AI, Part 1: We’re only in the an­tithe­sis phase. [A syn­the­sis is in the fu­ture.]

Bill Benzon16 Nov 2023 12:34 UTC
4 points
0 comments12 min readLW link

[Question] When did Eliezer Yud­kowsky change his mind about neu­ral net­works?

[deactivated]14 Nov 2023 21:24 UTC
31 points
15 comments1 min readLW link

AISC Pro­ject: Model­ling Tra­jec­to­ries of Lan­guage Models

NickyP13 Nov 2023 14:33 UTC
25 points
0 comments12 min readLW link

[Question] Vec­tor search on a large dataset?

camsdixon10 Nov 2023 18:43 UTC
−1 points
2 comments1 min readLW link

Me­tac­u­lus In­tro­duces AI-Pow­ered Com­mu­nity In­sights to Re­veal Fac­tors Driv­ing User Forecasts

ChristianWilliams10 Nov 2023 17:57 UTC
6 points
0 comments1 min readLW link
(www.metaculus.com)

ChatGPT’s On­tolog­i­cal Land­scape

Bill Benzon1 Nov 2023 15:12 UTC
7 points
0 comments4 min readLW link

Grokking Beyond Neu­ral Networks

Jack Miller30 Oct 2023 17:28 UTC
9 points
0 comments2 min readLW link
(arxiv.org)

math ter­minol­ogy as convolution

bhauth30 Oct 2023 1:05 UTC
34 points
1 comment4 min readLW link
(www.bhauth.com)

Grokking, mem­o­riza­tion, and gen­er­al­iza­tion — a discussion

29 Oct 2023 23:17 UTC
63 points
10 comments23 min readLW link

[Question] Non­lin­ear limi­ta­tions of ReLUs

magfrump26 Oct 2023 18:51 UTC
13 points
1 comment1 min readLW link

An­nounc­ing Epoch’s newly ex­panded Pa­ram­e­ters, Com­pute and Data Trends in Ma­chine Learn­ing database

25 Oct 2023 2:55 UTC
18 points
0 comments1 min readLW link
(epochai.org)

Re­veal­ing In­ten­tion­al­ity In Lan­guage Models Through AdaVAE Guided Sampling

jdp20 Oct 2023 7:32 UTC
117 points
14 comments22 min readLW link

Fea­tures and Ad­ver­saries in MemoryDT

20 Oct 2023 7:32 UTC
30 points
6 comments25 min readLW link

Brains, Planes, Blimps, and Algorithms

ai dan18 Oct 2023 21:26 UTC
1 point
0 comments6 min readLW link

ChatGPT Plays 20 Ques­tions [some­times needs help]

Bill Benzon17 Oct 2023 17:30 UTC
5 points
3 comments12 min readLW link

Unity Gridworlds

WillPetillo15 Oct 2023 4:36 UTC
8 points
0 comments1 min readLW link

Un­der­stand­ing LLMs: Some ba­sic ob­ser­va­tions about words, syn­tax, and dis­course [w/​ a con­jec­ture about grokking]

Bill Benzon11 Oct 2023 19:13 UTC
5 points
0 comments5 min readLW link

Linkpost: Are Emer­gent Abil­ities in Large Lan­guage Models just In-Con­text Learn­ing?

Erich_Grunewald8 Oct 2023 12:14 UTC
12 points
6 comments2 min readLW link
(arxiv.org)

Mech In­terp Challenge: Oc­to­ber—De­ci­pher­ing the Sorted List Model

CallumMcDougall3 Oct 2023 10:57 UTC
23 points
0 comments3 min readLW link

Re­vis­it­ing the Man­i­fold Hypothesis

Aidan Rocke1 Oct 2023 23:55 UTC
10 points
19 comments4 min readLW link

Ba­sic Math­e­mat­ics of Pre­dic­tive Coding

Adam Shai29 Sep 2023 14:38 UTC
49 points
6 comments9 min readLW link

Dis­cur­sive Com­pe­tence in ChatGPT, Part 2: Me­mory for Texts

Bill Benzon28 Sep 2023 16:34 UTC
1 point
0 comments3 min readLW link

In­fluence func­tions—why, what and how

Nina Rimsky15 Sep 2023 20:42 UTC
69 points
6 comments8 min readLW link

Mech In­terp Challenge: Septem­ber—De­ci­pher­ing the Ad­di­tion Model

CallumMcDougall13 Sep 2023 22:23 UTC
35 points
0 comments4 min readLW link

Ex­pand­ing the Scope of Superposition

Derek Larson13 Sep 2023 17:38 UTC
10 points
0 comments4 min readLW link

Ex­plain­ing grokking through cir­cuit efficiency

8 Sep 2023 14:39 UTC
98 points
8 comments3 min readLW link
(arxiv.org)

Re­port on An­a­lyz­ing Con­no­ta­tion Frames in Evolv­ing Wikipe­dia Biographies

Maira30 Aug 2023 22:02 UTC
1 point
0 comments4 min readLW link

Ap­ply to a small iter­a­tion of MLAB to be run in Oxford

27 Aug 2023 14:21 UTC
12 points
0 comments1 min readLW link

Is this the be­gin­ning of the end for LLMS [as the royal road to AGI, what­ever that is]?

Bill Benzon24 Aug 2023 14:50 UTC
3 points
16 comments3 min readLW link

Causal­ity and a Cost Se­man­tics for Neu­ral Networks

scottviteri21 Aug 2023 21:02 UTC
22 points
1 comment9 min readLW link

Google Deep­Mind’s RT-2

SandXbox11 Aug 2023 11:26 UTC
9 points
1 comment1 min readLW link
(robotics-transformer2.github.io)

The po­si­tional em­bed­ding ma­trix and pre­vi­ous-to­ken heads: how do they ac­tu­ally work?

AdamYedidia10 Aug 2023 1:58 UTC
26 points
4 comments13 min readLW link

Mech In­terp Challenge: Au­gust—De­ci­pher­ing the First Unique Char­ac­ter Model

CallumMcDougall9 Aug 2023 19:14 UTC
34 points
1 comment3 min readLW link

Trad­ing off com­pute in train­ing and in­fer­ence (Overview)

Pablo Villalobos31 Jul 2023 16:03 UTC
31 points
1 comment7 min readLW link
(epochai.org)

AI Safety 101 : In­tro­duc­tion to Vi­sion Interpretability

28 Jul 2023 17:32 UTC
41 points
0 comments1 min readLW link
(github.com)

Visi­ble loss land­scape bas­ins don’t cor­re­spond to dis­tinct algorithms

Mikhail Samin28 Jul 2023 16:19 UTC
65 points
13 comments4 min readLW link

Thoughts on Loss Land­scapes and why Deep Learn­ing works

beren25 Jul 2023 16:41 UTC
52 points
4 comments18 min readLW link

How LLMs are and are not myopic

janus25 Jul 2023 2:19 UTC
122 points
14 comments8 min readLW link

GPT-2′s po­si­tional em­bed­ding ma­trix is a helix

AdamYedidia21 Jul 2023 4:16 UTC
42 points
18 comments4 min readLW link

Spec­u­la­tive in­fer­ences about path de­pen­dence in LLM su­per­vised fine-tun­ing from re­sults on lin­ear mode con­nec­tivity and model souping

RobertKirk20 Jul 2023 9:56 UTC
38 points
2 comments5 min readLW link

LLM mis­al­ign­ment can prob­a­bly be found with­out man­ual prompt engineering

ProgramCrafter8 Jul 2023 14:35 UTC
1 point
0 comments1 min readLW link

VC The­ory Overview

Joar Skalse2 Jul 2023 22:45 UTC
10 points
2 comments11 min readLW link

faster la­tent diffusion

bhauth2 Jul 2023 1:30 UTC
10 points
8 comments2 min readLW link
(www.bhauth.com)

Ele­ments of Com­pu­ta­tional Philos­o­phy, Vol. I: Truth

1 Jul 2023 11:44 UTC
11 points
6 comments1 min readLW link
(compphil.github.io)

Challenge pro­posal: small­est pos­si­ble self-hard­en­ing back­door for RLHF

Christopher King29 Jun 2023 16:56 UTC
7 points
0 comments2 min readLW link

re­solv­ing some neu­ral net­work mysteries

bhauth19 Jun 2023 0:09 UTC
44 points
6 comments2 min readLW link
(www.bhauth.com)

The (lo­cal) unit of in­tel­li­gence is FLOPs

boazbarak5 Jun 2023 18:23 UTC
40 points
7 comments5 min readLW link

Tu­tor-GPT & Ped­a­gog­i­cal Reasoning

courtlandleer5 Jun 2023 17:53 UTC
26 points
3 comments4 min readLW link

Neu­roevolu­tion, So­cial In­tel­li­gence, and Logic

vinnik.dmitry0731 May 2023 17:54 UTC
1 point
0 comments10 min readLW link

Align­ing an H-JEPA agent via train­ing on the out­puts of an LLM-based “ex­em­plary ac­tor”

Roman Leventov29 May 2023 11:08 UTC
12 points
10 comments30 min readLW link

Solv­ing the Mechanis­tic In­ter­pretabil­ity challenges: EIS VII Challenge 2

25 May 2023 15:37 UTC
71 points
1 comment13 min readLW link

Solv­ing the Mechanis­tic In­ter­pretabil­ity challenges: EIS VII Challenge 1

9 May 2023 19:41 UTC
119 points
1 comment10 min readLW link

Lan­guage mod­els can ex­plain neu­rons in lan­guage models

nz9 May 2023 17:29 UTC
23 points
0 comments1 min readLW link
(openai.com)

Against sac­ri­fic­ing AI trans­parency for gen­er­al­ity gains

Ape in the coat7 May 2023 6:52 UTC
3 points
0 comments2 min readLW link

Resi­d­ual stream norms grow ex­po­nen­tially over the for­ward pass

7 May 2023 0:46 UTC
72 points
24 comments11 min readLW link

[Question] Nat­u­ral Selec­tion vs Gra­di­ent Descent

CuriousApe111 May 2023 22:16 UTC
4 points
3 comments1 min readLW link

Im­ple­ment­ing a Trans­former from scratch in PyTorch—a write-up on my experience

Mislav Jurić25 Apr 2023 20:51 UTC
16 points
0 comments10 min readLW link

Sub­jec­tive AI/​ML Digest: April II

Boris T24 Apr 2023 18:33 UTC
1 point
0 comments1 min readLW link
(borisagain.substack.com)

Ar­chi­tec­ture-aware op­ti­mi­sa­tion: train ImageNet and more with­out hyperparameters

Chris Mingard22 Apr 2023 21:50 UTC
6 points
2 comments2 min readLW link

Neu­ral net­work poly­topes (Co­lab note­book)

Zach Furman21 Apr 2023 22:42 UTC
11 points
0 comments1 min readLW link
(colab.research.google.com)

Ap­prox­i­ma­tion is ex­pen­sive, but the lunch is cheap

19 Apr 2023 14:19 UTC
68 points
3 comments16 min readLW link

AI Align­ment Re­search Eng­ineer Ac­cel­er­a­tor (ARENA): call for applicants

CallumMcDougall17 Apr 2023 20:30 UTC
100 points
9 comments7 min readLW link

Mechanis­ti­cally in­ter­pret­ing time in GPT-2 small

16 Apr 2023 17:57 UTC
68 points
6 comments21 min readLW link

An­nounc­ing Epoch’s dash­board of key trends and figures in Ma­chine Learning

Jsevillamol13 Apr 2023 7:33 UTC
35 points
7 comments1 min readLW link
(epochai.org)

The sur­pris­ing pa­ram­e­ter effi­ciency of vi­sion models

beren8 Apr 2023 19:44 UTC
77 points
28 comments4 min readLW link

[Question] Where to be­gin in ML/​AI?

Jake the Student6 Apr 2023 20:45 UTC
8 points
4 comments1 min readLW link

[Question] Is “Re­cur­sive Self-Im­prove­ment” Rele­vant in the Deep Learn­ing Paradigm?

DragonGod6 Apr 2023 7:13 UTC
32 points
36 comments7 min readLW link

Sin­gu­lar­i­ties against the Sin­gu­lar­ity: An­nounc­ing Work­shop on Sin­gu­lar Learn­ing The­ory and Alignment

1 Apr 2023 9:58 UTC
87 points
0 comments1 min readLW link
(singularlearningtheory.com)

Imi­ta­tion Learn­ing from Lan­guage Feedback

30 Mar 2023 14:11 UTC
71 points
3 comments10 min readLW link

[Question] Why no ma­jor LLMs with mem­ory?

Kaj_Sotala28 Mar 2023 16:34 UTC
41 points
15 comments1 min readLW link

Prac­ti­cal Pit­falls of Causal Scrubbing

27 Mar 2023 7:47 UTC
87 points
17 comments13 min readLW link

Em­piri­cal risk min­i­miza­tion is fun­da­men­tally confused

Jesse Hoogland22 Mar 2023 16:58 UTC
32 points
5 comments1 min readLW link

Google’s PaLM-E: An Em­bod­ied Mul­ti­modal Lan­guage Model

SandXbox7 Mar 2023 4:11 UTC
86 points
7 comments1 min readLW link
(palm-e.github.io)

Is there a ML agent that aban­dons it’s util­ity func­tion out-of-dis­tri­bu­tion with­out los­ing ca­pa­bil­ities?

Christopher King22 Feb 2023 16:49 UTC
1 point
7 comments1 min readLW link

The shal­low re­al­ity of ‘deep learn­ing the­ory’

Jesse Hoogland22 Feb 2023 4:16 UTC
34 points
11 comments3 min readLW link
(www.jessehoogland.com)

Be­hav­ioral and mechanis­tic defi­ni­tions (of­ten con­fuse AI al­ign­ment dis­cus­sions)

LawrenceC20 Feb 2023 21:33 UTC
33 points
5 comments6 min readLW link

Paper: The Ca­pac­ity for Mo­ral Self-Cor­rec­tion in Large Lan­guage Models (An­thropic)

LawrenceC16 Feb 2023 19:47 UTC
65 points
9 comments1 min readLW link
(arxiv.org)

GPT-175bee

8 Feb 2023 18:58 UTC
119 points
13 comments1 min readLW link

A multi-dis­ci­plinary view on AI safety research

Roman Leventov8 Feb 2023 16:50 UTC
43 points
4 comments26 min readLW link

In­ter­view Daniel Mur­fet on Univer­sal Phenom­ena in Learn­ing Machines

Alexander Gietelink Oldenziel6 Feb 2023 0:00 UTC
44 points
1 comment16 min readLW link

ChatGPT in­ti­mates a tan­ta­l­iz­ing fu­ture; its core LLM is or­ga­nized on mul­ti­ple lev­els; and it has bro­ken the idea of think­ing.

Bill Benzon24 Jan 2023 19:05 UTC
5 points
0 comments5 min readLW link

Bioin­for­mat­ics 101

iy3d22 Jan 2023 2:36 UTC
5 points
0 comments4 min readLW link

Neu­ral net­works gen­er­al­ize be­cause of this one weird trick

Jesse Hoogland18 Jan 2023 0:10 UTC
157 points
26 comments53 min readLW link
(www.jessehoogland.com)

Spec­u­la­tion on Path-Depen­dance in Large Lan­guage Models.

NickyP15 Jan 2023 20:42 UTC
16 points
2 comments7 min readLW link

[Question] How Does the Hu­man Brain Com­pare to Deep Learn­ing on Sam­ple Effi­ciency?

DragonGod15 Jan 2023 19:49 UTC
10 points
6 comments1 min readLW link

Tracr: Com­piled Trans­form­ers as a Lab­o­ra­tory for In­ter­pretabil­ity | Deep­Mind

DragonGod13 Jan 2023 16:53 UTC
62 points
12 comments1 min readLW link
(arxiv.org)

Scal­ing laws vs in­di­vi­d­ual differences

beren10 Jan 2023 13:22 UTC
44 points
21 comments7 min readLW link

Paper: Su­per­po­si­tion, Me­moriza­tion, and Dou­ble Des­cent (An­thropic)

LawrenceC5 Jan 2023 17:54 UTC
53 points
11 comments1 min readLW link
(transformer-circuits.pub)

From Si­mon’s ant to ma­chine learn­ing, a parable

Bill Benzon4 Jan 2023 14:37 UTC
6 points
5 comments2 min readLW link

Ba­sic Facts about Lan­guage Model Internals

4 Jan 2023 13:01 UTC
130 points
18 comments9 min readLW link

Touch re­al­ity as soon as pos­si­ble (when do­ing ma­chine learn­ing re­search)

LawrenceC3 Jan 2023 19:11 UTC
107 points
7 comments8 min readLW link

On the Im­por­tance of Open Sourc­ing Re­ward Models

elandgre2 Jan 2023 19:01 UTC
17 points
5 comments6 min readLW link

[Question] Book recom­men­da­tions for the his­tory of ML?

Eleni Angelou28 Dec 2022 23:50 UTC
2 points
2 comments1 min readLW link

Durkon, an open-source tool for In­her­ently In­ter­pretable Modelling

abstractapplic24 Dec 2022 1:49 UTC
29 points
0 comments4 min readLW link

Pro­lifer­at­ing Education

Haris Rashid20 Dec 2022 19:22 UTC
−1 points
2 comments5 min readLW link
(www.harisrab.com)

Refram­ing in­ner alignment

davidad11 Dec 2022 13:53 UTC
53 points
13 comments4 min readLW link

Neu­ral net­works bi­ased to­wards ge­o­met­ri­cally sim­ple func­tions?

DavidHolmes8 Dec 2022 16:16 UTC
16 points
2 comments3 min readLW link

Ma­chine Learn­ing Consent

jefftk8 Dec 2022 3:50 UTC
38 points
14 comments3 min readLW link
(www.jefftk.com)

Mesa-Op­ti­miz­ers via Grokking

orthonormal6 Dec 2022 20:05 UTC
36 points
4 comments6 min readLW link

Ap­ply for the ML Up­skil­ling Win­ter Camp in Cam­bridge, UK [2-10 Jan]

hannah wing-yee2 Dec 2022 20:45 UTC
3 points
0 comments2 min readLW link

Multi-Com­po­nent Learn­ing and S-Curves

30 Nov 2022 1:37 UTC
61 points
24 comments7 min readLW link

Us­ing mechanis­tic in­ter­pretabil­ity to find in-dis­tri­bu­tion failure in toy transformers

Charlie George28 Nov 2022 19:39 UTC
6 points
0 comments4 min readLW link

Why square er­rors?

Aprillion (Peter Hozák)26 Nov 2022 13:40 UTC
41 points
11 comments2 min readLW link

Eng­ineer­ing Monose­man­tic­ity in Toy Models

18 Nov 2022 1:43 UTC
75 points
7 comments3 min readLW link
(arxiv.org)

[Question] Why don’t we have self driv­ing cars yet?

Linda Linsefors14 Nov 2022 12:19 UTC
22 points
16 comments1 min readLW link

Cau­tion when in­ter­pret­ing Deep­mind’s In-con­text RL paper

Sam Marks1 Nov 2022 2:42 UTC
103 points
6 comments4 min readLW link

Re­in­force­ment Learn­ing Goal Mis­gen­er­al­iza­tion: Can we guess what kind of goals are se­lected by de­fault?

25 Oct 2022 20:48 UTC
14 points
2 comments4 min readLW link

What will the scaled up GATO look like? (Up­dated with ques­tions)

Amal 25 Oct 2022 12:44 UTC
34 points
22 comments1 min readLW link

Learn­ing so­cietal val­ues from law as part of an AGI al­ign­ment strategy

John Nay21 Oct 2022 2:03 UTC
5 points
18 comments54 min readLW link

GD’s Im­plicit Bias on Separable Data

Xander Davies17 Oct 2022 4:13 UTC
25 points
0 comments7 min readLW link

Six (and a half) in­tu­itions for KL divergence

CallumMcDougall12 Oct 2022 21:07 UTC
153 points
25 comments10 min readLW link1 review
(www.perfectlynormal.co.uk)

QAPR 4: In­duc­tive biases

Quintin Pope10 Oct 2022 22:08 UTC
67 points
2 comments18 min readLW link

Paper: Dis­cov­er­ing novel al­gorithms with AlphaTen­sor [Deep­mind]

LawrenceC5 Oct 2022 16:20 UTC
82 points
18 comments1 min readLW link
(www.deepmind.com)

Paper+Sum­mary: OMNIGROK: GROKKING BEYOND ALGORITHMIC DATA

Marius Hobbhahn4 Oct 2022 7:22 UTC
46 points
11 comments1 min readLW link
(arxiv.org)

No free lunch the­o­rem is irrelevant

Catnee4 Oct 2022 0:21 UTC
18 points
7 comments1 min readLW link

If you want to learn tech­ni­cal AI safety, here’s a list of AI safety courses, read­ing lists, and resources

KatWoods3 Oct 2022 12:43 UTC
12 points
3 comments1 min readLW link

Four us­ages of “loss” in AI

TurnTrout2 Oct 2022 0:52 UTC
43 points
18 comments4 min readLW link

[Question] What Is the Idea Be­hind (Un-)Su­per­vised Learn­ing and Re­in­force­ment Learn­ing?

Morpheus30 Sep 2022 16:48 UTC
9 points
6 comments2 min readLW link

In­ter­est­ing pa­pers: for­mally ver­ify­ing DNNs

the gears to ascension30 Sep 2022 8:49 UTC
13 points
0 comments3 min readLW link

linkpost: loss basin visualization

Nathan Helm-Burger30 Sep 2022 3:42 UTC
14 points
1 comment1 min readLW link

LOVE in a sim­box is all you need

jacob_cannell28 Sep 2022 18:25 UTC
63 points
72 comments44 min readLW link1 review

My Thoughts on the ML Safety Course

zeshen27 Sep 2022 13:15 UTC
50 points
3 comments17 min readLW link

Sum­mary of ML Safety Course

zeshen27 Sep 2022 13:05 UTC
7 points
0 comments6 min readLW link

[MLSN #5]: Prize Compilation

Dan H26 Sep 2022 21:55 UTC
14 points
1 comment2 min readLW link

Brief Notes on Transformers

Adam Jermyn26 Sep 2022 14:46 UTC
46 points
3 comments2 min readLW link

Trends in Train­ing Dataset Sizes

Pablo Villalobos21 Sep 2022 15:47 UTC
25 points
2 comments5 min readLW link
(epochai.org)

Lev­er­ag­ing Le­gal In­for­mat­ics to Align AI

John Nay18 Sep 2022 20:39 UTC
11 points
0 comments3 min readLW link
(forum.effectivealtruism.org)

D&D.Sci Septem­ber 2022: The Allo­ca­tion Helm

abstractapplic16 Sep 2022 23:10 UTC
32 points
33 comments1 min readLW link

A mar­ket is a neu­ral network

David Hugh-Jones15 Sep 2022 21:53 UTC
6 points
4 comments8 min readLW link

[Question] Are Speed Su­per­in­tel­li­gences Fea­si­ble for Modern ML Tech­niques?

DragonGod14 Sep 2022 12:59 UTC
9 points
7 comments1 min readLW link

Deep Q-Net­works Explained

Jay Bailey13 Sep 2022 12:01 UTC
55 points
4 comments22 min readLW link

Can you force a neu­ral net­work to keep gen­er­al­iz­ing?

Q Home12 Sep 2022 10:14 UTC
2 points
10 comments5 min readLW link

Path de­pen­dence in ML in­duc­tive biases

10 Sep 2022 1:38 UTC
68 points
13 comments10 min readLW link

Fram­ing AI Childhoods

David Udell6 Sep 2022 23:40 UTC
37 points
8 comments4 min readLW link

Sur­vey of NLP Re­searchers: NLP is con­tribut­ing to AGI progress; ma­jor catas­tro­phe plausible

Sam Bowman31 Aug 2022 1:39 UTC
92 points
6 comments2 min readLW link

Break­ing down the train­ing/​de­ploy­ment dichotomy

Erik Jenner28 Aug 2022 21:45 UTC
30 points
3 comments3 min readLW link

The Shard The­ory Align­ment Scheme

David Udell25 Aug 2022 4:52 UTC
47 points
32 comments2 min readLW link

Stable Diffu­sion has been released

P.22 Aug 2022 19:42 UTC
15 points
7 comments1 min readLW link
(stability.ai)

A Mechanis­tic In­ter­pretabil­ity Anal­y­sis of Grokking

15 Aug 2022 2:41 UTC
368 points
47 comments36 min readLW link1 review
(colab.research.google.com)

Steganog­ra­phy in Chain of Thought Reasoning

A Ray8 Aug 2022 3:47 UTC
61 points
13 comments6 min readLW link

A Data limited future

Donald Hobson6 Aug 2022 14:56 UTC
52 points
25 comments2 min readLW link

Trans­former lan­guage mod­els are do­ing some­thing more general

Numendil3 Aug 2022 21:13 UTC
53 points
6 comments2 min readLW link

chin­chilla’s wild implications

nostalgebraist31 Jul 2022 1:18 UTC
410 points
128 comments11 min readLW link1 review

Quan­tum Ad­van­tage in Learn­ing from Experiments

Dennis Towne27 Jul 2022 15:49 UTC
5 points
5 comments1 min readLW link
(ai.googleblog.com)

[Question] Does agent foun­da­tions cover all fu­ture ML sys­tems?

Jonas Hallgren25 Jul 2022 1:17 UTC
2 points
0 comments1 min readLW link

Find­ing Skele­tons on Rashomon Ridge

24 Jul 2022 22:31 UTC
30 points
2 comments7 min readLW link

[Question] Im­pact of ” ‘Let’s think step by step’ is all you need”?

yrimon24 Jul 2022 20:59 UTC
20 points
2 comments1 min readLW link

Ma­chine Learn­ing Model Sizes and the Pa­ram­e­ter Gap [abridged]

Pablo Villalobos18 Jul 2022 16:51 UTC
20 points
0 comments1 min readLW link
(epochai.org)

Safety Im­pli­ca­tions of LeCun’s path to ma­chine intelligence

Ivan Vendrov15 Jul 2022 21:47 UTC
102 points
18 comments6 min readLW link

Grouped Loss may dis­fa­vor dis­con­tin­u­ous capabilities

Adam Jermyn9 Jul 2022 17:22 UTC
14 points
2 comments4 min readLW link

Train first VS prune first in neu­ral net­works.

Donald Hobson9 Jul 2022 15:53 UTC
20 points
5 comments2 min readLW link

Race Along Rashomon Ridge

7 Jul 2022 3:20 UTC
50 points
15 comments8 min readLW link

Deep neu­ral net­works are not opaque.

jem-mosig6 Jul 2022 18:03 UTC
22 points
14 comments3 min readLW link

Re­mak­ing Effi­cien­tZero (as best I can)

Hoagy4 Jul 2022 11:03 UTC
35 points
9 comments22 min readLW link

Yann LeCun, A Path Towards Au­tonomous Ma­chine In­tel­li­gence [link]

Bill Benzon27 Jun 2022 23:29 UTC
5 points
1 comment1 min readLW link

The Limits of Automation

milkandcigarettes23 Jun 2022 18:03 UTC
5 points
1 comment5 min readLW link
(milkandcigarettes.com)

The in­or­di­nately slow spread of good AGI con­ver­sa­tions in ML

Rob Bensinger21 Jun 2022 16:09 UTC
173 points
62 comments8 min readLW link

Key Papers in Lan­guage Model Safety

aogara20 Jun 2022 15:00 UTC
39 points
1 comment22 min readLW link

Re­search Ques­tions from Stained Glass Windows

StefanHex8 Jun 2022 12:38 UTC
4 points
0 comments2 min readLW link

Miriam Ye­vick on why both sym­bols and net­works are nec­es­sary for ar­tifi­cial minds

Bill Benzon6 Jun 2022 8:34 UTC
1 point
0 comments4 min readLW link

Machines vs Memes Part 3: Imi­ta­tion and Memes

ceru231 Jun 2022 13:36 UTC
7 points
0 comments7 min readLW link

Machines vs Memes Part 1: AI Align­ment and Memetics

Harriet Farlow31 May 2022 22:03 UTC
18 points
1 comment6 min readLW link

CNN fea­ture vi­su­al­iza­tion in 50 lines of code

StefanHex26 May 2022 11:02 UTC
17 points
4 comments5 min readLW link

Google’s Ima­gen uses larger text encoder

Ben Livengood24 May 2022 21:55 UTC
27 points
2 comments1 min readLW link

The No Free Lunch the­o­rems and their Razor

Adrià Garriga-alonso24 May 2022 6:40 UTC
56 points
3 comments9 min readLW link

[Question] Why does gra­di­ent de­scent always work on neu­ral net­works?

MichaelDickens20 May 2022 21:13 UTC
15 points
11 comments1 min readLW link

We have achieved Noob Gains in AI

phdead18 May 2022 20:56 UTC
117 points
20 comments7 min readLW link

Pre­dict­ing the Elec­tions with Deep Learn­ing—Part 1 - Results

Quentin Chenevier14 May 2022 12:54 UTC
0 points
0 comments1 min readLW link

Con­di­tions for math­e­mat­i­cal equiv­alence of Stochas­tic Gra­di­ent Des­cent and Nat­u­ral Selection

Oliver Sourbut9 May 2022 21:38 UTC
61 points
19 comments8 min readLW link1 review
(www.oliversourbut.net)

A Bird’s Eye View of the ML Field [Prag­matic AI Safety #2]

9 May 2022 17:18 UTC
163 points
6 comments35 min readLW link

[Question] Why hasn’t deep learn­ing gen­er­ated sig­nifi­cant eco­nomic value yet?

Alex_Altair30 Apr 2022 20:27 UTC
114 points
88 comments2 min readLW link

[Question] What is a train­ing “step” vs. “epi­sode” in ma­chine learn­ing?

Evan R. Murphy28 Apr 2022 21:53 UTC
10 points
4 comments1 min readLW link

dalle2 comments

nostalgebraist26 Apr 2022 5:30 UTC
183 points
14 comments13 min readLW link
(nostalgebraist.tumblr.com)

Make a neu­ral net­work in ~10 minutes

Arjun Yadav26 Apr 2022 5:24 UTC
8 points
0 comments4 min readLW link
(arjunyadav.net)

Skil­ling-up in ML Eng­ineer­ing for Align­ment: re­quest for comments

23 Apr 2022 15:11 UTC
19 points
0 comments1 min readLW link

Ex­plor­ing toy neu­ral nets un­der node re­moval. Sec­tion 1.

Donald Hobson13 Apr 2022 23:30 UTC
12 points
7 comments8 min readLW link

Play­ing with DALL·E 2

Dave Orr7 Apr 2022 18:49 UTC
165 points
118 comments6 min readLW link

How to train your trans­former

p.b.7 Apr 2022 9:34 UTC
6 points
0 comments8 min readLW link

New Scal­ing Laws for Large Lan­guage Models

1a3orn1 Apr 2022 20:41 UTC
243 points
22 comments5 min readLW link

Les­sons After a Cou­ple Months of Try­ing to Do ML Research

KevinRoWang22 Mar 2022 23:45 UTC
70 points
8 comments6 min readLW link

One pos­si­ble ap­proach to de­velop the best pos­si­ble gen­eral learn­ing algorithm

martillopart14 Mar 2022 19:24 UTC
3 points
0 comments7 min readLW link

Com­pute Trends — Com­par­i­son to OpenAI’s AI and Compute

12 Mar 2022 18:09 UTC
23 points
3 comments3 min readLW link

What we know about ma­chine learn­ing’s repli­ca­tion crisis

Younes Kamel5 Mar 2022 23:55 UTC
36 points
4 comments6 min readLW link
(youneskamel.substack.com)

An­ti­cor­re­lated Noise In­jec­tion for Im­proved Generalization

tailcalled20 Feb 2022 10:15 UTC
2 points
9 comments1 min readLW link

[Question] Is the com­pe­ti­tion/​co­op­er­a­tion be­tween sym­bolic AI and statis­ti­cal AI (ML) about his­tor­i­cal ap­proach to re­search /​ en­g­ineer­ing, or is it more fun­da­men­tally about what in­tel­li­gent agents “are”?

Edward Hammond17 Feb 2022 23:11 UTC
1 point
1 comment2 min readLW link

Com­pute Trends Across Three eras of Ma­chine Learning

16 Feb 2022 14:18 UTC
92 points
13 comments2 min readLW link

A com­pila­tion of mi­suses of statistics

Younes Kamel14 Feb 2022 21:53 UTC
4 points
11 comments13 min readLW link
(youneskamel.substack.com)

Ques­tion 1: Pre­dicted ar­chi­tec­ture of AGI learn­ing al­gorithm(s)

Cameron Berg10 Feb 2022 17:22 UTC
13 points
1 comment7 min readLW link

ML Sys­tems Will Have Weird Failure Modes

jsteinhardt26 Jan 2022 1:40 UTC
57 points
8 comments6 min readLW link
(bounded-regret.ghost.io)

Emo­tions = Re­ward Functions

jpyykko20 Jan 2022 18:46 UTC
16 points
10 comments5 min readLW link

Truth­ful LMs as a warm-up for al­igned AGI

Jacob_Hilton17 Jan 2022 16:49 UTC
65 points
14 comments13 min readLW link

Fu­ture ML Sys­tems Will Be Qual­i­ta­tively Different

jsteinhardt11 Jan 2022 19:50 UTC
118 points
10 comments5 min readLW link
(bounded-regret.ghost.io)

Reg­u­lariza­tion Causes Mo­du­lar­ity Causes Generalization

dkirmani1 Jan 2022 23:34 UTC
50 points
7 comments3 min readLW link

Re­in­force­ment Learn­ing Study Group

Kay Kozaronek26 Dec 2021 23:11 UTC
20 points
8 comments1 min readLW link

Re­searcher in­cen­tives cause smoother progress on bench­marks

ryan_greenblatt21 Dec 2021 4:13 UTC
20 points
4 comments1 min readLW link

Ev­i­dence Sets: Towards In­duc­tive-Bi­ases based Anal­y­sis of Pro­saic AGI

bayesian_kitten16 Dec 2021 22:41 UTC
22 points
10 comments21 min readLW link

Un­der­stand­ing and con­trol­ling auto-in­duced dis­tri­bu­tional shift

L Rudolf L13 Dec 2021 14:59 UTC
32 points
4 comments16 min readLW link

Magna Alta Doctrina

jacob_cannell11 Dec 2021 21:54 UTC
58 points
7 comments28 min readLW link

See­ing the In­visi­ble (And How to Think About Ma­chine Learn­ing)

Filip Dousek8 Dec 2021 21:04 UTC
3 points
0 comments3 min readLW link

Be­hav­ior Clon­ing is Miscalibrated

leogao5 Dec 2021 1:36 UTC
76 points
3 comments3 min readLW link

A Gen­er­al­iza­tion of ROC AUC for Bi­nary Classifiers

Adam Scherlis4 Dec 2021 21:47 UTC
10 points
0 comments2 min readLW link
(adam.scherlis.com)

Effi­cien­tZero: How It Works

1a3orn26 Nov 2021 15:17 UTC
292 points
50 comments29 min readLW link1 review

My ML Scal­ing bibliography

gwern23 Oct 2021 14:41 UTC
35 points
9 comments1 min readLW link
(www.gwern.net)

Bor­ing ma­chine learn­ing is where it’s at

George3d620 Oct 2021 11:23 UTC
28 points
16 comments3 min readLW link
(cerebralab.com)

[MLSN #1]: ICLR Safety Paper Roundup

Dan H18 Oct 2021 15:19 UTC
59 points
1 comment2 min readLW link

NLP Po­si­tion Paper: When Com­bat­ting Hype, Pro­ceed with Caution

Sam Bowman15 Oct 2021 20:57 UTC
46 points
14 comments1 min readLW link

[Pro­posal] Method of lo­cat­ing use­ful sub­nets in large models

Quintin Pope13 Oct 2021 20:52 UTC
9 points
0 comments2 min readLW link

NVIDIA and Microsoft re­leases 530B pa­ram­e­ter trans­former model, Me­ga­tron-Tur­ing NLG

Ozyrus11 Oct 2021 15:28 UTC
51 points
36 comments1 min readLW link
(developer.nvidia.com)

Au­to­mated Fact Check­ing: A Look at the Field

Hoagy6 Oct 2021 23:52 UTC
12 points
0 comments8 min readLW link

Prefer­ences from (real and hy­po­thet­i­cal) psy­chol­ogy papers

Stuart_Armstrong6 Oct 2021 9:06 UTC
15 points
0 comments2 min readLW link

Model­ling and Un­der­stand­ing SGD

J Bostock5 Oct 2021 13:41 UTC
8 points
0 comments3 min readLW link

An anal­y­sis of the Less Wrong D&D.Sci 4th Edi­tion game

Maxwell Peterson4 Oct 2021 0:03 UTC
18 points
7 comments5 min readLW link

Un­solved ML Safety Problems

jsteinhardt29 Sep 2021 16:00 UTC
59 points
2 comments3 min readLW link
(bounded-regret.ghost.io)

Neu­ral net /​ de­ci­sion tree hy­brids: a po­ten­tial path to­ward bridg­ing the in­ter­pretabil­ity gap

Nathan Helm-Burger23 Sep 2021 0:38 UTC
21 points
2 comments12 min readLW link

Vir­tual Ma­chine Learn­ing Con­fer­ences: The Good and the Bad

libai29 Aug 2021 19:26 UTC
4 points
0 comments3 min readLW link

Au­tore­gres­sive Propaganda

lsusr22 Aug 2021 2:18 UTC
25 points
3 comments3 min readLW link

New GPT-3 competitor

Quintin Pope12 Aug 2021 7:05 UTC
32 points
10 comments1 min readLW link

[Question] Ques­tion about Test-sets and Bayesian ma­chine learn­ing

Haziq Muhammad9 Aug 2021 17:16 UTC
2 points
8 comments1 min readLW link

Deep­Mind: Gen­er­ally ca­pa­ble agents emerge from open-ended play

Daniel Kokotajlo27 Jul 2021 14:19 UTC
247 points
53 comments2 min readLW link
(deepmind.com)

Ex­per­i­men­ta­tion with AI-gen­er­ated images (VQGAN+CLIP) | So­larpunk air­ships flee­ing a dragon

Kaj_Sotala15 Jul 2021 11:00 UTC
44 points
4 comments2 min readLW link
(kajsotala.fi)

The Effi­cient Mar­ket Hy­poth­e­sis in Research

libai8 Jul 2021 17:00 UTC
11 points
9 comments3 min readLW link

Pa­ram­e­ter counts in Ma­chine Learning

19 Jun 2021 16:04 UTC
47 points
16 comments7 min readLW link

“De­ci­sion Trans­former” (Tool AIs are se­cret Agent AIs)

gwern9 Jun 2021 1:06 UTC
37 points
4 comments1 min readLW link
(sites.google.com)

Thoughts on the Align­ment Im­pli­ca­tions of Scal­ing Lan­guage Models

leogao2 Jun 2021 21:32 UTC
82 points
11 comments17 min readLW link

SGD’s Bias

johnswentworth18 May 2021 23:19 UTC
61 points
16 comments3 min readLW link

Up­dat­ing the Lot­tery Ticket Hypothesis

johnswentworth18 Apr 2021 21:45 UTC
73 points
41 comments2 min readLW link

Place-Based Pro­gram­ming—Part 2 - Functions

lsusr16 Apr 2021 0:25 UTC
14 points
0 comments3 min readLW link

Place-Based Pro­gram­ming—Part 1 - Places

lsusr14 Apr 2021 22:18 UTC
29 points
18 comments2 min readLW link

Opinions on In­ter­pretable Ma­chine Learn­ing and 70 Sum­maries of Re­cent Papers

9 Apr 2021 19:19 UTC
141 points
17 comments102 min readLW link

The Ja­panese Quiz: a Thought Ex­per­i­ment of Statis­ti­cal Epistemology

DanB8 Apr 2021 17:37 UTC
11 points
0 comments9 min readLW link

I Trained a Neu­ral Net­work to Play Helltaker

lsusr7 Apr 2021 8:24 UTC
29 points
5 comments3 min readLW link

Pre­dic­tive Cod­ing has been Unified with Backpropagation

lsusr2 Apr 2021 21:42 UTC
174 points
51 comments2 min readLW link

[Link] Whit­tle­stone et al., The So­cietal Im­pli­ca­tions of Deep Re­in­force­ment Learning

Aryeh Englander10 Mar 2021 18:13 UTC
11 points
1 comment1 min readLW link
(jair.org)

The case for al­ign­ing nar­rowly su­per­hu­man models

Ajeya Cotra5 Mar 2021 22:29 UTC
184 points
75 comments38 min readLW link1 review

Mul­ti­modal Neu­rons in Ar­tifi­cial Neu­ral Networks

Kaj_Sotala5 Mar 2021 9:01 UTC
57 points
2 comments2 min readLW link
(distill.pub)

Ma­chine learn­ing could be fun­da­men­tally unexplainable

George3d616 Dec 2020 13:32 UTC
26 points
15 comments15 min readLW link
(cerebralab.com)

[Linkpost] AlphaFold: a solu­tion to a 50-year-old grand challenge in biology

adamShimi30 Nov 2020 17:33 UTC
54 points
22 comments1 min readLW link
(deepmind.com)

Model Depth as Panacea and Obfuscator

abstractapplic9 Nov 2020 0:02 UTC
8 points
3 comments15 min readLW link

Fre­quen­tist prac­tice in­cor­po­rates prior in­for­ma­tion all the time

Maxwell Peterson7 Nov 2020 20:43 UTC
18 points
0 comments4 min readLW link

the scal­ing “in­con­sis­tency”: openAI’s new insight

nostalgebraist7 Nov 2020 7:40 UTC
148 points
14 comments9 min readLW link
(nostalgebraist.tumblr.com)

Does SGD Pro­duce De­cep­tive Align­ment?

Mark Xu6 Nov 2020 23:48 UTC
96 points
9 comments16 min readLW link

“model scores” is a ques­tion­able concept

Maxwell Peterson6 Nov 2020 3:19 UTC
26 points
0 comments6 min readLW link

Su­per­vised learn­ing of out­puts in the brain

Steven Byrnes26 Oct 2020 14:32 UTC
28 points
9 comments10 min readLW link

[Question] GPT-3 + GAN

stick10917 Oct 2020 7:58 UTC
4 points
3 comments1 min readLW link

[Question] Why isn’t JS a pop­u­lar lan­guage for deep learn­ing?

Will Clark8 Oct 2020 14:36 UTC
12 points
21 comments1 min readLW link

My (Mis)Ad­ven­tures With Al­gorith­mic Ma­chine Learning

AHartNtkn20 Sep 2020 5:31 UTC
16 points
4 comments41 min readLW link

“Learn­ing to Sum­ma­rize with Hu­man Feed­back”—OpenAI

[deleted]7 Sep 2020 17:59 UTC
57 points
3 comments1 min readLW link

Us­ing GPT-N to Solve In­ter­pretabil­ity of Neu­ral Net­works: A Re­search Agenda

3 Sep 2020 18:27 UTC
67 points
11 comments2 min readLW link

in­ter­pret­ing GPT: the logit lens

nostalgebraist31 Aug 2020 2:47 UTC
202 points
34 comments11 min readLW link

Pong from pix­els with­out read­ing “Pong from Pix­els”

Ian McKenzie29 Aug 2020 17:26 UTC
17 points
1 comment7 min readLW link

Tech­ni­cal model re­fine­ment formalism

Stuart_Armstrong27 Aug 2020 11:54 UTC
19 points
0 comments6 min readLW link

Model splin­ter­ing: mov­ing from one im­perfect model to another

Stuart_Armstrong27 Aug 2020 11:53 UTC
79 points
10 comments33 min readLW link

Alex Ir­pan: “My AI Timelines Have Sped Up”

Vaniver19 Aug 2020 16:23 UTC
43 points
20 comments1 min readLW link
(www.alexirpan.com)

Search ver­sus design

Alex Flint16 Aug 2020 16:53 UTC
100 points
40 comments36 min readLW link1 review

Matt Botv­inick on the spon­ta­neous emer­gence of learn­ing algorithms

Adam Scholl12 Aug 2020 7:47 UTC
153 points
87 comments5 min readLW link

In­ter­pretabil­ity in ML: A Broad Overview

lifelonglearner4 Aug 2020 19:03 UTC
53 points
5 comments15 min readLW link

is gpt-3 few-shot ready for real ap­pli­ca­tions?

nostalgebraist3 Aug 2020 19:50 UTC
31 points
5 comments9 min readLW link
(nostalgebraist.tumblr.com)

[Question] What are the most im­por­tant pa­pers/​post/​re­sources to read to un­der­stand more of GPT-3?

adamShimi2 Aug 2020 20:53 UTC
22 points
4 comments1 min readLW link

UML IV: Lin­ear Predictors

Rafael Harth8 Jul 2020 19:06 UTC
15 points
0 comments9 min readLW link

GAN Discrim­i­na­tors Don’t Gen­er­al­ize?

tryactions8 Jun 2020 20:36 UTC
18 points
7 comments2 min readLW link

An Illus­trated Proof of the No Free Lunch Theorem

lifelonglearner8 Jun 2020 1:54 UTC
19 points
0 comments1 min readLW link
(mlu.red)

How can In­ter­pretabil­ity help Align­ment?

23 May 2020 16:16 UTC
37 points
3 comments9 min readLW link

UML final

Rafael Harth8 Mar 2020 20:43 UTC
22 points
1 comment14 min readLW link

UML XIII: On­line Learn­ing and Clustering

Rafael Harth1 Mar 2020 18:32 UTC
13 points
0 comments14 min readLW link

If I were a well-in­ten­tioned AI… I: Image classifier

Stuart_Armstrong26 Feb 2020 12:39 UTC
35 points
4 comments5 min readLW link

UML XII: Di­men­sion­al­ity Reduction

Rafael Harth23 Feb 2020 19:44 UTC
9 points
0 comments9 min readLW link

UML XI: Near­est Neigh­bor Schemes

Rafael Harth16 Feb 2020 20:30 UTC
15 points
3 comments9 min readLW link

Per­cep­trons Explained

lifelonglearner14 Feb 2020 17:34 UTC
13 points
2 comments1 min readLW link
(owenshen24.github.io)

A Sim­ple In­tro­duc­tion to Neu­ral Networks

Rafael Harth9 Feb 2020 22:02 UTC
34 points
13 comments18 min readLW link

UML IX: Ker­nels and Boosting

Rafael Harth2 Feb 2020 21:51 UTC
13 points
1 comment10 min readLW link

If Van der Waals was a neu­ral network

George3d628 Jan 2020 18:38 UTC
18 points
3 comments11 min readLW link
(blog.cerebralab.com)

[Question] Al­gorithms vs Compute

johnswentworth28 Jan 2020 17:34 UTC
26 points
11 comments1 min readLW link

UML VIII: Lin­ear Pre­dic­tors (2)

Rafael Harth26 Jan 2020 20:09 UTC
9 points
2 comments10 min readLW link

New pa­per: The In­cen­tives that Shape Behaviour

RyanCarey23 Jan 2020 19:07 UTC
23 points
5 comments1 min readLW link
(arxiv.org)

UML VII: Meta-Learning

Rafael Harth19 Jan 2020 18:23 UTC
14 points
0 comments15 min readLW link

Ar­tifi­cial In­tel­li­gence and Life Sciences (Why Big Data is not enough to cap­ture biolog­i­cal sys­tems?)

HansNauj15 Jan 2020 1:59 UTC
6 points
3 comments6 min readLW link

[Question] How do you do hy­per­pa­ram­e­ter searches in ML?

lsusr13 Jan 2020 3:45 UTC
9 points
3 comments1 min readLW link

UML VI: Stochas­tic Gra­di­ent Descent

Rafael Harth12 Jan 2020 21:59 UTC
13 points
0 comments10 min readLW link

UML V: Con­vex Learn­ing Problems

Rafael Harth5 Jan 2020 19:47 UTC
14 points
0 comments10 min readLW link

Un­der­stand­ing Ma­chine Learn­ing (III)

Rafael Harth25 Dec 2019 18:55 UTC
16 points
2 comments11 min readLW link

Un­der­stand­ing Ma­chine Learn­ing (II)

Rafael Harth22 Dec 2019 18:28 UTC
24 points
4 comments10 min readLW link

Un­der­stand­ing Ma­chine Learn­ing (I)

Rafael Harth20 Dec 2019 18:22 UTC
44 points
12 comments11 min readLW link

In­duc­tive bi­ases stick around

evhub18 Dec 2019 19:52 UTC
64 points
15 comments3 min readLW link

Un­der­stand­ing “Deep Dou­ble Des­cent”

evhub6 Dec 2019 0:00 UTC
146 points
51 comments5 min readLW link4 reviews

[1911.08265] Mas­ter­ing Atari, Go, Chess and Shogi by Plan­ning with a Learned Model | Arxiv

DragonGod21 Nov 2019 1:18 UTC
52 points
4 comments1 min readLW link
(arxiv.org)

Neu­ral nets as a model for how hu­mans make and un­der­stand vi­sual art

Owain_Evans9 Nov 2019 16:53 UTC
28 points
7 comments2 min readLW link
(owainevans.github.io)

AlphaS­tar: Im­pres­sive for RL progress, not for AGI progress

orthonormal2 Nov 2019 1:50 UTC
113 points
58 comments2 min readLW link1 review

[Question] Can this model grade a test with­out know­ing the an­swers?

Elizabeth31 Aug 2019 0:53 UTC
20 points
3 comments1 min readLW link

Ta­boo­ing ‘Agent’ for Pro­saic Alignment

Hjalmar_Wijk23 Aug 2019 2:55 UTC
56 points
10 comments6 min readLW link

A Primer on Ma­trix Calcu­lus, Part 2: Ja­co­bi­ans and other fun

Matthew Barnett15 Aug 2019 1:13 UTC
22 points
7 comments6 min readLW link

“De­sign­ing agent in­cen­tives to avoid re­ward tam­per­ing”, DeepMind

gwern14 Aug 2019 16:57 UTC
28 points
15 comments1 min readLW link
(medium.com)

Why Gra­di­ents Van­ish and Explode

Matthew Barnett9 Aug 2019 2:54 UTC
25 points
9 comments3 min readLW link

Which of these five AI al­ign­ment re­search pro­jects ideas are no good?

rmoehn8 Aug 2019 7:17 UTC
25 points
13 comments1 min readLW link

Self-Su­per­vised Learn­ing and AGI Safety

Steven Byrnes7 Aug 2019 14:21 UTC
29 points
9 comments12 min readLW link

Re­think­ing Batch Normalization

Matthew Barnett2 Aug 2019 20:21 UTC
20 points
5 comments8 min readLW link

Cross-Val­i­da­tion vs Bayesian Model Comparison

johnswentworth21 Jul 2019 18:14 UTC
25 points
2 comments4 min readLW link

Let’s Read: Su­per­hu­man AI for mul­ti­player poker

Yuxi_Liu14 Jul 2019 6:22 UTC
56 points
6 comments8 min readLW link

Ma­chine Learn­ing Pro­jects on IDA

24 Jun 2019 18:38 UTC
49 points
3 comments2 min readLW link

“The Bit­ter Les­son”, an ar­ti­cle about com­pute vs hu­man knowl­edge in AI

the gears to ascension21 Jun 2019 17:24 UTC
52 points
14 comments4 min readLW link
(www.incompleteideas.net)

On AI and Compute

johncrox3 Apr 2019 19:00 UTC
36 points
10 comments8 min readLW link

Declar­a­tive Mathematics

johnswentworth21 Mar 2019 19:05 UTC
58 points
10 comments3 min readLW link

Some thoughts af­ter read­ing Ar­tifi­cial In­tel­li­gence: A Modern Approach

swift_spiral19 Mar 2019 23:39 UTC
38 points
4 comments2 min readLW link

Com­plex­ity Penalties in Statis­ti­cal Learning

michael_h6 Feb 2019 4:13 UTC
31 points
3 comments6 min readLW link

Learn­ing with catastrophes

paulfchristiano23 Jan 2019 3:01 UTC
27 points
9 comments4 min readLW link

Rein­ter­pret­ing “AI and Com­pute”

habryka25 Dec 2018 21:12 UTC
30 points
9 comments1 min readLW link
(aiimpacts.org)

Rea­sons com­pute may not drive AI ca­pa­bil­ities growth

Tristan H19 Dec 2018 22:13 UTC
42 points
10 comments8 min readLW link

Pro­saic AI alignment

paulfchristiano20 Nov 2018 13:56 UTC
46 points
10 comments8 min readLW link

Com­pet­i­tive Mar­kets as Distributed Backprop

johnswentworth10 Nov 2018 16:47 UTC
52 points
10 comments4 min readLW link1 review

Speci­fi­ca­tion gam­ing ex­am­ples in AI

Samuel Rødal10 Nov 2018 12:00 UTC
24 points
6 comments1 min readLW link
(docs.google.com)

Dis­cus­sion on the ma­chine learn­ing ap­proach to AI safety

Vika1 Nov 2018 20:54 UTC
27 points
3 comments4 min readLW link

The Un­rea­son­able Effec­tive­ness of Deep Learning

Richard_Ngo30 Sep 2018 15:48 UTC
85 points
5 comments13 min readLW link
(thinkingcomplete.blogspot.com)

Deep learn­ing—deeper flaws?

Richard_Ngo24 Sep 2018 18:40 UTC
39 points
17 comments4 min readLW link
(thinkingcomplete.blogspot.com)

Mak­ing a Differ­ence Tem­pore: In­sights from ‘Re­in­force­ment Learn­ing: An In­tro­duc­tion’

TurnTrout5 Jul 2018 0:34 UTC
33 points
6 comments8 min readLW link

Ma­chine Learn­ing Anal­ogy for Med­i­ta­tion (illus­trated)

abramdemski28 Jun 2018 22:51 UTC
97 points
48 comments1 min readLW link

OpenAI re­leases func­tional Dota 5v5 bot, aims to beat world cham­pi­ons by August

habryka26 Jun 2018 22:40 UTC
53 points
12 comments1 min readLW link
(blog.openai.com)

Begin­ning Ma­chine Learning

crybx30 Apr 2018 15:54 UTC
12 points
4 comments6 min readLW link

Us­ing ra­tio­nal­ity to de­bug Ma­chine Learning

Dr_Manhattan10 Apr 2018 20:03 UTC
20 points
3 comments1 min readLW link
(amid.fish)

Op­ti­miz­ing a Week of Ma­chine Learn­ing Learning

Raemon9 Jan 2018 6:55 UTC
8 points
2 comments3 min readLW link

Mas­ter­ing Chess and Shogi by Self-Play with a Gen­eral Re­in­force­ment Learn­ing Algorithm

DragonGod6 Dec 2017 6:01 UTC
13 points
4 comments1 min readLW link
(arxiv.org)

Deep­Mind ar­ti­cle: AI Safety Gridworlds

scarcegreengrass30 Nov 2017 16:13 UTC
25 points
6 comments1 min readLW link
(deepmind.com)

LDL 7: I wish I had a map

magfrump30 Nov 2017 2:03 UTC
13 points
2 comments3 min readLW link

LDL 4: Big data is a pain in the ass

magfrump25 Oct 2017 20:59 UTC
6 points
0 comments3 min readLW link

LDL 2: Non­con­vex Optimization

magfrump20 Oct 2017 18:20 UTC
13 points
13 comments4 min readLW link

Ex­am­ples of AI’s be­hav­ing badly

Stuart_Armstrong16 Jul 2015 10:01 UTC
41 points
41 comments1 min readLW link

The Brain as a Univer­sal Learn­ing Machine

jacob_cannell24 Jun 2015 21:45 UTC
187 points
171 comments19 min readLW link

[Link] Word-vec­tor based DL sys­tem achieves hu­man par­ity in ver­bal IQ tests

jacob_cannell13 Jun 2015 23:38 UTC
17 points
8 comments1 min readLW link

Con­cept Safety: Pro­duc­ing similar AI-hu­man con­cept spaces

Kaj_Sotala14 Apr 2015 20:39 UTC
51 points
45 comments8 min readLW link

Us­ing ma­chine learn­ing to pre­dict ro­man­tic com­pat­i­bil­ity: em­piri­cal results

JonahS17 Dec 2014 2:54 UTC
37 points
18 comments11 min readLW link

Con­nec­tion­ism: Model­ing the mind with neu­ral networks

Scott Alexander19 Jul 2011 1:16 UTC
59 points
20 comments8 min readLW link

[Link] Com­puter im­proves its Civ­i­liza­tion II game­play by read­ing the manual

Kaj_Sotala13 Jul 2011 12:00 UTC
49 points
5 comments4 min readLW link

The Ma­chine Learn­ing Per­son­al­ity Test

PhilGoetz4 Aug 2009 23:36 UTC
31 points
34 comments6 min readLW link

Link: In­ter­view with Vladimir Vapnik

Daniel_Burfoot25 Jul 2009 13:36 UTC
22 points
6 comments2 min readLW link

Log­i­cal or Con­nec­tion­ist AI?

Eliezer Yudkowsky17 Nov 2008 8:03 UTC
42 points
26 comments9 min readLW link

Sel­ling Nonapples

Eliezer Yudkowsky13 Nov 2008 20:10 UTC
75 points
78 comments7 min readLW link

The Weighted Ma­jor­ity Algorithm

Eliezer Yudkowsky12 Nov 2008 23:19 UTC
23 points
96 comments10 min readLW link

Worse Than Random

Eliezer Yudkowsky11 Nov 2008 19:01 UTC
46 points
102 comments12 min readLW link

Mag­i­cal Categories

Eliezer Yudkowsky24 Aug 2008 19:51 UTC
71 points
133 comments9 min readLW link

The “Out­side the Box” Box

Eliezer Yudkowsky12 Oct 2007 22:50 UTC
89 points
51 comments2 min readLW link

“In­duc­tive Bias”

Eliezer Yudkowsky8 Apr 2007 19:52 UTC
39 points
24 comments3 min readLW link
No comments.