RSS

Ma­chine Learn­ing (ML)

TagLast edit: 29 May 2022 18:22 UTC by Raemon

Machine Learning refers to the general field of study that deals with automated statistical learning and pattern detection by non-biological systems. It can be seen as a sub-domain of artificial intelligence that specifically deals with modeling and prediction through the knowledge extracted from training data. As a multi-disciplinary area, it has borrowed concepts and ideas from other areas like pure mathematics and cognitive science.

Understanding different machine learning algorithms

The most widely used distinction is between unsupervised (e.g. k-means clustering, principal component analysis) vs supervised (e.g. Support Vector Machines, logistic regression) methods. The first approach identifies interesting patterns (e.g. clusters and latent dimensions) in unlabeled training data, whereas the second takes labeled training data and tries to predict the label for unlabeled data points from the same distribution.

Another important distinction relates to the bias/​variance tradeoff—some machine learning methods are are capable of recognizing more complex patterns, but the tradeoff is that these methods can overfit and generalize poorly if there’s noise in the training data—especially if there’s not much training data available.

There are also subfields of machine learning devoted to operating on specific kinds of data. For example, Hidden Markov Models and recurrent neural networks operate on time series data. Convolutional neural networks are commonly applied to image data.

Applications

The use of machine learning has been widespread since its formal definition in the 50’s. The ability to make predictions based on data has been extensively used in areas such as analysis of financial markets, natural language processing and even brain-computer interfaces. Amazon’s product suggestion system makes use of training data in the form of past customer purchases in order to predict what customers might want to buy in the future.

In addition to its practical usefulness, machine learning has also offered insight into human cognitive organization. It seems likely machine learning will play an important role in the development of artificial general intelligence.

Further Reading & References

See Also

Paper: Dis­cov­er­ing novel al­gorithms with AlphaTen­sor [Deep­mind]

LawrenceC5 Oct 2022 16:20 UTC
80 points
18 comments1 min readLW link
(www.deepmind.com)

Pre­dic­tive Cod­ing has been Unified with Backpropagation

lsusr2 Apr 2021 21:42 UTC
165 points
44 comments2 min readLW link

Play­ing with DALL·E 2

Dave Orr7 Apr 2022 18:49 UTC
165 points
116 comments6 min readLW link

An Illus­trated Proof of the No Free Lunch Theorem

lifelonglearner8 Jun 2020 1:54 UTC
17 points
0 comments1 min readLW link
(mlu.red)

Matt Botv­inick on the spon­ta­neous emer­gence of learn­ing algorithms

Adam Scholl12 Aug 2020 7:47 UTC
147 points
87 comments5 min readLW link

Us­ing GPT-N to Solve In­ter­pretabil­ity of Neu­ral Net­works: A Re­search Agenda

3 Sep 2020 18:27 UTC
67 points
12 comments2 min readLW link

the scal­ing “in­con­sis­tency”: openAI’s new insight

nostalgebraist7 Nov 2020 7:40 UTC
146 points
14 comments9 min readLW link
(nostalgebraist.tumblr.com)

I Trained a Neu­ral Net­work to Play Helltaker

lsusr7 Apr 2021 8:24 UTC
29 points
5 comments3 min readLW link

Effi­cien­tZero: How It Works

1a3orn26 Nov 2021 15:17 UTC
271 points
42 comments29 min readLW link

What we know about ma­chine learn­ing’s repli­ca­tion crisis

Younes Kamel5 Mar 2022 23:55 UTC
35 points
4 comments6 min readLW link
(youneskamel.substack.com)

The No Free Lunch the­o­rems and their Razor

Adrià Garriga-alonso24 May 2022 6:40 UTC
47 points
3 comments9 min readLW link

Un­solved ML Safety Problems

jsteinhardt29 Sep 2021 16:00 UTC
57 points
2 comments3 min readLW link
(bounded-regret.ghost.io)

Magna Alta Doctrina

jacob_cannell11 Dec 2021 21:54 UTC
37 points
7 comments28 min readLW link

Reg­u­lariza­tion Causes Mo­du­lar­ity Causes Generalization

dkirmani1 Jan 2022 23:34 UTC
49 points
7 comments3 min readLW link

One pos­si­ble ap­proach to de­velop the best pos­si­ble gen­eral learn­ing algorithm

martillopart14 Mar 2022 19:24 UTC
3 points
0 comments7 min readLW link

Neu­ral nets as a model for how hu­mans make and un­der­stand vi­sual art

Owain_Evans9 Nov 2019 16:53 UTC
28 points
7 comments2 min readLW link
(owainevans.github.io)

UML VI: Stochas­tic Gra­di­ent Descent

Rafael Harth12 Jan 2020 21:59 UTC
13 points
0 comments10 min readLW link

[Question] How do you do hy­per­pa­ram­e­ter searches in ML?

lsusr13 Jan 2020 3:45 UTC
9 points
3 comments1 min readLW link

Ma­chine Learn­ing Anal­ogy for Med­i­ta­tion (illus­trated)

abramdemski28 Jun 2018 22:51 UTC
87 points
48 comments1 min readLW link

Dis­cus­sion on the ma­chine learn­ing ap­proach to AI safety

Vika1 Nov 2018 20:54 UTC
26 points
3 comments4 min readLW link

UML IV: Lin­ear Predictors

Rafael Harth8 Jul 2020 19:06 UTC
15 points
0 comments9 min readLW link

Un­der­stand­ing Ma­chine Learn­ing (I)

Rafael Harth20 Dec 2019 18:22 UTC
44 points
11 comments11 min readLW link

Un­der­stand­ing Ma­chine Learn­ing (II)

Rafael Harth22 Dec 2019 18:28 UTC
24 points
4 comments10 min readLW link

Un­der­stand­ing Ma­chine Learn­ing (III)

Rafael Harth25 Dec 2019 18:55 UTC
16 points
0 comments11 min readLW link

UML V: Con­vex Learn­ing Problems

Rafael Harth5 Jan 2020 19:47 UTC
14 points
0 comments10 min readLW link

UML VII: Meta-Learning

Rafael Harth19 Jan 2020 18:23 UTC
14 points
0 comments15 min readLW link

UML VIII: Lin­ear Pre­dic­tors (2)

Rafael Harth26 Jan 2020 20:09 UTC
9 points
2 comments10 min readLW link

UML IX: Ker­nels and Boosting

Rafael Harth2 Feb 2020 21:51 UTC
13 points
1 comment10 min readLW link

A Sim­ple In­tro­duc­tion to Neu­ral Networks

Rafael Harth9 Feb 2020 22:02 UTC
34 points
13 comments18 min readLW link

UML XI: Near­est Neigh­bor Schemes

Rafael Harth16 Feb 2020 20:30 UTC
15 points
3 comments9 min readLW link

UML XII: Di­men­sion­al­ity Reduction

Rafael Harth23 Feb 2020 19:44 UTC
9 points
0 comments9 min readLW link

UML XIII: On­line Learn­ing and Clustering

Rafael Harth1 Mar 2020 18:32 UTC
13 points
0 comments14 min readLW link

UML final

Rafael Harth8 Mar 2020 20:43 UTC
22 points
1 comment14 min readLW link

[Link] Word-vec­tor based DL sys­tem achieves hu­man par­ity in ver­bal IQ tests

jacob_cannell13 Jun 2015 23:38 UTC
17 points
8 comments1 min readLW link

AlphaS­tar: Im­pres­sive for RL progress, not for AGI progress

orthonormal2 Nov 2019 1:50 UTC
113 points
58 comments2 min readLW link1 review

Un­der­stand­ing “Deep Dou­ble Des­cent”

evhub6 Dec 2019 0:00 UTC
135 points
51 comments5 min readLW link4 reviews

Let’s Read: Su­per­hu­man AI for mul­ti­player poker

Yuxi_Liu14 Jul 2019 6:22 UTC
56 points
6 comments8 min readLW link

OpenAI re­leases func­tional Dota 5v5 bot, aims to beat world cham­pi­ons by August

habryka26 Jun 2018 22:40 UTC
53 points
12 comments1 min readLW link
(blog.openai.com)

“The Bit­ter Les­son”, an ar­ti­cle about com­pute vs hu­man knowl­edge in AI

the gears to ascenscion21 Jun 2019 17:24 UTC
50 points
14 comments4 min readLW link
(www.incompleteideas.net)

[1911.08265] Mas­ter­ing Atari, Go, Chess and Shogi by Plan­ning with a Learned Model | Arxiv

DragonGod21 Nov 2019 1:18 UTC
52 points
4 comments1 min readLW link
(arxiv.org)

In­ter­pretabil­ity in ML: A Broad Overview

lifelonglearner4 Aug 2020 19:03 UTC
52 points
5 comments15 min readLW link

If I were a well-in­ten­tioned AI… I: Image classifier

Stuart_Armstrong26 Feb 2020 12:39 UTC
35 points
4 comments5 min readLW link

Search ver­sus design

Alex Flint16 Aug 2020 16:53 UTC
89 points
40 comments36 min readLW link1 review

Con­cept Safety: Pro­duc­ing similar AI-hu­man con­cept spaces

Kaj_Sotala14 Apr 2015 20:39 UTC
50 points
45 comments8 min readLW link

in­ter­pret­ing GPT: the logit lens

nostalgebraist31 Aug 2020 2:47 UTC
157 points
32 comments11 min readLW link

“In­duc­tive Bias”

Eliezer Yudkowsky8 Apr 2007 19:52 UTC
36 points
24 comments3 min readLW link

Cross-Val­i­da­tion vs Bayesian Model Comparison

johnswentworth21 Jul 2019 18:14 UTC
25 points
2 comments4 min readLW link

Su­per­vised learn­ing of out­puts in the brain

Steven Byrnes26 Oct 2020 14:32 UTC
27 points
9 comments10 min readLW link

Does SGD Pro­duce De­cep­tive Align­ment?

Mark Xu6 Nov 2020 23:48 UTC
83 points
6 comments16 min readLW link

Mul­ti­modal Neu­rons in Ar­tifi­cial Neu­ral Networks

Kaj_Sotala5 Mar 2021 9:01 UTC
57 points
2 comments2 min readLW link
(distill.pub)

[Link] Whit­tle­stone et al., The So­cietal Im­pli­ca­tions of Deep Re­in­force­ment Learning

Aryeh Englander10 Mar 2021 18:13 UTC
11 points
1 comment1 min readLW link
(jair.org)

Opinions on In­ter­pretable Ma­chine Learn­ing and 70 Sum­maries of Re­cent Papers

9 Apr 2021 19:19 UTC
132 points
16 comments102 min readLW link

Place-Based Pro­gram­ming—Part 1 - Places

lsusr14 Apr 2021 22:18 UTC
29 points
19 comments2 min readLW link

Place-Based Pro­gram­ming—Part 2 - Functions

lsusr16 Apr 2021 0:25 UTC
14 points
0 comments3 min readLW link

The Brain as a Univer­sal Learn­ing Machine

jacob_cannell24 Jun 2015 21:45 UTC
156 points
169 comments19 min readLW link

SGD’s Bias

johnswentworth18 May 2021 23:19 UTC
60 points
16 comments3 min readLW link

Ex­per­i­men­ta­tion with AI-gen­er­ated images (VQGAN+CLIP) | So­larpunk air­ships flee­ing a dragon

Kaj_Sotala15 Jul 2021 11:00 UTC
44 points
4 comments2 min readLW link
(kajsotala.fi)

Deep­Mind: Gen­er­ally ca­pa­ble agents emerge from open-ended play

Daniel Kokotajlo27 Jul 2021 14:19 UTC
247 points
53 comments2 min readLW link
(deepmind.com)

New GPT-3 competitor

Quintin Pope12 Aug 2021 7:05 UTC
32 points
10 comments1 min readLW link

Au­tore­gres­sive Propaganda

lsusr22 Aug 2021 2:18 UTC
25 points
3 comments3 min readLW link

Neu­ral net /​ de­ci­sion tree hy­brids: a po­ten­tial path to­ward bridg­ing the in­ter­pretabil­ity gap

Nathan Helm-Burger23 Sep 2021 0:38 UTC
21 points
2 comments12 min readLW link

Model­ling and Un­der­stand­ing SGD

Jemist5 Oct 2021 13:41 UTC
8 points
0 comments3 min readLW link

Prefer­ences from (real and hy­po­thet­i­cal) psy­chol­ogy papers

Stuart_Armstrong6 Oct 2021 9:06 UTC
15 points
0 comments2 min readLW link

Au­to­mated Fact Check­ing: A Look at the Field

Hoagy6 Oct 2021 23:52 UTC
12 points
0 comments8 min readLW link

NVIDIA and Microsoft re­leases 530B pa­ram­e­ter trans­former model, Me­ga­tron-Tur­ing NLG

Ozyrus11 Oct 2021 15:28 UTC
51 points
36 comments1 min readLW link
(developer.nvidia.com)

NLP Po­si­tion Paper: When Com­bat­ting Hype, Pro­ceed with Caution

Sam Bowman15 Oct 2021 20:57 UTC
46 points
15 comments1 min readLW link

[MLSN #1]: ICLR Safety Paper Roundup

Daniel Hendrycks18 Oct 2021 15:19 UTC
59 points
1 comment2 min readLW link

Bor­ing ma­chine learn­ing is where it’s at

George3d620 Oct 2021 11:23 UTC
28 points
16 comments3 min readLW link
(cerebralab.com)

My ML Scal­ing bibliography

gwern23 Oct 2021 14:41 UTC
35 points
9 comments1 min readLW link
(www.gwern.net)

Un­der­stand­ing and con­trol­ling auto-in­duced dis­tri­bu­tional shift

LRudL13 Dec 2021 14:59 UTC
26 points
3 comments16 min readLW link

Re­searcher in­cen­tives cause smoother progress on bench­marks

ryan_greenblatt21 Dec 2021 4:13 UTC
20 points
4 comments1 min readLW link

Fu­ture ML Sys­tems Will Be Qual­i­ta­tively Different

jsteinhardt11 Jan 2022 19:50 UTC
113 points
10 comments5 min readLW link
(bounded-regret.ghost.io)

Emo­tions = Re­ward Functions

jpyykko20 Jan 2022 18:46 UTC
16 points
10 comments5 min readLW link

ML Sys­tems Will Have Weird Failure Modes

jsteinhardt26 Jan 2022 1:40 UTC
54 points
8 comments6 min readLW link
(bounded-regret.ghost.io)

An­ti­cor­re­lated Noise In­jec­tion for Im­proved Generalization

tailcalled20 Feb 2022 10:15 UTC
2 points
9 comments1 min readLW link

New Scal­ing Laws for Large Lan­guage Models

1a3orn1 Apr 2022 20:41 UTC
220 points
21 comments5 min readLW link

How to train your trans­former

p.b.7 Apr 2022 9:34 UTC
6 points
0 comments8 min readLW link

Ex­plor­ing toy neu­ral nets un­der node re­moval. Sec­tion 1.

Donald Hobson13 Apr 2022 23:30 UTC
12 points
7 comments8 min readLW link

Make a neu­ral net­work in ~10 minutes

Arjun Yadav26 Apr 2022 5:24 UTC
8 points
0 comments4 min readLW link
(arjunyadav.net)

dalle2 comments

nostalgebraist26 Apr 2022 5:30 UTC
183 points
13 comments13 min readLW link
(nostalgebraist.tumblr.com)

A Bird’s Eye View of the ML Field [Prag­matic AI Safety #2]

9 May 2022 17:18 UTC
125 points
5 comments35 min readLW link

We have achieved Noob Gains in AI

phdead18 May 2022 20:56 UTC
114 points
21 comments7 min readLW link

Google’s Ima­gen uses larger text encoder

Ben Livengood24 May 2022 21:55 UTC
27 points
2 comments1 min readLW link

[Question] Im­pact of ” ‘Let’s think step by step’ is all you need”?

yrimon24 Jul 2022 20:59 UTC
19 points
2 comments1 min readLW link

Key Papers in Lan­guage Model Safety

aogara20 Jun 2022 15:00 UTC
37 points
1 comment22 min readLW link

The in­or­di­nately slow spread of good AGI con­ver­sa­tions in ML

Rob Bensinger21 Jun 2022 16:09 UTC
161 points
66 comments8 min readLW link

Re­mak­ing Effi­cien­tZero (as best I can)

Hoagy4 Jul 2022 11:03 UTC
33 points
9 comments22 min readLW link

Train first VS prune first in neu­ral net­works.

Donald Hobson9 Jul 2022 15:53 UTC
20 points
5 comments2 min readLW link

Safety Im­pli­ca­tions of LeCun’s path to ma­chine intelligence

Ivan Vendrov15 Jul 2022 21:47 UTC
89 points
16 comments6 min readLW link

[Question] Does agent foun­da­tions cover all fu­ture ML sys­tems?

Jonas Hallgren25 Jul 2022 1:17 UTC
2 points
0 comments1 min readLW link

chin­chilla’s wild implications

nostalgebraist31 Jul 2022 1:18 UTC
361 points
114 comments11 min readLW link

A Data limited future

Donald Hobson6 Aug 2022 14:56 UTC
52 points
25 comments2 min readLW link

A Mechanis­tic In­ter­pretabil­ity Anal­y­sis of Grokking

15 Aug 2022 2:41 UTC
337 points
39 comments42 min readLW link
(colab.research.google.com)

Stable Diffu­sion has been released

P.22 Aug 2022 19:42 UTC
15 points
7 comments1 min readLW link
(stability.ai)

Break­ing down the train­ing/​de­ploy­ment dichotomy

Erik Jenner28 Aug 2022 21:45 UTC
29 points
4 comments3 min readLW link

Sur­vey of NLP Re­searchers: NLP is con­tribut­ing to AGI progress; ma­jor catas­tro­phe plausible

Sam Bowman31 Aug 2022 1:39 UTC
89 points
6 comments2 min readLW link

A mar­ket is a neu­ral network

David Hugh-Jones15 Sep 2022 21:53 UTC
6 points
4 comments8 min readLW link

D&D.Sci Septem­ber 2022: The Allo­ca­tion Helm

abstractapplic16 Sep 2022 23:10 UTC
31 points
33 comments1 min readLW link

[MLSN #5]: Prize Compilation

Dan H26 Sep 2022 21:55 UTC
14 points
1 comment2 min readLW link

LOVE in a sim­box is all you need

jacob_cannell28 Sep 2022 18:25 UTC
58 points
69 comments44 min readLW link

In­ter­est­ing pa­pers: for­mally ver­ify­ing DNNs

the gears to ascenscion30 Sep 2022 8:49 UTC
12 points
0 comments3 min readLW link

linkpost: loss basin visualization

Nathan Helm-Burger30 Sep 2022 3:42 UTC
14 points
1 comment1 min readLW link

Four us­ages of “loss” in AI

TurnTrout2 Oct 2022 0:52 UTC
42 points
17 comments5 min readLW link

Paper+Sum­mary: OMNIGROK: GROKKING BEYOND ALGORITHMIC DATA

Marius Hobbhahn4 Oct 2022 7:22 UTC
44 points
11 comments1 min readLW link
(arxiv.org)

QAPR 4: In­duc­tive biases

Quintin Pope10 Oct 2022 22:08 UTC
63 points
2 comments18 min readLW link

GD’s Im­plicit Bias on Separable Data

Xander Davies17 Oct 2022 4:13 UTC
23 points
0 comments7 min readLW link

Cau­tion when in­ter­pret­ing Deep­mind’s In-con­text RL paper

Sam Marks1 Nov 2022 2:42 UTC
102 points
6 comments4 min readLW link

[Question] Why don’t we have self driv­ing cars yet?

Linda Linsefors14 Nov 2022 12:19 UTC
21 points
16 comments1 min readLW link

Why square er­rors?

Aprillion (Peter Hozák)26 Nov 2022 13:40 UTC
39 points
10 comments2 min readLW link

How can In­ter­pretabil­ity help Align­ment?

23 May 2020 16:16 UTC
37 points
3 comments9 min readLW link

GAN Discrim­i­na­tors Don’t Gen­er­al­ize?

tryactions8 Jun 2020 20:36 UTC
18 points
7 comments2 min readLW link

The “Out­side the Box” Box

Eliezer Yudkowsky12 Oct 2007 22:50 UTC
87 points
34 comments2 min readLW link

Us­ing ra­tio­nal­ity to de­bug Ma­chine Learning

Dr_Manhattan10 Apr 2018 20:03 UTC
20 points
3 comments1 min readLW link
(amid.fish)

Declar­a­tive Mathematics

johnswentworth21 Mar 2019 19:05 UTC
57 points
10 comments3 min readLW link

Ta­boo­ing ‘Agent’ for Pro­saic Alignment

Hjalmar_Wijk23 Aug 2019 2:55 UTC
54 points
10 comments6 min readLW link

Deep­Mind ar­ti­cle: AI Safety Gridworlds

scarcegreengrass30 Nov 2017 16:13 UTC
24 points
5 comments1 min readLW link
(deepmind.com)

Com­pet­i­tive Mar­kets as Distributed Backprop

johnswentworth10 Nov 2018 16:47 UTC
50 points
10 comments4 min readLW link1 review

Ma­chine Learn­ing Pro­jects on IDA

24 Jun 2019 18:38 UTC
49 points
3 comments2 min readLW link

Mag­i­cal Categories

Eliezer Yudkowsky24 Aug 2008 19:51 UTC
65 points
133 comments9 min readLW link

Con­nec­tion­ism: Model­ing the mind with neu­ral networks

Scott Alexander19 Jul 2011 1:16 UTC
59 points
20 comments8 min readLW link

Deep learn­ing—deeper flaws?

Richard_Ngo24 Sep 2018 18:40 UTC
39 points
17 comments4 min readLW link
(thinkingcomplete.blogspot.com)

Sel­ling Nonapples

Eliezer Yudkowsky13 Nov 2008 20:10 UTC
71 points
78 comments7 min readLW link

[Question] What are the most im­por­tant pa­pers/​post/​re­sources to read to un­der­stand more of GPT-3?

adamShimi2 Aug 2020 20:53 UTC
22 points
4 comments1 min readLW link

[Link] Com­puter im­proves its Civ­i­liza­tion II game­play by read­ing the manual

Kaj_Sotala13 Jul 2011 12:00 UTC
49 points
5 comments4 min readLW link

Some thoughts af­ter read­ing Ar­tifi­cial In­tel­li­gence: A Modern Approach

swift_spiral19 Mar 2019 23:39 UTC
38 points
4 comments2 min readLW link

is gpt-3 few-shot ready for real ap­pli­ca­tions?

nostalgebraist3 Aug 2020 19:50 UTC
31 points
5 comments9 min readLW link
(nostalgebraist.tumblr.com)

Worse Than Random

Eliezer Yudkowsky11 Nov 2008 19:01 UTC
44 points
102 comments12 min readLW link

In­duc­tive bi­ases stick around

evhub18 Dec 2019 19:52 UTC
63 points
14 comments3 min readLW link

Rea­sons com­pute may not drive AI ca­pa­bil­ities growth

Kythe19 Dec 2018 22:13 UTC
42 points
10 comments8 min readLW link

Begin­ning Ma­chine Learning

crybx30 Apr 2018 15:54 UTC
12 points
4 comments6 min readLW link

Pro­saic AI alignment

paulfchristiano20 Nov 2018 13:56 UTC
40 points
10 comments8 min readLW link

[Question] Al­gorithms vs Compute

johnswentworth28 Jan 2020 17:34 UTC
26 points
11 comments1 min readLW link

Com­plex­ity Penalties in Statis­ti­cal Learning

michael_h6 Feb 2019 4:13 UTC
31 points
3 comments6 min readLW link

Mak­ing a Differ­ence Tem­pore: In­sights from ‘Re­in­force­ment Learn­ing: An In­tro­duc­tion’

TurnTrout5 Jul 2018 0:34 UTC
33 points
6 comments8 min readLW link

New pa­per: The In­cen­tives that Shape Behaviour

RyanCarey23 Jan 2020 19:07 UTC
23 points
5 comments1 min readLW link
(arxiv.org)

On AI and Compute

johncrox3 Apr 2019 19:00 UTC
36 points
10 comments8 min readLW link

Rein­ter­pret­ing “AI and Com­pute”

habryka25 Dec 2018 21:12 UTC
30 points
10 comments1 min readLW link
(aiimpacts.org)

The Ma­chine Learn­ing Per­son­al­ity Test

PhilGoetz4 Aug 2009 23:36 UTC
31 points
34 comments6 min readLW link

Op­ti­miz­ing a Week of Ma­chine Learn­ing Learning

Raemon9 Jan 2018 6:55 UTC
8 points
2 comments3 min readLW link

Alex Ir­pan: “My AI Timelines Have Sped Up”

Vaniver19 Aug 2020 16:23 UTC
43 points
20 comments1 min readLW link
(www.alexirpan.com)

“De­sign­ing agent in­cen­tives to avoid re­ward tam­per­ing”, DeepMind

gwern14 Aug 2019 16:57 UTC
28 points
15 comments1 min readLW link
(medium.com)

Mas­ter­ing Chess and Shogi by Self-Play with a Gen­eral Re­in­force­ment Learn­ing Algorithm

DragonGod6 Dec 2017 6:01 UTC
13 points
4 comments1 min readLW link
(arxiv.org)

Why Gra­di­ents Van­ish and Explode

Matthew Barnett9 Aug 2019 2:54 UTC
25 points
9 comments3 min readLW link

Ex­am­ples of AI’s be­hav­ing badly

Stuart_Armstrong16 Jul 2015 10:01 UTC
41 points
37 comments1 min readLW link

Speci­fi­ca­tion gam­ing ex­am­ples in AI

Samuel Rødal10 Nov 2018 12:00 UTC
24 points
6 comments1 min readLW link
(docs.google.com)

Learn­ing with catastrophes

paulfchristiano23 Jan 2019 3:01 UTC
27 points
9 comments4 min readLW link

LDL 7: I wish I had a map

magfrump30 Nov 2017 2:03 UTC
13 points
2 comments3 min readLW link

Self-Su­per­vised Learn­ing and AGI Safety

Steven Byrnes7 Aug 2019 14:21 UTC
29 points
9 comments12 min readLW link

LDL 2: Non­con­vex Optimization

magfrump20 Oct 2017 18:20 UTC
13 points
13 comments4 min readLW link

Which of these five AI al­ign­ment re­search pro­jects ideas are no good?

rmoehn8 Aug 2019 7:17 UTC
25 points
13 comments1 min readLW link

Model splin­ter­ing: mov­ing from one im­perfect model to another

Stuart_Armstrong27 Aug 2020 11:53 UTC
74 points
10 comments33 min readLW link

Tech­ni­cal model re­fine­ment formalism

Stuart_Armstrong27 Aug 2020 11:54 UTC
19 points
0 comments6 min readLW link

Pong from pix­els with­out read­ing “Pong from Pix­els”

Ian McKenzie29 Aug 2020 17:26 UTC
15 points
1 comment7 min readLW link

Us­ing ma­chine learn­ing to pre­dict ro­man­tic com­pat­i­bil­ity: em­piri­cal results

JonahS17 Dec 2014 2:54 UTC
37 points
18 comments11 min readLW link

Log­i­cal or Con­nec­tion­ist AI?

Eliezer Yudkowsky17 Nov 2008 8:03 UTC
39 points
26 comments9 min readLW link

Ar­tifi­cial In­tel­li­gence and Life Sciences (Why Big Data is not enough to cap­ture biolog­i­cal sys­tems?)

HansNauj15 Jan 2020 1:59 UTC
6 points
3 comments6 min readLW link

LDL 4: Big data is a pain in the ass

magfrump25 Oct 2017 20:59 UTC
6 points
0 comments3 min readLW link

“Learn­ing to Sum­ma­rize with Hu­man Feed­back”—OpenAI

Rekrul7 Sep 2020 17:59 UTC
57 points
3 comments1 min readLW link

My (Mis)Ad­ven­tures With Al­gorith­mic Ma­chine Learning

AHartNtkn20 Sep 2020 5:31 UTC
16 points
4 comments41 min readLW link

[Question] Why isn’t JS a pop­u­lar lan­guage for deep learn­ing?

Will Clark8 Oct 2020 14:36 UTC
12 points
21 comments1 min readLW link

[Question] GPT-3 + GAN

stick10917 Oct 2020 7:58 UTC
4 points
4 comments1 min readLW link

Per­cep­trons Explained

lifelonglearner14 Feb 2020 17:34 UTC
13 points
2 comments1 min readLW link
(owenshen24.github.io)

The Weighted Ma­jor­ity Algorithm

Eliezer Yudkowsky12 Nov 2008 23:19 UTC
23 points
96 comments10 min readLW link

If Van der Waals was a neu­ral network

George3d628 Jan 2020 18:38 UTC
18 points
3 comments11 min readLW link
(blog.cerebralab.com)

“model scores” is a ques­tion­able concept

Maxwell Peterson6 Nov 2020 3:19 UTC
26 points
0 comments6 min readLW link

Fre­quen­tist prac­tice in­cor­po­rates prior in­for­ma­tion all the time

Maxwell Peterson7 Nov 2020 20:43 UTC
18 points
0 comments4 min readLW link

Model Depth as Panacea and Obfuscator

abstractapplic9 Nov 2020 0:02 UTC
8 points
3 comments15 min readLW link

[Question] Can this model grade a test with­out know­ing the an­swers?

Elizabeth31 Aug 2019 0:53 UTC
20 points
3 comments1 min readLW link

Re­think­ing Batch Normalization

Matthew Barnett2 Aug 2019 20:21 UTC
20 points
5 comments8 min readLW link

Link: In­ter­view with Vladimir Vapnik

Daniel_Burfoot25 Jul 2009 13:36 UTC
22 points
6 comments2 min readLW link

[Linkpost] AlphaFold: a solu­tion to a 50-year-old grand challenge in biology

adamShimi30 Nov 2020 17:33 UTC
54 points
22 comments1 min readLW link
(deepmind.com)

Min­i­mal Maps, Semi-De­ci­sions, and Neu­ral Representations

Zachary Robertson6 Dec 2020 15:15 UTC
30 points
2 comments4 min readLW link

Ma­chine learn­ing could be fun­da­men­tally unexplainable

George3d616 Dec 2020 13:32 UTC
26 points
15 comments15 min readLW link
(cerebralab.com)

The case for al­ign­ing nar­rowly su­per­hu­man models

Ajeya Cotra5 Mar 2021 22:29 UTC
187 points
74 comments38 min readLW link

The Ja­panese Quiz: a Thought Ex­per­i­ment of Statis­ti­cal Epistemology

DanB8 Apr 2021 17:37 UTC
11 points
0 comments9 min readLW link

Up­dat­ing the Lot­tery Ticket Hypothesis

johnswentworth18 Apr 2021 21:45 UTC
73 points
41 comments2 min readLW link

Thoughts on the Align­ment Im­pli­ca­tions of Scal­ing Lan­guage Models

leogao2 Jun 2021 21:32 UTC
79 points
11 comments17 min readLW link

“De­ci­sion Trans­former” (Tool AIs are se­cret Agent AIs)

gwern9 Jun 2021 1:06 UTC
37 points
4 comments1 min readLW link
(sites.google.com)

Pa­ram­e­ter counts in Ma­chine Learning

19 Jun 2021 16:04 UTC
47 points
16 comments7 min readLW link

The Effi­cient Mar­ket Hy­poth­e­sis in Research

libai8 Jul 2021 17:00 UTC
11 points
9 comments3 min readLW link

In­stru­men­tal Con­ver­gence: Power as Rademacher Complexity

Zachary Robertson12 Aug 2021 16:02 UTC
6 points
0 comments3 min readLW link

[Question] Ques­tion about Test-sets and Bayesian ma­chine learn­ing

Haziq Muhammad9 Aug 2021 17:16 UTC
2 points
8 comments1 min readLW link

Vir­tual Ma­chine Learn­ing Con­fer­ences: The Good and the Bad

libai29 Aug 2021 19:26 UTC
4 points
0 comments3 min readLW link

An anal­y­sis of the Less Wrong D&D.Sci 4th Edi­tion game

Maxwell Peterson4 Oct 2021 0:03 UTC
14 points
7 comments5 min readLW link

[Pro­posal] Method of lo­cat­ing use­ful sub­nets in large models

Quintin Pope13 Oct 2021 20:52 UTC
9 points
0 comments2 min readLW link

A Primer on Ma­trix Calcu­lus, Part 2: Ja­co­bi­ans and other fun

Matthew Barnett15 Aug 2019 1:13 UTC
22 points
7 comments6 min readLW link

A Gen­er­al­iza­tion of ROC AUC for Bi­nary Classifiers

Adam Scherlis4 Dec 2021 21:47 UTC
6 points
0 comments2 min readLW link
(adam.scherlis.com)

Be­hav­ior Clon­ing is Miscalibrated

leogao5 Dec 2021 1:36 UTC
52 points
3 comments3 min readLW link

See­ing the In­visi­ble (And How to Think About Ma­chine Learn­ing)

Filip Dousek8 Dec 2021 21:04 UTC
3 points
0 comments3 min readLW link

Ev­i­dence Sets: Towards In­duc­tive-Bi­ases based Anal­y­sis of Pro­saic AGI

bayesian_kitten16 Dec 2021 22:41 UTC
22 points
10 comments21 min readLW link

Re­in­force­ment Learn­ing Study Group

Kay Kozaronek26 Dec 2021 23:11 UTC
20 points
9 comments1 min readLW link

Truth­ful LMs as a warm-up for al­igned AGI

Jacob_Hilton17 Jan 2022 16:49 UTC
65 points
14 comments13 min readLW link

Ques­tion 1: Pre­dicted ar­chi­tec­ture of AGI learn­ing al­gorithm(s)

Cameron Berg10 Feb 2022 17:22 UTC
12 points
1 comment7 min readLW link

A com­pila­tion of mi­suses of statistics

Younes Kamel14 Feb 2022 21:53 UTC
4 points
11 comments13 min readLW link
(youneskamel.substack.com)

Com­pute Trends Across Three eras of Ma­chine Learning

16 Feb 2022 14:18 UTC
91 points
13 comments2 min readLW link

[Question] Is the com­pe­ti­tion/​co­op­er­a­tion be­tween sym­bolic AI and statis­ti­cal AI (ML) about his­tor­i­cal ap­proach to re­search /​ en­g­ineer­ing, or is it more fun­da­men­tally about what in­tel­li­gent agents “are”?

Edward Hammond17 Feb 2022 23:11 UTC
1 point
1 comment2 min readLW link

Com­pute Trends — Com­par­i­son to OpenAI’s AI and Compute

12 Mar 2022 18:09 UTC
23 points
3 comments3 min readLW link

Les­sons After a Cou­ple Months of Try­ing to Do ML Research

KevinRoWang22 Mar 2022 23:45 UTC
68 points
8 comments6 min readLW link

Skil­ling-up in ML Eng­ineer­ing for Align­ment: re­quest for comments

23 Apr 2022 15:11 UTC
19 points
0 comments1 min readLW link

[Question] What is a train­ing “step” vs. “epi­sode” in ma­chine learn­ing?

Evan R. Murphy28 Apr 2022 21:53 UTC
9 points
4 comments1 min readLW link

[Question] Why hasn’t deep learn­ing gen­er­ated sig­nifi­cant eco­nomic value yet?

Alex_Altair30 Apr 2022 20:27 UTC
112 points
95 comments2 min readLW link

Con­di­tions for math­e­mat­i­cal equiv­alence of Stochas­tic Gra­di­ent Des­cent and Nat­u­ral Selection

Oliver Sourbut9 May 2022 21:38 UTC
49 points
11 comments10 min readLW link

Pre­dict­ing the Elec­tions with Deep Learn­ing—Part 1 - Results

Quentin Chenevier14 May 2022 12:54 UTC
0 points
0 comments1 min readLW link

The Un­rea­son­able Effec­tive­ness of Deep Learning

Richard_Ngo30 Sep 2018 15:48 UTC
85 points
5 comments13 min readLW link
(thinkingcomplete.blogspot.com)

CNN fea­ture vi­su­al­iza­tion in 50 lines of code

StefanHex26 May 2022 11:02 UTC
17 points
4 comments5 min readLW link

[Question] Why does gra­di­ent de­scent always work on neu­ral net­works?

MichaelDickens20 May 2022 21:13 UTC
15 points
11 comments1 min readLW link

Machines vs Memes Part 1: AI Align­ment and Memetics

Harriet Farlow31 May 2022 22:03 UTC
16 points
0 comments6 min readLW link

Machines vs Memes Part 3: Imi­ta­tion and Memes

ceru231 Jun 2022 13:36 UTC
5 points
0 comments7 min readLW link

Miriam Ye­vick on why both sym­bols and net­works are nec­es­sary for ar­tifi­cial minds

Bill Benzon6 Jun 2022 8:34 UTC
1 point
0 comments4 min readLW link

Trans­former Re­search Ques­tions from Stained Glass Windows

StefanHex8 Jun 2022 12:38 UTC
4 points
0 comments2 min readLW link

The Limits of Automation

milkandcigarettes23 Jun 2022 18:03 UTC
5 points
1 comment5 min readLW link
(milkandcigarettes.com)

Yann LeCun, A Path Towards Au­tonomous Ma­chine In­tel­li­gence [link]

Bill Benzon27 Jun 2022 23:29 UTC
5 points
1 comment1 min readLW link

Deep neu­ral net­works are not opaque.

jem-mosig6 Jul 2022 18:03 UTC
22 points
14 comments3 min readLW link

Race Along Rashomon Ridge

7 Jul 2022 3:20 UTC
49 points
15 comments8 min readLW link

Grouped Loss may dis­fa­vor dis­con­tin­u­ous capabilities

Adam Jermyn9 Jul 2022 17:22 UTC
14 points
2 comments4 min readLW link

Find­ing Skele­tons on Rashomon Ridge

24 Jul 2022 22:31 UTC
30 points
2 comments7 min readLW link

Ma­chine Learn­ing Model Sizes and the Pa­ram­e­ter Gap [abridged]

Pablo Villalobos18 Jul 2022 16:51 UTC
20 points
0 comments1 min readLW link
(epochai.org)

Quan­tum Ad­van­tage in Learn­ing from Experiments

Dennis Towne27 Jul 2022 15:49 UTC
5 points
5 comments1 min readLW link
(ai.googleblog.com)

Trans­former lan­guage mod­els are do­ing some­thing more general

Numendil3 Aug 2022 21:13 UTC
44 points
6 comments2 min readLW link

Steganog­ra­phy in Chain of Thought Reasoning

A Ray8 Aug 2022 3:47 UTC
49 points
13 comments6 min readLW link

Re­in­force­ment Learn­ing Goal Mis­gen­er­al­iza­tion: Can we guess what kind of goals are se­lected by de­fault?

25 Oct 2022 20:48 UTC
9 points
1 comment4 min readLW link

The Shard The­ory Align­ment Scheme

David Udell25 Aug 2022 4:52 UTC
47 points
33 comments2 min readLW link

Fram­ing AI Childhoods

David Udell6 Sep 2022 23:40 UTC
37 points
8 comments4 min readLW link

Path de­pen­dence in ML in­duc­tive biases

10 Sep 2022 1:38 UTC
43 points
13 comments10 min readLW link

Can you force a neu­ral net­work to keep gen­er­al­iz­ing?

Q Home12 Sep 2022 10:14 UTC
2 points
10 comments5 min readLW link

Deep Q-Net­works Explained

Jay Bailey13 Sep 2022 12:01 UTC
36 points
4 comments22 min readLW link

[Question] Are Speed Su­per­in­tel­li­gences Fea­si­ble for Modern ML Tech­niques?

DragonGod14 Sep 2022 12:59 UTC
8 points
5 comments1 min readLW link

Lev­er­ag­ing Le­gal In­for­mat­ics to Align AI

John Nay18 Sep 2022 20:39 UTC
11 points
0 comments3 min readLW link
(forum.effectivealtruism.org)

Trends in Train­ing Dataset Sizes

Pablo Villalobos21 Sep 2022 15:47 UTC
24 points
2 comments5 min readLW link
(epochai.org)

Brief Notes on Transformers

Adam Jermyn26 Sep 2022 14:46 UTC
32 points
2 comments2 min readLW link

Sum­mary of ML Safety Course

zeshen27 Sep 2022 13:05 UTC
6 points
0 comments6 min readLW link

My Thoughts on the ML Safety Course

zeshen27 Sep 2022 13:15 UTC
49 points
3 comments17 min readLW link

[Question] What Is the Idea Be­hind (Un-)Su­per­vised Learn­ing and Re­in­force­ment Learn­ing?

Morpheus30 Sep 2022 16:48 UTC
9 points
6 comments2 min readLW link

If you want to learn tech­ni­cal AI safety, here’s a list of AI safety courses, read­ing lists, and resources

KatWoods3 Oct 2022 12:43 UTC
12 points
3 comments1 min readLW link

No free lunch the­o­rem is irrelevant

Dmitry Savishchev4 Oct 2022 0:21 UTC
12 points
7 comments1 min readLW link

Six (and a half) in­tu­itions for KL divergence

TheMcDouglas12 Oct 2022 21:07 UTC
111 points
16 comments10 min readLW link
(www.perfectlynormal.co.uk)

Learn­ing so­cietal val­ues from law as part of an AGI al­ign­ment strategy

John Nay21 Oct 2022 2:03 UTC
3 points
18 comments54 min readLW link

What will the scaled up GATO look like? (Up­dated with ques­tions)

Amal 25 Oct 2022 12:44 UTC
33 points
20 comments1 min readLW link

Eng­ineer­ing Monose­man­tic­ity in Toy Models

18 Nov 2022 1:43 UTC
70 points
6 comments3 min readLW link
(arxiv.org)

Us­ing mechanis­tic in­ter­pretabil­ity to find in-dis­tri­bu­tion failure in toy transformers

Charlie George28 Nov 2022 19:39 UTC
6 points
0 comments4 min readLW link

Multi-Com­po­nent Learn­ing and S-Curves

30 Nov 2022 1:37 UTC
57 points
24 comments7 min readLW link

Ap­ply for the ML Up­skil­ling Win­ter Camp in Cam­bridge, UK [2-10 Jan]

hannah wing-yee2 Dec 2022 20:45 UTC
3 points
0 comments2 min readLW link