Done! There aren’t enough mysterious old wizards.
P.
Karma: 521
You know of a technology that has at least a 10% chance of having a very big novel impact on the world (think the internet or ending malaria) that isn’t included in this list, very similar, or downstream from some element of it: AI, mind uploads, cryonics, human space travel, geo-engineering, gene drives, human intelligence augmentation, anti-aging, cancer cures, regenerative medicine, human genetic engineering, artificial pandemics, nuclear weapons, proper nanotech, very good lie detectors, prediction markets, other mind-altering drugs, cryptocurrency, better batteries, BCIs, nuclear fusion, better nuclear fission, better robots, AR, VR, room-temperature superconductors, quantum computers, polynomial time SAT solvers, cultured meat, solutions to antibiotic resistance, vaccines to some disease, optical computers, artificial wombs, de-extinction and graphene.
Bad options included just in case someone thinks they are good.
Public mechanistic interpretability research is net positive in expectation.
Cultural values are something like preferences over pairs of social environments and things we actually care about. So it makes sense to talk about jointly optimizing them.
If we had access to a brain upload (and maybe a world simulator too) we could in principle extract something like a utility function, and the theory behind it relates more to agents in general than it does to humans in particular.
Research into getting a mechanistic understanding of the brain for purposes of at least one of: understanding how values/empathy works in people, brain uploading or improving cryonics/plastination is net positive and currently greatly underfunded.
Came here to comment that. It seems much more efficient to learn the cognitive strategies smart people use than to try to figure them out from scratch. Ideally, you would have people of different skill levels solve problems (and maybe even do research) while thinking out loud and describing or drawing the images they are manipulating. I know this has been done at least for chess, and it would be nice to have it for domains with more structure. Then you could catalog these strategies and measure the effectiveness of teaching the system 2 process (the whole process they use, not only the winning path) and explicitly train in isolation the individual system 1 steps that make it up.
Also “indivudals”.
Doesn’t minimizing the L1 norm correspond to performing MLE with laplacian errors?
Do you know whether this will be cancelled given the FTX situation?
If the optimal norm is below the minimum you can achieve just by re-scaling, you are trading-off training set accuracy for weights with a smaller norm within each layer. It’s not that weird that the best known way of making this trade-off is by constrained optimization.
And a 3D one by optimizing a differentiable volumetric representation using 2D diffusion: https://dreamfusionpaper.github.io/
And here we have another one: https://phenaki.video/
It’s not quite what you want, but there’s this: https://forum.effectivealtruism.org/community#individuals and this: https://eahub.org/
Emad from Stability AI (the people behind Stable Diffusion) says that they will make a model better than this.
Make-A-Video by Meta AI
RatSLAM: Using Models of Rodent Hippocampus for Robot Navigation
Building Collision Simulations: An Introduction to Computer Graphics
Self-Driving Cars [S1E4: RALPH]
What is the graph of x^a when a is not an integer? An unusual look at familiar functions #some2
Advanced 4. Monte Carlo Tree Search
Researchers Use Group Theory to Speed Up Algorithms — Introduction to Groups
The Traveling Salesman Problem: When Good Enough Beats Perfect
AI Alignment & AGI Fire Alarm—Connor Leahy
Vulcan | The Planet That Didn’t Exist
How MRI Works—Part 1 - NMR Basics
The Enduring Mystery of Jack the Ripper
Mechanical Computer (All Parts) - Basic Mechanisms In Fire Control Computers
Feynman’s Lost Lecture (ft. 3Blue1Brown)
ML Tutorial: Gaussian Processes (Richard Turner)
Google I/O 2013 - Project Ground Truth: Accurate Maps Via Algorithms and Elbow Grease
DeepMind’s AlphaFold 2 Explained! AI Breakthrough in Protein Folding! What we know (& what we don’t)
Deep Blue | Down the Rabbit Hole
Illustrated Guide to Transformers Neural Network: A step by step explanation
Cicada 3301 (All Clues and How They Were Solved)
Andrew Ng: Deep Learning, Self-Taught Learning and Unsupervised Feature Learning
Margo Seltzer—Automatically Scalable Computation—Code Mesh 2017
A Grape Made of… Meat?? - Tissue Recellularization
What Bodies Think About: Bioelectric Computation Outside the Nervous System—NeurIPS 2018
Understanding Sensor Fusion and Tracking, Part 2: Fusing a Mag, Accel, & Gyro Estimate
Drone Control and the Complementary Filter
World Record Progression: Super Mario Bros
Why Do We Age ? Cellular Aging (HD)
The Complex Hack That Stopped a Secret Nuclear Power Plant
Richard Szeliski—“Visual Reconstruction and Image-Based Rendering” (TCSDLS 2017-2018)
Might be the same talk as this one: https://youtu.be/0VIUbIzv_wc
What are the others?
But the outcome IS uncertain. I want to know how low the karma threshold can go before the website gets nuked. There are other fun games, but this one is unique to LW and seems like an appropriate way of celebrating Petrov Day.
It depends on what you know about the model and the reason you have to be concerned in the first place (if it’s just “somehow”, that’s not very convincing).
You might be worried that training it leads to the emergence of inner-optimizers, be them ones that are somehow “trying” to be good at prediction in a way that might generalize to taking real-life actions, approximating the searchy part of the humans they are trying to predict, or just being RL agents. If you are just using basically standard architectures with a lot more compute, these all seem unlikely. But if I were you, I might try to test its ability to perform well in a domain it has never seen, where humans start by performing poorly but very quickly learn what to do (think about video games with new mechanics). If it does well, you have a qualitatively new thing on your hands, don’t deploy, study it instead. If a priori for some reason you think it could happen, and only a small subset of all the data is necessary to achieve that, do a smaller training run first with that data.
Or you might be worried about mostly external consequentialist cognition (think explicit textual it-then-elses). In that case, existing systems can already do it to some extent, and you should worry about how good its reasoning actually is, so perform capability evaluations. If it looks that there is some way of getting it to do novel research by any known method or that it’s getting close, don’t deploy, otherwise someone might figure out how to use it to do AI research, and then you get a singularity.
And in any case, you should worry about the effects your system will have on the AI race. Your AI might not be dangerous, but if it is a good enough lawyer or programmer that it starts getting many people out of their jobs, investment in AI research will increase a lot and someone will figure out how to create an actual AGI sooner than they would otherwise.
Edit: And obviously you should also test how useful it could be for people trying to do mundane harm (e.g. with existing pathogens) and, separately, there might not be a hard threshold on how good a model is at doing research that it starts being dangerous, so they might get there little by little and you would be contributing to that.
Edit in response to the second clarification: Downscale the relevant factors, like amount of training data, number of parameters and training time, or use a known-to-be-inferior architecture until the worrying capabilities go away. Otherwise, you need to solve the alignment problem.
Edit in response to Beth Barnes’s comment: You should probably have people reviewing outputs to check the model behaves well, but if you actually think you need measures like “1000 workers with technical undergrad degrees, paid $50/hr” because you are worried it somehow kills you, then you simply shouldn’t deploy it. It’s absurd to have the need to check whether a commercial product is an existential threat, or anything close to that.