One lesson you could take away from this is “pay attention to the data, not the process”—this happened because the data had longer successes than failures. If successes were more numerous than failures, many algorithms would have imitated those as well with null reward.
Tao Lin
Karma: 484
I think the “fraction of Training compute” going towards agency vs nkn agency will be lower in video models than llms, and llms will likely continue to be bigger, so video models will stay behind llms in overall agency
Helpfullness finetuning might make these models more capable when they’re on the correct side of the debate. Sometimes RLHF(like) models simply perform worse on tasks they’re finetuned to avoid even when they don’t refuse or give up. Would be nice to try base model debaters
A core advantage of bandwidth limiting over other cybersec interventions is its a simple system we can make stronger arguments about, implemented on a simple processor, without the complexity and uncertainty of modern processors and OSes
no clock speed stays the same, but clock cycle latency of communication between regions increases. Just like CPUs require more clock cycles to access memory than they used to.
do we have any reason to believe that particular election won’t be close
I’d expect artificial sweeteners are already very cheap, and most people want more tested chemicals.
There exists an Effective Altruism VR discord group. It used to have regular VRChat meetups in like 2021 but doesn’t have much activity now i think
I’d be interested in experiments with more diverse data. Maybe this only works because the passages are very short and simple and uniform, and are using very superposition-y information that wouldn’t exist in longer and more diverse text
i thought about this for a minute and landed on no counting for lorentz factor. Things hitting on the side have about the same relative velocity as things hitting from the front . Because they’re hitting the side they could either bounce off or dump all their tangent kinetic energy into each other. like because all the relative velocity is tangent, they could in principle interact without exchanging significant energy. But probably the side impacts are just as dangerous. Which might make them more dangerous because you have less armor on the side
probes probably want a very skinny aspect ratio. If cosmic dust travels at 20km/s, that’s 15k times slower than the probe is travelling, so maybe that means the probe should be eg 10cm wide and 1.5km long
important to note that gpt4 is more like 300x scale equivalent of gpt3, not 100x, based on gpt4 being trained with (rumored) 2e25 flops vs contemporary gpt3-level models (llama2-7b) being trained on 8e22 flops ( 250 times the compute for that particular pair)
Some months before release they had a RLHF-ed model, where the RLHF was significantly worse on most dimensions than the model they finally released. This early RLHF-ed model was mentioned in eg Sparks of AGI.
if AI does change the offence defence balance, it could be because defending an AI (that doesnt need to protect humans) is fundamentally different than defending humans, allowing the AI to spend much less on defence
video can get extremely expensive without specific architectural support. Eg a folder of images takes up >10x the space of the equivalent video, and using eg 1000 tokens per frame for 30 frames/second is a lot of compute
looks slightly behind gpt-4-base in benchmarks. On the tasks where gemini uses chain-of-thought best-of-32 with optimized prompts it beats gpt-4-base, but ones where it doesnt its same or behind
E.g. suppose some AI system was trained to learn new video games: each RL episode was it being shown a video game it had never seen, and it’s supposed to try to play it; its reward is the score it gets. Then after training this system, you show it a whole new type of video game it has never seen (maybe it was trained on platformers and point-and-click adventures and visual novels, and now you show it a first-person-shooter for the first time). Suppose it could get decent at the first-person-shooter after like a subjective hour of messing around with it. If you saw that demo in 2025, how would that update your timelines?
Time constraints may make this much harder. Like a lot of games require multiple inputs per second (eg double jump) and at any given time the AI with the best transfer learning will be far too slow for inference to play as well as a human. (you could slow the game down of course)
Leela Zero uses MCTS, it doesnt play superhuman in one forward pass (like gpt-4 can do in some subdomains) (i think, didnt find any evaluations of Leela Zero at 1 forward pass), and i’d guess that the network itself doesnt contain any more generalized game playing circuitry than an llm, it just has good intuitions for Go.
Nit:
Subjectively there is clear improvement between 7b vs. 70b vs. GPT-4, each step 1.5-2 OOMs of training compute.
1.5 to 2 OOMs? 7b to 70b is 1 OOM of compute, adding in chinchilla efficiency would make it like 1.5 OOMs of effective compute, not 2. And llama 70b to gpt-4 is 1 OOM effective compute according to openai naming—llama70b is about as good as gpt-3.5. And I’d personally guess gpt4 is 1.5 OOMs effective compute above llama70b, not 2.
I think the heuristic “people take AI risk seriously in proportion to how seriously they take AGI” is a very good one.
Agree. Most people will naturally buy AGI Safety if they really believe in AGI. No AGI->AGI is the hard part, not AGI->AGI Safety.
I’ve recently gotten into partner dancing and I think it’s a pretty superior activity