Occasionally think about topics discussed here. Will post if I have any thoughts worth sharing. Write CRUD for a living.
Bjartur Tómas
Karma: 500
This is my favourite LW post in a long while. Trying to think what the shoot-the-moon strat would be for AI risk, ha.
(2009) Shane Legg—Funding safe AGI
[Question] What would it look like if it looked like AGI was very near?
Fair enough. “Silly” is out.
Going Out With Dignity
Irrational Modesty
I don’t know too much about it. But I do know it was used extensively by Shell; they credited it with allowing them to respond to the Oil Shock much quicker than their competitors. They had analyzed the symptoms of a similar scenario (which was considered extremely outlandish at the time of scenario’s creation) and begin to notice eerie similarities between those symptoms and their present reality.
I see it as a sort of social technology that tries to assist an organization (and perhaps an individual) in resisting becoming the proverbial slowly-boiling frog.
As to evidence of its efficacy, I am only aware of anecdotal evidence. There appears to be an extensive Wikipedia page on the topic but I have not read it—my knowledge comes mostly from hearing Vernor Vinge speak about the technique, as he assisted in scenario-creation for several companies.
Ever since I heard Vinge speak about this, I have occasionally tried to think about the present as if it were a scenario I developed in the past: what sort of scenario would it be, how surprised would my past self be, and so on. Seeing how much The Pile improved GPT-J’s performance on this task trigged such thoughts.
AI-Based Code Generation Using GPT-J-6B
I did not write this post. Just thought it was interesting/relevant for LessWrong.
A Breakdown of AI Chip Companies
Just posting in case you did not get my PM. It has my email in it.
This is probably not a meta enough comment, but I have been using kettlebells since the pandemic and I think they are the highest ROI form of exercise I have ever tried. I do 5 minutes of kettlebell swings with a 60 pound bell 3 times a day: before work, on my lunch break, and after work. My strength has significantly increased and it feels like a good cardio workout too.
My big problem with exercise is not the discomfort but the monotony. Swings are much more exhausting than most exercises and are also a hybrid of lifting and cardio, making them very efficient.
Parameter vs Synapse?
Your estimates of hardware advancement seem higher than most people’s. I’ve enjoyed your comments on such things and think there should be a high-level, full length post on them, especially with widely respected posts claiming much longer times until human-level hardware.Would be willing to subsidize such a thing if you are interested. Would pay 500 USD to yourself or a charity of your choice for a post on the potential of ASICS, Moore’s law, how quickly we can overcome the memory bandwidth bottlenecks and such things. Would also subsidize a post estimating an answer this question, too: https://www.lesswrong.com/posts/7htxRA4TkHERiuPYK/parameter-vs-synapse
Thoughts on the Repugnant Conclusion
One thing we have to account for is advances architecture even in a world where Moore’s law is dead, to what extent memory bandwidth is a constraint on model size, etc. You could rephrase this as how much of an “architecture overhang” exists. One frame to view this through is in era the of Moore’s law we sort of banked a lot of parallel architectural advances as we lacked a good use case for such things. We now have such a use case. So the question is how much performance is sitting in the bank, waiting to be pulled out in the next 5 years.
I don’t know how seriously to take the AI ASIC people, but they are claiming very large increases in capability, on the order of 100-1000x in the next 10 years, if this is a true this is a multiplier on top of increased investment. See this response from a panel including big-wigs at NVIDIA, Google, and Cerebras about projected capabilities: https://youtu.be/E__85F_vnmU?t=4016. On top of this, one has to account, too, for algorithmic advancement: https://openai.com/blog/ai-and-efficiency/
Another thing to note is though by parameter count, the largest modern models are 10000x smaller than the human brain, if one buys that parameter >= synapse idea (which most don’t but is not entirely off the table), the temporal resolution is far higher. So once we get human-sized models, they may be trained almost comically faster than human minds are. So on top an architecture overhang we may have this “temporal resolution overhang”, too, where once models are as powerful as the human brain they will almost certainly be trained much faster. And on top of this there is an “inference overhang” where because inference is much, much cheaper than training, once you are done training an economically useful model, you will almost tautologically have a lot of compute to exploit it with.
Hopefully I am just being paranoid (I am definitely more of a squib than a wizard in these domains), but I am seeing overhangs everywhere!
I have created this Google Calendar link if anyone wants to quickly setup a reminder: https://calendar.google.com/event?action=TEMPLATE&tmeid=MjM5cnQ3cmUwcW5lNXIxM3Nva2pqYTI4MXUgZGtrcXJnamwyM2R1aG8wcjIxZ3ZudWRuMjhAZw&tmsrc=dkkqrgjl23duho0r21gvnudn28%40group.calendar.google.com
What would be a good exit plan? If you’ve thought about this, can you share your plan and/or discuss (privately) my specific situation?′
+1 for this. Would love to talk to other people seriously considering exit. Maybe we could start a Telegram or something.
They already assigned >90% probability that GPT-2 models something like how speech production works.
Is that truly the case? I recall reading Corey Washington a former linguist (who left the field for neuroscience in frustration with its culture and methods) claim that when he was a linguist the general attitude was there was no way in hell something like GPT-2 would ever work even close to the degree that it does.
Found it:
Steve: Corey’s background is in philosophy of language and linguistics, and also neuroscience, and I have always felt that he’s a little bit more pessimistic than I am about AGI. So I’m curious — and answer honestly, Corey, no revisionist thinking — before the results of this GPT-2 paper were available to you, would you not have bet very strongly against the procedure that they went through working?
Corey: Yes, I would’ve said no way in hell actually, to be honest with you.
Steve: Yes. So it’s an event that caused you to update your priors.
Corey: Absolutely. Just to be honest, when I was coming up, I was at MIT in the mid ’80s in linguistics, and there was this general talk about how machine translation just would never happen and how it was just lunacy, and maybe if they listened to us at MIT and took a little linguistics class they might actually figure out how to get this thing to work, but as it is they’re going off and doing this stuff which is just destined to fail. It’s a complete falsification of that basic outlook, which I think, — looking back, of course — had very little evidence — it had a lot of hubris behind it, but very little evidence behind it.
I was just recently reading a paper in Dutch, and I just simply… First of all, the OCR recognized the Dutch language and it gave me a little text version of the page. I simply copied the page, pasted it into Google Translate, and got a translation that allowed me to basically read this article without much difficulty. That would’ve been thought to be impossible 20, 30 years ago — and it’s not even close to predicting the next word, or writing in the style that is typical of the corpus.
Regarding your podcast example, I have some thoughts:
Psychometrics is both correct and incredibly unpopular—this means there is possibly an arbitrage here for anyone willing to believe in it.
Very high IQ people are rare and often have hobbies that are considered low-status in the general population. Searching for low-status signals that are predictive of cognitive ability looks to be an efficient means of message targeting.
It is interesting to note that Demis Hassibias’s prodigious ability was obvious to anyone paying attention to board games competitions in the late 90s. It may have been high ROI to sponsor the Mind Sports Olympiad at that time just for a small shot at influencing someone like Demis. There are likely other low-status signals of cognitive ability that will allow us to find diamonds in the rough.
Those who do well in strategic video games, board games, and challenging musical endeavors may be worth targeting. (Heavy metal for example—being very low-status and extremely technical musically—is a good candidate for being underpriced).
With this in mind, one obvious idea for messaging is to run ads. Unfortunately, high-impact people almost certainly have ad-blockers on their phones and computers.
However, the podcast space offers a way around this. Most niche 3rd party apps allow podcasters to advertise their podcasts on the podcast search pages. On the iPhone, at least, these cannot be adblocked trivially.
As the average IQ of a 3rd-party podcast app user is likely sligher higher than those who use first-party podcast apps, the audience is plausibly slightly enriched for high-impact people already. By focusing ads on podcast categories that are both cheap and good proxies for listener’s IQs (especially of the low-status kind mentioned above) one may be able to do even better.
I have been doing this for the AXRP podcast on the Overcast podcast app, and it has worked out to about ~5 dollars per subscriber. I did this without asking the permission of the podcast’s host.
Due to the recurring nature of podcasts and the parasocial relationship podcast listeners develop to the hosts of podcasts, it is my opinion their usefulness as a propaganda and inculcation tool is underappreciated at this time. It is very plausible to me that 5 dollars per subscriber may indeed be very cheap for the right podcast.
Directly sponsoring niche podcasts with extremely high-IQ audiences may be even more promising. There are likely mathematics, music theory, games and puzzle podcasts that are small enough to have not attracted conventional advertisers but are enriched enough in intelligent listeners to be a gold mine from this perspective.
I do not think I am a particularly good fit for this project. My only qualification is I am the only person I am aware of who is running such a project. Someone smarter with a better understanding of statistics would plausibly do far better. Perhaps if you have an application by a higher-quality person with a worse idea, you can give them my project. Then I can use my EA budget on something even crazier!