Economist.
Sherrinford
Karma: 1,207
“The US government and leading AI companies have already jointly decided to race hard and win.”
Have they ever defined what exactly this race is? What does its finishing line, or the post-race state of the world look like? What are they trying to accomplish?
My understanding of some things in AI development, which is probably completely wrong:
OpenAI wants to automate AI research.
This is bad for AI researchers, including those working at OpenAI, unless they own shares in the company.
The decision-makers have wriggled out of the non-profit status.
This could potentially mean that they are inclined and have the ability to evade other obligations as well.
If I were an AI developer at OpenAI without a deeper understanding of company structures, corporate law, etc., I would wonder if the decision-makers there could find a way to ensure that they ultimately pocket all the money and I get nothing, even if I currently own shares in the company.
Thanks for your posts, Jeff!
But I have a question on your photo usage—what became of your ghiblification project?
I may be wrong here, but IIRC the trade-off is:
Brighter skin = more Vitamin D from a given amount of sunlight
Darker skin = better protection against skin cancer
So in an area with less sunlight, evolution leads to people with brighter skin color and vice versa. In the modern world where people are mobile across continents, they need better information about technology that can help against both problems (like supplements and protection against skin cancer).
That is an interesting aspect, but I assume that assumes that the “dead or obsolete” part is not influenceable?
Right—the ought has more, additional requirements that I left implicit here.
If this is your timeline:
“I think AGI by end of 2027 should be ~8% now
I think I’d forecast: ~2026-2030 -- AI replaces ~all AI researchers
~2027-2033 -- AI replaces ~all white collar industry
~2032-2040 -- AI replaces ~all human industry
~2033-2042 -- All humans dead or obsolete”Then what does that imply you should be doing right now?
While we don’t have a counterfactual history for the post-WW2 decades, this interpretation seems at least plausible. At the same time, there were almost-catastrophic events in those decades that suggest that our timeline contains a good amount of luck. (Additionally, I also hope that there will not be a nuclear war in the future, but a situation in which several major powers are armed with nuclear weapons does not seem as stable to me as the Cold War was.)
I doubt that this is a good description of current Eastern Germany. I assume the reaction to vaccination campaigns during the pandemic would have been different. Here you can see the share of the population vaccinated (April 2023): https://impfdashboard.de/ Shares in the west were, in general, higher than in the East. This is of course only one piece of data, but in general it seems appropriate to be more careful with respect to such claims.
with its still-docile population accustomed to state directives, than in democratic Japan
This is a very strong claim. I’d be interested in your sources.
Some of the theory-heavy or ideology-heavy classes went over my head (though I noticed some of the more practiced rationalists enjoyed them, so it’s potentially an experience gap problem).
Which ones did you find theory-heavy or ideology-heavy?
I agree with this—I enjoyed reading the story, but I am confused by the way the world is deduced.
If you have a database of all possible universes and atmospheres and whatever, then I understand that things in the game are evidence for a the game being made by people evolving on a planet like ours.
Still, the evidence seems week, because the whole game has low-resolution graphics, is 2-dimensional and not very detailed. I assume we experience it as a representation of things that could technically happen in a cartoon version of our world only because we do not just see what happens on the screen but add our experience with it and therefore it is not so important that the game itself is not realistic.
But now suppose you do not know what air is, and have no idea of our physics and chemistry. You don’t know, for example, what the clouds on the screenshots represent. You don’t know why some of the things on the screen look like brick walls to us. What kind of pattern is that? Under these circumstances, does that deduction of
“Carbon-based chemistry on a rocky body orbiting a stable energy source. Atmosphere dense enough for pressure wave propagation. Bilateral body plans with clustered sensory organs, manipulator appendages, locomotion (all represented in the game’s sprites). Large brains (the symbolic language and tool-building demand it). Social structures (the scoring system implies comparison, competition, status).”
make sense? Why carbon? Why does the being that is reasoning here know the concept of sound if it has never experienced an atmosphere or sound itself? Why do the sensory organs have to be clustered, why should the units moving through the game represent people or animals (we could also be playing pacman or space invaders here)?
So yes, engineer with amnesia seems to point in the right direction, but maybe a god with amnesia would be necessary?
The idea to have a “agree/disagree” voting button seems better than just vote/downvote. However, it can be just a button for unproductively signaling “this post’s overall direction is different from my prior opinions”, instead of writing down your reasons for disagreeing, and thus instead of enabling productive discourse and learning. This seems much worse for disagreement than agreement, but unnecessary “ingroup signaling to yourself” may happen in both directions.
“It is to train up and hire health care workers who can speak Low German.”
Could you please write a bit about how many potential health care workers who speak Low German are there in Canada? How easy is it to do that?
Due to Raemon’s call for more reviews, “especially critical ones”, I decided to read and review this one, so yes, I am interested in the topic. The post lists many interesting aspects of the discussed regulation.
Nonetheless, I think the post would not be a good choice for the 2024 Review.
Firstly, the post is a post about American politics and American policy, and I hope the perspective or target audience of LW is not such that is assumed that readers of LW posts naturally understand “we”, “our failures”, “we have required” etc as “American”.
Secondly, while it is possible that the policy change would yield the benefits promised here, there are strong claims in the beginning of the article (“Repeal would restore America’s oceangoing trade between its ports and rebuild its merchant marine, noticeably impacting the price level and GDP growth.”) and you need to read much further to get to a point where sources are presented and discussed.
Thirdly, at least in the first few sections, the causality between the things in “What is the Jones Act?” and “What is the Effect of the Jones Act?” is not explained clearly. “The Jones Act has failed to protect America’s shipbuilding and merchant marine.” Okay, so how did it destroy America’s shipbuilding and merchant marine? It seems that you already need a clear model of the effects of protectionism in your head to understand what the author is saying.
I read only the first sections including “What Else Happens When We Ship More Goods Between Ports?” I will probably continue reading the post later, but for the 2024 Review review, I will stop here.
Thanks, good to know. So I assume there is an incentive difference between monetary incentives that can be distributed in such a way, and the incentive of being able to say that you won a tournament (maybe also as a job qualification).
I’m a bit confused about forecasting tournaments and would appreciate any comments:
Suppose you take part in such a tournament.
You could predict as accurately as you can and get a good score. But let’s say there are some other equally good forecasters in the tournament and it becomes a random draw who wins. On expectation, all forecasters of the same quality have the same forecasts. If there are many good forecasters, your chances of winning become very low.
However, you could include some outlier predictions in your predictions. Then you lower your expected accuracy, but you also incrase your chances of winning the tournament if these outlier probabilities come true.
Therefore, I would expect a lot of noise in the relation between forecasting quality and being a tournament winner.
Yes, a one-time investment is different from permanent effort. But they both can be very costly. In my original post, I meant all things that raise the cost / time investment / whatever of being a parent (and did not use the term “parenting”). I also think it is totally great if people invest a lot of time, effort, money, and other things into having great children. But I think there is a tension with saying that society should lower the demands on parents, so that people have more kids. Parts of the same community seem to have really high standards, and other parts say that we need to lower the standards, and I do not yet see how the tension is resolved.
How much time do you need for this per week, including preparation etc?
Okay, but then what? Do they all (US companies + US government) believe that—once they have actually built AGSI—there will be a static endpoint at which all their competitors say: “Okay, you’ve invented AGSI now; we all give up”? If so: In what competition, exactly, would they be giving up? Would the rest of the world then stop conducting further AI research? Or would the OpenAI/Anthropic/Google/Musk AI then be deployed to sabotage AI research in the rest of the world? I don’t understand the scenario.