It’s not clear here, but if you read the linked post it’s spelled out (the two are complementary really). The thesis is that it’s easy to do a narrow AI that knows only about chess, but very hard to make an AGI that knows the world, can operate in a variety of situations, but only cares about chess in a consistent way.
I think this is correct at least with current AI paradigms, and it has both some reassuring and some depressing implications.
What you say is even more true than you think. We would have had “relativity” in 1906, if you are satisfied with an experimentally indistinguishable theory which kept the ether as a conventional choice (a degree of difference similar to the one between interpretations of quantum mechanics). Poincaré had already submitted a paper in 1905 before seeing Einstein’s, building on Lorentz’s previous work. Now, Einstein’s theory is preferable for several reasons, but ultimately the difference is small.
If you look you find similar stories for Newton, Mendeleev, obviously Darwin, and others. There are some counterexamples, but ultimately we should take Newton seriously: the height of the shoulders you stand on is more important than your own for determining how far you can see.
I mostly agree with the other commenters that the story does not show the qualitative changes we may expect to see from autonomous weapons. But I found it a very good short story nevertheless, and believable as well. I think it could serve well if broadly diffused, by getting someone to think about the topic for the first time before going into scenarios farther away from what they are used to.
The problem is, licensed people have made an investment and expect to repay it by reaping profits from the protected market. Some have borrowed money to get in and may have to file for personal bankruptcy. So they will oppose the reform by any means at their disposal, for which I don’t blame them (even if it is obviously against the general interest).
Such a reform would be doable in the following cases (1) it compensates the losers in some way (2) it’s so gradual that current licensed will mostly retire before it’s fully implemented (3) it is decided by a political faction that has no interest in the votes of the licensed and no sympathy for their concerns, while the licensed have no “hard power” to block the reform (and this third will never be fulfilled for a blanket effort on all licenses: in practice you get a party punching down on the least powerful people in the opponent’s coalition).
As you see, it’s a whole other order of complication with respect to the case presented in the post...
Although they never take the whole supercomputer, so if you have the whole supercomputer for yourself and the calculations do not depend on each other you can run many in parallel
That’s one simulation though. If you have to screen hundreds of candidate structures, and simulate every step of the process because you cannot run experiments, it becomes years of supercomputer time.
Good post, thank you for it. Linking this will save me a lot of time when commenting...
However I think that the banking case is not a good application. When one bank fails, it makes much more likely that other banks will fail immediately after. So it is perfectly plausible that two banks are weak for unrelated reasons, and that when one fails this pushes the other under as well.
The second one does not even have to be that weak. The twentieth could be perfectly healthy and still fail in the panic (it’s a full blown financial crisis at this point!)
That seem reasonable, but it will probably change a number of correct answers (to tricky questions) as well if asked whether it’s certain. One should verify that the number of incorrect answers fixed is significantly larger than the number of errors introduced.
But it might be difficult to devise a set of equally difficult questions for which the first result is different. Maybe choose questions where different instances give different answers, and see if asking a double check changes the wrong answers but not the correct ones?
I always thought Hall’s point about nanotech was trivially false. Nanotech research like he wanted it died out in the whole world, but he explains it by US-specific factors. Why didn’t research continue elsewhere? Plus, other fields that got large funding in Europe or Japan are alive and thriving. How comes?
That doesn’t mean that a government program which sets up bad incentives cannot be worse than useless. It can be quite damaging, but not kill a technologically promising research field worldwide for twenty years.
That was quite different though (spoiler alert)
A benevolent conspiracy to hide a dangerous scientific discovery by lying about the state of the art and denying resources to anyone whose research might uncover the lie. Ultimately failing because apparently unrelated advances made rediscovering the true result too easy.
I always saw it as a reply to the idea that physicists could have hidden the possibility of an atomic bomb for more than a few years.
I like the idea! Just a minor issue with the premise:
“Either I’d find out he’s wrong, and there is no problem. Or he’s right, and I need to reevaluate my life priorities.”
There is a wide range of opinions, and EY’s has one of the most pessimistic ones. It may be the case that he’s wrong on several points, and we are way less doomed than he thinks, but that the problem is still there and a big one as well.
(In fact, if EY is correct we might as well ignore the problem, as we are doomed anyway. I know this is not what he thinks, but it’s the consequence I would take from his predictions)
I am left wondering if when GPT3 does few-shot arithmetics, it is actually fitting a linear model on the examples to predict the next token. I.e. the GPT3 weights do not “know” arithmetics, but they know how to fit, and that’s why they need a few examples before they can tell you the answer to 25+17: they need to know what function of 25 and 17 to return.
It is not that crazy given my understanding of what a transformer does, which is in some sense returning a function of the most recent input which depends on earlier inputs. Or am I confusing them with a different NN design?
I agree it would be very good, and possibly an economic no-brainer. My point is just that what is discussed in the post works for a political no-brainer, by which I mean something that no one would bother to oppose. To get what you want you need a real political campaign, or a large scale economic education campaign. Even then it’s difficult, imo, unless your proposals fit one of the cases I mention above.
That said, of you are thinking of the US there is an easy proposal to be done for medicine, which is making medical school equivalent to a college degree and eliminating the requirement of having already done college before to enter (see https://slatestarcodex.com/2015/06/06/against-tulip-subsidies/, which notes it’s done that way in Europe, I add it’s the same for law school etc.). It’s not an earth-shaking reform but it could work exactly for that reason.
On Prussia:
they managed to have almost the same GNP as France while keeping larger military spending, it’s not surprising that they won the war
of course, it may be surprising that they managed to get there. Given the model, you would expect that they sacrificed internal stability, but in fact it was France that was the most unstable country in that period! (Revolution, Napoleon, restoration, second Republic, second empire)
you could say the political instability may have really hindered France, forcing higher consumption spending, but how comes this was true only post-napoleon?
back to Prussia: it is the case that they never needed to maintain superiority over Austria and France over a long period. Nearly everyone in Germany wanted to unify, the question was how/under whom. The Prussians in 1970 needed few quick victories to convince everyone that they were the only choice. For this reason they could focus on the short term. After that, they absorbed the rest of Germany which had focused on economy. Compare the parallel unification of Italy under Piedmont/Sardinia, a much weaker power that played a similar strategy.
It’s not a coincidence that Hegel came up with the Zeitgeist idea exactly in 1800s Germany...
My overall take is that this is an useful starting point, and that structural factors are often underestimated, but the model is too simplified to actually make predictions with any confidence.
My feeling is that many of the people which did not benefit tend to “generalise from one example” and assume that’s true for most kids. Actually, I (despite being generally pro-schooling) would say something stronger than you: there is a minority of people who are actually harmed by school compared to a reasonable counterfactual (e.g. home-schooling for some). Plus, many kids can see easily where the system is failing them, less easily where it’s working.
Thanks for the review!
Regarding the “countering racism” doubts, I can see how the results should disprove at least some racist worldviews.
I think that an interpretation of human history among racists is the following: the population splits in to clusters, these clusters diverge in different “races”, eventually one emerges as “the best” and out-competes or replaces all others, before splitting again. Historically, this view was used to justify aggressive expansionism, opposition to intermarriage, and opposition to any policy that could slow this process by helping races which were seen as lesser.
I think what he wants to say is that this picture is not supported by the genetic data, which shows instead population clusters which split and merge and split again among different lines, on a fairly fast timescale and without one population replacing the other (except arguably for the Neanderthals, but even then not completely). In other words, there’s no darwinian selection at the racial level, and there has almost never been.
Consider also that when a zero-sum game is embedded in a positive-sum one, often the most effective way to negotiate is to threaten to walk away from the positive-sum game if you don’t get a bigger share of the spoils (e.g. threaten to leave the job if you don’t get a rise). The simplified version is the ultimatum game: https://en.wikipedia.org/wiki/Ultimatum_game.
This also means that holding a positive-sum trade sacred has the side effect of freezing the zero-sum part of it to the status quo.
I don’t think it would change your conclusion but your kettle was not very scaly. My gets much worse than that, with the resistence entirely covered by a thick layer, despite descaling 3-4 times per year. It depends on the calc content of your tap water. I still don’t think it affects energy use (maybe?), but the taste can be noticeable and I feel tea is actually harder to digest if I put off the descaling.
Also, you can use citric acid instead of vinegar. Better for the environment, less damaging to the kettle and it doesn’t smell :)
I think having an opinion on this requires much more technical knowledge than GPT4 or DALLE 3. I for one don’t know what to expect. But I upvoted the post, because it’s an interesting question.
(Phd in condensed matter simulation) I agree with everything you wrote where I know enough (for readers, I don’t know anything about lead contacts and several other experimental tricky points, so my agreement should not be counted too much).
I just add on the simulation side (Q3): this is what you would expect to see in a room-T superconductor unless it relies on a completely new mechanism. But, this is something you see also in a lot of materials that superconduct at 20K or so. Even in some where the superconducting phase is completely suppressed by magetism or structural distortions or any other phase transition. In addition, DFT+U is a quick-and-dirty approach for this kind of problem, as fits the speed at which the preprint was put out. So from the simulation bayesian evidence in favor but very weak