I think having an opinion on this requires much more technical knowledge than GPT4 or DALLE 3. I for one don’t know what to expect. But I upvoted the post, because it’s an interesting question.
jacopo
Karma: 131
I agree with you actually. My point is that in fact you are implicitly discounting EY pessimism—for example, he didn’t release a timeline but often said “my timeline is way shorter than that” with respect to 30-years ones and I think 20-years ones as well. The way I read him, he thinks we personally are going to die from AGI, and our grandkids will never be born, with 90+% probability, and that the only chances to avoid it is that are either someone having a plan already three years ago which has been implemented in secret and will come to fruition next year, or some large out-of-context event happens (say, nuclear or biological war brings us back to the stone age).
My no-more-informed-than-yours opinion is that he’s wrong on several points, but correct on others. From this I deduce that the risk of very bad outcomes is real and not negligible, but the situation is not as desperate and there are probably actions that will improve our outlook significantly. Note that in the framework “either EY is right or he’s wrong and there’s nothing to worry about” there’s no useful action, only hope that he’s wrong because if he’s right we’re screwed anyway.
Implicitly, this is your world model as well from what you say. Discussing this then may look like nitpicking, but whether Yudkowsky or Ngo or Christiano are correct about possible scenarios changes a lot about which actions are plausibly helpful. Should we look for something that has a good chance to help in an “easier” scenario, rather than concentrate efforts on looking for solutions that work on the hardest scenario, given that the chance of finding one is negligible? Or would that be like looking for the keys under the streetlight?
I like the idea! Just a minor issue with the premise:
“Either I’d find out he’s wrong, and there is no problem. Or he’s right, and I need to reevaluate my life priorities.”
There is a wide range of opinions, and EY’s has one of the most pessimistic ones. It may be the case that he’s wrong on several points, and we are way less doomed than he thinks, but that the problem is still there and a big one as well.
(In fact, if EY is correct we might as well ignore the problem, as we are doomed anyway. I know this is not what he thinks, but it’s the consequence I would take from his predictions)
I think that you need to distinguish two different goals:
the very ambitious goal of eliminating any risk of misaligned AI doing any significant damage. If even possible, that would require an aligned AI with much stronger capabilities than the misaligned one (or many aligned AIs such that their combined capabilities are not easily matched)
the more limited goal to reduce extinction risk by AGI to a low enough level (say, comparable to asteroid risk or natural pathogen risk). This might manageble with the help of lesser AIs, depending on time to prepare
Addendum: if you want to bring legislation more in line with voters’ preferences issue by issue, avoiding the distortion from coalition building, Swiss-style referenda seem to work to an acceptable degree http://www.lesswrong.com/posts/x6hpkYyzMG6Bf8T3W/swiss-political-system-more-than-you-ever-wanted-to-know-i
The biggest obstacle to your idea is, I think, the executive. In parlamentary systems the government answers to the parliament, and needs MPs support to continue—indeed, the Israeli maneuvering that you cite is related to making the government collapse, not to political parties. So as a first thing, you need a presidential system. But even then, MPs would probably organize as for or against the president—I imagine that the president’s role in drafting and proposing legislation would be even higher than in present day US, as the coordination of MPs via the parties would be missing.
The second factor is the actual parties, i.e. the organisation of people that want to be politically active (in the current system these people select, support, control and in a few cases eventually become MPs, but that is only part of what the do). A lot of this activity will always be there and is important—what makes the party is members and of some political ideology, the MPs are not in principle needed. You want to separate these orgs from what happens in parliament, but it’s not clear if it’s possible—many candidates are always internal, people who have participated to party activity for years before running and continue to discuss their views with other members on a regular basis.
Personally, I think if we eliminated the parties we would probably be worse off, because they would be replaced by worse (less transparent) coalition building. But I would be curious to see you flesh out your ideas!
You are correct (QM-based simulation of materials is what I do). The caveat is that exact simulations are so slow that they are impossible, that would not be the case with quantum computing I think. Fortunately, we have different levels of approximation for different purposes that work quite well. And you can use QM results to fit faster atomistic potentials.
Note that there could still be some priors on some functions being more probable, or some more complex case being plainly impossible to fit because there’s no way to get there from the meta-model that is the trained NN.
I am left wondering if when GPT3 does few-shot arithmetics, it is actually fitting a linear model on the examples to predict the next token. I.e. the GPT3 weights do not “know” arithmetics, but they know how to fit, and that’s why they need a few examples before they can tell you the answer to 25+17: they need to know what function of 25 and 17 to return.
It is not that crazy given my understanding of what a transformer does, which is in some sense returning a function of the most recent input which depends on earlier inputs. Or am I confusing them with a different NN design?
Ahh sorry! Going back to read it was pretty clear from the text. I was tricked by the figure where the embedding is presented first. Again, good job! :)
Cool work!
Can I ask a couple of questions about the DR+clustering approach?
If I understand correctly, you do the clustering in a 2D space obtained with UMAP (ignore this if I am wrong). Are you sure you are not losing important information with such a low dimension? I say this because you show that one dimension is strongly correlated with style (academic vs forum/blog) and the second may be somewhat correlated with time. I remember that an argument exists for using n-1 dimensions when looking for n clusters, although that was probably using linear DR techniques and might not apply to UMAP. But it would be interesting to check if using higher n_components (3 to 5) results in the same clustering or generates some new insight.
Another thing you could check is using GMM instead of k-means. My (limited) experience is that if the embedding dimension is low you get better results this way. But, again, I was clustering downstream of linear DR.
I agree. In fact, you could say that Mélenchon and le Pen are closer to each other on economic and possibly foreign policy, and very far from Macron. So not unreasonable that some votes would transfer from one to the other. Huge differences on everything else of course (immigration, but also law and order, education, culture, …) I disagree on Hollande and generally center-left. Hollande had to juggle a very broad coalition as you say. He ended up hated by everyone because his way to handle it was not finding a middle ground, but campaigning as Mélenchon lite and then attempting to govern as Macron lite. Then he tried to dump the responsibility of the turnaround to financial markets and EU. After this, any possibility of a center-left coalition with an actual center-left agenda was dead and buried...
I think if you look up antifragile investment you find a lot of discussion of exactly this problem. As far as I understand, the idea is that most investments have limited downsides (at most, you lose what you put in) but may have limitless upsides in low-probability scenarios. Then you can make many small investments of this kind, so that when ones pays off, it’s more than enough to pay you back from the loss of the rest. Taking your example of the nuclear bunker, if you could build one with 1% of your wealth or less, in this frame of mind probably you should. Or less dramatically, invest a bit into any technology that could reach world-changing level even if it looks unlikely, plus buy a house as well as a cabin far away from any city. Learn a little bit of skills that could be useless or incredibly useful, depending on the future.
The answers you are suggesting are more related to safe/robust investment (I’m uncertain on the correct term), i.e. investment which should be useful whatever happens, but has a capped upside. Both maintain good health and buy a house count in this category. I’m not more qualified to give specific advice here than anyone else, but if you ask yourself “what would a standard prudent person do” you’re basically there.
Robust investment is what most people do in practice, and it’s probably a good thing because I think antifragile investment is too easy to screw up. But under some assumption on the nature of the high variance, which I think match the kind of future scenarios that you listed, well-thought antifragile strategies could be much better. At the very least, you can copy the idea of not putting all eggs in one basket, even if you don’t go all the way to make many small gambles in the expectation that at least one will pay off.
Interesting post! I like the picture you draw. But you should consider the possibility that it was not a Rome-unique factor, but the intersection of multiple things of which each one was true for multiple ancient states, but all of them only for Rome. In particular I have the impression that the subjects of the Persian empire were pretty happy with it and flourishing under its rule. To be clear, it was nothing like citizenship, because Persia was a kingdom and not a republican city-state. But between the investment model and the pillaging model that you mention, it was closer to the first. At some point, the Jews thought Cyrus was the Messiah! And in some ways, it’s not a surprise that the second-greatest empire of western antiquity had some things in common with the first.
I like to think it in this way: the determinant is the product of the eigenvalues of a matrix, which you can conveniently compute without reducing the matrix to diagonal form. All interesting properties of the determinant are very easy (and often trivial!) to show for the product of the eigenvalues.
More in the spirit of your post, I don’t remember how hard it is to show that the determinant is invariant under unitary transformation, but not too hard I think. It’s not the only invariant of course (the trace is as well, I don’t remember if there are others). But you could definitely start from the product of eigenvalues idea and make it invariant to get the formula for det.
Interesting read, but I don’t think the initial example and the following are very much connected. The shift of opinion about ww2 has presumably happened without fabricated evidence or misinformation about factual events. USSR and USA played a very different role in the defeat of Germany, so asking “which contributed the most” is sensitive to shifting narratives and highlighting of different events. Similar questions from more distant past: who was to blame for ww1? Was Napoleon spreading modernity and equality in Europe, or ruthlessly subjugating neighbors? Were the middle ages a dark age? Was the western Roman empire brought down by barbarians, or mainly by other factors? For all these you can have different answers without fabricated evidence, just by shifting some facts forward and neglecting others. That’s not to say that having tamper-proof historical sources is not important, just that it’s not sufficient. And personally I think most manipulation happens at the broader narrative level (in the past and in the present).
Or more generally, X sends a costly signal of his belief in P. If X is the state (as in example 2) a bet is probably impractical, but doing anything that would be costly if X is false should work. But for this, it makes a big difference in what sense Y does not trust X. If Y thinks X may deceive, costly signals are good. If Y thinks X is stupid or irrational or similar, showing belief in P is useless.
I mostly agree with the other commenters that the story does not show the qualitative changes we may expect to see from autonomous weapons. But I found it a very good short story nevertheless, and believable as well. I think it could serve well if broadly diffused, by getting someone to think about the topic for the first time before going into scenarios farther away from what they are used to.
I notice that while a lot of the answer is formal and well-grounded, “stories have the minimum level of internal complexity to explain the complex phenomena we experience” is itself a story :) Personally, I would say that any gear-level model will have gaps in the understanding, and trying to fill these gaps will require extra modeling which also has gaps, and so on forever. My guess is that part of our brain will constantly try to find the answers and fill the holes, like a small child asking “why x? …and why y?”. So if a more practical part of us wants to stop investigating, it plugs the holes with fuzzy stories which sound like understanding. Obviously, this is also a story, so discount it accordingly...
They commented without reading the post I guess...