Tails come apart. The most successful actresses are not the most attractive women.
On average, the most successful actresses are certainly more attractive than their male counterparts at least. While both tend to be good looking, a man who is not attractive but has other talents (funny, athletic, good at physical acting, charismatic) tends to have a better chance at fame than a woman with the same talents and the same flaws.
She’s an outlier, but an outlier in the sense that someone who’s 6′2″ is an outlier, not an outlier like Michael Jordan is an outlier.
I think this might be selling Marilyn Monroe short, because from what I’ve heard, she was also extremely charismatic and skilled at building her image. Which is kind of a different but parallel point to yours—lots of women can have pretty faces and pretty bodies, but what really makes a few like her stand above and beyond (besides the luck of being in the right place at the right time to become world famous) is also the talent to leverage that and make it shine. A model for example needs to know how to pose; a well-rounded variety act needs to be pretty, emotionally intelligent, good at conversation, at coordination, with the patience and persistence to suffer through all sorts of treatments and make-up… there’s a lot! It’s just not all physical.
No, I don’t think there’s anything like that. I do wonder about deriving the same result for the divergence. I have no idea how hard that would be; it might even be quite easy. Possibly even reduces to something more Bayes-like in case of commutating operators. I’ll try.
The actual formula is in the paper. I explained the process that it is obtained from. The formula for the posterior looks quite abstruse, required me to explain more notation and ultimately doesn’t give any particular useful intuitions on its face so I omitted it. You can also find it in my code.
Quoting from the paper:
Fidelity is one of the most natural measures of the closeness between quantum states and has found countless applications in quantum information theory.
I agree that this sort of quantum relative entropy should also be doable. It’s possible that the result would be the same. I guess an easy check would be to perturb the posterior and check whether this measure also has a minimum around the same point.
When you say defensiveness, does that include something like “act as though you’ve been attacked viciously by a person who is biased against you because they’re bad”? Because that, to me, is the defensiveness behavior I’d find the most suspicious (other facets of defensiveness less so).
I mean, suppose I belong to ideology A, and someone from rival ideology B accuses me of something we both agree is bad, but I do not believe I have done.
Given our relative positions and probably pre-existing animosity and mistrust, it’s entirely possible for me to genuinely believe “he is a B, therefore his being <long list of ideological grievances I have with Bs that make me believe their epistemics and/or morals are bad> makes him more likely to accuse me of random stuff just because I’m an A”, and say so. Meanwhile the B may as well have had the same kind of bias in deciding that I could probably be guilty of the thing he accused me of. If that is the case, we could both be in good faith and thinking the other is in bad faith, both biased, and the situation be perfectly symmetrical. But if you look with suspicion at defensiveness you give by default the advantage to whoever strikes first, which creates also a nasty incentive to do so. And in fact we do see this happening a lot in public discourse.
Namely that a majority of US or Chinese population would support risking nuclear war on the other population, just to get slightly better living standards for themselves.
I don’t know about nuclear war, though I would remind you of the worrying amount of “nuclear winter wouldn’t be so bad” takes when tensions were a bit higher with Russia due to the Ukraine war. But obviously throughout history massive popular pro-war and interventionist movements have existed, even for blatantly braindead choices (see: Italy throwing itself into WW1 in 1915 after having managed to stay out, or for a more recent examples, support for the Afghanistan war after 9/11).
I don’t think nuclear war would ever happen because someone actively wants to just push a button and go full Armageddon on the other side. The risk is more that you progressively slide through various steps of escalation until one side starts feeling so existentially threatened that they start using nuclear weapons, and from there things escalate even further until a mix of panic and fog of war triggers the catastrophe. And each of those steps can at the time seem reasonable and rational, given limited knowledge. Besides, my point isn’t that people would want this necessarily out of nowhere, but that limiting secrecy does nothing to prevent politicians from still being hawkish and short-sighted and riling up their voting base with misleading or false information; it only prevents them from having an easier out later.
I mean, yeah, obviously I get why he’s bringing up specifically the case of persuasion. But even if you didn’t ever get to agents that powerful or far-sighted, you still have the problem that if you make a therapist-AI an assistant to a scientist, the AI will just say “oh yes you must be so right” when the scientist is asking about a wrong hypothesis and ultimately lead to making junk science. Not as serious a problem but still fundamentally undermines its goal (and if this becomes very common, risks undermining science as a whole, and being a different path to loss of control and decline).
I’m not seeing the connection from no private room to more war.
I think in practice people seem to reward “strength” very often and that would encourage leaders to keep acting as aggressively as many do now on the world stage, but also without the secret channels to actually defuse the resulting tensions. Hence the stuff that is said openly could end up mattering more and be taken more seriously, because it’s now the only channel.
I think he’s hitting at a fundamental point though: the jobs are very different and in a sensible world in which we care about results they would never be performed by the same exact model (at most by two different fine-tunings of the same base model). But in practice what we’re getting is a bunch of swiss army knife models that are then embedded with minimal changes in all sorts of workflows. And so if they’re trained to be therapists, personality-wise, they’ll keep acting like therapists even in contexts where it’s inappropriate and harmful.
It is good actually that you can’t do these things. Most people don’t support threatening nuclear war on the other empire in order to extract favourable deals for your empire.
My impression is that very often the threats of nuclear war are what’s public, and the backroom deals to actually avoid those are what’s secret. This seems remarkably optimistic because while nuclear war specifically might not be something anyone wants (and it’s questionable too; see how many Russian politicians seem to think that bragging about nuking the West will make them more popular and do so as part of their propaganda), other kinds of war absolutely can be. There was a lot of support for war post 9/11, for example. So if anything I fear that the opposite would happen—with everything in the open where everyone has always to worry about not losing face, and no room to secretly say “no ok I was just kidding let’s find a deal” you’d get more war, not less.
I don’t think you could realistically get to the point where you can do international policy, trade policy, military planning and R&D, etc, without secrecy. I guess however there’s always the good old analogue paper files and folders.
But also, if AI is so powerful, we’d need to see where the chips fall once people also start deploying them for defense.
While the scenario is worrying, I have to say there’s something funny about North Korea being reduced to a point where a few US tech job salaries are a significant help to their national budget.
Well, if one tries to assess their performance compared to their potential, observing your peers can be a way to guess that. It’s not necessarily a matter of competitiveness as much as “this is the reference class I’m in, so this is roughly the kind of thing I can reasonably aspire to”. But if you’re on a class of your own anyway then you can’t even appeal to that—only look at the task.
I think the fundamental problem is that at Von Neumann’s level you don’t compare yourself against your peers, but against the vastness of the task at hand—which is the understanding of a part or possibly even the total of nature, logic, and reality itself. And so you can’t do anything but come up short, because well, a single human life even with the greatest intellect that ever lived is barely enough to put a microscopic dent in it.
I find it very amusing that everyone proudly boasts of being “Agile” when they have all the nimble swiftness in decision and action of a 1975 Soviet Russia agricultural committee.
Having used Cursor and VSCode with Github Copilot I feel like a huge part of the problem here isn’t even the LLMs per se: it’s the UX.
The default here is “you get a suggestion whether you asked for it or not, and if you press Tab it gets added”. Who even thought that was a good idea? Sometimes I press Tab because I need to indent four spaces, not because I want to write whatever random code the LLM thinks is appropriate. And this seems incredibly wasteful, continuously sending queries to the API, often for stuff I simply don’t need, with who knows how big context length that isn’t necessary! A huge part of the benefit I get from LLM assistants are simple cases of “here is a function called a very obvious thing that does exactly that obvious thing” (which is really nothing more than “grab an existing code snipped and adapt the names to my conventions”), or “follow this repetitive pattern to do the same thing five times” (again a very basic, very context-dependent automatic task). Other high value stuff includes writing docstrings and routine unit tests for simple stuff. Meanwhile when I need to code something that takes a decent amount of thought I am grateful for the best thing that these UX luckily do include, the “snooze” function to just shut the damn thing up for a while.
As I see it, the correct UX for an LLM code assistant would be:
only operate on demand
have a few basic tasks (like “write docstring”) possible to invoke on a given line where your cursor is, grabbing a smart context, using their own specific prompt
ability to define your own custom tasks flexibly, or download them as plugins
But use something like the command palette Ctrl+Shift+P for it. There’s probably even smarter and more efficient stuff that can be done via clever use of RAG, embedding models, etc. The current approach is among the laziest possible and definitely suffers from problems, yeah.
I don’t think it matters much for practical purposes. It could be that some problems are theoretically solvable by human intelligence but we realistically lack the time to do so in the age of the universe, or that they just can’t be solved by us, and either way an ASI that solves them in a day leaves us in the dust. The reason why becomes secondary at that point.
I feel like one problem with solving problems intelligently is that it’s rarely as easy as tackling a tedious task in small bits—you need an intuition to see the whole path in a sort of coarse light, and then refine on each individual step. So there’s a fast algorithm that goes “I know I can do this, I don’t know how yet” and then we slowly unpack the relevant bits. And I think there might be a qualitative effect to e.g. being able to hold more steps in memory simultaneously or such.
Oh, that sounds interesting! Definitely gonna check this out.