Yep. Where this deviated from my notes, I approve (purely in terms of the time travel logic, that is). Seems like OpenAI is way ahead on time-travel logic, which is evidence that it is significantly ahead on “general reasoning”.
abramdemski
Karma: 21,884
They do deliberately try to set up an “I’ll get in the box if I don’t see myself get out” sort of situation in the movie, though they don’t succeed, and they don’t seem to realize that it would result in 0-2-0-2-… across metatime.
Good point about how permanent increases have to be as improbable as permanent decreases! I should’ve gotten that from what you were saying earlier. I suppose that leaves me with the “movies follow interesting timelines” theory, where it’s just a convention of the film to look at the timelines where characters multiply.
The characters in the movie take a lot of precautions to isolate themselves from their time-clones, meaning that they don’t really know whether they got out of the box at the start. Therefore, they just have faith in the plan and jump in the box at the end of the loop. So long as they don’t create any obvious paradoxes (“break symmetry” as they call it), everything works out from their perspective, and they can assume it’s consistent-timeline travel rather than branching, so they don’t think they’re creating a timeline in which they mysteriously vanish.
When they start creating paradoxes, of course, they should realize. The fact that they don’t think about it this way fits with the general self-centeredness of the characters, however.
I agree that it makes sense to think of this probabilistically, but we can also think of it as just all timelines existing. I’m happy to excuse the events of the movie as showing one particularly interesting timeline out of the many. It makes sense that the lens of the film isn’t super interested in the timelines which end up lacking one of the viewpoint characters.
If we do think of it probabilistically, though, are the events of the movie so improbable that we should reject them? By my thinking, the movie still fits well with that. Depending on how you think the probabilities should work out, it seems like that first timeline where the person just vanishes is low-probability, particularly if they create a relatively consistent time-loop. In a simple consistent loop, only the original branch has them vanish, while each other branch looks like an internally consistent timeline, and spawns another just like itself. The probability of a timeline like “one Abe, then two Abes, then back to one” seems high, if Abe is careful to avoid paradoxes. With paradoxes, the high-probability timelines get chaotic, which is what we see in the movie (and in the comic I linked).
I’m not sure why you say it’s hard to explain with branching timelines. To me this is just branching timelines. The movie voiceover states at one point that the last version of events seems to be the one that holds true, meaning that you see the last branching timeline, usually the one with the most Bobs. I don’t think you have to belive this part of the voiceover, though; this is just the opinion of someone trying to make sense of events. You could instead say that the movie has a convention of showing us later splits rather than earlier.
Canada is doing a big study to better understand the risks of AI. They aren’t shying away from the topic of catastrophic existential risk. This seems like good news for shifting the Overton window of political discussions about AI (in the direction of strict international regulations). I hope this is picked up by the media so that it isn’t easy to ignore. It seems like Canada is displaying an ability to engage with these issues competently.
This is an opportunity for those with technical knowledge of the risks of artificial intelligence to speak up. Making such knowledge legible to politicians and the general public is an important part of civilization being able to deal with AI in a sane manner. If you can state the case well, you can apply to speak to the committee:
Send a request to ETHI@parl.gc.ca, stating:
which study you want to participate in (Challenges Posed by Artificial Intelligence and its Regulation)
who you are and why the committee should care about what you have to say
what you want to talk about
indicate what language(s) you can testify (english/french) and virtually vs in-person
Luc Theriault is responsible for this study taking place.
I don’t think the ‘victory condition’ of something like this is a unilateral Canadian ban/regulation—rather, Canada and other nations need to do something of the form “If [some list of other countries] pass [similar regulation], Canada will [some AI regulation to avoid the risks posed by superintelligence]”.Here’s a relatively entertaining second hour of proceedings from 26 January:
https://youtu.be/W0qMb1qGwFw?si=EqgPSHRt_AYuGgu8&t=4123
Full videos:
https://www.youtube.com/watch?v=W0qMb1qGwFw&t=30s
https://www.youtube.com/watch?v=mow9UFdxiIw&t=30s
https://www.youtube.com/watch?v=ipMS1S5oOlg&t=19s
3. How does that handle ontology shifts? Suppose that this symbolic-to-us language would be suboptimal for compactly representing the universe. The compression process would want to use some other, more “natural” language. It would spend some bits of complexity defining it, then write the world-model in it. That language may turn out to be as alien to us as the encodings NNs use.
The cheapest way to define that natural language, however, would be via the definitions that are the simplest in terms of the symbolic-to-us language used by our complexity-estimator. This rules out definitions which would look to us like opaque black boxes, such as neural networks.
I note that this requires a fairly strong hypothesis: the symbolic-to-us language apparently has to be interpretable no matter what is being explained in that language. It is easy to imagine that there exist languages which are much more interpretable than neural nets (EG, English). However, it is much harder to imagine that there is a language in which all (compressible) things are interpretable.
Python might be more readable than C, but some Python programs are still going to be really hard to understand, and not only due to length. (Sometimes terser programs are the more difficult to understand.)
Perhaps the claim is that such Python programs won’t be encountered due to relevant properties of the universe (ie, because the universe is understandable).
Yes, I think what I’ve described here shares a lot with Bengio’s program.
The closest we can get is a little benchmark where the models are supposed to retrieve “a needle out of a haystack”. Stuff like a big story of 1 million tokens and they are supposed to retrieve a fact from it
This isn’t “the closest we can get”. Needle-in-a-haystack tests seem like a sensible starting point, but testing long-context utilization in general involves synthesis of information, EG looking at a novel or series of novels and answering reading comprehension questions. There are several benchmarks of this sort, EG:
https://epoch.ai/benchmarks/fictionlivebench
This is my inclination, but a physicalist either predicts that the phenomenology would in fact change, or perhaps asserts that you’re deluded about your phenomenal experience when you think that the experience is the same despite substrate shifts. My understanding of cube_flipper’s position is that they anticipate changes in the substrate to change the qualia.
From a physicalist’s perspective, you’re essentially making predictions based on your theory of phenomenal consciousness, and then arguing that we should already update on those predictions ahead of time, since they’re so firm. I’m personally sympathetic to this line of argument, but it obviously depends on some assumptions which need to be articulated, and which the physicalist would probably not be happy to make.
Today’s Inkhaven post is an edit to yesterday’s, adding more examples of legitimacy-making characteristics, so I’m posting it in shortform so that I can link it separately:
Here are some potential legitimacy-relevant characteristics:
The reasoning is logically valid.
The assumptions of the argument are credible (this splits into many characteristics we can name:)
The assumptions are simple.
There are few assumptions.
The assumptions have very few degrees of freedom.
The assumptions are agreed upon by many humans.
The assumptions are a strong consensus in the relevant field(s).
The assumptions are very probable according to best existing models.
The assumptions have high-quality citations backing them up.
The assumptions have been very useful (EG productive axioms in mathematics).
The assumptions are not jointly contradictory.
The reasoning is probabilistically valid.
The reasoning is not Dutch-bookable.
The reasoning is very difficult to Dutch-book (bounded cognition).
The reasoning is accuracy-maximizing.
The reasoning is a plausible extension of logic.
The reasoning employs credible priors.
The prior is close to human priors, or priors humans justifiably endorse.
The priors have rigorous frequentist justification (EG probability of a prime number based on the prime number theorem).
The priors have empirical validation.
The priors have a maximum-entropy justification.
The priors are dominant over other priors one might want to use.
The probability of the conclusion is very high.
The argument is very strong as a statistical test.
The bias is low.
The variance is low.
Confidence intervals are small.
The test has a low false positive rate.
The test has a low false negative rate.
The test is the best test to use out of alternative tests.
Robustness to outliers.
Robustness to a variety of distributions.
Good rate of convergence.
Yeah, the logic still can’t handle arbitrary truth-functions; it only works for continuous truth-functions. To accept this theory, one must accept this limitation. A zealous proponent of the theory might argue that it isn’t a real loss, perhaps arguing that there isn’t really a true precise zero, that’s just a model we use to understand the semantics of the logic. What I’ll say is that this is a real compromise, just a lesser compromise than many other theories require. We can construct truth-functions arbitrarily close to a zero detector, and their corresponding Strengthened Liar will be arbitrarily close to false.
Seems to me like both.
No disagreement with the broad statements, but I note that your words do not particularly register the point that good conversation itself might be a turnon and lack thereof a turnoff? IE your post presents a puzzle: what’s with the banter → sex thing? I’m suggesting that many people might want to talk first as an inherent preference. Sure, there might be ways around that, but you weren’t asking for something with no loopholes, you were asking about the banter → sex thing.
Not really an experienced player of the relevant games, but I personally have turned down an obvious sex invitation with someone who I was otherwise interested in because too little conversation (and don’t regret this choice). I am not very interested in sex with someone who I can’t have a good conversation with. I feel like a lot of the intrigue of an intimate encounter is conversational intimacy. I’ve never experienced the chat at party → sex pipeline, however. Only [chat online for multiple months]->sex.
I also don’t believe that insider trading is immoral. Insider trading increases the accuracy of the stock prices available to the public, which is the public good that equity trading provides. For this reason, prediction markets love insider trading. The reason it’s illegal is to protect retail investors, but why do they get privileged over everyone else? Another reason insider trading is immoral is that it robs the company of proprietary information (if you weren’t a limb of The Company, you wouldn’t know the merger is happening). That’s fair, but in that case doing it officially for The Company should be allowed, and it’s not. In this example ChatGPT arguably helped steal information from LING, but did so in service of the other company, so I guess it’s kind of an immoral case—but would be moral if LING is also insider-trading on it.
The problem with insider trading, in my view, is that someone with an important role in the company can short the stock and then do something really bad that tanks the value of the company. The equilibrium in a market that allows insider trading involves draconian measures within the companies themselves to prevent this sort of behavior (or else, no multi-person ventures that can be publicly traded).
This is an instance of the more general misalignment of prediction markets: whenever there’s something on a prediction market that is quite improbable in ordinary circumstances but could be caused to happen by a single actor or a small number of people, there’s profit to be made by sewing chaos.
all the relationships between components, etc, I made “explicit”
Is there a typo here? “are made explicit” perhaps?
or that there is some set of rules such that following those rules (/modifying the expression according to those rules) is guaranteed to preserve (something like) the expression’s “truth value”?
That’s correct. More generally (since the concept also applies to noun phrases) guaranteed to preserve its “value” whatever type that may be. This “value” is something like what-it-points-at, semantic reference.
Yep, will fix.
(Also it’s kinda iffy that weak disjunction is a stronger statement than strong disjunction...)
Yeah! I’m just going with what Wikipedia said there (unless I’ve made an error), but I had the same thought.
Fixed!
I’m not sure I understand the argument here correctly. It seems like the intended argument is something like this:
“Omega has access to an infinite number of fair coinflips. Alice can do no better than guess, and Alice cannot guess every coin-flip correctly. Omega knows how Alice will guess, and also knows how each coinflip will land. Therefore, Omega can choose to ask Alice about only the coinflips Alice will guess incorrectly (of which there will be at least one). Alice therefore surely loses money from bets placed.”
This argument uses the assumption that Alice can’t change eir beliefs in response to learning that Omega has proposed specific bets and not others. This might seem concerning, because it seems like precisely what Alice should do, if Alice understands the situation: Alice should expect to lose any bet proposed by Omega. However, this assumption is perfectly normal for Dutch Book arguments. Such an objection would rule out all the usual Dutch Books. I think the classic Dutch Book arguments in fact illustrate a useful idea, even with this ‘flaw’, so I allow it.
More concerningly, the argument assumes Omega has knowledge of how the coins will land. This is a significant departure from classical Dutch Books. It seems clear that a bookie can reliably make money from gamblers if the bookie knows which horse will win which race; this is not, in the classical way of thinking, a testament to the irrationality of the gamblers. It appears to me that this is all that is happening in the above argument.
A second quibble is that in classical Dutch Book arguments, the bookie will surely make money. In the argument above, the bookie only almost surely makes money: since Omega relies on Alice making a bad guess, Omega makes money with probability 1, but not with (logical) certainty.
Considering these two violations of the pre-existing norms of Dutch Books, what should we make of the proposed Dutch Book argument? It intuitively makes sense to me that Infrabayes might be supported by a sort of almost-dutch-book argument. It offers a fresh perspective; perhaps we need to slightly modify the pre-existing norms wrt Dutch Books to see the benefits of infrabayes.
(An analogy: intuitionistic bayesianism generalizes the usual dutch books by allowing bets to fail to pay out, cleanly justifying the possibility of probabilities that do not sum to 1.)
I am mostly unbothered by weakening surely to almost-surely. Losing money with probability 1 seems almost exactly as bad as losing money with logical certainty. However, I haven’t thought deeply about the consequences of such a move. Perhaps this allows some unsavory “Dutch Book” arguments.
Allowing the bookie to know more than the gambler seems far more worrying, but perhaps justifiable. The classical Bayesian really does need to rule out such a case, but perhaps this is precisely because they are not infrabayesian. One might argue that infrabayes is precisely the generalization in belief-structures required to handle this generalization of dutch-books.
Personally, it seems to me like a more natural way to handle bookies who know more is to drop the earlier-mentioned assumption that the gambler’s probabilities are independent of what bets the bookie proposes. If gamblers know that the bookies at the horse-race know which horses are going to win, then they should update upon seeing what bets those bookies are willing to take. The assumption to the contrary was only tenable in the context of bookies who don’t know anything the gamblers don’t.
Perhaps, then, the content of the argument is that infrabayesianism can handle knowledgeable bookies in a different way: though we could perhaps handle such cases by dropping the no-update-on-bets-offered assumption, doing so might not result in a very nice theory. Instead, infrabayesianism recommends a strict preference for mixed strategies. I’m not against the idea of a strict preference for mixed strategies, but it also doesn’t jump out at me as the natural way to handle this dutch-book argument as I understand it: after all, we could just as well suppose that Omega can predict the randomness behind the mixed strategy.
I came upon this post because the more recent What is Inadequate about Bayesianism for AI Alignment cited this as the source of its Dutch Book against bayesians. However, the Dutch Book argument made there is somewhat different. That version relies on a “causal” assumption that Omega’s choices are probabilistically independent of the gambler’s. This assumption seems inherently contrary to the problem description (since Omega can predict the gambler’s choices, and uses those predictions to make its choices). Again, maybe the point is that it is theoretically useful: although the “correct” way (according to me) to deal with such cases is to drop the independence assumption, it turns out that we can work out a beautiful and useful theory without doing so.