“If a tree falls in the woods, but no one is around to hear it, does it make a sound?” doesn’t sound like an argument, but a question. “Yes, because the presence of a person with ears doesn’t affect the physical behavior of the air” or “No, because air waves shouldn’t be considered sound until they interact with a mind” are arguments.
Or do you mean “argument” in the sense of a debate or discussion (as in “we’re having an argument about X”)?
Could one approach to detecting biases be to look for “dominated strategies”?
For instance, suppose the human model is observed making various trades, exchanging sets of tokens for other sets of tokens, and the objective of the machine is to infer “intrinsic values” for each type of token.
(Maybe conditional on certain factors, i.e “An A is valuable, but only if you have a B”, or “a C is only valuable on Tuesday”).
Then if the human trades an A and an E for a B, a B for a C, and a C for an A, but then trades an A for ten Es, we can infer that the human has some form of bias, maybe neglecting tokens with small value (not realizing that the value of an E matters until you have ten of them), or maybe an “eagerness” to make trades.
This clearly relies on some “Strong assumptions” (for instance, that tokens are only valuable in themselves—that executing a trade has no inherent value).
This is great.
A point which helped me understand number 6: If you ask someone “why do you believe X”, since you’re presumably going to update your probability of X upwards if they give a reason, you should update downwards if they don’t give a reason. But you probably already updated upwards as soon as they said “I believe X”, and there is no theorem which says this update has to be smaller than the latter update. So you can still end up with a higher or equal probability of X compared to where you were at the beginning of the conversation.
I tend to favor your own approach—think about whatever I’m working on.
The solution to not having enough questions is to always keep a question around which is A: hard enough that you’re unlikely to solve it during a brief wait, and B: in a state where you can work on it without something to write on. Combining these two is not always easy, so you sometimes need to plan ahead.
Departing a bit from the question as stated, adding a phone(and headphones), I’ve also found that listening to audiobooks is a good way to use e.g. a commute.
I added some clarification, but you are right.
(Since x5−10 has the root 5√10, it’s clearly not true that all fifth-degree polynomials have this property)
“If you’ve never missed a flight, you spend too much time hanging around in airports” ~ “If you’ve never been publicly proven wrong, you don’t state your beliefs enough” ?
(There was a LaTeX error in my comment, which made it totally illegible. But I think you managed to resolve my confusion anyway).
I see! It’s not provable that Provable(A=10⇒U=10) implies A=10. It seems like it should be provable, but the obvious argument relies on the assumption that, if *A=10⇒U=0 is provable, then it’s not also provable that A=10⇒U=10 - in other words, that the proof system is consistent! Which may be true, but is not provable.
The asymmetry between 5 and 10 is that, to choose 5, we only need a proof that 5 is optimal, but to choose 10, we need to not find a proof that 5 is optimal. Which seems easier than finding a proof that 10 is optimal, but is not provably easier.
I think I don’t understand the Löb’s theorem example.
If A=5⇒U=5∧A=10⇒U=0 is provable, then A=5, so it is true (because the statement about A=10 is vacuously true). Hence by Löb’s theorem, it’s provable, so we get A=5.
If A=10⇒U=0∧A=10⇒U=10 is provable, then it’s true, for the dual reason. So by Löb, it’s provable, so A=10.
The broader point about being unable to reason yourself out of a bad decision if your prior for your own decisions doesn’t contain a “grain of truth” makes sense, but it’s not clear we can show that the agent in this example will definitely get stuck on the bad decision—if anything, the above argument seems to show that the system has to be inconsistent! If that’s true, I would guess that the source of this inconsistency is assuming the agent has sufficient reflective capacity to prove “If I can prove A=5, then A=5. Which would suggest learning the lesson that it’s hard for agents to reason about their own behaviour with logical consistency.
I think I managed to avoid the Inbox Zero thing by not reading my emails, if the little bit of text that Gmail displays is enough for be to be confident that I don’t need to read or respond to the mail. This means that I have a huge, constantly growing number of unread mails in my inbox, so the idea of getting it down to zero isn’t really attractive.
I still check my email unnecessarily often, but I don’t feel a compulsion to read any new mails immediately.
Belief: There is no amount of computing power which would make AlphaGo Zero(AGZ) turn the world into computronium in order to make the best possible Go moves (even if we assume there is some strategy which would let the system achieve this, like manipulating humans with cleverly chosen Go moves).
My reasoning is that AGZ is trained by recursively approximating a Monte Carlo Tree Search guided by its current model (very rough explanation which is probably missing something important). And it seems the “attractor” in this system is “perfect Go play”, not “whatever Go play leads to better Go play in the future”. There is no way for a system like this to learn that humans exist, or that it’s running on a computer of a certain type, or even to conceptualize that certain moves may alter certain parameters of the system, because these things aren’t captured in the MCTS, only the rules of Go.
This isn’t an argument against dangerous AGI in general—I’m trying to clarify my thinking about the whole “Tool AI vs Agent AI” thing, before I read reframing superintelligence.
Am I right? And is this a sound argument?