If you think people weigh logical counter factual to make decisions, I don’t think you have talked to many people.
See my post which discusses how we understand it from behavioral economics. The standard view is pretty simple: people place real utility on complex values and actions. I discuss the example of voting in detail.
I had another comment with a general overview of my own issues with FDT which are different. But I want to ago through and address some other points that are more tangential.
The standard jumping off point for decision theories is the set of fair problems with perfect information about the overall scenario
This is quite different then how most decision theory and game theory courses will introduce normal-form games and these aren’t always assumed. These also have nothing to do with “fairness,” fair problems are not typically a formal basis.
If we assume an imperfect predictor, then Newcomb’s problem is not a perfect information, normal-form game since the predictor is acting without common knowledge of other player’s strategies. If the predictor is a perfect predictor, then it can be understood as a normal-form game, but there is no decisions (by definition, whatever it predicted will always be right so you have no choice over the outcome, it is predetermined as whatever Omega predicted). The original framing of Newcomb’s problem is evidentiary, it does not assume perfect knowledge of how it is making its predictions or what their accuracy is, it only assumes a strong evidentiary basis to believe the predictions are accurate.[1]
For reference, a normal-form game is typically defined with the below assumptions:
To someone who adopts evidential decision theory, you could reasonably say the set of fair problems is just every problem for which the outcomes have evidentiary dependence on your decision. The CDTer could similarly say the set of fair problems is those with causal dependence on your decisions. FDT to my understanding would consider the set of fair problems those problems where the outcomes are logically dependent on your choices. None of these would inherently consider newcomb’s problem unfair, it just depends on how you judge the outcomes. CDT is going to judge the outcome just by the consequences of your decision, not by some assumed logical relationship.
As described by Nozick: “You know that this being has often correctly predicted your choices in the past (and has never, so far as you know, made an incorrect prediction about your choices), and furthermore you know that this being has often correctly predicted the choices of other people, many of whom are similar to you, in the particular situation to be described below”
My recent critique of FDT (coming from an economic background) is that in economics we have 2 real uses for a decision theory.
Descriptive (e.g., describing how people actually behave and interact around uncertain utility expectations resulting from their decisions)
Prescriptive (i.e., how people should act to optimize their utility expectations).
FDT seems to be obviously much less useful for 1 and not clearly more useful for 2, when trying to apply it to real world models it claims to perform in. Even granting in the abstract the assertion that FDT’s answers are more correct (which is not an objective criteria), it doesn’t follow that we should prefer it. It seems facially (and formally) less useful for deriving prescriptions even in examples proponents cite.
Edit: to be clear, it doesn’t claim to be useful as a descriptive theory. Advocates claim it’s advantage is prescriptive. This would be fine if it could offer better prescriptions. But on the issues proponents claim it offers better prescriptions, the same proponents (e.g. Yudkowsky) citing those issues (e.g. voting) are unable to actually articulate better prescriptions. They, in fact, admit that FDT does not offer a clear way of prescribing actions, while other decision theories do offer clear prescriptions.
I think it matters for a lot of people. A lot of people would say the odds are substantially lower given international action.
“I’m worried about AI risk” is not any less vague than “P(doom) = 50%”.
I think this misses the point. “I am worried about AI risk” doesn’t seem to give itself any undue weight: it doesn’t feign being more confident or certain of what the risk is.
If your actions can change the probability of an outcome, then you can’t meaningfully have a belief about the probability.
I think the problem getting at is that people’s idea of P(doom) is often really P(doom|inaction) not a cumulative estimate of probability over the full set of probable actions and all interactions. It is frequently given implicitly assuming no interactions with people’s discussions or reactions to risk.
Necro, but this seems to lead to other weird conclusions if we assume their belief is true as described by the statement: “Both the seller and the buyer believe the seller’s prediction to be accurate with probability 0.75.” This requires b1 and b2 to be viewed distinctly buy the buyer (if they are equally likely to pick box 1 and box 2, their is no solution). However, whatever their preference is we can plug back into their odds (not causally determined!) and get different expected values.
Using b to denote the buyer’s decision to buy box 1 or 2 and g to denote the seller’s prediction.:
If we grant your scenario, we know that the CDT agent will buy a box thus reasons p(b1) + p (b2) = 1.
For the belief to be correct, it must be the case that p(b1|g1)+p(b2|g2)= 0.75 and p(b1|g1) > 0 and p(b2|g2) > 0
If their accuracy doesn’t vary, it follows that p(g1|b1) = 0.75 and p(g2|b2)=0.75 and p(g0|b0) = 0.75.
Given Baye’s theorem can also say: p(g1) = p(g1|b1) *p(b1) / p(b1|g1) and p(g2) = p(g2|b2) *p(b2) / p(b2|g2)
Let’s imagine for ease that the buyer cannot distinguish between boxes (so p(b1) = p(b2) = 0.5).
We can then do some algebra: p(g1) = 0.35 / p(b1|g1)
Since(skipping over more algebra) we can show: p(g2) = 0.35 / (0.75 - p(b1|g1))
We can then see:
1> 0.35 / p(b1|g1) → p(b1|g1) >.35
1>0.35 / (0.75 - p(b1|g1)) > 0 → p(b1|g1) < 0.4
Thus: 1<p(g1)<.875 and 1<p(g2)<.875. Since p(g1) + p(g2) ⇐ 1 this is impossible.
Let’s imagine the other extreme, p(b1) = 1 and p(b2) = 0 (so they will always take first the box sequentially as the calculate it has a positive expected value).
We can then solve as: g(1) = 0.75 / p(b1|g1)
Since p(b1) = 1, it follows p(b1|g1) = 1, so p(g1) = 0.75. And, in this case, p(g2) = 0.
But in this case, we can go plug it back into the original problem as get the EV for the CDT agent as EV(b1) = 0.25*3 − 1 = − 0.25. EV(b2) = 3-2 = 2.
For them to be correct about the 75% accuracy, requires that the CDT agent both treat the boxes differently (i.e. they are more likely to choose one or the other) and for the CDT agent to not consider how that effects the odds of each box, since whichever box they prefer picking will have a expected value of < 0 before considering any causality.
It seems like your complaint is just with the term sense data then, and I don’t see why. You can replace the word sense with “experienced” and the meaning is identical. Sense data refers to the information we obtain through our experienced perceptions. Sense data refers to the experience not the raw information flowing through the nervous system.
since in fact a larger part of your experience depends on the processing than the inputs.
I don’t thjnk this is a fact. Our experience is one thing dependent on the entire process. If you cut off the front end, there is no information, if you cut off the backend the same is true.
Take your example of a camera. Which is more responsible for a photo: the incoming light, the CCD, the wiring (inc. A/D converter, line driver, frame driver), or the software the converts those signals into a display? Intuitively, my feeling would be the software is the least important (you could derive the information from the raw signals, but not the other way) but as far as understanding what captures the image you have to have the entire process for it to work.
Yes, I think this is worth pointing out, but what I wanted this to be focusing on the qualitive way people tend to use Bayes in citing their confidence as a percentage (without differentiating the source of their estimate) which gives rise to appearing excessively confident and uncritical.
Moving it to talk about a beta distribution is just a step back, since before hand Al would have good reason to think the probability was 50% while Bri’s estimate is purely a guess (as she lacks any data to derive the likely outcomes).
Yes the thought experiment is meaningful based on the fact that they are the same. It serves to answer the question “why talk about qualia at all rather than just calling experiences sense data?
If something is identical, it doesn’t make much sense, imo, to differentiate them like this. It makes more sense to say “sense data is the total product of information transmitted and interpreted by our nervous system” then to say “sense data is the information recorded by our peripheral nervous system, which is experienced by our central nervous system which poses qualia”. The actual sense data we are talking about is our perceptions not the uninterpreted chemical systems. SO it doesn’t make sense to consider them separate entities when talking about the perceived data.
It is not obvious to me that imagining flipping one’s optic nerves is meaningfulling different in terms of our experience than flipping around one’s neurons. If a brain interprets a signal from an optic nerve as recieving red light (when the light is of wave length normally called green) where the signal is from a normal optic nerve, that seems identical in terms of experience to an optic nerve sending a signal that the brain interprets as red because the optic nerves that transmit the signals are switched. It is like saying 1+2 is meaningfully different from 2+1. The ordering doesn’t matter if the end results are indistinguishable. There is no discernible difference in the results so it seems to me they should be thought of as the same, it seems nonsensical to define the first one as something like “experiences from different qualia” and the second as “mere differences from optic nerves.” In no way does the former seem indicative of a deeper difference that I can identify.
I was going to write up an essay earlier on the topic but held off because real work and I saw you had tweeted about doing a write up covering similar topics. It seems to me what people get “wrong” about the zombie, conciousness etc debate is a matter of first principles.
I think we (understandably) over-empasize definitions sometimes, which becomes a key problem. Try to approach the problem from both directions, from the bottom modeling and applying models to the world and from the top (more akin to Descartes’ approach) observing and making inductive inferences from there. Neither side can provide us strict ‘proof’ but provide useful ways to think about the problem.
From the bottom up:
Imagine at the level of first principles modeling the abstract terms without reference to the physical. This is what math is about: I can do calculations using the Pythagorean theorem (a^2 + b^2 = c^2) to make useful calculations of distances when I map that relationship onto real world things. Defining a in terms of real world phenomena would be less than unhelpful.
What is conciousness in such a model? You might say your model includes some functions that model themselves and you are calling that conciousness, okay how helpful is that to understanding the world? How well does it map on to what we see in the world?
My senses are aware of something, call it qualia perhaps, does this map on to the same term?
We can think of our senses from the top down instead:
“I” have this experience. What is the basis for the experience? It seems to be have strong causal ties to neurological activity which and responses to stimulus which I can observe from my nervous system and brain studies. Okay, do others have an analogous experience? Well they, like me, report having something similiar, and when we cut open their brains and look at how they respond to stimuli so it is reasonable to say that the explanation that best fits those observations is that there are analogous experiences going on in them. Okay, do animals have the same? Well, they seem to share the same mechanism and many of the responses to stimuli to various extents, some are nearly identical to us. Some seem to also be able to perform similiar tasks which require responding to stimuli and modeling themselves. So to varying extents we can deduce with various confidences this data is best explained by some sort of analogous experience. Okay what about a camera? Not so much, the same analyses cannot be applied. How about an LLM? Maybe we see some of the same actions, possibly we even can get it modeling itself, but we don’t have the same mechanisms and at a granual level the responses to stimuli are quite different (they lack nervous systems, of course) so we cannot deduce an analogous experience.
------
After all that we are left with the “so what?” What does it mean if something is concious? Are we going to apply what we infer to utilitarian calculations and say “if i infer you have qualia, then I place utility on your experience so it is wrong to kill and eat you, but if you lack qualia then your experience is without utility so if it improves my utility killing and eating you is a good thing, actually”? Does your understanding of the world with it lead to different policies and approaches? These involve quite a bit of subjective reasoning, so I will leave any reader to try to think them through and answer themselves.
Not sure what you mean by that.
I think I should have discussed the implicit point about information states more explicitly, I sort of assumed it was a point people familiar with probabilities would understand but I think not enumerating the point made my post more confusing.
When we talk about probability in determined and determinist states depends on information states.
The probability of it being tails at the point the question is asked is 100% since it landed tails up and heads down. But Al and Bri don’t have the information to make that assessment, so instead they have to assess the probability another way. What Al is estimating is not the “real world probability of this particular event” but instead “if I found myself in a situation with the same knowledge, how often in those situations would I expect the coin to be heads up or tails up?” His estimate of 51% of the time it would be heads up would correct and reasonable in the frequentist view in the but wrong to the real world.
By contrast, Bri’s estimate of 90% is likely unreasonable but correct to the real world data.
The point being that bad priors can appear correct when generally wrong. As a consequence, priors that are not objectively verified should be viewed with a great deal more skepticism even if they may appear to be correct. I had another section which I was too lazy to finish writing up (as it really should be done more mathically rigorously) extending this to causal implications through the example of monkey’s using typewriters. I might get around to it at some point.
Might make it longer at some point, as there are a lot more nuances, but in case you are interested, I made a post discussing the use of decision theory and my view about how FDT fails to provide the same use, even utilizing the real world cases that Yudkowsky tends to cite for preferring FDT: https://www.lesswrong.com/posts/ALvpivH4Ys6Jp7CKK/the-disutility-of-fdt-on-utility-functions-and-voting
It is to prescribe behaviors which I expect to be similar to ethical ones, like “Don’t loot the other universe even if it’s inhabited only by a paperclip optimizer”.
Also (apologies for the double comment), while directionally correct, I think this may be a bit of a misrepresentation of the aim of their decision theory (Yudkowsky and Soares would have to clarify their exact intent).
Decision theories, including FDT, take utility functions as external to the theories themselves. If an agent gained utility by anti-social, destructive acts, every decision theory would prescribe positive values to those acts by definition. I don’t think FDT is intended to contradict this definition (indeed, their paper implicitly seems to accept it). Where I think you are directionally correct is that it aims to recommend the decision that would optimize utility from cooperating where greater (the first dilemma they describe in their paper is a clear example of this). If one’s ethical values are encapsulated within a utility function, than whatever optimizes utility should also be the most ethical (and if you are a utilitarian, the optimal ethical and optimal utility decisions are definitionally identical).
I should have been clear there, I wasn’t trying to imply that the descriptive utility was a claim they made. i meant that in the sense that “it fails to serve a purpose which traditional decision theories are intended for (and are useful at)” not that “it fails to serve the purpose it is designed for.”
I would argue it fails on its own terms (at least sometimes) as a normative theory (which is a stated goal for the theory given by Yudkowsky & Soares) but as you say it isn’t trying to be a descriptive theory. It not being useful as a descriptive theory is just a point of comparison of its merits against traditional models.
Edit: Also, in case it isn’t obvious, I have an economic background so my primary interest in it would be if it outperforms the uses of decision theory (both normative and descriptive) in economics.
I was thinking of doing more research and a write up, but someone may have covered it or knows of a good reference (I have seen there is some related discussions here on uncertainty and Bayesian updating).
Priors are not all equal, there is a big (rather obvious) difference between a prior that is based on empirical data and quantitatively robust models, and priors that are based on personal guesses and intuitions. Personally, with my own empirical and skeptical training as background, I find conflating these somewhat unhelpful and giving the appearance of having more confidence than one really should at times (that is, having a 90% confidence derived from empirical analysis seems very different than having a 90% confidence derived from unenumerated priors).
Is there a good discussion anyone is aware of delineating these? It seems often, particularly among those applying bayesian updating to lay settings, priors are not clearly differentiated on an evidentiary basis.
I think it depends on what you are including by analogy, when making general cases, we have to pretty much exclusively deal with analogs.
The most basic case might be stated as:
If ∀x(a⊃x⊢b) ⊃ ∀y(a⊃y⊢b)
y1⊢a
∴ y1⊢b
That is, e.g.
If all for all convicted criminals, killing someone implies they are bad, then for cops killing someone implies they are bad.
Officer Bob killed someone.
Therefore, officer Bob is bad.
If it is the case that for convicted criminals killing someone is only bad because, e.g., there wasn’t a justification, then (1) is false (and you would have to include that as a qualifiers to make it true). But it is trivial to accept as true.
Thank you for the references! Aella’s case is interesting a very good example of how design matters to how you should think about the problem, particularly her statement (emphasis added):
in general whenever I’ve asked variations on this set of questions, the number of correct votes roughly increases with the difficulty of the problem, before hitting a ceiling when the difficulty is too high, and slowly decreases with that difficulty.
In a lot of survey design the opposite is a much larger problem, particularly if you incentivize responses (i.e. paying people for completing the survey). The more complex you make a question, the more often you will have lazy respondents just pick the first question instead of engaging with what is being asked. Because she is just posing a twitter poll with little incentive to answer, I imagine most lazy respondents simply do not read the question.
@Karl Krueger, thank you for the typo corrections and reacts! I deleted the “linked landlines” part since it was confusing and added nothing, but, for reference, I was referring to a practice of address-based sampling (ABS) where addresses are sometimes matched out to landlines to make it easier to contact them for responses. These often leads to particular sampling bias (which can sometimes be addressed by creating separate sets of addresses that can be matched and those that cannot). It is sort of a middle ground between randomly sampling addresses and landline surveys (randomly sampling numbers has a lot of issues).
Since surveys using landlines are generally more prone to bogus responses, I had referenced that in my example, but on reflection is a needless nuance that doesn’t change the point being made.
I will probably try to reword a few things pursuant to some other areas you marked tomorrow, time permitting.
Apologies for the double post, but to be a bit more precise, since my last comment was somewhat dismissive. As I discuss in my post, the reason people will give for voting even when they do not expect their vote to have an impact in excess of the cost of voting is because they place some value on voting. This is entirely consistent with standard behavioral models and is causal.
People get a warm and fuzzy feeling for engaging in pro-social, altruistic behavior. This has a real utility value. They also get signaling benefits from engaging in public pro-social behavior. This also has real utility benefits. Any serious model of real world utility will include these effects.
FDT does not offer clear intuitions with regards to behavior like voting. Even proponents (like Yudkowsky) will confess they are unable to offer a way of valuing voting under FDT. But some plain intuitions from FDT directly contradict observed behaviors and the explicit views of FDT advocates as I mention in my previous post.
FDT would imply “the more people that are similar to you, the more value you should place to vote” which runs counter to my intuition and the intuition of proponents like Yudkowsky under FDT (who proports to, at least circumstantially, endorse not voting “if you don’t expect any of the elections to be close”).