MichaelA

Karma: 826

MichaelA 16 Aug 2019 1:08 UTC
14 points
in reply to: waveman’s comment on: The Robbers Cave Experiment
I know I’m very late to this thread, but I wanted to mention that this article also provides reasons to not place too much weight on Sherif’s results. (Although of course the broad inferences drawn from his results might happen to be true anyway.) In particular, the article suggests Sherif had attempted a similar study earlier (in another location, with other boys), did not find the results he wanted (despite manipulation), and then suppressed that attempt’s results:
Despite his pretence of leaving the 11-year-olds to their own devices, Sherif and his research staff, posing as camp counsellors and caretakers, interfered to engineer the result they wanted. He believed he could make the two groups, called the Pythons and the Panthers, sworn enemies via a series of well-timed “frustration exercises”. These included his assistants stealing items of clothing from the boys’ tents and cutting the rope that held up the Panthers’ homemade flag, in the hope they would blame the Pythons. One of the researchers crushed the Panthers’ tent, flung their suitcases into the bushes and broke a boy’s beloved ukulele. To Sherif’s dismay, however, the children just couldn’t be persuaded to hate each other.
After losing a tug-of-war, the Pythons declared that the Panthers were in fact the better team and deserved to win. The boys concluded that the missing clothes were the result of a mix-up at the laundry. And, after each of the Pythons swore on a Bible that they didn’t cut down the Panthers’ flag, any conflict “fizzled”. By the time of the incident with the suitcases and the ukulele, the boys had worked out that they were being manipulated. Instead of turning on each other, they helped put the tent back up and eyed their “camp counsellors” with suspicion. “Maybe you just wanted to see what our reactions would be,” one of them said.
That said, the article doesn’t seem to provide evidence of substantial manipulation during the Robbers Cave study itself. So perhaps the conclusion to draw from Sherif’s pair of studies is that, under such conditions, intense intergroup conflict will arise naturally roughly half the time.

MichaelA 30 Dec 2019 0:41 UTC
1 point
in reply to: TurnTrout’s comment on: Editor Mini-Guide
As a reader, I’d find it quite useful to be able to jump back from the footnote to where I was in the main text. Not being able to do so slows me down, especially on long articles and when reading on an iPad (where going back to “find in page” every time is a bit annoying).
As a (just beginning to be a) writer, it’d be nice to have a footnote option pop up when I highlight text (like italics, linking, etc.), rather than having to go into markdown mode for that. (Or am I just missing the easy, WYSIWYG-mode way of doing footnotes?)

MichaelA 30 Dec 2019 1:11 UTC
2 points
in reply to: Steven Byrnes’s comment on: Editor Mini-Guide
A few things I’ll add to habryka’s response, based on what did/didn’t seem to work for me:
- I was struggling to make longnotes work (they were working as footnotes, but were just removing the paragraphs after the first one), but I discovered there’s also something called “bignotes”, which is basically identical but did work for me. Info here.
- You can write “[^bignote2]” and “[^bignote2]:” (and then 3, 4, etc.) to have more than 1 bignote. (This wasn’t in the link I provided above, but I guessed it might work and it did. I assume the same would apply for longnotes, but I haven’t been able to get them to work.)
- “indenting” for longnotes/bignotes seems to require precisely the number of spaces used in the examples on this page and the github link (this many: ” ”), or at least not just 1 space, and “tab” doesn’t work.
What links here?
- MichaelA's comment on Thoughts on The Weapon of Openness by Will Bradshaw (EA Forum; 15 Feb 2020 8:36 UTC; 9 points)

MichaelA 30 Dec 2019 1:14 UTC
2 points
on: Editor Mini-Guide
Is there a way to switch quickly between markdown and regular editing mode, without having to go into user settings? (I.e., switch while editing an article.)

MichaelA 31 Dec 2019 0:00 UTC
2 points
in reply to: Pattern’s comment on: Making decisions under moral uncertainty
Thanks for your kind words and your feedback/commentary!
(I’ll split my reply into multiple comments to make following the threads easier.)
In this sense, this brand of non-consequentialist theories seems to be an amalgamation of ‘moral theories’.
I’m not sure I see what you mean by that. Skippable guesses to follow:
Do you mean something like “this brand of non-consequentialist theories seem to basically just be a collection of common sense intuitions”? If so, I think that’s part of the intention for any moral theory.
Or do you mean something like that, plus that that brand of non-consequentialist theory hasn’t abstracted away from those intuitions much (such that they’re liable to something like overfitting, whereas something like classical utilitarianism errs more towards underfitting by stripping everything down to one single strong intuition[1]), and wouldn’t provide preference orderings that satisfy axioms of rationality/expected utility[2]? If so, I agree with that too, and that’s why I personally find something like classical utilitarianism far more compelling, but it’s also an “issue” of lot of smart people of aware of and yet still endorse the non-consequentialist theories, so I think it’s still important for our moral uncertainty framework to be able to handle such theories.
Or do you mean something like “this brand of non-consequentialist theory is basically what you’d get if you averaged (or took a credence-weighted average) across all moral theories”? Is so, I’m pretty sure I disagree, and one indication that this is probably incorrect is that accounting for moral uncertainty seems likely to lead to fairly different results than just going with an ordinal Kantian theory.
Or is intention behind words not a multiple choice test, in which case please provide your short-answer and/or essay response :p

[1] My thinking here is influenced by pages 26-28 of Nick Beckstead’s thesis, though it was a while ago that I read them.
[2] Disclaimer: I don’t yet understand those axioms in detail myself; I think I get the gist, but often when I talk about them it’s more like I’m extrapolating what conclusions smart people would draw based on others I’ve seen them draw, rather than knowing what’s going on under the hood.
In this case, one relevant smart person is MacAskill, who says in his thesis: “Many theories do provide cardinally measurable choice-worthiness: in general, if a theory orders empirically uncertain prospects in terms of their choice-worthiness, such that the choice-worthiness relation satisfies the axioms of expected utility theory [footnote mentioning von Neumann et al.], then the theory provides cardinally measurable choice-worthiness.” This seems to me to imply (as a matter of how people speak, not by actual logic) that theories that aren’t cardinal, like the hypothesised Kantian theory, don’t meet the axioms of expected utility theory.

MichaelA 31 Dec 2019 0:13 UTC
3 points
in reply to: Said Achmiz’s comment on: Making decisions under moral uncertainty
Good point. I wondered if maybe this should instead be called “Overview of moral uncertainty” or “Making decisions under moral uncertainty” (edit: I’ve now changed it to the latter title, both here and on the EA forum, partly due to your feedback).
Do you mean adding a paragraph or two at the start, or a whole other post?
Part of me wants to say I don’t think there’s really that much to say about moral uncertainty itself, before getting into how to handle it. I also think it’s probably best explained through examples, which it therefore seems efficient to combine with examples of handling it (e.g., the Devon example both illustrates an instance of moral uncertainty, and how to handle it, saving the reader time). But maybe in that case I should explicitly note early in the post that I’ll mostly illustrate moral uncertainty through the examples to come, rather than explaining it abstractly up front.
But I am also considering writing a whole post on different types/sources of moral uncertainty (particularly integrating ideas from posts by Justin Shovelain , Kaj_Sotala, Stuart_Armstrong, an anonymous poster, and a few other places. This would for example discuss how it can be conceptualised under moral realism vs under antirealism. So maybe I’ll try write that soon, and then provide near the start of this post a very brief summary of (and link to) that.

MichaelA 31 Dec 2019 0:20 UTC
2 points
in reply to: Pattern’s comment on: Making decisions under moral uncertainty
It would be useful to have an example of how Variance Voting works. Also, the examples for the other methods are fantastic!
Thanks! I try to use concrete examples wherever possible, and especially when dealing with very abstract topics. Basically, if it was a real struggle for me to get to an understanding from existing sources, that signals to me that examples would be especially useful here, to make things easier on others.
In the case of Variance Voting, I think I stopped short of fully getting to an internalised, detailed understanding, partly because MacAskill doesn’t actually provide any numerical examples (only graphical illustrations and an abstract explanation). I’ll try read the paper he links to and then update this with an example.

MichaelA 31 Dec 2019 0:28 UTC
2 points
in reply to: Pattern’s comment on: Making decisions under moral uncertainty
What is sensitivity analysis?
This is a somewhat hard to read (at least for me) Wikipedia article on sensitivity analysis. It’s a common tool; my extension of it to moral uncertainty would basically boil down to “Do what people usually advice, but for moral uncertainty too.” I’ll link this comment (and that part of this post) to my post on that once I’ve written it.
Also, sensitivity analysis is extremely easy in Guesstimate (though I’m not yet sure precisely how to interpret the results). Here’s the Guesstimate model that’ll be the central example in my upcoming post. To do a sensitivity analysis, just go to the variable of interest (in this case, the key outcome is “Should Devon purchase a fish meal (0) or a plant-based meal (1)?”), click the cog/speech bubble, click “Sensitivity”. On all the variables feeding into this variable, you’ll now see a number in green showing how sensitive the output is to this input.
In this case, it appears that the variables the outcome is most sensitive too (and thus that are likely most worth gathering info on) are the empirical question of how many fish hedons the fish meal would cause, followed at some distance by the moral question of choice-worthiness of each fish hedon according to T1 (how much does that theory care about fish?) and the empirical question of how much human hedons the fish meal would cause (how much would Devon enjoy the meal?).

And is there any literature or information on how value of information analysis can account for things like unknown unknowns?
Very good question. I’m not sure, but I’ll try to think about and look into that.

MichaelA 31 Dec 2019 0:48 UTC
2 points
in reply to: Pattern’s comment on: Making decisions under moral uncertainty
“or even whether I should spend much time making that decision, because that might trade off to some extent with time and money I could put towards longtermist efforts (which seem more choice-worthy according to other moral theories I have some credence in).”
What longtermist efforts might there be according to the theory that (if you were certain of) you’d choose to be vegan?
Not sure I understand this question. I’ll basically expand on/explain what I said, and hope that answers your question somewhere. (Disclaimer: This is fairly unpolished, and I’m trying more to provide you an accurate model of my current personal thinking than provide you something that appears wise and defensible.)
I currently have fairly high credence in the longtermism, which I’d roughly phrase as the view that, even in expectation and given difficulties with distant predictions, most of the morally relevant consequences of our actions lie in the far future (meaning something like “anywhere from 100 years from now till the heat death of the universe”). In addition to that fairly high credence, it seems to me intuitively that the “stakes are much higher” on longtermism than on non-longtermist theories (e.g., a person-affecting view that only cares about people alive today). (I’m not sure if I can really formally make sense of that intuition, because maybe those theories should be seen as incomparable and I should use variance voting to give them equal say, but at least for now that’s my tentative impression.)
I also have probably somewhere between 10-90% credence that at least many nonhuman animals are morally conscious in a relevant sense, with non-negligible moral weights. And this theory again seems to suggest much higher stakes than a human-only view would (there are a bunch of reasons one might object to this, and they do lower my confidence, but it still seems like it makes more sense to say that animal-inclusive views say there’s all the potential value/disvalue the human-only view said there was, plus a whole bunch more). I haven’t bothered pinning down my credence much here, because multiplying 0.1 by the amount of suffering caused by an individual’s contributions to factory farming if that view is correct seems already enough to justify vegetarianism, making my precise credence less decision-relevant. (I may use a similar example in my post on value of information, now I think about it.)
As MacAskill notes in that paper, while vegetarianism is typically cheaper than meat-eating, strict vegetarianism will be almost certainly at least slightly more expensive (or inconvenient/time-consuming) that “vegetarian except when it’s expensive/difficult”. A similar thing would likely apply more strongly to veganism. So the more I more towards strict veganism, the more time and money it costs me. It’s not much, and a very fair argument could be made that I probably spend more on other things, like concert tickets or writing these comments. But it still does trade off somewhat against my longtermist efforts (currently centring on donating to the EA Long Term Future Fund and gobbling up knowledge and skills so I can do useful direct work, but I’ll also be starting a relevant job soon).
To me, it seems that the stakes under “longtermism plus animals matter” seem higher than just under “animals matter”. Additionally, I have fairly high credence in longtermism, and no reason to believing conditioning on animals mattering makes longtermism less likely (so even if I accept “animals matter”, I’d have basically exactly the same fairly high credence in longtermism as before).
It therefore seems that a heuristic MEC type of thinking should make me lean towards what longtermism says I should do, though with “side constraints” or “low hanging fruits being plucked” from a “animals matter” perspective. This seems extra robust because, even if “animals matter”, I still expect longtermism is fairly likely, and then a lot of the policies that seem wise from a longtermist angle (getting us to existential safety, expanding our moral circle, raising our wisdom and ability to prevent suffering and increase joy) seem fairly wise from an animals matter angle too (because they’d help us help nonhumans later). (But I haven’t really tried to spell that last assumption out to check it makes sense.)

MichaelA 31 Dec 2019 0:54 UTC
2 points
in reply to: Pattern’s comment on: Making decisions under moral uncertainty
(Link from earlier in the post: Will MacAskill’s 2014 thesis. Alas, its table of contents doesn’t entirely use the same terminology, so its not clear where Hybrid procedures are discussed. However, if they are discussed after Variance Voting, as in the post, then this may be somewhere after page 89.)
Oh, good point, should’ve provided page numbers! Hybrid procedures (or Hybrid Views) are primarily discussed in pages 117-122. I’ve now edited my post to include those page numbers. (You can also use the command+f/control+f to find key terms in the thesis you’re after.)

MichaelA 31 Dec 2019 3:56 UTC
4 points
in reply to: Said Achmiz’s comment on: Making decisions under moral uncertainty
Ok, this has increased the likelihood I’ll commit the time to writing that other post. I think it’ll address some of the sorts of questions you list, but not all of them.
One reason is that I’m not a proper expert on this.
Another reason is that I think that, very roughly speaking, answers to a lot of questions like that would be “Basically import what we already know about regular/factual/empirical uncertainty.” For moral realists, the basis for the analogy seems clear. For moral antirealists, one can roughly imagine dealing with moral uncertainty as something like trying to work out the fact of the matter about one’s own preferences, or one’s idealised preferences (something like CEV). But that other post I’ll likely write should flesh this out a bit more.

MichaelA 31 Dec 2019 6:03 UTC
2 points
in reply to: Pattern’s comment on: Making decisions under moral uncertainty
What I am familiar with of Kant’s reasoning was a bit consequentialist, and if “this leads to a bad circumstance under some circumstance → never do it even under circumstances when doing it leads to bad consequences” (which means the analysis could come to a different conclusion if it was done in a different order or reversed the action/inaction related bias) is dropped in favor of “here are the reference classes, use the policy with the highest expected utility given this fixed relationship between preference classes and policies” then it can be made into one that might meet the axioms.
I think that’s one way one could try to adapt Kantian theories, or extrapolate certain key principles from them. But I don’t think it’s what the theories themselves say. I think what you’re describing lines up very well with rule utilitarianism.
(Side note: Personally, “my favourite theory” would probably be something like two-level utilitarianism, which blends both rule and act utilitarianism, and then based on moral uncertainty I’d add some side constraints/concessions to deontological and virtue ethical theories—plus just a preference for not doing anything too drastic/irreversible in case the “correct” theory is one I haven’t heard of yet/no one’s thought of yet.)

MichaelA 31 Dec 2019 22:22 UTC
3 points
on: Making decisions under moral uncertainty
As I hope for this to remain a useful, accessible summary of these ideas, I’ve made various edits as I’ve learned more and gotten feedback, and expect to continue to do so. So please keep the feedback coming, so I can make this more useful for people!
(In the interest of over-the-top transparency, here’s the version from when this was first published.)

MichaelA 1 Jan 2020 13:13 UTC
2 points
in reply to: MichaelA’s comment on: Making decisions under moral uncertainty
I’ve now substantially updated/overhauled this article, partly in response to your feedback. One big thing was reading more about variance voting/normalisation and related ideas, and, based on that, substantially changing how I explain that idea and adding a (somewhat low confidence) worked example. Hope that helps make that section clearer.
If there are things that still seem unclear, and especially if anyone thinks I’ve made mistakes in the Variance Voting part, please let me know.

MichaelA 3 Jan 2020 0:03 UTC
5 points
in reply to: Kaj_Sotala’s comment on: Making decisions under moral uncertainty
I’m now definitely planning to write the above-mentioned post, discussing various definitions, types, and sources of moral uncertainty. As this will require thinking more deeply about that topic, I’ll be able to answer more properly once I have. (And I’ll also comment a link to post that in this thread.)

For now, some thoughts in response to your comment, which are not fully-formed and not meant to be convincing; just meant to indicate my current thinking and maybe help me formulate that other post through discussion:
- Well, I did say “part of me”… :)
- I think the last two sentences of your comment raise an interesting point worth taking seriously
- I do think there are meaningfully different ways we can think about what moral uncertainty is, and that a categorisation/analysis of the different types and “sources” (i.e., why is one morally uncertain) could advance one’s thinking (that’s what that other post I’ll write will aim to do)
- I think the main way it would help advance one’s thinking is by giving clues as to what one should do to resolve one’s uncertainty.
  - This seems to me to be what your post focuses on, for each of the three types you mention. E.g., to paraphrase (to relate this more to humans than AI, though both applications seem important), you seem to suggest that if the source of the uncertainty is our limited self-knowledge, we should engage in processes along the lines of introspection. In contrast, if the source of our uncertainty is that we don’t know what we’d enjoy/value because we haven’t tried it, we should engage with the world to learn more. This all seems right to me.
- But what I cover in this post and the following one is how to make decisions when one is morally uncertain. I.e., imagining that you are stuck with uncertainty, what do you do? This is a different question to “how do I get rid of this uncertainty and find the best answer” (resolving it).
  - (Though in reality the best move under uncertainty may actually often be to gather more info—which I’ll discuss somewhat in my upcoming post on the applying value of information analysis to this topic—in which case the matter of how to resolve the uncertainty becomes relevant again.)
- I’d currently guess (though I’m definitely open to being convinced otherwise) that the different types and sources of moral uncertainty don’t have substantial bearing on how to make decisions under moral uncertainty. This is for three main reasons:
  - Analogy to empirical uncertainty: There are a huge number of different reasons I might be empirically uncertain—e.g., I might not have enough data on a known issue, I might have bad data on the issue, I might have the wrong model of a situation, I might not be aware of a relevant concept, I might have all the right data and model but limited processing/computation ability/effort. And this is certainly relevant to the matter of how to resolve uncertainty. But as far as I’m aware, expected value reasoning/expected utility theory is seen as the “rational” response in any case of empirical uncertainty. (Possibly excluding edge cases like Pascal’s wagers, which in any case seem to be issues related to the size of the probability rather than to the type of uncertainty.) It seems that, likewise, the “right” approach to making decisions under moral uncertainty may apply regardless of the type/source of that uncertainty (especially because MEC was developed by conscious analogy to approaches for handling empirical uncertainty).
  - The fact that most academic sources I’ve seen about moral uncertainty seem to just very briefly discuss what moral uncertainty is, largely through analogy to empirical uncertainty and an example, and then launch into how to make decisions when morally uncertain. (Which is probably part of why I did it that way too.) It’s certainly possible that there are other sources I haven’t seen which discuss how different types/sources might suggest different approaches would be best. It’s also certainly possible the academics all just haven’t thought of this issue, or haven’t taken it seriously enough. But to me this is at least evidence that the matter of types/sources of moral uncertainty shouldn’t affect how one makes decisions under moral uncertainty.
    (That said, I’m a little surprised I haven’t yet seen academic sources analysing different types/sources of moral uncertainty in order to discuss the best approaches for resolving it. Maybe they feel that’s effectively covered by more regular moral philosophy work. Or maybe it’s the sort of thing that’s better as a blog post than an academic article.)
  - I can’t presently see a reason why one should have a different decision-making procedure/aggregation procedure/approach under moral uncertainty depending on the type/source of uncertainty. (This is partly double-counting the points about empirical uncertainty and academic sources, but here I’m also indicating I’ve tried to think this through myself at least a bit.)
What links here?
- MichaelA's comment on Moral uncertainty vs related concepts by MichaelA (12 Jan 2020 2:29 UTC; 5 points)

MichaelA 3 Jan 2020 6:59 UTC
1 point
in reply to: MichaelA’s comment on: Making decisions under moral uncertainty
(Also, should’ve mentioned more explicitly—I’d be interested in hearing people’s thoughts on that “current thinking” of mine, to inform the “What is moral uncertainty?” post I’m working on.)

MichaelA 3 Jan 2020 9:22 UTC
2 points
in reply to: jmh’s comment on: What to do when both morally and empirically uncertain
Good question. I’m working on a post on what moral uncertainty actually is (which I’ll hopefully publish tomorrow), and your question has made me realise it’d be worth having a section on the matter of “risk vs (Knightian) uncertainty” in there.
Personally, I was effectively using “uncertainty” in a sense that incorporates both of the concepts you refer to, though I hadn’t been thinking explicitly about the matter. (And some googling suggests academic work on moral uncertainty almost never mentions the potential distinction. When the term “moral risk” is used, it just means the subset of moral uncertainty where one may be doing something bad, rather than being set against something like Knightian uncertainty.)
This distinction is something I feel somewhat confused about, and it seems to me that confusion/disagreement about the distinction is fairly common more widely, including among LW/EA-related communities. E.g., the Wikipedia article on Knightian uncertainty says: “The difference between predictable variation and unpredictable variation is one of the fundamental issues in the philosophy of probability, and different probability interpretations treat predictable and unpredictable variation differently. The debate about the distinction has a long history.”
But here are some of my current (up for debate) thoughts, which aren’t meant to be authoritative or convincing. (This turned out to be a long comment, because I used writing it as a chance to try to work through these ideas, and because my thinking on this still isn’t neatly crystallised, and because I’d be interested in people’s thoughts, as that may inform how I discuss this matter in that other post I’m working on.)
Terms
My new favourite source on how the terms “risk” and “uncertainty” should probably (in my view) be used is this, from Ozzie Gooen. Ozzie notes the distinction/definitions you mention being common in business and finance, but then writes:
Disagreeing with these definitions are common dictionaries and large parts of science and mathematics. In the Merriam-Webster dictionary, every definition of ‘risk’ is explicitly about possible negative events, not about general things with probability distributions. (https://www.merriam-webster.com/dictionary/risk)
There is even a science explicitly called “uncertainty quantification”, but none explicitly called “risk quantification”.
This is obviously something of a mess. Some business people get confused with mathematical quantifications of uncertainty, but other people would be confused by quantifications of socially positive “risks”.
Ozzie goes on to say:
Douglas Hubbard came up with his own definitions of uncertainty and risk, which are what inspired a very similar set of definitions on Wikipedia (the discussion page specifically mentions this link).
From Wikipedia:
Uncertainty
The lack of certainty. A state of having limited knowledge where it is impossible to exactly describe the existing state, a future outcome, or more than one possible outcome.
Measurement of uncertainty
A set of possible states or outcomes where probabilities are assigned to each possible state or outcome — this also includes the application of a probability density function to continuous variables.
Risk
A state of uncertainty where some possible outcomes have an undesired effect or significant loss.
Measurement of risk
A set of measured uncertainties where some possible outcomes are losses, and the magnitudes of those losses — this also includes loss functions over continuous variables.
So according to these definitions, risk is essentially a strict subset of uncertainty.
(Douglas Hubbard is the author of How to Measure Anything, which I’d highly recommend for general concepts and ways of thinking useful across many areas. Great summary here.)
That’s how I’d also want to use those terms, to minimise confusion (despite still causing it among some people from e.g. business and finance).
Concepts
But that was just a matter of what words are probably least likely to lead to conclusion. A separate point is whether I was talking about what you mean by “risk” or what you mean by “uncertainty” (i.e., “Knightianuncertainty”). I do think a distinction between these concepts “carves reality at the joints” to some extent—it points in the direction of something real and worth paying attention to—but I also (currently) think it’s too binary, and can be misleading. (Again, all of this comment up for debate, and I’d be interested in people checking my thinking!)
Here’s one description of the distinction (from this article):
In the case of risk, the outcome is unknown, but the probability distribution governing that outcome is known. Uncertainty, on the other hand, is characterised by both an unknown outcome and an unknown probability distribution. For risk, these chances are taken to be objective, whereas for uncertainty, they are subjective. Consider betting with a friend by rolling a die. If one rolls at least a four, one wins 30 Euros (or Pounds, Dollars, Yen, Republic Dataries, Bitcoins, etc.). If one rolls lower, one loses. If the die is unbiased, one’s decision to accept the bet is taken with the knowledge that one has a 50 per cent chance of winning and losing. This situation is characterised by risk. However, if the die has an unknown bias, the situation is characterised by uncertainty. The latter applies to all situations in which one knows that there is a chance of winning and losing but has no information on the exact distribution of these chances.”
This sort of example is common, and it seems to me that it’d make more sense to not see a difference in kind between the type of estimate/probability/uncertainty, but a difference of degree of our confidence in/basis for that estimate/probability. And this seems to be a fairly common view, at least among people using a Bayesian framework. E.g., Wikipedia states:
Bayesian approaches to probability treat it as a degree of belief and thus they do not draw a distinction between risk and a wider concept of uncertainty: they deny the existence of Knightian uncertainty. They would model uncertain probabilities with hierarchical models, i.e. where the uncertain probabilities are modelled as distributions whose parameters are themselves drawn from a higher-level distribution (hyperpriors)
And in the article on Knightian uncertainty, Wikipedia says “Taleb asserts that Knightian risk does not exist in the real world, and instead finds gradations of computable risk”. I haven’t read anything by Taleb and can’t confirm that that’s his view, but that view makes a lot of sense to me.
For example, in the typical examples, we can never really know with certainty that a coin/dice/whatever is unbiased. So it can’t be the case that “the probability distribution governing that outcome is known”, except in the sense of being known probabilistically, in which case it’s not a totally different situation to a case of an “unknown” distribution, where we still may have some scrap of evidence or meaningful prior, and even if not, we can use some form of uninformative prior (e.g., uniform prior; it seems to me that uninformative priors are another debated topic, and my knowledge there is quite limited).
Something that seems relevant from here is a great quote from Tetlock, along the lines of (I can’t remember or find his exact phrasing) lots of people claiming you can’t predict x, you can’t predict y, these things are simply too unforeseeable, and Tetlock deciding to make a reference class for “Things people say you can’t predict”, with this leading to fairly good predictions. (I think this was from Superforecasting, but it might’ve been Expert Political Judgement. If anyone can remember/find the quote, please let me know.)
Usefulness of a distinction
But as I said, I do think the distinction still points in the direction of something important (I’d just want it to do so in terms of quantitative degrees, rather than qualitative kinds). Holden of GiveWell (at the time) had some relevant posts that seem quite interesting and generated a lot of discussion, e.g. this and this.
One reason I think it’s useful to consider our confidence in our estimates is the optimizer’s curse (also discussed here). If we’re choosing from a set of things based on which one we predict will be “best” (e.g., the most cost-effectiveness of a set of charities), we should expect to be disappointed, and this issue is more pronounced the more “uncertain” our estimates are (not as in “probabilities closer to 0”, but as in “Probabilities we have less basis for”). (I’m not sure how directly relevant the optimizer’s curse is for moral uncertainty, though. Maybe something more general like regression to the mean is more relevant.)
But I don’t think issues like that mean different types/levels of uncertainty need to be treated in fundamentally different ways. We can just do things like adjusting our initial estimates downwards based on our knowledge of the optimizer’s curse. Or, relatedly, model combination and adjustment, which I’ve had a go at for the Devon example in this Guesstimate model.
Concepts I find very helpful here are credence resilience and (relatedly) a distinction between the balance of the evidence and the weight of the evidence (explained here). These (I think) allow us to discuss the sort of thing “risk vs Knightian uncertainty” is trying to get at, but in a less binary way, and in a way that more clearly highlights how we should respond and other implications (e.g., that value of information will typically be higher when our credences are less “resilient”, which I’ll discuss a bit in my upcoming post on relating value of information to moral uncertainty).

Hope that wall of text (sorry!) helps clarify my current thinking on these issues. And as I said earlier, I’d be interested in people’s thoughts on what I’ve said here, to inform how I (hopefully far more concisely) discuss this matter in my later post.

MichaelA 3 Jan 2020 10:12 UTC
1 point
in reply to: MichaelA’s comment on: What to do when both morally and empirically uncertain
To be specific regarding the model combination point (which is probably the way in which the risk vs Knightian uncertainty distinction is most likely to be relevant to moral uncertainty): I now very tentatively think it might make sense to see whatever “approach” we use to moral uncertainty (whether accounting for empirical uncertainty or not) as one “model”, and have some other model(s) as well. These other models could be an alternative approach to moral uncertainty, or a sort of “common sense” moral theory (of a form that can’t be readily represented in the e.g. MEC model, or that Devon thinks shouldn’t be represented in that model because he’s uncertain about that model), or of something like “Don’t do anything too irreversible because there might be some better moral theory out there that we haven’t heard of yet”.
Then we could combine the two (or more) models, with the option of weighting them differently if we have different levels of confidence in each one. (E.g., because we’re suspicious of the explicit MEC approach. Or on the other hand because we might think our basic, skipping-to-the-conclusion intuitions can’t be trusted much at all on matters like this, and that a more regimented approach that disaggregates/factors things out is more reliable). This is what I had a go at showing in this model.
For example, I think I’d personally find it quite plausible to take the results of (possibly normalised) MEC/MEC-E quite seriously, but to still think there’s a substantial chance of unknown unknowns, making me want to combine those results with something like a “Try to avoid extremely counter-common-sense actions or extremely irreversible consequences” model.
But I only thought of this when trying to think through how to respond to your comment (and thus re-reading things like the model combination post and the sequence vs cluster thinking post), and I haven’t seen something like it discussed in the academic work on moral uncertainty I’ve read, so all of the above is just putting an idea out there for discussion.

MichaelA 3 Jan 2020 23:34 UTC
1 point
in reply to: jmh’s comment on: What to do when both morally and empirically uncertain
With regards to rules, I think there is also something of an uncertainty reducing role in that rules will increase predictability of external actions. This is not a well thought out idea for me but seems correct, at least in a number of possible situations.
Are you talking in some like game-theoretic terms, like how pre-committing to one policy can make it easier for others to plan with that in mind, for cooperation to be achieved, to avoid extortion, etc.?
If so, that seems plausible and interesting (I hadn’t been explicitly thinking about that).
But I’d also guess that that benefit could be captured by the typical moral uncertainty approaches (whether they explicitly account for empirical uncertainty or not), as long as some theories you have credence in are at least partly consequentialist. (I.e., you may not need to use model combination and adjustment to capture this benefit, though I see no reason why it’s incompatible with model combination and adjustment either.)
Specifically, I’m thinking that the theories you have credence in could give higher choice-worthiness scores to actions that stick closer to what’s typical or you or of some broader group (e.g., all humans), or that stick closer to a policy you selected with a prior action, because of the benefits to cooperation/planning/etc. (In the “accounting for empirical uncertainty”, tweak that to the theories valuing those benefits as outcomes, and you predicting those actions will likely lead to those outcomes.)
Or you could set things up so there’s another “action” to choose from which represents “Select X policy, and sticking to it for the next few years except in Y subset of cases, in which situations you can run MEC again to see what to do”. Then it’d be something like evaluating whether to pre-commit to a form of rule utilitarianism in most cases.
But again, this is just me spit-balling—I haven’t seen this sort of thing discussed in the academic literature.

MichaelA 4 Jan 2020 22:25 UTC
1 point
in reply to: Kaj_Sotala’s comment on: Making decisions under moral uncertainty
Yes, I think thinking through that for that comment clarified things for myself as well! Once I’m further through this series, I’ll edit the first posts, and I’ve made a note to mention something like that in the first two posts.