ricraz
Karma: 188
NewTop
For future reference, after asking around elsewhere I learned that this has been discussed in a few places, and the term used for credences which are harder to shift is “resilient”. See this article, and the papers it links to: https://concepts.effectivealtruism.org/concepts/credal-resilience/
Interesting points! I hadn’t seriously considered the possibility of animals having more moral weight per capita than humans, but I guess it makes some sense, even if it’s implausible. Two points:
1. Are the ranges conditional on each species being moral patients at all? If not, it seems like there’d be enough probability mass on 0 for some of the less complex animals that any reasonable confidence interval should include it.
2. What are your thoughts on pleasure/pain asymmetries? Would your ranges for the moral weight of positive experiences alone be substantially different to the ones above? To me, it makes intuitive sense that animals can feel pain in roughly the same way we do, but the greatest happiness I experience is so wrapped up in my understanding of the overall situation and my expectations for the future that I’m much less confident that they can come anywhere close.
I don’t have much to add on the original question, but I do disagree about your last point:
In fact, any kind of second-order probability must be trivial. We have introspective access to our own beliefs. So given any statement about our beliefs we can say for certain whether or not it’s true. Therefore, any second-order probability will either be equal to 0 or 1.
There is a sense in which, once you say “my credence in X is Y”, then I can’t contradict you. But if I pointed out that actually, you’re behaving as if it is Y/2, and some other statements you made implied that it is Y/2, and then you realise that when you said the original statement, you were feeling social pressure to say a high credence even though it didn’t quite feel right—well, that all looks a lot like you being wrong about your actual credence in X. This may end up being a dispute over the definition of belief, but I do prefer to avoid defining things in ways where people must be certain about them, because people can be wrong in so many ways.
That makes sense to me, and what I’d do in practice too, but it still feels odd that there’s no theoretical solution to this question.
It seems like you’re describing a Bayesian probability distribution over a frequentist probability estimate of the “real” probability. Agreed that this works in cases which make sense under frequentism, but in cases like “Trump gets reelected” you need some sort of distribution over a Bayesian credence, and I don’t see any natural way to generalise to that.
Suppose I estimate the probability for event X at 50%. It’s possible that this is just my prior and if you give me any amount of evidence, I’ll update dramatically. Or it’s possible that this number is the result of a huge amount of investigation and very strong reasoning, such that even if you give me a bunch more evidence, I’ll barely shift the probability at all. In what way can I quantify the difference between these two things?
One possible way: add a range around it, such that you’re 90% confident your credence won’t move out of this range in the next D days. Problem: this depends heavily on whether you think you’ll be out looking for new evidence, or whether you’ll be locked in your basement during that time. But if we abstract to, say, “A normal period of D days”, it does provides some sense of how confident you are in your credence.
Another possible way: give numbers for how much your credence would shift, given B bits of information. Obviously 1 bit would shift your credence to 0 or 1. But how much would 0.5 bits move it? Etc. I actually don’t think this works, but wanted to get a second opinion.
A third proposal: specify the function that you’ll shift to if a randomly chosen domain expert told you that yours was a certain amount too high/low. This seems kinda arbitrary too though.
Firstly, excellent post! A cool idea, well-written, and very thought-provoking. Some thoughts on the robustness of the result:
Suppose that every individual were able to produce at least 1 unit of resources throughout their life. Then total utility is monotonically increasing in the number of people, and you have the repugnant conclusion again. How likely is this supposition? Assuming we have arbitrarily advanced technology, including AI, humans will be pretty irrelevant to the production of resources like food or compute (if we’re in simulation). But plausibly humans could still produce “goods” which are valuable to other humans, like friendship. Let’s plug this into your model above and see what happens. I’ll assume that humans need at least 1/K physical resources to survive, but otherwise their utility is logarithmic in the amount of physical resources + friendship that they get. Also, assume that every person receives as much friendship as they produce. So
, with an upper bound of N = KR. When F >= 1, then the optimal value of N is in fact KR, and so utility per person is
, which can be arbitrarily close to 0 (depending on K). When 0 ⇐ F < 1, then I think you get something like your original result again, but I’m not sure. Empirically, I expect that the best friends can produce F >> 1, i.e. if you had nothing except just enough food/water to keep yourself alive, but also you were the sole focus of their friendship, then you’d consider your life well worth living. Idk about average production, but hopefully that’ll improve in the future too. In summary, friendship may make things repugnant :P
Here’s another version of the repugnant conclusion and your argument. Suppose that the amount of resources used by each person is roughly fixed per unit time (because, say, we’re living in simulation), but that there’s a period of infancy and early childhood which uses up resources and isn’t morally valuable. Then the resources used up by one person are I + L, where I is the length of their infancy and L is the length of the rest of their life, but the utility gained from their life is a function of L alone. What function of L? Perhaps you think that it’s linear in L—for example, If you’re a hedonic utilitarian, it’s plausible that people will be just as happy later in their life as earlier. (In fact, right now, old people tend to be happiest). If so, you must endorse the anti-repugnant conclusion, where you’d prefer a population with very few very long-lived people, to minimise the fixed cost of infancy. If you’re a preference utilitarian, maybe you think that there’s diminishing marginal utility to having your preferences satisfied. It then follows that there’s an optimal point at which to kill people, which isn’t too soon (otherwise you’re incurring high fixed costs) and isn’t too late (otherwise people’s marginal utility diminishes too much) - a conclusion which is analogous to your result.
Strangely enough, TDT gets the wrong answer for Parfit’s Hitchhiker.
This confuses me, could you explain?
In 3, if P is high enough then it can be worth refusing to pay.
A world where a bystander has murdered a specific fat person by pushing him off a bridge to prevent a trolley from hitting five other specific people, and a world where a trolley was speeding toward a specific person, and a bystander has done nothing at all (when he could, at his option, have flipped a switch to make the trolley crush five other specific people instead), are very different worlds. That means that the action in question, and the omission in question, have different consequences.
Technically it is true that there are different consequences, but a) most consequentialists don’t think that the differences are very morally relevant, and b) you can construct examples where these differences are minimised without changing people’s responses very much. For instance, by specifying that you would be given amnesic drugs after the trolley problem, so that there’s no difference in your memories.
The total view (construed in the way that is implied by your comments) is not a plausible theory.
Yet many people seem to find it plausible, including me. Have you written up a justification of your view that you could point me to?
Is it really? Geoff Hinton thinks the future depends on some graduate student who is deeply suspicious of everything he says.
That suspicious grad student will be an outlier compared with all the others, who are still likely to be mistaken.
I think that’s true in the US, but not in most of Europe. E.g. in Switzerland a first year PhD student gets paid $40000 a year WITHOUT doing any teaching, and more if they teach. That’s unusually generous, but I think the setup isn’t uncommon.
Expensive in terms of time, perhaps, but almost all good universities in the US and continental Europe provide decent salaries to PhD students. UK is a bit more haphazard but it’s still very rare for UK PhDs to actually pay to be there, especially in technical fields.
Is epistemic logic useful for agent foundations?
A realistic example which matches this payoff matrix very well is when everyone is using the same means of transport, and so has to wait for the last to arrive before they can leave. A more speculative one: trying to create an environment of group emotional openness and vulnerability, where one person being sarcastic and prickly can ruin it for everyone.
I appreciate this post, and I thought you struck a good balance of explaining a standard concept using examples which made it interesting even if you already knew the concept.
Single neurons cannot represent two distinct kind of quantities, as would be required to do backprop (the presence of features and gradients for training).
I don’t understand why can’t you just have some neurons which represent the former, and some neurons which represent the latter?
The drop-out algorithm (which has been very popular, though it recently seems to have been largely replaced by batch normalisation).
Do you have any particular source for dropout being replaced by batch normalisation, or is it an impression from the papers you’ve been reading?
Thanks for the excellent post, Jacob. I think you might be placing too much emphasis on learning algorithms as opposed to knowledge representations, though. It seems very likely to me that at least one theoretical breakthrough in knowledge representation will be required to make significant progress (for one argument along these lines, see Pearl 2018). Even if it turns out that the brain implements backpropagation, that breakthrough will still be a bottleneck. In biological terms, I’m thinking of the knowledge representations as analogous to innate aspects of cognition impressed upon us by evolution, and learning algorithms as what an individual human uses to learn from their experiences.
Two examples which suggest that the former are more important than the latter. The first is the “poverty of stimulus” argument in linguistics: that children simply don’t hear enough words to infer language from first principles. This suggests that ingrained grammatical instincts are doing most of the work in narrowing down what the sentences they hear mean. Even if we knew that the kids were doing backpropagation whenever they heard new sentences, that doesn’t tell us much about how that grammatical knowledge works, because you can do backpropagation on lots of different things. (You know more psycholinguistics than I do, though, so let me know if I’m misrepresenting anything).
Second example: Hinton argues in this talk that CNNs don’t create representations of three-dimensional objects from two-dimensional pictures in the same way as the human brain does; that’s why he invented capsule networks, which (he claims) do use such representations. Both capsules and CNNs use backpropagation, but the architecture of capsules is meant to be an extra “secret sauce”. Seeing whether they end up working well on vision tasks will be quite interesting, because vision is better-understood and easier than abstract thought (for example, it’s very easy to theoretically specify how to translate between any two visual perspectives, it’s just a matrix multiplication).
Lastly, as a previous commentator pointed out, it’s not backpropagation but rather gradient descent which seems like the important factor. More specifically, recent research suggests that Stochastic Gradient Descent leads to particularly good outcomes, for interesting theoretical reasons (see Zhang 2017 and this blog post by Huzcar). Since the brain does online learning, if it’s doing gradient descent then it’s doing a variant of SGD. I discuss why SGD works well in more detail in the first section of this blog post.
I’d also add to 2: an intuitive explanation of eigenvectors and their properties in general. They seem to pop up everywhere in Linear Algebra and when then do, people gesture towards a set of intuitions about them that I haven’t managed to pick up.
FYI there’s more about “credal resilience” here (although I haven’t read the linked papers yet).