Vaniver
Karma: 26,892 (LW), 7 (AF)
NewTop
I read this as saying something like “This paper only makes sense if facts matter, separate to values.” It’s funny to me that this sentence felt necessary to be written.
I mean, it’s more something like “there’s a shared way in which facts matter,” right? If I mostly think in terms of material consumption by individuals, and you mostly think in terms of human dignity and relationships, the way in which facts matter for both of us is only tenuously related.
I think we’re using margins differently. Yes, you shouldn’t expect situations with x>1 to be durable, but you should expect x>1 before every charitable donation that you make. Otherwise you wouldn’t make the donation! And so x=1 is the ‘money in the bank’ valuation, instead of the upper bound.
Wait, are you claiming that humans have moral intuitions because it maximizes global utility? Surely moral intuitions have been produced by evolution.
No, I’m claiming that moral intuitions reflect the precomputation of higher-order strategic considerations (of the sort “if I let this person get away with stealing a bike, then I will be globally worse off even though I seem locally better off”).
I agree that you should expect evolution to create agents that maximize inclusive genetic fitness, which is quite different from global utility. But even if one adopts the frame that ‘utilitarian calculus is the standard of correctness,’ one can still use those moral intuitions as valuable cognitive guides, by directing attention towards considerations that might otherwise be missed.
On first-order effects, it seems that your preference rankings as are follows:
1) You have the widget, the commune has $80, your total satisfaction is $30+80x.
2a) You have nothing, the commune has $100, your total satisfaction is $100x.
2b) You have $100, the commune has nothing, your total satisfaction is $100.
3) You have the widget, a monopoly you don’t value has $80. Your total satisfaction is $30+80y.
By changing x and y, we represent your altruism to the other parties in the situation; if x is greater than 1, then you would rather give the commune money than have it yourself, but if x is above 1.5 then you’d rather just give the money to the commune than have a widget for yourself. For ys below 7/8ths, you’d rather not buy the widget. (The x and y I inferred from the question are slightly above 1 and slightly above 0, which suggests the best option is indeed 1.)
---
Why do humans have moral intuitions at all? I claim a major role is to represent higher order effects as shorthand. When you see a bike you don’t own, you might run the first order calculations and think it’s worth more to you than it is to whoever owns it, and so global utility is maximized by you stealing the bike. But a world in which agents reflexively don’t steal bikes has other benefits to it, such that the low-theft equilibrium might have higher global utility than the high-theft equilibrium. But you can’t get from the high-theft equilibrium to the low-theft equilibrium by making small pareto improvements.
And so if you notice you have moral intuitions that rise up whenever you run the numbers and decide you shouldn’t be upset that someone stole your bike, try to figure out what effects those intuitions are trying to have.
---
Why put economic transactions in a separate domain from charitable donations? There are a few related things to disentangle.
First, for you personally, it really doesn’t matter much. If you would rather pay your favorite charity $100 for a t-shirt with their logo on it, even though you normally wouldn’t pay $100 for a t-shirt, even though you could just give them the $100, then do it.
Second, for society as a whole, prices are a information-transmission mechanism, conveying how much caring something requires to produce, and how much people care about it being produced. Mucking with this mechanism to divert value flows generally destroys more than it creates, especially since the prices can freely fluctuate in response to changing conditions, whereas policies are stickier.
Possibly dowries have to be in cash and you don’t have liquidity.
Land dowries were common.
Dowries and inheritance are best thought of as the same thing, happening at different times; your sons have to wait until you die to come into full possession of your/their lands, but your daughters are ‘dead to you’ as soon as they get married. So the primary difference between sons and daughters is dynastic prestige; a son both maintains the wealth within the dynasty and accrues whatever dowry he can attract, whereas a daughter leaks wealth to another dynasty. (Indeed, when the mismatch was sufficiently large the man was typically forced to take his wife’s surname as a condition of being allowed to marry her, a sort of honorary swapping of the sexes.)
Interestingly, one of the things that happens here is that dowries are much more variable than male inheritance (which either gives almost all to the eldest son, or splits it almost equally, with deliberate splits being more rare); you can just send an unattractive daughter to a convent (tho this also typically required a dowry!) while giving your more promising daughters a larger share.
C. The proponents of the original arguments were misinterpreted, or overemphasised some of their beliefs at the expense of others, and actually these shifts are just a change in emphasis.
My interpretation of what happened here is that more narrow AI successes made it more convincing that one could reach ASI by building all of the components of it directly, rather than necessitating building an AI that can do most of the hard work for you. If it only takes 5 cognitive modules to take over the world instead of 500, then one no longer needs to posit an extra mechanism by which a buildable system is able to reach the ability to take over the world. And so from my perspective it’s mostly a shift in emphasis, with small amounts of A and B as well.
I’m objecting to the further implication that doing this makes it not a Bayes net.
I mean, white horses are not horses, right? [Example non-troll interpretations of that are “the set ‘horses’ only contains horses, not sets” and “the two sets ‘white horses’ and ‘horses’ are distinct.” An example interpretation that is false is “for all members X of the set ‘white horses’, X is not a member of the set ‘horses’.“]
To be clear, I don’t think it’s all that important to use influence diagrams instead of causal diagrams for decision problems, but I do think it’s useful to have distinct and precise concepts (such that if it even becomes important to separate the two, we can).
What is it that you want out of them being Bayes nets?
All the nodes in the network should be thought of as grounding out in imagination, in that it’s a world-model, not a world. Maybe I’m not seeing your point.
My point is that my world model contains both ‘unimaginative things’ and ‘things like world models’, and it makes sense to separate those nodes (because the latter are typically functions of the former). Agreed that all of it is ‘in my head’, but it’s important that the ‘in my head’ realm contain the ‘in X’s head’ toolkit.
I guess, philosophically, I worry that giving the nodes special types like that pushes people toward thinking about agents as not-embedded-in-the-world, thinking things like “we need to extend Bayes nets to represent actions and utilities, because those are not normal variable nodes”. Not that memoryless cartesian environments are any better in that respect.
I see where this is coming from, but I think it might also go the opposite direction. For example, my current guess of how counterfactuals/counterlogicals ground out is the imagination process; I implicitly or explicitly think of different actions I could take (or different ways math could be), and somehow select from those actions (hypotheses / theories); the ‘magic’ is all happening in my imagination instead of ‘in the world’ (noting that, of course, my imagination is being physically instantiated). Less imaginative reactive processes (like thermostats ‘deciding’ whether to turn on the heater or not) don’t get this treatment, and are better considered as ‘just part of the environment’, and if we build an imaginative process out of unimaginative processes (certainly neurons are more like thermostats than they are like minds) then it’s clear the ‘magic’ comes from the arrangement of them rather than the individual units.
Which suggests how the type distinction might be natural; places where I see decision nodes are ones where I expect to think about what action to take next (or expect some other process to think about what action to take next), or think that it’s necessary to think about how that thinking will go.
Aside: Bayes nets which are representing decision problems are usually called influence diagrams rather than Bayes nets. I think this convention is silly; why do we need a special term for that?
In influence diagrams, nodes have a type—uncertainty, decision, or objective. This gives you legibility, and makes it more obvious what sort of interventions are ‘in the spirit of the problem’ or ‘necessary to give a full solution.’ (It’s not obvious from the structure of the causal network that I should set ‘my action’ instead of ‘Omega’s prediction’ in Newcomb’s Problem; I need to read it off the labels. In an influence diagram, it’s obvious from the shape of the node.) This is a fairly small benefit, tho, and seems much less useful than the restriction on causal networks that the arrows imply causation.
[Edit] They also make it clearer how to do factorized decision-making with different states of local knowledge, especially when knowledge is downstream of earlier decisions you made; if you’re trying to reason about how a simple agent should deal with a simple situation, this isn’t that helpful, but if you’re trying to reason about many different corporate policies simultaneously, then something influence-diagram shaped might be better.
Suppose there was some doubt about whether it was genuinely conscious. Wouldn’t that amount to the question of whether or not it was a zombie?
No. There are a few places this doubt could be localized, but it won’t be in ‘whether or not zombies are possible.’ By definition we can’t get physical evidence about whether or not it’s a zombie (a zombie is in all physical respects similar to a non-zombie, except non-zombies beam their experience to a universe causally downstream of us, where it becomes “what it is like to be a non-zombie,” and zombies don’t), in exactly the same way we can’t get physical evidence about whether or not we’re zombies. In trying to differentiate between different physical outcomes, only physicalist theories are useful.
The doubt will likely be localized in ‘what it means to be conscious’ or ‘how to measure whether or not something is conscious’ or ‘how to manufacture consciousness’, where one hopes that answers to one question inform the others.
Perhaps instead the doubt is localized in ‘what decisions are motivated by facts about consciousness.’ If there is ‘something it’s like to be Alexa,’ what does that mean about the behavior of Amazon or its customers? In a similar way, it seems highly likely that the inner lives of non-human animals parallel ours in specific ways (and don’t in others), and even if we agree exactly on what their inner lives are like we might disagree on what that implies about how humans should treat them.
This is mostly just arguing over semantics.
If an argument is about semantics, this is not a good response. That is...
Just replace “philosophical zombie” with whatever your preferred term is for
An important part of normal human conversations is error correction. Suppose I say “three, as an even number, …“; the typical thing to do is to silently think “probably he meant odd instead of even; I will simply edit my memory of the sentence accordingly and continue to listen.” But in technical contexts, this is often a mistake; if I write a proof that hinges on the evenness of three, that proof is wrong, and it’s worth flagging the discrepancy and raising it.
Technical contexts also benefit from specificity of language. If I have a term used to refer to the belief that “three is even,” using that term to also refer to the belief that “three is odd” will be the source of no end of confusion. (“Threevenism is false!” “What do you mean? Of course Threevenism is true.“) So if there is a technical concept that specifically refers to X, using it to refer to Y will lead to the same sort of confusion; use a different word!
That is, on the object level: it is not at all sensible to think that philosophical zombies are useful as a concept; the idea is deeply confused. Separately, it seems highly possible that people vary in their internal experience, such that some people experience ‘qualia’ and other people don’t. If the main reason we think people have qualia is that they say that they do, and Dennett says that he doesn’t, then the standard argument doesn’t go through for him. Whether that difference will end up being deep and meaningful or merely cosmetic seems unclear, and more likely discerned through psychological study of multiple humans, in much the same way that the question of mental imagery was best attacked by a survey.
This variability suggests it’s likely a questionable thing to use as a foundation for other theories. For example, it seems to me like it would be unfortunate if someone thought it was fine to torture some humans and not others, because “only the qualia of being tortured is bad,” because it seems to me like torturing humans is likely bad for different reasons.
Subcommunities of AI researchers. A simple concrete example of gains from trade is when everyone uses the same library or conceptual methodology, and someone finds a bug. The primary ones of interest are algorithmic gains; the new thing used to do better lipreading can also be used by other researchers to do better on other tasks (or to further enhance this approach and push it further or lipreading).
I am amused that the footnotes are as long as the actual post.
Footnote 3 includes a rather salient point:
However, if you instead think that something like the typical amount of computing power available to talented researchers is what’s most important — or if you simply think that looking at the amount of computing power available to various groups can’t tell us much at all — then the OpenAI data seems to imply relatively little about future progress.
Especially in the light of this news item from Import AI #126:
The paper obtained state-of-the-art scores on lipreading, significantly exceeding prior SOTAs. It achieved this via a lot of large-scale infrastructure, combined with some elegant algorithmic tricks. But ultimately it was rejected from ICLR, with a comment from a meta-reviewer saying ‘Excellent engineering work, but it’s hard to see how others can build on it’, among other things.
It’s possible that we will see more divergence between ‘big compute’ and ‘small compute’ worlds in a way that one might expect will slow down progress (because the two worlds aren’t getting the same gains from trade that they used to).
In other words, if a philosophical zombie existed, there would likely be evidence that it was a philosophical zombie, such as it not talking about qualia. However, there are individuals who outright deny the existence of qualia, such as Daniel Dennett. Is it not impossible that individuals like Dennett are themselves philosophical zombies?
Nope, your “in other words” summary is incorrect. A philosophical zombie is not any entity without consciousness; it is an entity without consciousness that falsely perceives itself as having consciousness. An entity that perceives itself as not having consciousness (or not having qualia or whatever) is a different thing entirely.
I definitely don’t mean to imply that this is true. I personally don’t think that it is.
Your perception of them stays similar when you flip the signs? (“I don’t like watching TV, I only read novels” becomes “yep, that person is probably mistaken about what they want/like.“)
When it comes to the ‘ideas’ vs. ‘compute’ spectrum:
It seems to me like one of the main differences (but probably not the core one?) is whether or not whether or not something works seems predictable. Suppose Alice thinks that it’s hard to come up with something that works, but things that look like they’ll work do with pretty high probability, and suppose Bob thinks it’s easy to see lots of things that might work, but things that might work rarely do; I think Alice is more likely to think we’re ideas-limited (since if we had a textbook from the future, we could just code it up and train it real quick) and Bob is more likely to think we’re compute-limited (since our actual progress is going to look much more like ruling out all of the bad ideas that are in between us and the good ideas, and the more computational experiments we can run, the faster that process can happen).
I tend to be quite close to the end of the ‘ideas’ spectrum, tho the issue is pretty nuanced and mixed.
I think one of the things that’s interesting to me is not how much training time can be optimized, but ‘model size’—what seems important is not whether our RL algorithm can solve a double-pendulum lightning-quick but whether we can put the same basic RL architecture into an octopus’s body and have it figure out how to control the tentacles quickly. If the ‘exponential effort to get linear returns’ story is true, even if we’re currently not making the most of our hardware, gains of 100x in utilization of hardware only turn into 2 higher steps in the return space. I think the primary thing that inclines me towards the ‘ideas will drive progress’ view is that if there’s a method that’s exponential effort to linear returns and another method that’s, say, polynomial effort to linear returns, the second method should blow past the exponential one pretty quickly. (Even something that reduces the base of the exponent would be a big deal for complicated tasks.)
If you go down that route, then I think you start thinking a lot about the efficiency of other things (like how good human Go players are at turning games into knowledge) and what information theory suggests about strategies, and so on. And you also start thinking about how close we are—for a lot of these things, just turning up the resources plowed into existing techniques can work (like beating DotA) and so it’s not clear we need to search for “phase change” strategies first. (Even if you’re interested in, say, something like curing cancer, it’s not clear whether continuing improvements to current NN-based molecular dynamics predictors, causal network discovery tools, and other diagnostic and therapeutic aids will get to the finish line first as opposed to figuring out how to build robot scientists and then putting them to work on curing cancer.)
Elaborating on my comment (on the world where training time is the bottleneck, and engineers help):
To the extent major progress and flashy results are dependent on massive engineering efforts, that this seems like this lowers the portability of advances and makes it more difficult for teams to form coalitions. [Compare to a world where you just have to glue together different conceptual advances, and so you plug one model into another and are basically done.] This also means we should think about how progress happens in other fields with lots of free parameters that are sort of optimized jointly—semiconductor manufacturing is the primary thing that comes to mind, where you have about a dozen different fields of engineering that are all constrained by each other and the joint tradeoffs are sort of nightmarish to behold or manage. [Subfield A would be much better off if we switched from silicon to germanium, but everyone else would scream—but perhaps we’ll need to switch eventually anyway.] The more bloated all of these projects become, the harder it is to do fundamental reimaginings of how these things work (a favorite example of mine here is replacing matmuls in neural networks with bitshifts, also known as “you only wanted the ability to multiply by powers of 2, right?“, which seems like it is ludicrously more efficient and is still pretty trainable, but requires thinking about gradient updates differently, and the more effort you’ve put into optimizing how you pipe gradient updates around, the harder it is to make transitions like that).
This is also possibly quite relevant to safety; if it’s hard to ‘tack on safety’ at the end, then it’s important we start with something safe and then build a mountain of small improvements for it, rather than building the mountain of improvements for something that turns out to be not safe and then starting over.
Such CA is thought to result in diminished status and power for people in the “appropriated” culture.
I’m having a hard time separating this from the ‘offense’ argument that you’re not including. Like, The Simpsons introduces Apu, who is Indian and works at a convenience store. Written by and voice-acted by white Americans, he’s very much “Indian immigrants as seen from the outside” as opposed to “the self-representation of Indian immigrants”; as a character in a comedy show, he’s often a subject of mockery.
But someone being offended by Apu is what expecting this will lead to diminished status and power for Indian immigrants to America feels like from the inside. That makes me suspect that we should feel similarly about individuals taking offense claims of this category of CA, but I’m curious what makes you consider them separately.
It’s a straightforward application of the Copernican principle. Of course, that is not always the best approach.