I think Moral Mazes is a misleading meme that itself contributes to the problem and would much prefer splitting off the more specific purposes like Liability Laundering. I don’t recall to what extent the original sequence covers that such mazes are not a bug but a feature for upper management.
romeostevensit
Karma: 5,862
Effort is opaque and subject to goodharting if it becomes the measure of goodness. It generally eventually turns into suffering as currency, which is how hells are constructed.
“Any idiot can build a bridge that stands. It takes an engineer to build a bridge that barely stands.” -Provenance unknown
and I’ll take a crack at it:
If you invert and sense it still makes
a wild goose you soon will chase
off the top of my head:
3 shipping logistics
7-8 material suppliers
1 supplier of physical premises
1 supplier of machinery and support
2 payment services providers
1 banking service
1 loan servicer
3 e-commerce service providers
It seems like the frame of some of the critique is that humans are the authority on human values and want to ensure that the AI doesn’t escape that authority in illegible ways. To me it seems like the frame is more like we know that the sensors we have are only goodhartedly entangled with the things we care about and would ourselves prefer the less goodharted hypothetical sensors if we knew how to construct them. And that we’d want the AI to be inhabiting the same frame as us since, to take a page from Mad Investor Chaos, we don’t know how lies will propagate through an alien architecture.
I don’t know how ‘find less goodharted sensors’ is instantiated on natural hardware or might have toy versions implemented algorithmically, seems like it would be worth trying to figure out. In a conversation, John mentioned a type of architecture that is forced through an information bottleneck to find a minimal representation of the space. Seemed like a similar direction.
I think one of the important dynamics with debt is that it makes the race to the bottom worse because it lets people come up with clever ways of digging the bottom deeper. If there’s some well calibrated amount of debt to take on some people will, by virtue of miscalibration, take on more than that, and others in a position to profit from that will have an incentive to create distortions to make such miscalibration more likely. Even worse than with individuals, this happens at the level of organizations including governments, whom many rely on as monopoly providers of many important goods.
I think developmental models predate the relevant form of modernity. E.g. I expect to see psychological development with age in hunter gatherers and others not exposed to post-modernity.
We’ve got this thing called developmental psychology and also the fact that most breakthrough progress is made while young. What’s going on? If dev psych is about becoming a more well adjusted person what is it about being ‘well adjusted’ that makes breakthrough work less likely?
My guess is that it has to do with flexibility of cognitive representations. Having more degrees of freedom in your cognitive representation feels from the inside like more flexibility but looks from the outside like rationalization, like the person just has more ways of finding and justifying paths towards what they wanted in the first place, or justifying why something isn’t possible if it would be too much effort.
I think this should predict that breakthrough work will more often be done by disagreeable but high openness people. High enough openness to find and adopt good cognitive representations, but disagreeable enough to use them to refute things rather than justify things. The age effect would appear if cognitive representations tend to get more flexible with time and there is some sort of goldilocks effect.
“yeah, I don’t really mind being the evil thing. Seems okay to me.”
Regarding oneself as amoral seems to necessarily involve incoherence AFAICT. Like claiming you don’t have your own oughts or that there is no structure to your oughts. In this case
Like doing a good work of art or playing an instrument. It feels satisfying.
is regarded as a higher good than the judgment of others. Loudly signaling that you don’t care about the judgment of others seems to be a claim about what control surfaces you will and won’t expose to social feedback. Nevertheless, the common sense prior that you shouldn’t expect alliance with people who ‘don’t care about being evil’ to go well seems appropriate.
To put it less abstractly: believe people when they say they are defecting rather than believe it is nested layers of fun and interesting counter signaling.
If I get symptoms I will be taking some zembrin on the off chance that it shares some causal mechanisms with fluvoxamine. Annoyed that I can get neither fluvoxamine nor inhaled budesonide.
If we’re currently in a losing scenario then we want to increase the variance in our betting strategy. But most (all?) collective group decision procedures decrease the variance.
Immortals would be incentivized to invest in mitigations that don’t make any sense for mortals because they only kill you once every 1000 or more years.
I think this should also link to the two headed bacterium post since the immortals here are like persisters.
You may not be interested in mutually exclusive compression schemas, but mutually exclusive compression schemas are interested in you. One nice thing is that given that the schemas use an arbitrary key to handshake with there is hope that they can be convinced to all get on the same arbitrary key without loss of useful structure.
Spoiler tags are borked the way I’m using them.
anyway, another place to try your hand at calibration:
Humbali: No. You’re expressing absolute certainty in your underlying epistemology and your entire probability distribution
no he isn’t, why?
Humbali is asking for Eliezer to double count evidence. Consilience is hard if you don’t do your homework on provenance of heuristic and not just naively counting up outputs who themselves also didn’t do their homework.
Or in other words: “Do not cite the deep evidence to me, I was there when it was written”
And another place to take a whack at:
I’m not sure how to lead you into the place where you can dismiss that thought with confidence.
The particular cited example of statusy aliens seems like extreme hypothesis privileging, which often arises from reference class tennis.
I hope someone has taken seriously the idea of just paying top researchers a million a year to work on safety instead of capability. The last several times ‘pay for top talent’ was made in the context of ea in general very unconvincing excuses were given not to.
Are we talking about the same thing?
https://www.sciencedirect.com/science/article/am/pii/S0370157317301424
Tangential, but did you ever happen to read statistical physics of human cooperation?
Three pieces.
Creativity is a trainable skill.
Philosophy is a trainable skill and many ideas come about from turning a weird thought experiment into a writing piece via creativity.
Insights are a trainable skill via deliberate contemplation and inquiry.
For creativity I’d read John Cleese’s book. For philosophy I’d read the Stanford encyclopedia of philosophy focusing on following your own interests. For insight I’d look at Gendlin’s Focusing and Thinking at the Edge.
I guess there’s writing skill too, but that’s amenable to deliberate practice.
I’m not sure how to say it in another way. Moral mazes are selected to exist by management because they disperse liability for decisions that benefit management while harming employees, customers, and (often) shareholders. Maybe not 100%, some of their instantiation details I’m sure are just spandrels. But they have a purpose and fulfill that purpose.