Quinn
on wellunderstoodness
Do the best ideas float to the top?
This one had slipped by me, so thanks for pointing me to it. It’ll take me at least a week to read and digest. I’ll add a comment here (eventually) if I have anything to say.
Not much to add, but: Yes, you nailed it, I see it in the world all the time.
thanks for your comment.
Likewise, what level do you want a NAT to be implemented at? Personal behavior? Structure of group blog sites? Social norms?
-
personal behavior: probably not viable without a dystopian regime of microchips embedded into brains.
-
structure of group blog sites: maybe—these things have been suggested and tried, i.e. I can’t tell you how many times I’ve seen a reddit comment lamenting the incentives of their upvote system.
-
weirdly, I found out about the Brave browser last week (weird because it’s apparently been around for a while): attempting to overthrow advertising with an attention-measuring coin. This is great news!
-
I was thinking a lot about NAT reading this paper. In the context of debate judges, NAT is a bit of a “last minute jerry-rig / frantically shore up the levy” solution, something engineers would stumble upon in an elaborate and convoluted debugging process—the exact opposite of the kind of the solutions alignment researchers are interested in.
if an AI comes to you and says, “I would like to design the particle accelerator this way because,” and then makes to you an inscrutable argument about physics, you’re faced with this tough choice. You can either sign off on that decision and see if it has good consequences, or you can be like, no, don’t do that ’cause I don’t understand it. —Paul Christiano
Tim Wu’s “Is the first amendment obsolete?” is important and I think everybody should read it.
-
Sorry. The point was NAT,
density_{1,2,3}was devised scaffolding for the MVB (minimum viable blogpost). I imagine that NAT has already been discovered, discussed, problematized etc. somewhere but I couldn’t find it. I have a background assumption that attention economists are competent and well-intentioned people, so I trust that they have the situation under control.
IMO, this is what I briefly suggested by linking to Scott’s Against Murderism with the words “misleading compression”, i.e., I think describing a policy as murderistic and optimizing for stories are each instances of misleading compression.
If it’s only stories which matter, yet you split your efforts between stories and reality, then you will likely be outcompeted by someone who spent all of their resources on crafting good stories.
This is 100% what I find alarming about misinformation (both the malicious kind and the emergent/inadequate kind), and I don’t know a reason why alignment via debate would be resilient.
A skin-in-the-game vote multiplier based on age might look like
mean lifespan - your age. That’s the logical consequence of saying that people who have to put up with outcomes longer ought to weigh higher in shaping them. It should floor out at around 1 at the upper limit, and the lower limit should come from enforceability of anti-fraud measures (i.e. effectiveness at stopping parents from using kids who can’t walk yet for extra votes) instead of from anyone’s intuitions about when kids can think for themselves.If some experts got together and said that brain development, knowledge, wisdom, etc. peaks at N, then you’d want the multiplier to be convex with a max at N.
With functions like these, averages between them, etc. there’s a lot of material to play with, in terms of starting with one-person-one-vote and fixing it’s weirdness with multipliers.
Maybe the latest in voting theory or the current stage of quadratic voting research already considered all this and came up with something more promising.
Does having children whose future you care about also count as skin in the game?
Unclear. There’s a lot to unpack, because we don’t know the 1. narcissism or 2. epistemic competence distributions across parents. I.e., we can’t expect that what parents’ say are in their kids’ interests actually share their kids’ interests (either through willful misdirection or through earnest mistakes).
Or you can say that your skin-in-the-game factor is proprotional to how much you’ve already invested in the status quo. If you’ve spent 50 years working towards a goal it seems unfair that a 16-year old know-nothing should be able, on a whim, to throw all of that away.
I don’t mean to guilt-by-association dismiss this, but it strongly reminds me of the property/land interpretation of SITG-suffrage.
The risk of 16 year old know-nothings throwing things away on a whim is measured against the risk of bad “tradition is the democracy of the dead” / “most insolent of tyrannies is to govern from beyond the grave” scenarios. Which equilibrium is worse, a civilization unable to cooperate across lifetimes (because kids constantly throw everything away and start over, reinventing wheels and repeating mistakes), or one where adults only inherit agency at age 70 and by then all they care about is the same stuff the previous 70+ cohort cared about? I think “epistemically defer to the elderly when it seems wise to do so” is a more beneficial heuristic than “we owe the elderly deference for the sacrifices they made before I was born”, and if we’re going to bet on the distribution of how responsibly we expect these heuristics to scale, I’d much rather bet on the former.
A year or two before the paperclip version came out I played a lot of AdVenture Capitalist (and it’s sequel, wait for it, AdVenture Communist), was wondering to myself whether reinforcement learning researchers would find it interesting, and wondering if deep mind would start training up agents to compete in AdVenture Capitalist tournaments.
SITG-suffrage Sorry, by this point OP and I had established “right to vote weighted by stake” as a concept, using the words “skin-in-the-game”, so SITG was an acronym for skin-in-the-game, and suffrage referred to right to vote.
Parents are different from any other group in my comment because I was referencing Richard Kennaway’s question “Does having children whose future you care about also count as skin in the game?”
[Question] Have general decomposers been formalized?
[Question] How ought I spend time?
I’ll check out Lynette’s post.
I’d like to take a shot at technical AI alignment
Announcing the Technical AI Safety Podcast
When I submitted to pocketcasts it said we were already on it :) https://pca.st/9froevor
Chance that “AI safety basically [doesn’t need] to be solved, we’ll just solve it by default unless we’re completely completely careless”
Infodemics: with Jeremy Blackburn and Aviv Ovadya
7p on thursday the 14th for New York, 4p in San Fransisco
I don’t know a lot about evolution, but I suspect any benefits of building on memetics work directly would fall under the umbrella of “what about when we’re tipping the scale in favor of some ingroup?”. I defined density_3 as a placeholder for this along with all maximization related issues, and then said “we’ll ignore this for now and focus on more basic foundations”. I don’t know if I’ll return to it, but if I do, it’ll take me a really really long time.