What granularity of time are you talking about? When you “never maintain 1 and 2 at the same time”, is that any given minute, or any given decade?
I would say every couple months is an opportunity to either pivot or continue.
Quinn
Karma: 51
[Question] How ought I spend time?
Sorry, I think I might have a superficial understanding of encoders and embeddings. Would you be able to try pointing out for me how decomposition is performed in that case (or point me toward a favorite reading on the subject)? When I think of feeding a sentence into an encoder, I can think of multiple ways in which some compositional structure might be inferred.
I’m drawing up a proof of concept with seq2seq learners right now, but my hypothesis is that they will be inadequate decomposers suitable only for benchmarking a baseline.
[Question] Have general decomposers been formalized?
SITG-suffrage Sorry, by this point OP and I had established “right to vote weighted by stake” as a concept, using the words “skin-in-the-game”, so SITG was an acronym for skin-in-the-game, and suffrage referred to right to vote.
Parents are different from any other group in my comment because I was referencing Richard Kennaway’s question “Does having children whose future you care about also count as skin in the game?”
A year or two before the paperclip version came out I played a lot of AdVenture Capitalist (and it’s sequel, wait for it, AdVenture Communist), was wondering to myself whether reinforcement learning researchers would find it interesting, and wondering if deep mind would start training up agents to compete in AdVenture Capitalist tournaments.
Does having children whose future you care about also count as skin in the game?
Unclear. There’s a lot to unpack, because we don’t know the 1. narcissism or 2. epistemic competence distributions across parents. I.e., we can’t expect that what parents’ say are in their kids’ interests actually share their kids’ interests (either through willful misdirection or through earnest mistakes).
Or you can say that your skin-in-the-game factor is proprotional to how much you’ve already invested in the status quo. If you’ve spent 50 years working towards a goal it seems unfair that a 16-year old know-nothing should be able, on a whim, to throw all of that away.
I don’t mean to guilt-by-association dismiss this, but it strongly reminds me of the property/land interpretation of SITG-suffrage.
The risk of 16 year old know-nothings throwing things away on a whim is measured against the risk of bad “tradition is the democracy of the dead” / “most insolent of tyrannies is to govern from beyond the grave” scenarios. Which equilibrium is worse, a civilization unable to cooperate across lifetimes (because kids constantly throw everything away and start over, reinventing wheels and repeating mistakes), or one where adults only inherit agency at age 70 and by then all they care about is the same stuff the previous 70+ cohort cared about? I think “epistemically defer to the elderly when it seems wise to do so” is a more beneficial heuristic than “we owe the elderly deference for the sacrifices they made before I was born”, and if we’re going to bet on the distribution of how responsibly we expect these heuristics to scale, I’d much rather bet on the former.
A skin-in-the-game vote multiplier based on age might look like
mean lifespan—your age. That’s the logical consequence of saying that people who have to put up with outcomes longer ought to weigh higher in shaping them. It should floor out at around 1 at the upper limit, and the lower limit should come from enforceability of anti-fraud measures (i.e. effectiveness at stopping parents from using kids who can’t walk yet for extra votes) instead of from anyone’s intuitions about when kids can think for themselves.If some experts got together and said that brain development, knowledge, wisdom, etc. peaks at N, then you’d want the multiplier to be convex with a max at N.
With functions like these, averages between them, etc. there’s a lot of material to play with, in terms of starting with one-person-one-vote and fixing it’s weirdness with multipliers.
Maybe the latest in voting theory or the current stage of quadratic voting research already considered all this and came up with something more promising.
IMO, this is what I briefly suggested by linking to Scott’s Against Murderism with the words “misleading compression”, i.e., I think describing a policy as murderistic and optimizing for stories are each instances of misleading compression.
If it’s only stories which matter, yet you split your efforts between stories and reality, then you will likely be outcompeted by someone who spent all of their resources on crafting good stories.
This is 100% what I find alarming about misinformation (both the malicious kind and the emergent/inadequate kind), and I don’t know a reason why alignment via debate would be resilient.
Sorry. The point was NAT,
density_{1,2,3}was devised scaffolding for the MVB (minimum viable blogpost). I imagine that NAT has already been discovered, discussed, problematized etc. somewhere but I couldn’t find it. I have a background assumption that attention economists are competent and well-intentioned people, so I trust that they have the situation under control.
thanks for your comment.
Likewise, what level do you want a NAT to be implemented at? Personal behavior? Structure of group blog sites? Social norms?
-
personal behavior: probably not viable without a dystopian regime of microchips embedded into brains.
-
structure of group blog sites: maybe—these things have been suggested and tried, i.e. I can’t tell you how many times I’ve seen a reddit comment lamenting the incentives of their upvote system.
-
weirdly, I found out about the Brave browser last week (weird because it’s apparently been around for a while): attempting to overthrow advertising with an attention-measuring coin. This is great news!
-
I was thinking a lot about NAT reading this paper. In the context of debate judges, NAT is a bit of a “last minute jerry-rig / frantically shore up the levy” solution, something engineers would stumble upon in an elaborate and convoluted debugging process—the exact opposite of the kind of the solutions alignment researchers are interested in.
if an AI comes to you and says, “I would like to design the particle accelerator this way because,” and then makes to you an inscrutable argument about physics, you’re faced with this tough choice. You can either sign off on that decision and see if it has good consequences, or you can be like, no, don’t do that ’cause I don’t understand it. —Paul Christiano
Tim Wu’s “Is the first amendment obsolete?” is important and I think everybody should read it.
-
Not much to add, but: Yes, you nailed it, I see it in the world all the time.
This one had slipped by me, so thanks for pointing me to it. It’ll take me at least a week to read and digest. I’ll add a comment here (eventually) if I have anything to say.
I don’t know a lot about evolution, but I suspect any benefits of building on memetics work directly would fall under the umbrella of “what about when we’re tipping the scale in favor of some ingroup?”. I defined density_3 as a placeholder for this along with all maximization related issues, and then said “we’ll ignore this for now and focus on more basic foundations”. I don’t know if I’ll return to it, but if I do, it’ll take me a really really long time.
Do the best ideas float to the top?
I’m not trying to hold it constant, I’m just trying to understand a relatively low standard, because that’s the part I feel confused about. It seems relatively much easier to look at bad intellectual output and say how it could have been better, think about the thought processes involved, etc. Much harder to say what goes into producing output at all vs not doing so.
I think I understand the distinction, and I think if it was as simple as “people undershoot their actual capacities in favor of humility / don’t want to risk wasting anybody’s time” everyone would have adjusted social norms to remedy it by now.
thanks
I was thinking in a very different direction upon reading “a lot of people also find that writing down your ideas, causes you to have even more ideas.” I know what you mean in the context of a reinforcement system, but I think it misses the more pressing phenomena, at least in my experience of uncertainty whether i’m inventing or indulging, of working on ideas.
The “even more ideas” part sounds to me like a sort of (combinatorial) explosion, when my stroke of inspiration is much more problematic, much less elegant than I thought. Sometimes this also means much less original than I thought, but this isn’t a bad thing—convincing oneself that something is being discovered is often the most effective way at grokking it! You don’t really lose anything when you find out that it’s, in fact, old news.
Sometimes all this means is it will take more work than I thought, to follow the idea through. Other times it means this is the wrong rabbit hole.
But I think many of us AD(H)Ders develop a suspicion or even hostility to this “indulgent” signal, the internal phenomena of believing oneself to be creative, because they’ve too easily looted the reward (been superficially creative) at the expense of rigor (“why finish all those exercises in that boring book when this system i just wrote down is totally AGI already?”)
(At the same time, you and Lawrence Block are 100% right about nurturing/environment as well)
All in all, I can’t wrap my head around “what is the difference between a producer and a consumer of thought?” because the question as posed seems to hold rigor, even quality, constant/irrelevant.
(Many years ago a composer told me that when Schoenberg was at UCLA, young composers had to spend hours with him for 3+ years just analyzing mozart before he would consider looking at your music, compared to conservatories now you’re expressing yourself from day one. There is doubtless an analogy to AI risk—which culture is more productive?)
I’ll check out Lynette’s post.
I’d like to take a shot at technical AI alignment