Even in the world where everyone is perfectly aligned in their interests, it seems like credit for contributions still matters. It’s useful to know who did what in order to know who has what abilities and who should be assigned to do what thing in future.
DanielFilan
Karma: 9,213
In Australia, you’re ineligible to be a member of parliament if you’re a citizen of another country. Which sounds like a good rule until you realize how easy it is to become a citizen of another country, with changes like these.
AXRP Episode 49 - Caspar Oesterheld on Program Equilibrium
AXRP Episode 48 - Guive Assadi on AI Property Rights
I thought this was a cool post! I’d find it especially interesting to hear your flesh out how it applies to concepts like “local minimum” and the like.
Yep, see this paper—Bayesian updating is the same as having all the hypotheses in your head be agents in a prediction market using the Kelly criterion, where their inital wealths are your prior probabilities on those hypotheses, the market price of some observation is your marginal probability over that observation, and the wealths after the bets resolve are the posterior distributions.
Or in other words:
If we’re lucky the mercury in our thermometer moves up a constant amount each time, and we can avoid mixing things to calibrate our thermometer in the future. If we’re unlucky it doesn’t and we need to painstakingly calibrate all thermometers. If we’re really unlucky a scale that works on one substance doesn’t work on another and then we need to rethink this whole temperature thing again.
In fact you get unlucky on both counts.
I mean the actual temperature, which existed before humans had defined it. Maybe another non-circular way to say this is that you will get different results depending on whether you mix 99 parts ice and 1 part boiling water, vs other materials at the same temperature.
They discuss this in the book! The problem is that it might take different amounts of heat to raise the temperature by a fixed amount if you start from different temperature levels, so you can’t be sure the increments you get from this method are really equal.
On ‘Inventing Temperature’ and the realness of properties
We start off with some early values, and then develop instrumental strategies for achieving them. Those instrumental strategies become crystallized and then give rise to other instrumental strategies for achieving them, and so on. Understood this way, we can describe an organism’s goals/strategies purely in terms of which goals “have power over” which other goals, which goals are most easily replaced, etc, without needing to appeal to some kind of essential “terminalism” that some goals have and others don’t.
I think it would help me if you explained what you think it would mean for there to be an instrumental/terminal distinction, since to my eyes you’ve just spelled out the instrumental/terminal split.
Can you describe what you think the important inequivalencies are?
Am I right that this is about the version of the RAISE act that passed the senate, not the one that will be put into law?
AXRP Episode 47 - David Rein on METR Time Horizons
Anthropic wants to stay near the front of the pack at AI capabilities so that their empirical research is relevant, but not at the actual front of the pack to avoid accelerating race-dynamics.
— From an Anthropic employee in a private conversation, early 2023
Note that this is not a quote of an Anthropic employee in a private conversation. Instead, it is a quote of Ray Arnold describing his memory/summary of something an Anthropic told him in a private conversation.
I think it’s not just that it’s slower/deeper: my personal sense (which might be just a thing of not requiring much medical care between the ages of 5 and 30) is that the pace at which awesome new stuff is happening in medicines I can buy got much faster in the last few years. If my perception is right, it seems like that requires some explanation of “bio is slower/deeper and also 40 years ago there was a massive breakthru that took 40 years to percolate”, and not just “bio is slower/deeper”.
This year, we received a $1.6M matching grant from the Survival and Flourishing Fund, which means that the first $1.6M we receive in donations before December 31st will be matched 1:1. We will only receive the grant funds if it can be matched by donations.
Does anyone know (i.e. actually know, not just have a speculative guess) why SFF does this sort of thing? Naively it seems like a weird way to operate from a perspective of maximizing the good done by their donations. Is the reasoning something like “if other people donate to MIRI, that increases our estimate of how valuable MIRI is, and so we want to donate more to it”?
I wonder if it’s a thing where it’s taking a while for those things to hit. Like, mRNA vaccines are only a couple of years old, GLP-1RAs are in a gradual process of being rolled out, etc. If I think of the category of “awesome newish bio stuff I’d like to use”, it seems like most of it becomes widely available to consumers in the near future or last 5 years, with the exception of statins.
Another relevant consideration: I donated some money to political candidates and now I have a bunch of spam emails that cost me more than $3 of annoyance to deal with
I do think there’s also something like “robustness to everyone having the same values but a small number of people being maybe evil”