This changed my mind about the parent comment (I think the first paragraph would have done so, but the example certainly helped).
In general, I don’t mind added concreteness even at the cost of some valence-loading. But seeing how well “sanction” works and some other comments that seem to disagree on the exact meaning of “punch”, I guess not using “punch” would have been better
I did indeed! So I guess this game fails (5) out of Zvi’s criteria.
Does your program assume that the Kelly bet stays a fixed size, rather than changing?
Here’s a program you can paste in your browser that finds the expected value from following Kelly in Gurkenglas’ game (it finds EV to be 20)
https://pastebin.com/iTDK7jX6
(You can also fiddle with the first argument to experiment to see some of the effects when 4 doesn’t hold)
It sounds like in the first part of your post you’re disagreeing with my choice of reference class when using SSA? That’s reasonable. My intuition is that if one ends up using a reference class-dependent anthropic principle (like SSA) that transhumans would not be part of our reference class, but I suppose I don’t have much reason to trust this intuition.
On anthropic measure being tied to independently-intelligent minds, what is the difference between an independently- and dependently-intelligent mind? What makes you think the mind needs to be specifically independently-intelligent?
Yes, I suppose the only way that this would not be an issue is if the aliens are travelling at a very high fraction of the speed of light and inflation means that they will never reach spatially distant parts of the Universe in time for this to be an issue.
In SETI-attack, is the idea that the information signals are disruptive and cause the civilisations they may annihilate to be too disrupted (perhaps by war or devastating technological failures) to defend themselves?
Yeah, that’s a good point. I will amend that part at some point.
Also, the analysis might have some predictions if civilisations don’t pass through a (long) observable stage before they start to expand. It increases the probability that a shockwave of intergalactic expansion will arrive at Earth soon. Still, if the region of our past light cone where young civilisations might exist is small enough, we probably just lose information on where the filter is likely to be
I wonder if there are any plausible examples of this type where the constraints don’t look like ordering on B and search on A.
To be clear about what I mean about those constraints, here’s an example. One way you might be able to implement this function is if you can enumerate all the values of A and then pick the maximum B according to some ordering. If you can’t enumerate A, you might have some strategy for searching through it.
But that’s not the only feasible strategy. For example, if you can order B, take two elements of B to C and order C, you might do something like taking the element of B that, together with the value less than it, takes you to the greatest C.
My question is whether these weirder functions have any interest
I wasn’t aware that CFAR had workshops in Europe before this comment. I applied for a workshop off the back of this. Thanks!
I feel a pull towards downvoting this. I am not going to, because I think this was posted in good faith, and as you say, it’s clear a lot of time and effort has gone into these comments. That said, I’d like to unpack my reaction a bit. It may be you disagree with my take, but it may also be there’s something useful in it.
[EDIT: I should disclaim that my reaction may be biased from having recently received an aggressive comment.]
First, I should note that I don’t know why you did these edits. Did sarahconstantin ask you to? Did you think a good post was being lost behind poor presentation? Is it to spell out your other comment in more detail? Knowing the answer to this might have changed my reaction.
My most important concern is why this feedback was public. The only (charitable) reason I can think of is to give space for pushback of the kind that I am giving.
My other major concern is presentation. The sentence ‘I trust that you can see past the basic “I’m being attacked” feeling and can recognise the effort and time that has gone into the comments’ felt to me like a status move and potentially upsetting someone then asking them to say thank you.
It is probably true that those are the places with most engagement. However, as someone without Facebook, I’m always grateful for things (also) being posted in non-FB places (mailing lists work too, but there is a longer lag on finding out about things that way).
It seems like the images of the gears have disappeared. Are they still available anywhere? EDIT: They’re back!
If you can’t viscerally feel the difference between .1% and 1%, or a thousand and a million, you will probably need more of a statistics background to really understand things like “how much money is flowing into AI, and what is being accomplished, and what does it mean?”
I’m surprised at the suggestion that studying statistics strengthens gut sense of the significance of probabilities. I’ve updated somewhat towards that based on the above, but I would still expect something more akin to playing with and visualising data to be useful for this
It seems hard to me to get information out of the AI without also giving it information. That is, presumably we will configure parts of its environment to correspond to problems in our own world, which necessarily gives some information on our world.
I suppose another option would be that this is a proposal for running AGIs that just run without us ever getting information from. I don’t think that’s what you meant, but thought I’d check.
I suppose the problem comes when the AI starts to communicate with us. There would be a lot of information that they could exploit. Even if they don’t get any sense of our physics, if they are able to model us we might be in trouble. And even if we didn’t give them any direct communication (for example manifesting puzzles in their world, the solution of which would allow us to solve our own questions), they might promote simulation to a reasonable hypothesis.
Any disagreement with Robin Hanson (https://twitter.com/robinhanson/status/1022535475410731009)?
It seems to me that subsidy addresses V, II and maybe III, but not so much IV and I. Does that seem right?
(Also, I think he’s suggested non-anonymous trading as a cure for insider trading before)
On the other hand, there may well be a serious game-theoretic reason why it is “too harsh”: someone who is getting to cooperation from the system has no reason to cooperate in turn.
Is there a typo here? (“getting to cooperation” → “getting no cooperation”). And the idea is that there are other ways of making an impact on the world than the decisions of the “futarchy”, so people who have no stake in the futarchy could mess things up other ways, right?
It wasn’t clear to me from the paper if they thought values that came from our contingent history could be worth preserving or promoting. For example, they might think they engage our same moral intuitions as mammalian values without being worth defending
I felt similarly. I’d also like to see them dig more into their mammalian values + human cognition + evolution of human society/culture. Specifically, (1) defending the breakdown as a good account of human values and (2) separating out their claims about values in human cultures (and being a bit clearer about whether they claim that cultural values are less likely ‘true’ values, however that might be cashed out) and about values arising from historical incident
I’ve been trying this for a couple of weeks now. It’s hard! I often will have a missing link in the distraction chain: I know something that came at point X in the distraction chain and X-n, for n > 1. When I try and probe the missing part it’s pretty uncomfortable. Like using or poking a numb limb. It can be pretty aversive, so I can’t bring myself to do this meditation every time I meditate.