A.H.

• A.H. 2 Aug 2022 8:56 UTC

  3 points

  in reply to: Measure’s comment on: In­for­ma­tion is a Coun­ter­fac­tual Property

  I presume you are talking about the post What is Evidence?

  Yes, the ideas in that post are closely related to this one. I think that the main difference is that, in that post Eliezer is interested in epistemology whereas here I am interested in how the process works from a physics point of view. For example, in my post, there is no need for ‘information’ (as I am using the word) to be correlated with true beliefs.

• A.H. 2 Aug 2022 14:50 UTC

  1 point

  in reply to: Mo Putera’s comment on: In­for­ma­tion is a Coun­ter­fac­tual Property

  Hi, thanks for your question. I have a big piece covering all of this in more detail which I plan to post in a couple of days once I’ve finished writing it. In the meantime, please accept this ‘teaser’ of a few problems in the prevailing conception (PC):

  1. Dealing with hybrid systems. If we are operating in a regime where there are two contradictory sets of dynamical laws, we do not know what kind of evolution the system will follow. An example of such a system is one where both gravity (as governed by general relativity) and quantum mechanics are relevant. In such a cases, under the PC, it is difficult to make any predictions of what kind of behaviour systems will exhibit, since we lack the dynamical laws governing the system. However, by appealing to general counterfactual principles (the interoperability principle and the principle of locality), which cannot be stated in the PC, we can make predictions about such systems, even if we don’t know the form of the dynamical laws.

  2. The 2nd Law of Thermodynamics. Under the PC, the 2nd is difficult to express precisely, since all dynamical laws are reversible in time, but the 2nd law implies irreversible dynamics. This is normally dealt with by introducing some degree of imprecision or anthropocentrism (eg. through averaging or coarse graining, or describing the 2nd law in terms of our state of knowledge of the system). However, the 2nd law can be stated precisely as a counterfactual statement along the lines of ‘it is impossible to engineer a cyclic process which converts heat entirely into work’.

  3. The initial state problem. Under the PC, the state of a system can be explained in terms of its evolution, according to dynamical laws, from a previous state at an earlier time. This makes it difficult to explain early states of the universe: if a state can only be explained in terms of earlier states, then either the universe has an initial state, which we cannot explain (since there are no earlier state), or the universe does not have an initial state we have an infinite regress, explaining each state in terms of earlier states, going on forever. Neither of these options seem satisfactory.

• A.H. 4 Aug 2022 17:39 UTC

  1 point

  in reply to: Dacyn’s comment on: In­for­ma­tion is a Coun­ter­fac­tual Property

  I agree with your example and think that it touches on something important. However, in this post, I did not claim that the counterfactual condition was the only condition required for information transfer. You are correct to say that the lamp signal would not constitute information to someone who was unaware of the plan. But this is because, in that situation, there are other conditions that have not been met. Since the other person seeing the lamp signal would not react differently to the different signals, there is no causal link between the signal and them. This is also required for information transfer. I tried to explain this idea a bit more in my subsequent post.

  If you don’t like the idea of information being physical, rather than epistemological, then maybe you can think of this post as asking the question ‘what are the physical conditions that a system must satisfy in order to transmit epistemological information?’

• A.H. 4 Aug 2022 18:14 UTC

  1 point

  in reply to: TAG’s comment on: In­for­ma­tion is a Coun­ter­fac­tual Property

  I think I disagree with your characterisation of the split between ‘objective’ Shannon information and information as meaning, which requires interpretation.

  As you point at the end of your comment, Shannon information requires you to know the probability distribution from which your data is drawn. And probabilities are reflections of your own state of knowledge, which is subjective. (Or at least subjectively objective, if you are using ‘objective’ in that sense, then I guess I agree.) For example, if Alice sends Bob a string ’11111′, we might be tempted to say that she has sent Bob 5 bits of information, but if Bob knows that Alice can only send two possible strings ‘00000’ or ‘11111’, then he would say that she has only sent one bit. All signals, not just what you call ‘information as meaning’ require some degree of interpretation. And this interpretation, I argue, requires knowing the possible signals that could be sent, even if they are not actually sent. These possible signals are what I am calling counterfactuals.

  I’m not sure I understand your point about conditionals vs counterfactuals.

  AFAICS, that’s just a special case of the inverse relationship between probability and(Shannon) information. If the lamp is stuck “on”, the probability of an “on” signal is 1.000 and the information content is 0.000. So it’s not fundamentally about counterfactuals at all.

  I kind of agree with this, but it doesn’t tell the whole story. Consider the case where, instead of being stuck ‘on’, the lamp flickers randomly and is on 50% of the time and off 50% of the time. In this case, you would not be able to use the lamp to send information, even though the probability of an ‘on’ signal is 0.5 and, in one sense, the Shannon entropy would be maximal. To send information requires that it is possible for you to change the signal sent by the lamp. This is what I was trying to get at in this post. Another way of thinking about it is to say that you must have a causal effect on the state of the signal. In both the case where the lamp is stuck on and the case where it is flickering uncontrollably, you have no causal link to the state of the signal. I tried to explain the link between counterfactuals, information and causality in the subsequent post.

• A.H. 9 Aug 2022 11:04 UTC

  1 point

  in reply to: Charlie Steiner’s comment on: The use of Coun­ter­fac­tu­als in Physics

  I think this was well-written and clear, so good job there! I also happen to disagree with the contents.

  Thanks for your comment!

  First off, I’m highly suspicious of any definition of a “prevailing conception” of physics that excludes the second law of thermodynamics! It seems like in actual practice, sometimes people make predictions by simulation, (the “PC”) sometimes they make predictions by generalizing about the character of physical law (the quantum gravity example), and sometimes they do something in between those things and make abstractions/​generalizations but then treat those abstractions as tools to do simulation (condensed matter theorists I see you).

  Yeah, the term ‘prevailing conception’ is Deutsch’s. It refers specifically to formulating things in terms of initial conditions and dynamical law. I agree its not a great term, as it implies that all current physics comes under its umbrella, which, as you pointed out, is not true.

  And so what does it mean to recast physics in a different picture? Does this mean people are going to be rendered unable to do simple simulations about what actually happens when you shoot a particle at a barrier?

  The idea isn’t to throw away the dynamical laws picture, but to provide a different angle of attack on some problems that seem intractable when expressed in the PC.

  So then does it mean that we’re going to be able to make new exciting arguments that physicists were unable to make before?

  That’s the hope!

  I mean, I’d love it if this were true, but I’m skeptical. My cynical side expects that there will be few new sorts of arguments, but plenty of flag-planting on old sorts of arguments.

  Fair enough. I’m skeptical as well. Constructor theory has produced a couple of interesting results, but as far as I can see, nothing world-changing yet. But I am still convinced that the problems described here (eg. the incompatibility of reversible dynamics with irreversibility of the 2nd law) are real problems. Even if counterfactuals/​constructor theory don’t work (who knows?), we will need something new to address them!

• A.H. 9 Aug 2022 14:49 UTC

  1 point

  in reply to: supposedlyfun’s comment on: The use of Coun­ter­fac­tu­als in Physics

  Hi, thanks for the question. I am using the term ‘counterfactual’ (admittedly somewhat loosely) to describe facts that refer to whether things are possible or impossible, regardless of whether they actually happen.

  In the first post, I claimed that it is only meaningful to say that the lamp transmits information if it is possible for the lamp to be a in a different state. Conversely, if the lamp was broken, then it is impossible for the lamp to be in a different state, and information does not get transmitted. If you just describe the system in terms of what actually happens (ie. ‘the lamp is on’), you miss out on this fact. In the first post, I called statements about what actually happens in the system ‘factual statements’, and statements about what is possible/​impossible ‘counterfactual statements’.

  Similarly, in the case of the perpetual motion machine, you can make a factual statement about what actually happens (ie, some gears turn around and eventually the machine stops moving, failing to achieve perpetual motion), or you can make a counterfactual statement (that it is impossible to make perpetual motion machine, regardless of the specifications of that machine). In this post, I again claimed that just making the factual statement misses out on the important counterfactual claim.

  Of course, in the first post, when the lamp is broken, and we say it is ‘impossible’ to send another signal, this is specified by the parameters of the thought experiment, rather than the laws of physics (in practice, the laws of physics might not prevent you from fixing the lamp, for example). Whereas in the this this post, when we say it is ‘impossible’ to build a perpetual motion machine, the restriction does come from the laws of physics.

  Hope this helps clear things up!

• A.H. 10 Aug 2022 9:55 UTC

  1 point

  in reply to: supposedlyfun’s comment on: The use of Coun­ter­fac­tu­als in Physics

  Glad that confusion is removed!

  I think that it is the best word to use. When used as an adjective Collins defines ‘counterfactual’ as ‘expressing what has not happened but could, would, or might under differing conditions ‘. I think that this fits the way I was talking about it (eg. when referring to ‘counterfactual laws’). In the first post, I talk about whether the lamp ‘could would, or might’ have been in a different state. In this post, we talk about whether a perpetual motion machine ‘could would, or might’ work if it was made using a different configuration. (maybe some of the confusion comes from using ‘counterfactual’ as both an adjective and a noun?)

  Though if you have any suggestions on other words that might be clearer, let me know.

• A.H. 11 Aug 2022 14:27 UTC

  1 point

  in reply to: supposedlyfun’s comment on: The use of Coun­ter­fac­tu­als in Physics

  The form of a counterfactual law (“your perpetual motion machine won’t work even if you make that screw longer or do anything else different”) seems to be “A, no matter which parameter you change”.

  
  I don’t think this is right. As I am using it, ‘counterfactual’ refers to a statement about whether something is possible or impossible. Statements of the form “A, no matter which parameter you change” are not always like this. For example if A=’this ball has a mass of 10kg’. This is not a statement about what is possible or impossible. You could frame it as ‘it is impossible for this ball to have a mass other than 10kg, no matter which parameter you change’, but doesn’t give us any new information compared to the original statement.

  Another important feature is that the impossibility/​possibility is not restricted to specific dynamical laws. In your example ‘F=ma, even if the frictionless sphere is blue’, this statement is only true when Newton’s laws apply. But the statement ‘it is impossible to build a perpetual motion machine’ refers, in principle, to all dynamical laws-even ones we haven’t discovered yet-which is why principles like this may help guide our search for new laws.

• A.H. 11 Aug 2022 14:29 UTC

  1 point

  in reply to: TAG’s comment on: The use of Coun­ter­fac­tu­als in Physics

  I didn’t use ‘modal’ because that is used to refer to logical possibility/​impossibility, whereas I am interested in referring to physical possibility/​impossibility. Depending on your philosophical views, those two things may or may not be the same.

Is Con­struc­tor The­ory a use­ful tool for AI al­ign­ment?

• A.H. 30 Nov 2022 10:38 UTC

  3 points

  2

  in reply to: shminux’s comment on: Is Con­struc­tor The­ory a use­ful tool for AI al­ign­ment?

  Thanks for writing this. I wanted to write something about how Deutsch performs a bit a of motte-and-bailey argument (motte:‘there are some problems in physics which are hard to solve using the dynamical laws approach’. bailey:‘these problems can be solved using constructor theory specifically, rather than other approaches’). Your comment does a good job of making this case. In the end I didn’t include it, as the piece was already too long. I just wrote the sentence

  Pointing out problems in the dynamical laws approach to physics and trying to find solutions is useful, even if constructor theory turns out not to be the best solution to them.

  and left it at that.

• A.H. 24 Dec 2022 12:58 UTC

  1 point

  on: On ex­pected util­ity, part 2: Why it can be OK to pre­dictably lose

  Thanks for writing this. I think that the arguments in parts III and IV are particularly compelling and well-written.

• A.H. 16 Feb 2023 19:02 UTC

  1 point

  0

  on: The Geo­met­ric Expectation

  The infinite non-uniform discrete case is not much more difficult. If $X$ is a finite or countably infinite set, $f:X\to R^{\ge 0}$ assigns a nonnegative value to each $x\in X$, and $P$ is a probability distribution on $Y$, then $E_{x\sim P}f\left(x\right)={\sum }_{x\in X}P\left(x\right)f\left(x\right)$

  Very minor, but shouldn’t this read ”$P$ is a probability distribution on $X$” not $Y$?

• A.H. 23 Apr 2023 18:18 UTC

  3 points

  on: Harsanyi’s So­cial Ag­gre­ga­tion The­o­rem and what it means for CEV

  The link to Harsanyi’s paper doesn’t work for me. Here is a link that does, if anyone is looking for one:

  https://​​hceconomics.uchicago.edu/​​sites/​​default/​​files/​​pdf/​​events/​​Harsanyi_1955_JPE_v63_n4.pdf

• A.H. 4 May 2023 8:24 UTC

  1 point

  4

  on: [New] Re­jected Con­tent Section

  I think this is a good idea, thanks for implementing!

  Very minor but the link lesswrong.com/​​moderation#rejected-comments just goes to the same page as lesswrong.com/​​moderation#rejected-posts (the written address is correct but the hyperlink goes to the wrong page)

• A.H. 4 May 2023 10:27 UTC

  2 points

  0

  on: Geo­met­ric Ra­tion­al­ity is Not VNM Rational

  I am confused about something. You write that a preference ordering $L⪯M$ is geometrically rational if$G_{U\sim P}{E}_{O\sim L}U\left(O\right)\le G_{U\sim P}{E}_{O\sim M}U\left(O\right)$.

  This is compared to VNM rationality which favours $L⪯M$ if and only if $E_{O\sim L}U\left(O\right)\le E_{O\sim M}U\left(O\right)$.

  Why, in the the definition of geometric rationality, do we have both the geometric average and the arithmetic average? Why not just say “an ordering is geometrically rational if it favours $L⪯M$ if and only if $G_{O\sim L}U\left(O\right)\le G_{O\sim M}U\left(O\right)$ ” ?

  As I understand it, this is what Kelly betting does. It doesn’t favour lotteries over either outcome, but it does reject the VNM continuity axiom, rather than the independence axiom.

Align­ment as Func­tion Fitting

Op­ti­mized for Some­thing other than Win­ning or: How Cricket Re­sists Moloch and Good­hart’s Law

• A.H. 5 Jul 2023 13:44 UTC

  1 point

  0

  in reply to: RamblinDash’s comment on: Op­ti­mized for Some­thing other than Win­ning or: How Cricket Re­sists Moloch and Good­hart’s Law

  Interesting, thanks for sharing! Its cool to see how different games manage the conflict between coming up with innovative tactics (which for me is all part of the fun of sports) and exploiting the rules in a way that makes the game boring.

  Also thanks for the link to David Sirlin. I haven’t heard of him and the website looks interesting!

• A.H. 6 Jul 2023 14:35 UTC

  1 point

  0

  in reply to: Daniel Kokotajlo’s comment on: Op­ti­mized for Some­thing other than Win­ning or: How Cricket Re­sists Moloch and Good­hart’s Law

  I think the two-player-game-but-player2-gets-to-modify-the-rules is not a fair analogy here. Like I said it’s the cricket-loving public that decides, not player 2.

  Broadly, I agree with Richard Ngo’s characterisation. You are right that the ‘cricket loving public’ plays some part in determining what counts as ‘within the spirit’ but it is the decision of the players themselves that often is most important.

  How is this different from games with a referee? A foul is what the referee says it is; the spirit of cricket is what the cricket-lovers say it is. In both cases a savvy optimizer would start modelling the relevant humans and predicting what they would and wouldn’t judge illegal.
  
  I agree that different rules or optimization targets have different complexity levels, and the spirit of cricket seems more complicated than ordinary fouls which are more complicated than “did the ball hit the pegs.”

  I agree with you that the complexity is an important factor. I think you are correct that in principle this can still be Goodharted, but in practice it doesn’t seem to happen as it is much harder than Goodharting the written rules of the game, due to the increased complexity. There is nothing to prevent a superintelligent player from brainwashing the opposing team and general public to agreeing that their actions are legitimate. It’s just that doing this is a lot harder than normal ways of ‘gaming the system’. This is why I used the term ‘resists Goodharts law’ as opposed to ‘defeats Goodharts law’ or something similar.