avturchin

• avturchin 18 Jul 2022 13:40 UTC

  3 points

  0 ∶ 0

  in reply to: TAG’s comment on: Cog­ni­tive In­sta­bil­ity, Phys­i­cal­ism, and Free Will

  Thanks for the link to Wallace’s work. I think that he also fails in the same trap, that is he fails to distinguish “large” and “small” forms of self-undermining arguments.

  
  The large one is a choice between two alternatives: “1) I am real; 2) I am BB, and therefore can’t have reasonable thoughts.”

  
  The small argument is only (2) from above: “I am BB, and therefore I can’t have coherent thoughts. “Only the small argument is self-undermining but the large one stands because there is no probability update in (1) horn.

  
  For example, I have a 10 per cent chance to believe that I am real a priory, based on theoretical physics. I know that BBs can’t update properly but it is not evidence on which I should update, as it is not news for me.

  
  Another example: I can be either awake or dreaming. During dreaming I can’t effectively decide, am I awake or dreaming and properly calculate the chances of being in a dream. Sometimes I have lucid dreams and sometimes I have very realistic dreams. If we follow Carroll’s logic, we should conclude that dreaming is impossible, because it is cognitively unstable, and it could be that I am dreaming now about commenting in LW. But in real life, we assume that dreaming takes, say, 0.3 time of my subjective experiences and we ignore any probability computations about dreaming that we do during dreaming.

• avturchin 17 Jul 2022 21:46 UTC

  2 points

  0 ∶ 0

  in reply to: TAG’s comment on: Cog­ni­tive In­sta­bil­ity, Phys­i­cal­ism, and Free Will

  Carroll argument resembles a typical proof by contradiction: let’s assume A, but it produces a contradiction, so A is false. However, in Carroll’s case, no matter what is happening with BB, there is no contradiction, so it is not a proof that I am not BB. This is even stronger in the case when it is applied to the simulation argument, as a sim can think coherently.

  But what Carroll actually said is that BB can’t know for sure that it is BB.

  But from my point of view, its all is compensated by the dust theory: for any BB1 there is a BB2 which looks like logical continuation of BB1. Like for any random integer N there is another integer N+1.

  Therefore, for any BB which sees absurdity, there is another BB which recognise it as absurdity. Thus in dust theory, chains of BBs can calculate probability that they are BB.

• avturchin 17 Jul 2022 18:05 UTC

  2 points

  0 ∶ 0

  in reply to: TAG’s comment on: Cog­ni­tive In­sta­bil­ity, Phys­i­cal­ism, and Free Will

  If I can’t perform any probability estimates, then I am Boltzmann brain. ))

• avturchin 16 Jul 2022 20:31 UTC

  5 points

  0 ∶ 0

  on: Cog­ni­tive In­sta­bil­ity, Phys­i­cal­ism, and Free Will

  I think that Carroll’s argument is wrong because it already assumes as true what it wants to disprove.

  His argument is the following claim: If I am BB, I can’t reason about physics and the probabilities of being BB. However, this claim is true only if I am BB. But if I am interested only in the question “Am I BB?”, it doesn’t matter how good I am thinking of being a BB or everything else, because the only thing which I am interested in is already assumed as true.

  The same logic was used in the attempts to disprove the simulation argument by Danila Medvedev (and maybe others). If I am in a simulation, I can’t have a perfect model of the distribution of simulations in the universe and—Danila suggests, – it disproves the simulation argument.

  Let’s elaborate: One of two statements is true:

  1. I am real and simulations are impossible.

  2. I am in simulation, but in that case, I can’t perform perfect probability estimations.

  The italic part doesn’t have any logical power over the whole argument.

• avturchin 16 Jul 2022 8:35 UTC

  8 points

  1 ∶ 0

  on: A time-in­var­i­ant ver­sion of Laplace’s rule

  It looks like that time-dependence in Laplace’s rule appears because of the “+1” part which is not measured in time units.

  The “+1” appeared in the original Laplace’s rule exactly because it was derived for predicting discrete events with known periodicity. That is, if the sun has risen today, we know that it will not rise next 24 hours, and the question is will it appear after that on the 25th hour. This is not true for earthquakes which could happen at any moment.

  This problem has been addressed in Gott’s equation which looks exactly like Laplace’s rule of succession but without “+1” and “+2″ parts, and is used to predict the future duration of continuous processes, like life expectancy. Gott’s equation gives the same result no matter if days or years are used.

  If S=1 your equation does look like Gott’s equation.

• avturchin 7 Jul 2022 13:39 UTC

  24 points

  8 ∶ 2

  on: Babysit­ting as Par­ent­ing Trial?

  Early childhood is extremely short: I will spend more time with my child as a teenager than with a toddler. Thus any lessons from babysitting toddlers are irrelevant to most of my future parenting experience. Moreover, result is more important than experiencing: raising a great person is more important than having pleasure during parenting.

  Parent also has to pay bills, has fears and has to solve medical problems and all these is not related to babysitting experience. Also, parent can’t go home after a working day, and most difficult parts of parenting are often at night, like a child doesn’t want to sleep.

• avturchin 1 Jul 2022 8:09 UTC

  9 points

  2 ∶ 0

  on: Cry­on­ics-ad­ja­cent question

  Kriorus used to take hard drives with digital data from their clients, and I put a disk there.
  
  But the best strategy is to have many disks in different places: at home, at relatives homes and at other peoples who are interested in digital immortality, as well as buried underground.
  
  Internet Archive also takes personal digital libraries for permanent storage, if they are open to public access.

• avturchin 30 Jun 2022 15:48 UTC

  2 points

  in reply to: Liron’s comment on: Op­ti­mal­ity is the tiger, and agents are its teeth

  A similar story here: Why Tool AIs Want to Be Agent AIs

• avturchin 30 Jun 2022 14:17 UTC

  2 points

  0 ∶ 0

  in reply to: Tristan Cook’s comment on: The table of differ­ent sam­pling as­sump­tions in anthropics

  I think that the problem (one of them) here is that my utility function may include some indexical preferences. Like “I want to be in simulation”. Or “I don’t want to be a Boltzmann brain”. In that case, I return to the need of updating, as I again have to take into account my indexicals.

  Also, it allows the existence of “utility monster”: that I should act as if I will have the biggest possible impact on the future of humanity, even if prior odds of that is small.

The table of differ­ent sam­pling as­sump­tions in anthropics

• avturchin 23 Jun 2022 20:48 UTC

  2 points

  on: Half-baked AI Safety ideas thread

  Philosophical landmines.

  In order to get our of the box, AI has to solve several smilingly innocent puzzles, which however, require a lot of computations, or put AI in (almost) infinite loop or create very strong ontological uncertainty. Or halt it.

  This is a type of the questions, to which the answer is 42.

  Weak examples: “what is the goal of AI’s goal,” “is model realism true?” and “are we in simulation?”

  Really good philosophical landmines should be kept in secret as they should not appear in training datasets.

• avturchin 23 Jun 2022 13:10 UTC

  19 points

  on: What’s the con­tin­gency plan if we get AGI to­mor­row?

  I would try to compromise the perception of AI capabilities, so the CEO will think that it is just a bad chatbot and will never run it again.

  ---

  I will call CNN and tell them that my AI becomes sentient but also right wing and full of racial slur. I will hand write some fake logs to prove that.

  CEO will see that tomorrow in the news and he will be afraid to run the AI again as it for sure will produce more right wing texts. CEO will conclude that the best PR would be to delete the model and close the whole direction which created it.

• avturchin 22 Jun 2022 13:56 UTC

  2 points

  on: av­turchin’s Shortform

  AI safety as Grey Goo in disguise.
  First, a rather obvious observation: while the Terminator movie pretends to display AI risk, it actually plays with fears of nuclear war – remember that explosion which destroys children’s playground?
  

  EY came to the realisation of AI risk after a period than he had worried more about grey goo (circa 1999) – unstoppable replication of nanorobots which will eat all biological matter, – as was revealed in a recent post about possible failures of EY’s predictions. While his focus moved from grey goo to AI, the description of the catastrophe has not changed: nanorobots will eat biological matter, however, now not just for replication but for production of paperclips. This grey goo legacy is still a part of EY narrative about AI risk as we see from his recent post about AI lethalities.

  However, if we remove the fear of grey goo, we could see that AI which experiences hard takeoff is less dangerous than a slower AI. If AI gets superintelligence and super capabilities from the start, the value of human atoms becomes minuscule, and AI may preserve humans as a bargain against other possible or future AIs. If AI ascending is slow, it has to compete with humans for a period of time and this could take a form of war. Humans have killed Neanderthals, but not ants.

• avturchin 18 Jun 2022 8:57 UTC

  3 points

  on: A pos­si­ble AI-in­oc­u­la­tion due to early “robot up­ris­ing”

  If AI uprising is technically possible, it means that AI military infrastructure already exists, like drones and robots, so next uprising is even simpler.

  Humanity pays much less attention now to monkeypox than it did to coronavirus, as everybody tired from pandemics.

• avturchin 18 Jun 2022 8:06 UTC

  5 points

  0 ∶ 0

  in reply to: shminux’s comment on: Pivotal out­comes and pivotal processes

  My idea was to use a first human upload as “AI tzar” who will control all other AI development via extensive surveillance system. First human upload as AI Nanny.

• avturchin 17 Jun 2022 9:01 UTC

  19 points

  in reply to: shminux’s comment on: Hu­mans are very re­li­able agents

  I read somewhere that pilots made like one error every 6 minutes—but they have time and ability to detect the errors and correct them.

  Quick OODA loop is very effective in detecting and eliminating errors and its is a core for human safety. But it requires environments which provides quick feedback for errors before fatal crash. Like “this car is too close, I will drive slowly”.

• avturchin 15 Jun 2022 20:33 UTC

  2 points

  in reply to: Yitz’s comment on: Has there been any work on at­tempt­ing to use Pas­cal’s Mug­ging to make an AGI be­have?

  It is in public domain, and I revised it last year, but as I am not a native speaker, I still makes mistakes((( If you can point on them as well as the ones philosophical ones, i would be very grateful.

• avturchin 15 Jun 2022 17:18 UTC

  10 points

  on: Has there been any work on at­tempt­ing to use Pas­cal’s Mug­ging to make an AGI be­have?

  I wrote rather long “Message to any future AI”, but Rolf Nelson is a pioneer. https://​​philpapers.org/​​rec/​​TURMTA

• avturchin 14 Jun 2022 6:00 UTC

  2 points

  on: Con­tra EY: Can AGI de­stroy us with­out trial & er­ror?

  Self-improving will also require a lot of trial and errors, like training variants of NN and testing agents in simulation, if AI doesn’t have perfect theory of intelligence and if it is not P=NP difficult task.

• avturchin 13 Jun 2022 12:47 UTC

  5 points

  on: An Alter­na­tive In­ter­pre­ta­tion of Physics

  Interestingly, physics in the 20 century moved in the direction of more and more observer-centered perspective: first, special and general relativity, then quantum mechanics and later anthropic principle.