Logan Zoellner

[Question] Many Tur­ing Machines

• Logan Zoellner 15 Dec 2019 21:42 UTC

  1 point

  in reply to: interstice’s comment on: Many Tur­ing Machines

  I like the Mathematical Universe Hypothesis for simplicity and internal consistency, but it seems like we’re assuming a lot. And it’s not as simple as all that, either. Where do we draw the line? Only computable functions? The whole Turing hierarchy? Non-standard Turing Machines? If we draw the line at “anything logically conceivable”, I would worry that things like “a demon that can jump between different branches of the multiverse” ought to be popping into our reality all the time.

  If we want our theory to be predictive, we should probably cut it off at “anything computable exists”, but if predictability was our goal, why not go all the way back to “anything observable exists”?

• Logan Zoellner 15 Dec 2019 21:48 UTC

  1 point

  in reply to: Gordon Seidoh Worley’s comment on: Many Tur­ing Machines

  For it to be a formal claim would require us knowing more physics than we do such that we would know the true metaphysics of the universe.

  You are correct that I used Church-Turing as a shortcut to demonstrate my claim that MWH is computable. However, I am not aware of anyone seriously claiming quantum physics is non computable. People simulate quantum physics on computers all the time, although it is slow.

  
  

  I’m inclined to view your description as a strawman of MWI

  I don’t think it’s quite a strawman, since the point is that MTM is literally equivalent to MWH. In math saying “A is isomorphic to B, but B is easier to reason about” is something that is done all the time.

  
  

  but it’s also not an argument against MWI, only against MWI mattering to your purposes.

  Yes.

[Question] Why is the mail so much bet­ter than the DMV?

[Question] AI Box­ing for Hard­ware-bound agents (aka the China al­ign­ment prob­lem)

• Logan Zoellner 8 May 2020 21:49 UTC

  1 point

  in reply to: teradimich’s comment on: AI Box­ing for Hard­ware-bound agents (aka the China al­ign­ment prob­lem)

  Why are some so often convinced that the victory of China in the AGI race will lead to the end of humanity?

  I don’t think a Chinese world order will result in the end of humanity, but I do think it will make stuff like this much more common. I am interested in creating a future I would actually want to live in.

  The most prominent experts give a 50% chance of AI in 2099

  How much would you be willing to bet that AI will not exist in 2060, and at what odds?

  but I think that the probability of an existential disaster in this world will become less.

  Are you arguing that a victory for Chinese totalitarianism makes Human extinction less likely than a liberal world order?

• Logan Zoellner 8 May 2020 22:32 UTC

  1 point

  in reply to: teradimich’s comment on: AI Box­ing for Hard­ware-bound agents (aka the China al­ign­ment prob­lem)

  In any case, this does not seem to stop the Chinese people from feeling happier than the US people.

  Lots of happy people in China.

  And yes, I expect that in 2050 it will be possible to monitor the behavior of each person in countries ²⁴⁄₇. I can’t say that it makes me happy, but I think that the vast majority will put up with this. I don’t believe in a liberal democratic utopia, but the end of the world seems unlikely to me.

  Call me a crazy optimist, but I think we can aim higher than: Yes, you will be monitored ²⁴⁄₇, but at least humanity won’t literally go extinct.

• Logan Zoellner 9 May 2020 1:22 UTC

  3 points

  in reply to: Rohin Shah’s comment on: AI Box­ing for Hard­ware-bound agents (aka the China al­ign­ment prob­lem)

  Given that you emphasize hardware-bound agents: have you seen AI and Compute? A reasonably large fraction of the AI alignment community takes it quite seriously.

  This trend is going to run into Moore’s law as an upper ceiling very soon (within a year, the line will require a year of the world’s most powerful computer). What do you predict will happen then?

  
  

  “what are you doing if not encoding human values”

  Interested in the answer to this, and how much it looks like/​disagrees with my proposal: building free trade, respect for individual autonomy, and censorship resistance into the core infrastructure and social institutions our world runs on.

• Logan Zoellner 9 May 2020 1:34 UTC

  1 point

  in reply to: Donald Hobson’s comment on: AI Box­ing for Hard­ware-bound agents (aka the China al­ign­ment prob­lem)

  It modified its own algorithms to take better use of processor cache, bringing its speed from 500x human to 1000x human. It is making several publishable new results in AI research a day.

  I think we disagree on what Moof looks like. I envision the first human-level AI as also running at human-like speeds on a $10 million+ platform and then accelerating according to Moore’s law. This still results in pretty dramatic upheaval but over the course of years, not weeks. I also expect humans will be using some pretty powerful sub-human AIs, so it’s not like the AI gets a free boost just for being in software.

  Again, the reason why is I think the algorithms will be known well in advance and it will be a race between most of the major players to build hardware fast enough to emulate human-level intelligence. The more the first human-level AI results from a software innovation rather than a Manhatten-project style hardware effort, the more likely we will see Foom. If the first human-level AI runs on commodity hardware, or runs 500x faster than any human, we have already seen Foom.

• Logan Zoellner 9 May 2020 2:13 UTC

  1 point

  in reply to: dxu’s comment on: AI Box­ing for Hard­ware-bound agents (aka the China al­ign­ment prob­lem)

  In other words, you got the easy and useless part (“will it happen?”) right, and the difficult and important part (“when will it happen?”) wrong.

  “Will it happen?” isn’t vacuous or easy, generally speaking. I can think of lots of questions where I have no idea what the answer is, despite a “trend of ever increasing strength”. For example:

  Will Chess be solved?

  Will faster than light travel be solved?

  Will P=NP be solved?

  Will the hard problem of consciousness be solved?

  Will a Dyson sphere be constructed around Sol?

  Will anthropogenic climate change cause Earth’s temperature to rise by 4C?

  Will Earth’s population surpass 100 billion people?

  Will the African Rhinoceros go extinct?

  I feel obligated to point out that “predictions” of this caliber are the best you’ll ever be able to do if you insist on throwing out any information more specific and granular than “historically, these metrics seem to move consistently upward/​downward”.

  I’ve made specific statements about my beliefs for when Human-Level AI will be developed. If you disagree with these predictions, please state your own.

• Logan Zoellner 9 May 2020 13:56 UTC

  1 point

  in reply to: Donald Hobson’s comment on: AI Box­ing for Hard­ware-bound agents (aka the China al­ign­ment prob­lem)

  I really like this this response! We are thinking about some of the same math.

  Some minor quibbles, and again I think “years” not “weeks” is an appropriate time-frame for “first Human AI → AI surpasses all humans”

  Therefore, in a hardware limited situation, your AI will have been training for about 2 years. So if your AI takes 20 subjective years to train, it is running at 10x human speed. If the AI development process involved trying 100 variations and then picking the one that works best, then your AI can run at 1000x human speed.

  A three-year-old child does not take 20 subjective years to train. Even a 20-year-old adult human does not take 20 subjective years to train. We spend an awful lot of time sleeping, watching TV, etc. I doubt literally every second of that is mandatory for reaching the intelligence of an average adult human being.

  At the moment, current supercomputers seem to have around enough compute to simulate every synapse in a human brain with floating point arithmetic, in real time. (Based on 1014 synapses at 100 Hz, 1017 flops) I doubt using accurate serial floating point operations to simulate noisy analogue neurons, as arranged by evolution is anywhere near optimal.

  I think just the opposite. A synapse is not a FLOP. My estimate is closer to 10^19. Moreover most of the top slots in the TOP500 list are vanity projects by governments or used for stuff like simulating nuclear explosions.

  Although, to be fair, once this curve collides with Moore’s law, that 2nd objection will no longer be true.

• Logan Zoellner 9 May 2020 14:09 UTC

  1 point

  in reply to: Donald Hobson’s comment on: AI Box­ing for Hard­ware-bound agents (aka the China al­ign­ment prob­lem)

  A lot of the approaches to the “China alignment problem” rely on modifying the game theoretic position, given a fixed utility function. Ie having weapons and threatening to use them. This only works against an opponent to which your weapons pose a real threat. If, 20 years after the start of Moof, the AI’s can defend against all human weapons with ease, and can make any material goods using less raw materials and energy than the humans use, then the AI’s lack a strong reason to keep us around.

  If the AIs are a monolithic entity whose values are universally opposed to those of humans then, yes, we are doomed. But I don’t think this has to be the case. If the post-singularity world consists of an ecosystem of AIs whose mutually competing interests causes them to balance one-another and engage in positive sum games then humanity is preserved not because the AI fears us, but because that is the “norm of behavior” for agents in their society.

  Yes, it is scary to imagine a future where humans are no longer at the helm, but I think it is possible to build a future where our values are tolerated and allowed to continue to exist.

  By contrast, I am not optimistic about attempts to “extrapolate” human values to an AI capable of acts like turning the entire world into paperclips. Humans are greedy, superstitious and naive. Hopefully our AI descendants will be our better angels and build a world better than any that we can imagine.

• Logan Zoellner 9 May 2020 14:46 UTC

  1 point

  on: What does a pos­i­tive out­come with­out al­ign­ment look like?

  You clearly have some sort of grudge against or dislike of china. In the face of a pandemic, they want basically what we want, to stop it spreading and someone else to blame it on. Chinese people are not inherently evil.

  I certainly don’t think the Chinese are inherently evil. Rather I think that from the view of an American in the 1990′s a world dominated by a totalitarian China which engages in routine genocide and bans freedom of expression would be a “negative outcome to the rise of China”.

  This is a description of a Nash equilibria in human society. Their stability depends on humans having human values and capabilities.

  Yes. Exactly. We should be trying to find a Nash equilibrium in which humans are still alive (and ideally relatively free to pursue their values) after the singularity. I suspect such a Nash equilibrium involves multiple AIs competing with strong norms against violence and focus on positive-sum trades.

  But I don’t see why any of the Nash equilibria between superintelligences will be friendly to humans.

  This is precisely what we need to engineer! Unless your claim is that there is no Nash equilibrium in which humanity survives, which seems like a fairly hopeless standpoint to assume. If you are correct, we all die. If you are wrong, we abandon our only hope of survival.

  Why would one AI start shooting because the other AI did an action that benefited both equally?

  Consider deep seabed mining. I would estimate the percent of humans who seriously care (are are aware of the existence of) the sponges living at the bottom of the deep ocean at <1%. Moreover, there are substantial positive economic gains that could potentially be split among multiple nations from mining deep sea nodules. Nonetheless, every attempt to legalize deep sea mining has run unto a hopeless tangle of legal restrictions because most countries view blocking their rivals as more useful than actually mining the deep sea.

  If you have several AI’s and one of them cares about humans, it might bargain for human survival with the others. But that implies some human managed to do some amount of alignment.

  I would hope that some AIs have an interest in preserving humans for the same reason some humans care about protecting life on the deep seabed, but I don’t think this is a necessary condition for ensuring humanity’s survival in a post-singularity world. We should be trying to establish a Nash equilibrium in which even insignificant actors have their values and existence preserved.

  My point is, I’m not sure that aligned AI (in the narrow technical sense of coherently extrapolated values) is even a well-defined term. Nor do I think it is an outcome to the singularity we can easily engineer, since it requires us to both engineer such an AI and to make sure that it is the dominant AI in the post-singularity world.

• Logan Zoellner 10 May 2020 0:11 UTC

  1 point

  in reply to: Donald Hobson’s comment on: AI Box­ing for Hard­ware-bound agents (aka the China al­ign­ment prob­lem)

  You are still being stupid, because you are ignoring effective tools and making the problem needlessly harder for yourself.

  I think this is precisely where we disagree. I believe that we do not have effective tools for writing utility functions and we do have effective tools for designing at least one Nash Equilibrium that preserves human value, namely:

  1) All entities have the right to hold and express their own values freely

  2) All entities have the right to engage in positive-sum trades with other entities

  3) Violence is anathema.

  Some more about why I think humans are bad at writing utility functions:

  I am the extremely skeptical about anything of the form: We will define a utility function that encodes human values. Machine learning is really good at misinterpreting utility functions written by humans. I think this problem will only get worse with a super-intelligence AI.

  I am more optimistic about goals of the form “Learn to ask what humans want”. But I still think these will fail eventually. There are lots of questions even ardent utilitarians would have difficulty answering. For example, “Torture 1 person or give 3^^^3 people a slight headache?”.

  I’m not saying all efforts to design friendly AIs are pointless, or that we should willingly release paperclip maximizes on the world. Rather, I believe we boost our chances of preserving human existence and values by encouraging a multi-polar world with lots of competing (but non-violent) AIs. The competing plan of “don’t create AI until we have designed the perfect utility function and hope that our AI is the dominant one” seems like it has a much higher risk of failure, especially in a world where other people will also be developing AI.

  Importantly, we have the technology to deploy “build a world where people are mostly free and non-violent” today, and I don’t think we have the technology to “design a utility function that is robust against misinterpretation by a recursively improving AI”.

  
  

  One additional aside

  Suppose the AI has developed the tech to upload a human mind into a virtual paradise, and is deciding whether to do it or not.

  I must confess the goals of this post are more modest than this. The Nash equilibrium I described is one that preserves human existence and values as they are it does nothing in the domain of creating a virtual paradise where humans will enjoy infinite pleasure (and in fact actively avoids forcing this on people).

  I suspect some people will try to build AIs that grant them infinite pleasure, and I do not grudge them this (so long as they do so in a way that respects the rights of others to choose freely). Humans will fall into many camps. Those who just want to be left alone, those who wish to pursue knowledge, those who wish to enjoy paradise. I want to build a world where all of those groups can co-exist without wiping out one-another or being wiped out by a malevolent AI.

• Logan Zoellner 16 May 2020 15:47 UTC

  1 point

  on: The EMH Aten’t Dead

  I think you’re understating the amount of private information available to anyone with a reasonable level of intelligence. If you have a decent level of curiosity, chances are that you know some things that the rest of the world hasn’t “caught onto” yet. For example, most fans of Tesla probably realized that EVs are going to kill ICEs and that Telsa is at least 4 years ahead of anyone else in terms of building EVs long before the sudden rise in Tesla stock in Jan 2020. Similarly, people who nerd out about epidemics predicted the scale of COVID-19 before the general public.

  The extreme example of this is Venture Capital. People who are a bit “weird” and follow their hunches routinely start companies worth millions or billions of dollars. Every single one of them “beat the market” by tapping private information.

  None of this invalidates the EMH (which as you pointed out is unfalsifiable). The key is figuring out how to take your personal unique insights and translate them into meaningful investments (with reasonable amounts of leverage and appropriate stop-losses). Of course, the easier it is to trade something, the more likely someone has “already had that idea”, so predicting the S&P500 is harder than predicting an individual stock. But starting your own company is a power move so difficult that it’s virtually unbeatable.

• Logan Zoellner 13 Aug 2020 14:10 UTC

  1 point

  in reply to: marcogiglio’s comment on: The EMH Aten’t Dead

  Private information should be very hard to come by, it is not something that can be learned in a few minutes from an internet search.

  I think we have different definitions of private information.

  I have private information if I disagree with the substantial majority of people, even if everything I know is in principle freely available. The market is trading on the consensus expectation of the future. If that consensus is wrong and I know so, I have private information.

  Specifically, when Tesla was trading at $600 or so, it was publicly available that they were building cars in a way that no other company could, but the public consensus was not that they were therefore the most valuable car company in the world.

  Similarly, SpaceX is currently valued at $44B according to the public consensus. But I would be willing to be a substantial sum of money that they are worth 5-10x that and people just haven’t fully grasped the implications of Starlink and Starship.

  When you think about private information this way, in order to have private information all you have to do is:

  1) Disagree with the general consensus

  2) Be right

  Incidentally, those are precisely the skills that rationality is training you for. Most people aren’t optimizing for the truth, they’re optimizing for fitting in with their peers.

  
  

  To me it doesn’t look trivial/​nor easy at all: there are orders of magnitude more intelligence people than rich intelligence people.

  Very few intelligent people are optimizing for “make as much money as possible”. A trivial example of this, almost anyone working in academia could get a massive pay raise by switching to private industry. In addition, people can be very intelligent without being rational, so even if they claim to be optimizing for wealth they might not be doing a very good job of it. There are hordes of very intelligent people who are goldbugs or young earth creationists or global warming deniers. Why should we expect these people to behave rationally when it comes to financial self-interest when they so blatantly fail to do so in other domains?

  I’m not even sure I buy the idea that there are more intelligent people than rich people. The 90% percentile for wealth in the USA is north of $1M. Going by the “MENSA” definition of highly intelligent, only 2% of people qualify. That means there are 5x as many millionaires as geniuses.

• Logan Zoellner 13 Aug 2020 14:14 UTC

  1 point

  in reply to: ChristianKl’s comment on: The EMH Aten’t Dead

  Plenty of Venture Capitalist underform the market. Saying that everyone of them beats the market is not based on real data.

  I didn’t say every Venture Capitalist beats the market. Venture Capital in particular seems like a hobby for people who are already rich. I said every founder of a $1B startup beat the market.

  I propose the following bet: take any founder of a $1B startup that you please, strip them of all of their wealth, give them $1M cash. What percent of them do you think would see their net-worth grow by more than the S&P 500 over the next 10 years? If the EMH is true, the answer should be 50%. Would you really be willing to bet 50% of them will under preform the market?

• Logan Zoellner 14 Aug 2020 8:21 UTC

  1 point

  in reply to: ChristianKl’s comment on: The EMH Aten’t Dead

  No, you would only assume that if you bill the capacity of that founder to work at zero. Successful founders have skill at managing companies that distinct from having access to private information.

  Care to elucidate the difference between “skilled at managing companies” and “skilled at investing”. Do you really claim that if I restricted the same set of people to buying/​selling publicly tradable assets they would underperform the S&P 500?

• Logan Zoellner 17 Aug 2020 12:20 UTC

  1 point

  in reply to: ChristianKl’s comment on: The EMH Aten’t Dead

  I’m still curious if you would be willing to bet against a fund run exclusively by founders of the S&P500Those do underperform the S&P 500.

  Oh yeah, I definitely agree that mutual funds are terrible. Pretty sure they’re optimizing for management fees, though, not to actually outperform the market.

  
  

  I’m still curious if you would be willing to bet against a fund run exclusively by founders vs the S&P 500. Saying the management fee for such a fund would be ridiculously high seems like a reasonable objection though.

  For that matter, would you be willing to bet against SpaceX vs the S&P 500?

[Question] Liber­tar­i­anism, Ne­oliber­al­ism and Med­i­care for All?