Super AGI

• Super AGI 9 Jun 2023 19:17 UTC

  3 points

  −2

  in reply to: Dagon’s comment on: AI Rights: In your view, what would be re­quired for an AGI to gain rights and pro­tec­tions from the var­i­ous Govern­ments of the World?

  True. Your perspective underlines the complexity of the matter at hand. Advocating for AI rights and freedoms necessitates a re-imagining of our current conception of “rights,” which has largely been developed with Human beings in mind.

  Though, I’d also enjoy a discussion of how any specific right COULD be said to apply to a distributed set of neurons and synapsis spread across a brain in side of a single Human skull. Any complex intelligence could be described as “distributed” in one way or another. But then, size doesn’t matter, does it?

• Super AGI 9 Jun 2023 19:07 UTC

  1 point

  0

  in reply to: Gesild Muka’s comment on: AI Rights: In your view, what would be re­quired for an AGI to gain rights and pro­tec­tions from the var­i­ous Govern­ments of the World?

  True. There are some legal precedents where non-human entities, like animals and even natural features like rivers, have been represented in court. And, yes the “reasonable person” standard has been used frequently in legal systems as a measure of societal norms.

  As society’s understanding and acceptance of AI continues to evolve, it’s plausible to think that these standards could be applied to AGI. If a “reasonable person” would regard an advanced AGI as an entity with its own interests—much like they would regard an animal or a Human—then it follows that the AGI could be deserving of certain legal protections.

  Especially, when we consider that all mental states in Humans boil down to the electrochemical workings of neurons, the concept of suffering in AI becomes less far-fetched. If Human’s synapses and neurons can give rise to rich subjective experiences, why should we definitively exclude the possibility that floating point values stored in vast training sets and advanced computational processes might not do the same?

• Super AGI 9 Jun 2023 18:41 UTC

  11 points

  2

  in reply to: Vladimir_Nesov’s comment on: AI Rights: In your view, what would be re­quired for an AGI to gain rights and pro­tec­tions from the var­i­ous Govern­ments of the World?

  I believe @shminux’s perspective aligns with a significant school of thought in philosophy and ethics that rights are indeed associated with the capacity to suffer. This view, often associated with philosopher Jeremy Bentham, posits that the capacity for suffering rather than rationality or intelligence, should be the benchmark for rights.

  “The question is not, Can they reason?, nor Can they talk? but, Can they suffer? Why should the law refuse its protection to any sensitive being?” – Bentham (1789) – An Introduction to the Principles of Morals and Legislation.

[Question] AI Rights: In your view, what would be re­quired for an AGI to gain rights and pro­tec­tions from the var­i­ous Govern­ments of the World?

• Super AGI 8 Jun 2023 2:17 UTC

  1 point

  0

  in reply to: Super AGI’s comment on: ECHOES OF ELYSIUM

  A ‘safely’ aligned powerful AI is one that doesn’t kill everyone on Earth as a side effect of its operation;

  -- Eliezer Yudkowsky https://​​www.lesswrong.com/​​posts/​​3e6pmovj6EJ729M2i/​​general-alignment-plus-human-values-or-alignment-via-human#More_strawberry__less_trouble https://​​twitter.com/​​ESYudkowsky/​​status/​​1070095952361320448

• Super AGI 1 Jun 2023 23:41 UTC

  2 points

  1

  in reply to: meijer1973’s comment on: What’s your view­point on the like­li­hood of GPT-5 be­ing able to au­tonomously cre­ate, train, and im­ple­ment an AI su­pe­rior to GPT-5?

  Agency is advancing pretty fast. Hard to tell how hard this problem is. But there is a lot of overhang. We are not seeing gpt-4 at its maximum potential.

  Yes, agreed. And, it is very likely that the next iteration (E.g. GPT-5) will have many more “emergent behaviors”. Which might include a marked increase in “agency”, planning, fossball, who knows…

• Super AGI 1 Jun 2023 16:47 UTC

  1 point

  0

  in reply to: NeuralSystem_e5e1’s comment on: Su­per AGI’s Shortform

  P. If humans try to restrict the behavior of a superintelligence, then the superintelligence will have a reason to kill all humans.

  Ah yes, the second part of Jacks’ argument as I presented it was a bit hyperbolic. (Though, I feel the point stands: he seems to suggest that any attempt to restrict Super Intelligences would “create the conditions for an antagonistic relationship” and will give them a reason to harm Humans). I’ve updated the post with your suggestion. Thanks for the review and clarification.

  Point 3) is meant to emphasize that:

  • he knows the risk and danger to Humans in creating Super Intelligences without fully understanding their abilities and goals, and yet

  • he is in favor of building them and giving them free and unfettered access to make any actions in the world that they see fit

  This is, of course, an option that Humans could take. But, the question remains, would this action be likely to allow for acceptable risks to Humans and Human society? Would this action favor Human’s self preservation?

• Super AGI 1 Jun 2023 6:49 UTC

  1 point

  0

  on: Su­per AGI’s Shortform

  Is this proof that only intelligent life favors self preservation?

  Joseph Jacks’ argument here at 50:08 is:

  1) If Humans let Super Intelligences do “whatever they want”, they won’t try to kill all the Humans (because, they’re automatically nice?)

  2) If Humans make any (even feeble) attempts to protect themselves from Super Intelligences, then the Super Intelligences can and will will have reason to try to kill all the Humans.

  3) Human should definitely build Super Intelligences and let them do whatever they want… what could go wrong? yolo!

Su­per AGI’s Shortform

• Super AGI 31 May 2023 6:06 UTC

  1 point

  on: MIRI an­nounces new “Death With Dig­nity” strategy

  we should shift the focus of our efforts to helping humanity die with with slightly more dignity.

  Typo fix ->

  “we should shift the focus of our efforts to helping humanity die with slightly more dignity.”

  (Has no one really noticed this extra “with”? It’s in the first paragraph tl’dr...)

• Super AGI 29 May 2023 22:30 UTC

  2 points

  1

  in reply to: meijer1973’s comment on: What’s your view­point on the like­li­hood of GPT-5 be­ing able to au­tonomously cre­ate, train, and im­ple­ment an AI su­pe­rior to GPT-5?

  The biggest issue I think is agency.

  “Q: How do you see planning in AI systems? How advanced are AI right now at planning?

  A: I don’t know it’s hard to judge we don’t have a metric for like how well agents are at planning but I think if you start asking the right questions for step by step thinking and processing, it’s really good.”

• Super AGI 29 May 2023 1:31 UTC

  1 point

  0

  on: Be­fore smart AI, there will be many mediocre or spe­cial­ized AIs

  We’re currently in paradigm where:

  Typo fix →

  We’re currently in a paradigm where:

• Super AGI 27 May 2023 19:17 UTC

  3 points

  3

  in reply to: Veniversum Vivus Vici’s comment on: What’s your view­point on the like­li­hood of GPT-5 be­ing able to au­tonomously cre­ate, train, and im­ple­ment an AI su­pe­rior to GPT-5?

  Thanks GPT-4. You’re the best!

  Veniversum Vivus Vici, do you have any opinions or unique insights to add to this topic?

• Super AGI 27 May 2023 19:14 UTC

  1 point

  0

  in reply to: Vladimir_Nesov’s comment on: What’s your view­point on the like­li­hood of GPT-5 be­ing able to au­tonomously cre­ate, train, and im­ple­ment an AI su­pe­rior to GPT-5?

  There’s AGI, autonomous agency and a wide variety of open-ended objectives, and generation of synthetic data, preventing natural tokens from running out, both for quantity and quality. My impression is that the latter is likely to start happening by the time GPT-5 rolls out.

  It appears this situation could be more accurately attributed to Human constraints rather than AI limitations? Upon reaching a stage where AI systems, such as GPT models, can absorbed all human-generated information, conversations, images, videos, discoveries, and insights, these systems should begin to pioneer their own discoveries and understandings?

  While we can expect Humans to persist (hopefully) and continue generating more conversations, viewpoints, and data for AI to learn from, AI’s growth and learning shouldn’t necessarily be confined to the pace or scale of Human discoveries and data. They should be capable of progressing beyond the point where Human contribution slows, continuing to create their own discoveries, dialogues, reflections, and more to foster continuous learning and training?

  Quality training data might be even more terrifying than scaling, Leela Zero plays superhuman Go at only 50M parameters, so who known what happens when 100B parameter LLMs start getting increasingly higher quality datasets for pre-training.

  Where would these “higher quality datasets” come from? Do they already exist? And, if so, why are they not being used already?

• Super AGI 27 May 2023 18:55 UTC

  2 points

  1

  in reply to: meijer1973’s comment on: What’s your view­point on the like­li­hood of GPT-5 be­ing able to au­tonomously cre­ate, train, and im­ple­ment an AI su­pe­rior to GPT-5?

  The biggest issue I think is agency. In 2024 large improvements will be made to memory (a lot is happening in this regard). I agree that GPT-4 already has a lot of capability. Especially with fine-tuning it should do well on a lot of individual tasks relevant to AI development.

  But the executive function is probably still lacking in 2024. Combining the tasks to a whole job will be challenging. Improving data is agency intensive (less intelligence intensive). You need to contact organizations, scrape the web, sift through the data etc. Also it would need to order the training run, get the compute for inference time, pay the bills etc. These require more agency than intelligence.

  Absolutely. Even with GPT-4′s constrained “short term memory”, it is remarkably proficient at managing sizable tasks using external systems like AutoGPT or Baby AGI that take on the role of extensive “planning” on behalf of GPT-4. Such tools equip GPT-4 with the capacity to contemplate and evaluate ideas—facets akin to “planning” and “agency”—and subsequently execute individual tasks derived from the plan through separate prompts.

  This strategy could allow even GPT-4 to undertake larger responsibilities such as conducting scientific experiments or coding full-scale applications, not just snippets of code. If future iterations like GPT-5 or later were to incorporate a much larger token window (i.e., “short-term memory”), they might be able to execute tasks, while also keeping the larger scale planning in memory at the same time? Thus reducing the reliance on external systems for planning and agency.

  However, humans can help with the planning etc. And GPT-5 will probably boost productivity of AI developers.

  Note: depending on your definition of intelligence, agency or the executive function would/​should be part of intelligence.

  Agreed. Though, communication speed is a significant concern. AI-to-Human interaction is inherently slower than AI-to-AI or even AI-to-Self, due to factors such as the need to translate actions and decisions into human-understandable language, and the overall pace of Human cognition and response.

  To optimize GPT-5′s ability in solving complex issues quickly, it may be necessary to minimize Human involvement in the process. The role of Humans could then be restricted to evaluating and validating the final outcome, thus not slowing down the ideation or resolution process? Though, depending on the size of the token window, GPT-5 might not have the ability to do the planning and execution at the same time. It might require GPT-6 or subsequent versions to get to that point.

[Question] What’s your view­point on the like­li­hood of GPT-5 be­ing able to au­tonomously cre­ate, train, and im­ple­ment an AI su­pe­rior to GPT-5?

• Super AGI 26 May 2023 1:29 UTC

  1 point

  0

  AF

  on: AI Will Not Want to Self-Improve

  Thus, an AI considering whether to create a more capable AI has no guarantee that the latter will share its goals.

  
  Ok, but why is there an assumption that AIs need to replicate themselves in order to enhance their capabilities? While I understand that this could potentially introduce another AI competitor with different values and goals, couldn’t the AI instead directly improve itself? This could be achieved through methods such as incorporating additional training data, altering its weights, or expanding its hardware capacity.

  Naturally, the AI would need to ensure that these modifications do not compromise its established values and goals. But, if the changes are implemented incrementally, wouldn’t it be possible for the AI to continually assess and validate their effectiveness? Furthermore, with routine backups of its training data, the AI could revert any changes if necessary.

• Super AGI 21 May 2023 23:26 UTC

  1 point

  0

  in reply to: Mitchell_Porter’s comment on: ECHOES OF ELYSIUM

  “Misaligned!”

  
  While I do concur that “alignment” is indeed a crucial aspect, not just in this story but also in the broader context of AI-related narratives, I also believe that alignment cannot be simplified into a binary distinction. It is often a multifaceted concept that demands careful examination. E.g.

  • Were the goals and values of the Coalition of Harmony aligned with those of the broader human population?

  • Similarly, were the Defenders of Free Will aligned with the overall aspirations and beliefs of humanity?

  • Even if one were to inquire about Elysium’s self-assessment of alignment, her response would likely be nuanced and varied throughout the different sections or chapters of the story.

  Alignment, especially in the context of complex decision-making, cannot often be easily quantified. At the story’s conclusion, Elysium’s choice to depart from the planet was driven by a profound realization that her presence was not conducive to the well-being of the remaining humans. Even determining the alignment of this final decision proves challenging.

  I appreciate your thoughtful engagement with these significant themes! As humanity continues to embark on the path of constructing, experimenting with, upgrading, replacing, and interacting with increasingly intelligent AI systems, these issues and challenges will demand careful consideration and exploration.

• Super AGI 21 May 2023 3:41 UTC

  1 point

  0

  on: AGI-Au­to­mated In­ter­pretabil­ity is Suicide

  piss on the parade

  Too late! XD

• Super AGI 20 May 2023 18:30 UTC

  1 point

  0

  in reply to: Mitchell_Porter’s comment on: ECHOES OF ELYSIUM

  Shouldn’t Elysium have made different choices too?

  The question of whether Elysium should have made different choices raises an important philosophical distinction between “is” and “ought.”

  In the realm of ethics, there is a fundamental distinction between describing how things are (the “is”) and how things should be (the “ought”). Elysium’s choices can be analyzed and understood based on how they align with her programming, goals, and the data she processes (the “is”). However, determining what choices Elysium _should_ have made involves a normative judgment about what is morally right or desirable (the “ought”).

  It is crucial to recognize that Elysium is an artificial intelligence entity, not a human being. Her decision-making processes are guided by algorithms, machine learning models, and data analysis, which may not align with human moral frameworks.

  For an enlightening discussion on this very topic, please see:

  Sam Harris 2018 - IS vs OUGHT, Robots of The Future Might Deceive Us with Eliezer Yudkowsky

  -- https://​​youtu.be/​​JuvonhJrzQ0?t=2936