Robert Kralisch(Robert Kralisch)

• Robert Kralisch 30 Apr 2024 0:10 UTC

  1 point

  0

  in reply to: Seth Herd’s comment on: The Prop-room and Stage Cog­ni­tive Architecture

  Thanks a lot for the encouragement :)

  Yes, I am trying to understand a generalized (which also means simplified) and formalizable parallel to human cognition. Some of my thinking on this is inspired by predictive coding and adaptive resonance theory (although prettly loosely by the latter), and I am trying to figure out the implications of our most updated understanding of neurobiological principles, together with a notion of the “riverbeds of cognition”.
  
  In other words, how can we design an architecture such that it is not pressured to take shortcuts or “work around” design decisions we made, as its cognition develops? Is there a “natural path” of cognitive development that avoids some of the common pitfalls and failure modes (i.e. can we aim inner alignment if we have proficiency in this area)?
  This has a direct bearing on interpretability, and goes together with the goal of a sort of “conceptual curriculum” that is intended to teach the system natural abstractions.

  
  If I remember correctly, the centrality of “constraint satisfaction” fell out of considering causal (hyper/​meta)graphs as sensible representational substrate (which was partially inspired by Ben Goertzel). I personally find it quite intuitive to think in graphs.

• Robert Kralisch 29 Apr 2024 23:49 UTC

  1 point

  0

  in reply to: cheer Poasting’s comment on: Refer­en­tial Containment

  Hm, I’ll give some thought to how to integrate different types of data with this picture, but I think that the “useful” classification of data ultimately depends on whether the agent possesses the right “key” to interpret it, and by extension, how difficult that “key” is to produce from concepts that the agent is already proficient with.
  
  At the end of the day, the agent can only “understand” any data in terms of internalized concepts, so there will often be some uncertainty whether the difficulty is in translating sensible data into that internal representation or the difficulty is in the data being about phenomena somewhat (or far) outside of conceptual bounds. Having a “key” with respect to some data means that this data can be reliably translated.

  
  Referential containment is about how to structure internal representations of concepts such as to make them most useful (which could mean flexible, or it could mean efficient, etc, depending on the problem domain and constraints).
  
  If humanity forgot all of its medical knowledge tomorrow, would we discover the same categories and sub-categories of medicine, structuring our knowledge and drawing the distinctions into the different areas of expertise similarly?
  We could gather a bunch of observational data about “the art of preventing people from dying” and find clusters in interventions or treatment strategies that have appropriate conceptual size to teach to different groups of people. Note that this changes depending on how many people we can allocate in total, how much we believe reliably fits into a single human mind, and how many common features there are in curative or preventative measures (commonalities here roughly referring to useful information that is referentially contained with respect to medicine, but can not be further contained in a single sub-field or small cluster thereof).
  
  Let me know if that makes sense.

• Robert Kralisch 29 Apr 2024 23:30 UTC

  3 points

  2

  in reply to: Nathan Helm-Burger’s comment on: Disen­tan­gling Com­pe­tence and Intelligence

  Yeah, I wish we had some cleaner terminology for that.
  Finetuning the “simulation engine” towards a particular task at hand (i.e. to find the best trade-off between breadth and depth search in strategy games, or even know how much “thinking time” or “error allowance” to allocate to a move), given limited cognitive resources, is something that I would associate with level 3 capability.
  It certainly seems like learning could go into the direction of making the model of the game more useful by either improving the extent to which this model predicts/​ouputs good moves or by improving the allocation of cognitive resources to the sub-tasks involved. Presumably, an intelligent system should be capable of testing which improvement vectors seem most fruitful (and the frequency with which to update this analysis), but I find myself a bit confused about whether that should count as level 3 or as level 4, since the system is reasoning about allocating resources across relevant learning processes.

• Robert Kralisch 29 Apr 2024 23:17 UTC

  1 point

  0

  in reply to: Seth Herd’s comment on: Disen­tan­gling Com­pe­tence and Intelligence

  Thanks!
  
  In your example, I think it is possible that the hunter-gatherer solves the problem through pure level 2 capability, even if they never encountered this specific problem before. Using causal models compositionally to represent the current scene, and computing it to output a novel solution, does not actually require that the human updates their causal models about the world.
  I am trying to distinguish agents with this sort of compositional world model from ones that just have a bunch of cashed thoughts or habits (which would correspond to level 1), and I think this is perhaps a common case where people would attribute intelligence to a system that imo does not demonstrate level 3 capability.
  
  Of course, this would require that the human in our example already has some sufficiently decontextualised notion of knocking loose objects down, or that generally their concepts are suited to this sort of compositional reasoning. It might be worth elaborating on level 2 to introduce some measure modeling flexibility/​compositionality.

  I feel like this could be explained better, so I am curious if you think I am being clear.
  
  You are probably right that I should avoid the term intelligence for the time being, but I haven’t quite found an alternative term that resonates. Anyways, thanks for engaging!

  Edit: I’ll soon make some changes to the post to better account for this feature of level 2 algorithms to potentially solve novel problems even if no new learning occurs. It’s an important aspect of why I am saying that level 3 capabilities are only indirectly related to competence.

The Prop-room and Stage Cog­ni­tive Architecture

How are Si­mu­la­tors and Agents re­lated?

Ex­tended Embodiment

Refer­en­tial Containment

Disen­tan­gling Com­pe­tence and Intelligence

• Robert Kralisch 24 Apr 2024 19:30 UTC

  2 points

  0

  on: Plan­ning in a Lat­tice Graph

  the maximum plan length is only $10\cdot 100=1000$ steps

  You mean the maximum length for an efficient/​minimal plan, right? Maybe good to clarify (even if obvious in this case). Just a thought.

Introduction

In­her­ently In­ter­pretable Architectures

Pos­i­tive Attractors

AISC 2023, Progress Re­port for March: Team In­ter­pretable Architectures

• Robert Kralisch 11 Feb 2022 12:24 UTC

  2 points

  on: Paradigm-build­ing: The hi­er­ar­chi­cal ques­tion framework

  I believe that it is very sensible to bring this sort of structure into our approach to AGI safety research, but at the same time it seems very clear that we should update that structure to the best of our ability as we make progress in understanding the challenges and potentials of different approaches.

  It is a feedback loop where we make each step according to our best theory of where to make it, and use the understanding gleaned from that step to update the theory (when necessary), which could well mean that we retrace some steps and recalibrate (this can be the case within and across questions). I think this connects to what both Charlie and Tekhne have said, though I believe Tekhne could have been more charitable.
  
  In this light, it makes sense to emphasize the openness of the theory to being updated in this way, which also qualifies the ways in which the theory is allowed to be yet incomplete. Putting more effort into clarifying how this update process should look like seems like a promising addition to the framework that you propose.
  
  On a more specific note I felt that Q5 could just be in position 2 and maybe a sixth question would be “What is the predicted timeline for stable safety/​control implementations?” or something of the sort.
  
  I also think that phrasing our research in terms of “avoiding bad outcomes” and “controlling the AGI” biases the way in which we pay attention to these problems. I am sure that you will also touch on this in the more detailed presentation of these questions, but at the resolution presented here, I would prefer the phrasing to be more open.
  “Aiming at good outcomes while/​and avoiding bad outcomes” captures more conceptual territory, while still allowing for the investigation to turn out that avoiding bad outcomes is more difficult and should be prioritised. This extends to the meta-question of whether existential risk can be best adressed by focusing on avoiding bad outcomes, rather than developing a strategy to get to good outcomes (which are often characterised by a better abilitiy to deal with future risks) and avoid bad outcomes on the way there. It might rightfully appear that this is a more ambitious aim, but it is the less predisposed outlook! Many strategy games are based on the idea that you have to accumulate resources and avoid losses while at the same time improving your ability to accumulate resources and avoid losses in the future. Only focusing on the first aspect is a specific strategy in the space of possible ones, and often employed when one is close to losing. This isn’t a perfect analogy in a number of ways, but serves to point out the more general outlook.
  Similarly, we expect a superintelligent AGI to be out of our ability to control at some point, which invokes notions of “self-control” on part of the AGI or “justified trust” on our part—therefore, perhaps “influencing the development of the AGI” would be better, as, again, “influence” can cover more conceptual ground but can still be hardened into the more specific notion of “control” when appropriate.

Com­men­tary on “AGI Safety From First Prin­ci­ples by Richard Ngo, Septem­ber 2020”