Mainstream belief: Rational AI agents (situationally aware, optimizes decisions, etc.) are superior problem solvers, especially if they can logically motivate their reasoning.
Alternative possibility: Intuition, abstraction and polymathic guessing will outperform rational agents in achieving competing problem-solving outcomes. Holistic reasoning at scale will force-solve problems intractable by much more formal agents, or at least outcompete in speed/complexity.
2)
Mainstream belief: Non-sentient machines will eventually recursively self-improve to steer towards their goals
Alternative possibility (much lower but non-trivial p): Self-improvement coupled with goal persistence requires sophisticated, conscious self-awareness. No persistent self-improvement drive is likely to occur in non-sentient machines.
/
Mainstream beliefs are well-informed based on LLMs, but I think many AI people take them for granted at this point. Making belief into a premise is dangerous. This is uncharted territory and most humans-in-the-past appear silly and ignorant most of the time, after all, to most humans-in-the-future.
Two core beliefs about AI to question
Mainstream belief: Rational AI agents (situationally aware, optimizes decisions, etc.) are superior problem solvers, especially if they can logically motivate their reasoning.
Alternative possibility: Intuition, abstraction and polymathic guessing will outperform rational agents in achieving competing problem-solving outcomes. Holistic reasoning at scale will force-solve problems intractable by much more formal agents, or at least outcompete in speed/complexity.
2)
Mainstream belief: Non-sentient machines will eventually recursively self-improve to steer towards their goals
Alternative possibility (much lower but non-trivial p): Self-improvement coupled with goal persistence requires sophisticated, conscious self-awareness. No persistent self-improvement drive is likely to occur in non-sentient machines.
/
Mainstream beliefs are well-informed based on LLMs, but I think many AI people take them for granted at this point. Making belief into a premise is dangerous. This is uncharted territory and most humans-in-the-past appear silly and ignorant most of the time, after all, to most humans-in-the-future.