Self modifying code has been possible but not practical for as long as we have had digital computers. Now it has toolchains, use cases, and in the near future tens to hundreds of people will do it as their day job.
The strong version of my claim is that I expect to see the same kinds of failure modes we are concerned with in AGI pushed down to the level of consumer-grade software, at least in huge applications like social networks and self-driving cars.
I think it is now simple and cheap enough for a single research group to do something like:
Write a formal specification
Which employs learning for some simple purpose
And employs self-modification on one or more levels
Which is to say, it feels like we have enough tooling to start doing “Hello World” grade self-modification tests that account for every level of the stack, in real systems.
Self modifying code has been possible but not practical for as long as we have had digital computers. Now it has toolchains, use cases, and in the near future tens to hundreds of people will do it as their day job.
The strong version of my claim is that I expect to see the same kinds of failure modes we are concerned with in AGI pushed down to the level of consumer-grade software, at least in huge applications like social networks and self-driving cars.
I think it is now simple and cheap enough for a single research group to do something like:
Write a formal specification
Which employs learning for some simple purpose
And employs self-modification on one or more levels
Which is to say, it feels like we have enough tooling to start doing “Hello World” grade self-modification tests that account for every level of the stack, in real systems.