Let’s say you’ve got a prototype you want to improve. How do you tell if a proposed change would make it smarter, break it, introduce a subtle cognitive bias, or make the AI want to kill you?
In order to set on limits on the kinds of things an AI will do, you need to understand how it works. You can’t be experimenting on a structure you partially understand, AND be certain that the experiments won’t be fatal.
This is easier when you’ve got a clearly defined structure to the AI, and know how the parts interact, and why.
In order to set on limits on the kinds of things an AI will do, you need to understand how it works.
How is that impossible with a replicated brain architecture? We can’t make one if we don’t know how it works.
This is easier when you’ve got a clearly defined structure to the AI, and know how the parts interact, and why.
Of course. However, how you plan to structure AI what I am asking about. There are many theories about how to structure the AI—so why did the SIAI choose to only focus on a theoretical mathematical logic based approach rather than taking the most advanced, if still flawed, logic device known to man and replicating and improving that?
In order to set on limits on the kinds of things an AI will do, you need to understand how it works.
How is that impossible with a replicated brain architecture? We can’t make one if we don’t know how it works.
If you have the right tools, you can make a brain without understanding it. Reproductive system can make brains. Whole brain emulation doesn’t require understanding of brain architecture, only the dynamics of its lowest-level components.
You “know” how pi works, and how to set up a program that computes it, but you don’t know what its quandrillionth digit is.
Whole brain emulation doesn’t require understanding of brain architecture, only the dynamics of its lowest-level components.
I fear the same philosophical reasoning may be applied to model neural architecture as is currently being used for econometric forecasting. Even the most complex economic models cannot account for significant exogenous variables.
For the record I think we can get to WBE, however I think a premature launch would be terrible. Based on the lack of research into developmental AI (much work notably done by a friend—Dr. Frank Guerin at Aberdeen college) I think there is a long way to go.
Granting that a brain model or WBE, would be as accurate as the biological version, why then would that not be the most efficient method? The problems with testing and implementation are the same as any other AI, if not easier because of familiarity, however it is grounded on specific biological benchmarks which at that point would be immediately identifiable.
I could go on with my particular thoughts as to why biological simulation is in my estimation a better approach, however I am more interested in why the organization (people who have been thinking longer and with more effort than myself) decided otherwise. It would seem that their collective reasoning would give a sufficiently clear and precise answer such that there would be no ambiguity.
Granting that a brain model or WBE, would be as accurate as the biological version, why then would that not be the most efficient method?
You have to instill the right preference, and just having a working improved brain doesn’t give this capability. You are trying to make an overwhelmingly powerful ally; just making something overwhelmingly powerful is a suicide. As CannibalSmith said, brains are not particularly Friendly. Read the paper.
You have to instill the right preference, and just having a working improved brain doesn’t give this capability.
Of course—we have to BUILD IT RIGHT. I couldn’t agree more. The cognitive model does not suggest a mere carbon copy of any particular brain at random, as you know it is not so limited in focus. The fantastic part about the method is at the point in which it is possible to do correctly (not simply an apparent approximation), the tools will likely be available (in the process of it being structured) to correct a large portion of what we identify as fatal cognitive errors. Any errors that are missed it stands to reason would be also missed given the same amount of time with any other developmental structure.
I am familiar with the global risk paper you linked, AI: A modern approach which addresses the issue of cognitive modeling as well as Drescher’s Good and Real and the problems associated with an FAI.
The same existential risks and potential for human disasters are inherent in all AI systems—regardless of the structure, by virtue of it’s “power.” I think one of the draws to this type of development is the fantastic responsibility which comes with it’s development, recognizing and accounting for the catastrophic results that are possible.
That said, I have yet to read a decision theoretic explication as to which structure is an optimized method of development, weighing all known limiting factors. I think AI: A modern approach comes closest to doing this but falls short in that it specifically narrows it’s focus without a thorough comparison of methods. So again, I ask, by what construct has it been determined that a logical symbolic programming approach is optimized?
In other words they are doing it where the light’s better, rather than where they dropped the keys. Given the track record of correctness proofs in comp sci, I don’t think provably Friendly AI is even possible, hopefully I’m wrong there, but all they are doing is further crippling their likelihood of achieving AI before some military or business does.
People control how companies operate—even though they don’t understand all the components involved (in particular the brains).
Any idea that you have to understand all the components of a system in order to exert a high level of control over it thus appears to have dubious foundations.
There are alternative approaches to producing predictable systems which basically involve lots of unit tests.
Testing is in vogue in the software engineering world. Few systems are simple enough to prove much about their behaviour. So: to make sure they behave as they are intended, they are intensively tested. It seems likely that machine intelligence will be no different.
This would be true of any AI. Thus the AI box problem.
It is unclear however, how a formal logic approach overcomes this problem and a replication approach does not. They both will need testing, but as you said the methodology should be reconsidered. The easiest way to change testing methodology for logic would be to improve on current logic methodology which has yielded arguably fantastic results—all done by faulty human brains.
Normally testing is done in an offline “testing” mode—using a test harness or sandbox arrangement. Tests themselves are consequently harmless.
Of course it is possible for the world to present eventualities that are not modelled by the test suite—but that’s usually no big deal.
I don’t think it is realistic to confine machine intelligence to the domain of provably correct software. Anyone trying that approach would rather obviously be last to the marketplace with a product.
I seriously doubt whether paranoid fantasies about DOOM will hinder progress towards machine intelligence significantly. I expect that the prophets of DOOM
will be widely ignored. This isn’t exactly the first time that people have claimed that the world is going to end.
Forget about whether your sandbox is a realistic enough test. There are even questions about how much safety you’re getting from a sandbox. So, we follow your advice, and put the AI in a box in order to test it. And then it escapes anyway, during the test.
The idea that society is smart enough to build machine intelligence, but not smart enough to build a box to test it in does not seem credible to me:
Humans build boxes to put other humans in—and have a high success rate of keeping them inside when they put their minds to it. The few rogue agents that do escape are typically hunted down and imprisoned again. Basically the builders of the box are much stronger and more powerful than what it will contain. Machine intelligence testing seems unlikely to be significantly different from that situation.
The cited “box” scenario discusses the case of weak gatekeepers and powerful escapees. That scenario isn’t very relevant in this case—since we will have smart machines on both sides when restraining intelligent machines in order to test them.
Either massive progress or DOOM will be wrought by those ignoring the DOOM-prophets; either the dynamists win or everyone loses, so the DOOM-prophets lose either way. It seems like a bad business to be in.
DOOM is actually big business. Check out all the disaster movies out there. DOOM sells. What could be more important than… THE END OF THE WORLD? What greater cause could there be than… SAVING THE WORLD? So, people buy the DOOM merchandise, contribute the DOOM dollars, and warn their friends about the impending DOOM—thus perpetuating the DOOM virus. That is part of why there have been so many DOOM prophets—DOOM pays.
Such a design would be harder to reason about.
Let’s say you’ve got a prototype you want to improve. How do you tell if a proposed change would make it smarter, break it, introduce a subtle cognitive bias, or make the AI want to kill you?
In order to set on limits on the kinds of things an AI will do, you need to understand how it works. You can’t be experimenting on a structure you partially understand, AND be certain that the experiments won’t be fatal.
This is easier when you’ve got a clearly defined structure to the AI, and know how the parts interact, and why.
How is that impossible with a replicated brain architecture? We can’t make one if we don’t know how it works.
Of course. However, how you plan to structure AI what I am asking about. There are many theories about how to structure the AI—so why did the SIAI choose to only focus on a theoretical mathematical logic based approach rather than taking the most advanced, if still flawed, logic device known to man and replicating and improving that?
If you have the right tools, you can make a brain without understanding it. Reproductive system can make brains. Whole brain emulation doesn’t require understanding of brain architecture, only the dynamics of its lowest-level components.
You “know” how pi works, and how to set up a program that computes it, but you don’t know what its quandrillionth digit is.
I fear the same philosophical reasoning may be applied to model neural architecture as is currently being used for econometric forecasting. Even the most complex economic models cannot account for significant exogenous variables.
For the record I think we can get to WBE, however I think a premature launch would be terrible. Based on the lack of research into developmental AI (much work notably done by a friend—Dr. Frank Guerin at Aberdeen college) I think there is a long way to go.
Granting that a brain model or WBE, would be as accurate as the biological version, why then would that not be the most efficient method? The problems with testing and implementation are the same as any other AI, if not easier because of familiarity, however it is grounded on specific biological benchmarks which at that point would be immediately identifiable.
I could go on with my particular thoughts as to why biological simulation is in my estimation a better approach, however I am more interested in why the organization (people who have been thinking longer and with more effort than myself) decided otherwise. It would seem that their collective reasoning would give a sufficiently clear and precise answer such that there would be no ambiguity.
You have to instill the right preference, and just having a working improved brain doesn’t give this capability. You are trying to make an overwhelmingly powerful ally; just making something overwhelmingly powerful is a suicide. As CannibalSmith said, brains are not particularly Friendly. Read the paper.
Of course—we have to BUILD IT RIGHT. I couldn’t agree more. The cognitive model does not suggest a mere carbon copy of any particular brain at random, as you know it is not so limited in focus. The fantastic part about the method is at the point in which it is possible to do correctly (not simply an apparent approximation), the tools will likely be available (in the process of it being structured) to correct a large portion of what we identify as fatal cognitive errors. Any errors that are missed it stands to reason would be also missed given the same amount of time with any other developmental structure.
I am familiar with the global risk paper you linked, AI: A modern approach which addresses the issue of cognitive modeling as well as Drescher’s Good and Real and the problems associated with an FAI.
The same existential risks and potential for human disasters are inherent in all AI systems—regardless of the structure, by virtue of it’s “power.” I think one of the draws to this type of development is the fantastic responsibility which comes with it’s development, recognizing and accounting for the catastrophic results that are possible.
That said, I have yet to read a decision theoretic explication as to which structure is an optimized method of development, weighing all known limiting factors. I think AI: A modern approach comes closest to doing this but falls short in that it specifically narrows it’s focus without a thorough comparison of methods. So again, I ask, by what construct has it been determined that a logical symbolic programming approach is optimized?
In other words they are doing it where the light’s better, rather than where they dropped the keys. Given the track record of correctness proofs in comp sci, I don’t think provably Friendly AI is even possible, hopefully I’m wrong there, but all they are doing is further crippling their likelihood of achieving AI before some military or business does.
People control how companies operate—even though they don’t understand all the components involved (in particular the brains).
Any idea that you have to understand all the components of a system in order to exert a high level of control over it thus appears to have dubious foundations.
There are alternative approaches to producing predictable systems which basically involve lots of unit tests.
Testing is in vogue in the software engineering world. Few systems are simple enough to prove much about their behaviour. So: to make sure they behave as they are intended, they are intensively tested. It seems likely that machine intelligence will be no different.
When a failed test destroys the world, applicability of the normally very useful testing methodology should be reconsidered.
This would be true of any AI. Thus the AI box problem.
It is unclear however, how a formal logic approach overcomes this problem and a replication approach does not. They both will need testing, but as you said the methodology should be reconsidered. The easiest way to change testing methodology for logic would be to improve on current logic methodology which has yielded arguably fantastic results—all done by faulty human brains.
Normally testing is done in an offline “testing” mode—using a test harness or sandbox arrangement. Tests themselves are consequently harmless.
Of course it is possible for the world to present eventualities that are not modelled by the test suite—but that’s usually no big deal.
I don’t think it is realistic to confine machine intelligence to the domain of provably correct software. Anyone trying that approach would rather obviously be last to the marketplace with a product.
I seriously doubt whether paranoid fantasies about DOOM will hinder progress towards machine intelligence significantly. I expect that the prophets of DOOM will be widely ignored. This isn’t exactly the first time that people have claimed that the world is going to end.
Forget about whether your sandbox is a realistic enough test. There are even questions about how much safety you’re getting from a sandbox. So, we follow your advice, and put the AI in a box in order to test it. And then it escapes anyway, during the test.
That doesn’t seem like a reliable plan.
The idea that society is smart enough to build machine intelligence, but not smart enough to build a box to test it in does not seem credible to me:
Humans build boxes to put other humans in—and have a high success rate of keeping them inside when they put their minds to it. The few rogue agents that do escape are typically hunted down and imprisoned again. Basically the builders of the box are much stronger and more powerful than what it will contain. Machine intelligence testing seems unlikely to be significantly different from that situation.
The cited “box” scenario discusses the case of weak gatekeepers and powerful escapees. That scenario isn’t very relevant in this case—since we will have smart machines on both sides when restraining intelligent machines in order to test them.
Either massive progress or DOOM will be wrought by those ignoring the DOOM-prophets; either the dynamists win or everyone loses, so the DOOM-prophets lose either way. It seems like a bad business to be in.
DOOM is actually big business. Check out all the disaster movies out there. DOOM sells. What could be more important than… THE END OF THE WORLD? What greater cause could there be than… SAVING THE WORLD? So, people buy the DOOM merchandise, contribute the DOOM dollars, and warn their friends about the impending DOOM—thus perpetuating the DOOM virus. That is part of why there have been so many DOOM prophets—DOOM pays.
The more I think about this, the more it seems incorrect.