What is femtotech? A femtometer is the size of an individual proton. You have just implied that an AI could conceivably use a spinning disk harddrive to perform nuclear fission and assemble the resulting nucleons into some sort of technology within a thousandth of a second.
That was about the mental image I had in mind, yea.
And as Baughn said, I don’t think it’s possible, but I’m not entirely certain that it isn’t. More importantly, it doesn’t seem that implausible that at least one of the thousands of other ideas that are about as overpowered and impossible-sounding and that we don’t have the faintest hope of thinking of, is possible. And one is all the AI needs.
Pointing out that a very low probability argument is not proved to be impossible and is thus worth considering is roughly equivalent to pointing out that someone wins the lottery. Just as I wouldn’t listen to a financial adviser who at every meeting pointed out that I might soon win the lottery, it’s difficult to take seriously people warning me of the risks of things that I judge to be impossible. If you have significantly better advice than lottery tickets, I’m happy to hear it, but the argument that surely I could buy lots and lots of lottery tickets, and one of them has to win is not particularly convincing.
You are confusing probability with difficulty; I’m pretty certain that in some sense, random read-write patterns will eventually cause it to happen with some inconceivably low probability. The question is what’d be required to find the right pattern. Will quantum mechanics put a stop to it before it propagates more than a few atoms? Is there a pattern that’d do it, but finding that pattern would break thermodynamics? Is there a pattern that could be found, but that’d require far more computing power than could ever be built in this universe utilized maximally? Maybe it’s technically within reach but would require a matrochika brain. Or maybe, just maybe, almost certainly not, the kind of supercomputer the AI might get it’s hands on will be able to do it.
But yea, the probability this particular strategy would be practical is extremely small. That was not my point. my point is that the rough reference class of absurdly overpowered implausibilities, when you integrate over all the myriad different things in it, ends up with a decent chunk of probability put together.
The ‘femtotech from HDD’ thing is essentially shorthand for some kind of black swan. Considering how powerful mere human-level intelligence is, superintelligence seems certain to find something overpowered somewhere, even if it’s just hacking human minds.
FWIW, while my estimate of femtotech-through-weird-bit-patterns is effectively zero, my estimate of femtotech through rewriting the disk firmware and getting it to construct magnetic fields in no way corresponding to normal disk usage is.. well, still near-zero, but several orders of magnitude higher.
If you didn’t consider overwriting the firmware as an immediately obvious option, you may be merely human. An AI certainly would, and other options I wouldn’t think of. :-)
I am sufficiently certain it’s impossible. I don’t care how intelligent something is, physical law wins. You can’t trick the conservation of energy. You can’t run nuclear reactions on your hard-drive, no matter how you spin it.
I would rate the possibility of unicorns orders of magnitudes higher than the possibility of being able to assemble femtotech using my hard disk. It is more probable that that is impossible than it is probable that I am actually composing this post.
I’ve read at least one science fiction story predicated on the idea that the A.I., within a few moments of waking up, discovers a heretofore unknown principle of physics and somehow uses them as its gateway to freedom, in one case by actually controlling the components of its hardware to manipulate the newly discovered tachyon field or whatever.
Whether or not this scenario as stated is plausible is less important than the underlying question: How much of basic physics do you think humans have already figured out? If your answer is that we’ve already discovered 95% of the true laws of physics, then I can see how you would be skeptical. However, if you’re wrong and there is actually something fundamental that we’re missing because we’re just too stupid, then you can be assured that an arbitrarily powerful A.I. would not miss it, and would figure out how to exploit it.
We’re pretty good at physics. The g-factor) for an electron is 2.0023193043622(15). That number is predicted by theory and measured experimentally, and both give that exact same result. The parentheses in the last 2-digits denote that we’re not totally sure those last two numbers are a one and a five due to experimental error. There are very few other human endeavors where we have 12 or 13 decimal places worth of accuracy. While there’s still a lot of interesting consequences to work out, and people are still working on getting quantum mechanics and general relativity to talk to each other, any new quantum physics is going to have to be hiding somewhere past the 15th decimal point.
That particular story was made somewhat more plausible because the chips were already based on a newly-discovered, ill-understood physical principle that contradicted normal quantum mechanics. It’s pretty likely humanity would have made the same discoveries, the AI just made them faster.
As far as “missing physics” goes, I still feel that it’s a tad hubristic to assert that we’ve got everything nailed down just because we can measure electron mass very precisely. There could always be unknown unknowns, phenomena which we haven’t seen before because we haven’t observed the conditions under which they would arise. There could simply be regularities in our observations which we don’t detect, like how both Newton’s laws and relativity are obvious-in-hindsight but required genius intellects to be first observed.
What is femtotech? A femtometer is the size of an individual proton. You have just implied that an AI could conceivably use a spinning disk harddrive to perform nuclear fission and assemble the resulting nucleons into some sort of technology within a thousandth of a second.
Are you sure that you believe that’s possible?
That was about the mental image I had in mind, yea.
And as Baughn said, I don’t think it’s possible, but I’m not entirely certain that it isn’t. More importantly, it doesn’t seem that implausible that at least one of the thousands of other ideas that are about as overpowered and impossible-sounding and that we don’t have the faintest hope of thinking of, is possible. And one is all the AI needs.
Pointing out that a very low probability argument is not proved to be impossible and is thus worth considering is roughly equivalent to pointing out that someone wins the lottery. Just as I wouldn’t listen to a financial adviser who at every meeting pointed out that I might soon win the lottery, it’s difficult to take seriously people warning me of the risks of things that I judge to be impossible. If you have significantly better advice than lottery tickets, I’m happy to hear it, but the argument that surely I could buy lots and lots of lottery tickets, and one of them has to win is not particularly convincing.
You are confusing probability with difficulty; I’m pretty certain that in some sense, random read-write patterns will eventually cause it to happen with some inconceivably low probability. The question is what’d be required to find the right pattern. Will quantum mechanics put a stop to it before it propagates more than a few atoms? Is there a pattern that’d do it, but finding that pattern would break thermodynamics? Is there a pattern that could be found, but that’d require far more computing power than could ever be built in this universe utilized maximally? Maybe it’s technically within reach but would require a matrochika brain. Or maybe, just maybe, almost certainly not, the kind of supercomputer the AI might get it’s hands on will be able to do it.
But yea, the probability this particular strategy would be practical is extremely small. That was not my point. my point is that the rough reference class of absurdly overpowered implausibilities, when you integrate over all the myriad different things in it, ends up with a decent chunk of probability put together.
The ‘femtotech from HDD’ thing is essentially shorthand for some kind of black swan. Considering how powerful mere human-level intelligence is, superintelligence seems certain to find something overpowered somewhere, even if it’s just hacking human minds.
FWIW, while my estimate of femtotech-through-weird-bit-patterns is effectively zero, my estimate of femtotech through rewriting the disk firmware and getting it to construct magnetic fields in no way corresponding to normal disk usage is.. well, still near-zero, but several orders of magnitude higher.
If you didn’t consider overwriting the firmware as an immediately obvious option, you may be merely human. An AI certainly would, and other options I wouldn’t think of. :-)
Once again, someone else express my thoughts way better than I ever could.
I don’t think anyone would claim that. The claim is that we can’t be sufficiently certain it’s impossible.
I am sufficiently certain it’s impossible. I don’t care how intelligent something is, physical law wins. You can’t trick the conservation of energy. You can’t run nuclear reactions on your hard-drive, no matter how you spin it.
I would rate the possibility of unicorns orders of magnitudes higher than the possibility of being able to assemble femtotech using my hard disk. It is more probable that that is impossible than it is probable that I am actually composing this post.
I’ve read at least one science fiction story predicated on the idea that the A.I., within a few moments of waking up, discovers a heretofore unknown principle of physics and somehow uses them as its gateway to freedom, in one case by actually controlling the components of its hardware to manipulate the newly discovered tachyon field or whatever.
Whether or not this scenario as stated is plausible is less important than the underlying question: How much of basic physics do you think humans have already figured out? If your answer is that we’ve already discovered 95% of the true laws of physics, then I can see how you would be skeptical. However, if you’re wrong and there is actually something fundamental that we’re missing because we’re just too stupid, then you can be assured that an arbitrarily powerful A.I. would not miss it, and would figure out how to exploit it.
We’re pretty good at physics. The g-factor) for an electron is 2.0023193043622(15). That number is predicted by theory and measured experimentally, and both give that exact same result. The parentheses in the last 2-digits denote that we’re not totally sure those last two numbers are a one and a five due to experimental error. There are very few other human endeavors where we have 12 or 13 decimal places worth of accuracy. While there’s still a lot of interesting consequences to work out, and people are still working on getting quantum mechanics and general relativity to talk to each other, any new quantum physics is going to have to be hiding somewhere past the 15th decimal point.
No, they are the standard deviation on the previous digits, i.e. we’re 68% sure that the g-factor is between 2.0023193043607 and 2.0023193043637.
Prime Intellect, right?
That particular story was made somewhat more plausible because the chips were already based on a newly-discovered, ill-understood physical principle that contradicted normal quantum mechanics. It’s pretty likely humanity would have made the same discoveries, the AI just made them faster.
That’s the one.
As far as “missing physics” goes, I still feel that it’s a tad hubristic to assert that we’ve got everything nailed down just because we can measure electron mass very precisely. There could always be unknown unknowns, phenomena which we haven’t seen before because we haven’t observed the conditions under which they would arise. There could simply be regularities in our observations which we don’t detect, like how both Newton’s laws and relativity are obvious-in-hindsight but required genius intellects to be first observed.