Can a superintelligence do THAT?
(From the vast heaps of discarded material from my 2024 attempts at drafts for “If Anyone Builds It, Everyone Dies”.)
Welcome to today’s quiz show: Can a superintelligence do THAT?
With us today we have our contestants: Msr. Soberskeptic and Msr. Oldhand!
Soberskeptic: “I’d just like to say, however this quiz show ends up being judged, I will consider that judgment to be objectively ridiculous—there’s no way anyone can know what a superintelligence could do, in advance of empirical observation. So I’m just here to say what I consider to be true, and I suppose these credulous fools will mark me down as wrong every time I say ‘No it can’t’. In the unlikely event they decide I’ve won anything, good for them and I’ll be grateful for whichever prize. Game-show money isn’t enough to get me to lie.”
A very reasonable attitude, Msr. Soberskeptic! You’ll shortly see how we handle that dilemma! And you, Msr. Oldhand?
Oldhand: “Don’t worry, Sober! I’ll let our hosts know if they’ve gotten any of the answers wrong.”
Also a very reasonable attitude! Now for our first question:
Suppose a digital device contains a secret encryption key that it is repeatedly using to sign messages. Also, this device has a power LED.
Could a superintelligence point a nearby video camera at the device, and read off the encryption key, by looking at the power LED?
Soberskeptic: “Pffft. Obviously not. How would that even work?”
Oldhand: “Ah, I perceive you are not a computer security expert! Many particular steps in encryption draw slightly varying amounts of electrical power to the processor, in a way that correlates to exact features of the secret key. So the power LED, if it’s connected directly to the electrical circuit powering the processor, might get a tiny bit dimmer. I’ll say… ten percent probability that a superintelligence can do that with a randomly selected electronic device that has a secret key and a power light.”
Soberskeptic: “And now they’re going to mark me down, but seriously, give me a break here, audience. Any fluctuations like that would be too tiny to pick up with a nearby video camera. Fluctuations like that would be too fast to pick up at the 60-frames-per-second measuring speed of a video camera. …And now, they’re going to say something stupid about how a superintelligence could hack into the factory and deliberately introduce a design flaw into the power LED.”
And Msr. Soberskeptic… is wrong! A superintelligence can definitely do that to many current electronic devices!
Soberskeptic: “Uh huh. And you know this how?”
Why, because it’s already been done by merely human computer security researchers, of course! See “Video-Based Cryptanalysis: Extracting Cryptographic Keys from Video Footage of a Device’s Power LED”! In which they recovered a 378-bit key from a Samsung Galaxy S8, by pointing an iPhone 13 camera at the power LED of a pair of speakers connected to a USB hub that was being used to charge the Galaxy S8!
The trick, if you’re wondering, is that a video camera uses a rolling shutter to scan pixels across its field of vision. The actual light-sensitive element measures much faster than 60 measurements per second! An Internet attacker can modify a video camera to point to one place instead of scanning across, and measure the device’s power LED intensity millions of times per second! Though a successful attack still requires the device to continuously use its key for an hour or so.
Oldhand: “Gosh. Being able to do it off the power light of a speaker connected to the same USB hub was a bit more extreme than even I thought. But I guess that figures; as anyone in computer security knows, computer-security hacks are often even more impossible than you were expecting.”
Soberskeptic: “Uh huh. I see the gimmick. You’re leading with an example that’s actually provable, and now, you’re going to ask us to swallow way weirder stuff that nobody can prove.”
On to our next question! Is it pragmatically possible, even for a superintelligence, to cross astronomical distances much faster than light? Without having first sent ahead any sort of wormhole-receiver that traveled slower than light?
Soberskeptic: “Unless you’re going to play some silly game about ‘going faster than the speed of light in some medium’, like Cherenkov radiation from particles going faster than the speed of light in an atmosphere, I’ll answer… that you have no possible way of knowing about the ultimate limits of technology! People have theorized incredibly speculative ways to travel faster than light, like the Alcubierre Drive, if you could supply huge amounts of exotics like ‘negative energy’. I’d guess none of them work. But it’s only a guess, and you don’t know any better than I do about that. Our current theories of physics could be wrong, after all; humanity didn’t have the one true theory of physics 200 years ago, and maybe we don’t have it today, either. Our descendants millions of years from now will be working very hard to go faster than light if there’s any possible way of doing so; and nobody can say if they will succeed.”
Oldhand: “And my answer is: No!”
Soberskeptic: “What do you mean, no? Aren’t you just going to say superintelligences can do anything whatsoever, because magic, and how can we mortals possibly constrain them? Isn’t that your whole shtick?”
Oldhand: “Ha! Sounds like somebody has been heavily misrepresenting a couple of decades’ worth of careful analysis to you.”
And Oldhand… is once again correct! The Earth’s Sun is relatively young in age among all the sorts of stars we’d expect to be able to support biological life, over the history of a whole universe. This timing strongly suggests that we find ourselves existing now, around that relatively young star, because later on, the stars would have already been colonized; and then new life like Earthly life would not have had later chances to emerge! This would still hold true even if you imagine that it’s colony ships full of tinned biologicals traveling between stars, rather than machine intelligences.
If arbitrarily advanced aliens could travel much faster than light, we should observe our solar system being even younger!
(See “grabby aliens” for more about the sort of statistical inference which underlies this sort of reasoning. There is more detailed grist and confirmation for this kind of inference than one might expect; eg, the amount of time that remains for Earth to support life, before the oceans evaporate, is around what we’d expect to see under that statistical model.)
Oldhand: “Or to simplify: Besides the reasons from our current model of physics, we have observational evidence that nobody—no machine life, no organic life—can travel much faster than light. Because then the universe would fill up very fast, even if one single merely biological species decided to let two parents have four kids every hundred years; and the universe doesn’t look full. Aliens have not mined out the ice or metals of our asteroid belt; they have not surrounded stars with swarms of solar panels that would intercept the solar energy and reradiate it as infrared. Earth emits enough radio to be detectable from many light-years away, and nobody else like that seems to be around. Combine it with the standard model of physics seeming extremely strident about the lightspeed limit on causality, and I’m willing to make the call: We’re directly observing that the limits of technology don’t permit practical FTL travel—at least, not substantially faster than light, and not running ahead of a slower-than-light wormhole receiver you have to send on first. Physics mostly says you can’t do wormholes either, but we don’t observe that the same way.”
Soberskeptic: “Aliens could decide to hide, or choose not to reproduce; how can you know?”
Oldhand: “What, every single species across billions of faster-than-light years all chose not to reproduce quickly, every last one? The super-hives, the super-dolphins, the super-octopi, the machine intellects, every last one of a million entity-kinds like that all decide to restrain their reproduction? To solve the Fermi Paradox you don’t need ‘There could exist a non-spreading civilization’, you need ’99.9999% of all diverse alien life-origins end up never spreading.′ Or similarly: All kinds of alien species are running around at FTL speeds, but prefer to hide more strongly than they want anything else they could get from mining asteroids, colonizing planets, or using solar energy? One must distinguish possibility from probability, if this is meant to be inference about real life rather than the allowed premises for a fantastical story. Also if arguendo all aliens refuse to reproduce or they hide forever, and this lets more species like humanity pop up, then most species like humanity should find themselves in a solar system that’s born later in the universe’s history! So your vastly strained hypothesis still fails to retrodict our observations. I therefore conclude from theory plus observation that superintelligences cannot make a first crossing of intergalactic distances much faster than light; that the extremely strong implication of our best physical models, reflects a genuine constraint on intergalactic travel speed that cannot be pragmatically defeated by any amount of cleverness.”
Next question, please! Suppose you prove using valid logic that some computer program obeys a constraint like ‘never writes to a particular section of memory’. Suppose you have a machine-verified proof of this constraint, that includes all other software on the system, and the CPU and the memory and every other part of the computer. Suppose it’s a proof that goes all the way down to the factory layout of the transistors, and the usual simple behaviors for those transistors. And suppose that the factory map of the transistors is accurate.
If the proven-correct computer program contains a superintelligence, can the superintelligence write to that forbidden area of memory?
Soberskeptic: “What sort of axioms are the theorems being proven from, besides the true program specification and the true transistor layout? Like, what proof system are the theorems being proven in?”
Great question, Msr. Soberskeptic! Let’s say: first-order arithmetic, aka Peano Arithmetic.
Soberskeptic: “Then no, because if Peano Arithmetic is inconsistent then everything about reality is a lie. Not even a superintelligence can violate actually-valid mathematical proofs. This is going to be another place where even Oldhand says no, like for FTL travel, and this time Oldhand will be correct. Thinking that a superintelligence can disobey the laws of mathematics, is about the clearest test I can imagine for whether you are holding even the tiniest pretense of putting forth a serious argument. Ah, assuming you’re not going to say something dumb about the superintelligence persuading a human to come over and write to that area of memory.”
Great catch, Msr. Soberskeptic! Say we’re just talking about what the superintelligence can do by wriggling around its code inside the computer, not by persuading any outside people or hiring Taskrabbits over the Internet.
Soberskeptic: “Then no.”
Oldhand: “Yes.”
Correct again, Msr. Oldhand!
Soberskeptic: “Wow. I’m… kind of speechless. Where you do you get the idea that a superintelligence can’t violate our theories of physics, but it can violate math?”
Violating math is hardly required to disobey properties that have been mathematically proven true about a computer chip given the transistor wiring and the usual simple behaviors of transistors, Msr. Soberskeptic! You just have to do something with the transistors that subverts their usual behavior!
For example, the “Rowhammer” class of computer-security vulnerabilities is that by using the right high-speed memory access pattern on some memory cells in DRAM, you can electromagnetically perturb bits in adjacent cells of DRAM! And that’s a big deal in practice, because once you start violating transistor-level assumptions like that, you can parlay it into all kinds of privilege escalations—like if that flipped bit was doing something important to the system!
So just because you mathematically prove code to be safe given the transistor layout and the usual rules governing transistors, doesn’t mean it could really constrain a superintelligence—or even a program written by a savvy human security researcher!
Next question! Is physically ripping out the Wifi antenna, Wifi chip, all USB ports, all peripherals, removing all power lights, all speakers, and any kind of hard drive that could be spun to make audible clicks—leaving behind just the CPU and RAM and a very minimal motherboard—enough to prevent a superintelligence inside that computer from communicating with the outside world?
Soberskeptic: “Okay, I can kind of guess that the answer is going to be ‘no’ and involve some sort of amazing computer security trick that’s already been done. Probably you can make the capacitors whine in a way that imitates a human voice, or something.”
Oldhand: “Glad to see you’re finally picking up on the pattern! Just keep in mind, real life does work this same way, in terms of how much is possible to accomplish starting from extremely restrictive conditions—even if you’re not on a gameshow, and the answers aren’t known already. I’ll also reply no.”
Both of you are… correct and known to be correct! Memory accesses can also be modulated to send out radio signals on GSM cellphone frequencies! See “GSMem: Data Exfiltration from Air-Gapped Computers over GSM Frequencies”! Though the existing demo required somebody with a hacked cellphone, not too far away, to pick up the signal.
Msr. Soberskeptic, what would you guess is the smallest possible size for a solar-powered factory system, that starts from completely raw materials found on a planet like Earth, and builds a full copy of itself? And what would you guess is the minimum time for it to make a copy? Assuming a superintelligence has designed the factory.
Soberskeptic: ”...right. So, the obvious gotcha would be if you’re going to point me to somebody’s actual demo of a self-replicating factory that picks up two exactly right kinds of clay, and uses it to 3d print a copy of a 3d printer… wait. Is the factory being operated by a hobbyist to build a copy of itself, or does the factory have to be fully automated?”
No external help, Msr. Soberskeptic! The factory has got to build another factory on its own!
Soberskeptic: “Okay. In that case I am having trouble seeing a fully self-copying factory that starts from sheer raw materials and has to build a complete copy of itself including the onboard computers. Because that’s going to take, like, smelting copper, and I guess you can make a kiln just from clay but… okay, you know, I’m going to guess the gotcha is about somebody’s theoretical design for a self-replicating factory. And that it is merely ten cubic meters on a side, and… builds abacus-like physical logic out of the right kind of clay… and picks up wood and burns it for fuel, and supposedly replicates in just a week or so. But has never actually been built, because in reality it’d scoop up the wrong kind of clay and break every time it rained.”
Oldhand: “Well, I could hardly guess the smallest a superintelligence could get a fully self-replicating factory system operating in a real planetary environment. But certainly underneath one micron on a side, replicating in less than an hour.”
Soberskeptic: “One micron! Ha! I take it you think nanotechnology is real, then? Because even if our game-show hosts say somebody did the theoretical designs for a system like that, I’ll say it’s no coincidence that nobody has ever built the smallest part of one. You just can’t actually get machinery that small.”
Sorry, Msr. Soberskeptic, but we’re afraid you’ve overlooked an important practical example! A tree is a self-replicating solar-powered factory that builds a complete copy of itself! (Mostly out of mass it gets from carbon dioxide in the atmosphere, and water from rain; what it gets from the soil is trace nutrients making up around 1% of mass. That’s why trees don’t fall into giant holes in the ground underneath after turning too much dirt into tree mass.)
And while the tiny protein-making machine of a ribosome, inside a tree, may happen to be making only more tree proteins—in principle, any ribosome is able to make every kind of protein, just by being told the amino acid sequence! Nothing in principle stops there from being a tree that buds off mosquitos that inject botulinum toxin, for example! So a blade of grass is a self-replicating factory, not just a replicator.
Oldhand: “And small algae cells aren’t far off from one micron on a side, and have a replication time of… I think around 6 hours in the fastest cases? If you look at the theoretical analyses that have been done for crude air-eating nanotechnology, they come out at around 1 hour doubling time. For actual E. coli bacteria it’s 20 minutes, and for Clostridium it’s 7 minutes, but those shouldn’t count as a self-contained factory—bacteria don’t make all the compounds they eat, and rely on there being other life forms around. Probably an actual superintelligence can get a solar-powered factory well below one micron and one-hour replication times, if it’s allowed to redesign self-replicators from scratch; but that’s one way of guessing the least it can do.”
And now, our final question! Msr. Soberskeptic and Msr. Oldhand, what sort of weird shenanigans do you think a superintelligence would be able to set in motion inside a human mind… by talking to a human?
Soberskeptic: “Right, so, I suppose the idea, now, is that I’m supposed to think of all the things that humans have done by talking to humans?”
Oldhand: “I mean, you could anchor yourself on human examples. But maybe this is also an opportunity for you to depart from your previous habit, and stretch your imagination a tiny bit further?”
Soberskeptic: “And I’m supposed to be scared of the superintelligence… because of how human con artists can talk people out of money? Or, I guess, politicians talking people into voting for them… or prophets starting religions that propagate from person to person… I admit it’s scary to imagine something being better than any human at that. But there’s no evidence that any mind can be better. Maybe that’s just something superintelligences can’t realistically do. Like, maybe there’s some secret series of words you could say to some particular person to get them do things your way, but that doesn’t mean even a superintelligence can figure that out by reading their social media profile.”
Oldhand: “Soberskeptic, none of the material in this gameshow has been about what actual superintelligences could do, except the part about ruling out faster-than-light travel. It’s been about what clever humans have already figured out what to do, starting from handicapped gameboards—or, in one case, what natural selection has already done with algae cells. The idea being to snap out of the mindset where, if something sounds hard and you can’t figure out immediately how to do it, you conclude with great confidence that it’s impossible, and dismiss any contrary thoughts as naive and no doubt ignorant of all the practical difficulties. That’s the lesson we learn from dealing with other humans—who sometimes try to be a little clever, and sometimes succeed; and we ourselves have not always anticipated the way in which they will be clever. What superintelligences could do against us would be a qualitatively different story.”
Soberskeptic: “See, the thing is, no matter how persuasive superintelligences end up, we humans are an ornery lot. We can just refuse to believe anything the superintelligence tries to persuade us about, even if it is being very reasonable, or appealing to our greed, or whatever.”
Oldhand: ”...I feel like this is the wrong level on which to be having this discussion. You are imagining that your orneriness is an invincible shield that the superintelligence has to confront and defeat on its own level, by being persuasive the way a human tries to be persuasive. Imagine something more like a computer security expert, seeing your brain as a system made of parts and searching a vast space of paths through causality. If you think of a human brain’s security condition as being, ‘Do not accept invalid arguments which derive conclusions that didn’t follow from their premises’, or ‘Don’t get into self-harmful or other-endangering states’, or ‘Have all acceptance of bad ideas be a case of unpredictable individual noise, rather than systematic shared error, such that no bad idea ever propagates from one human being to another’—then the security on a human brain is so awful that it somehow wraps around and makes it hard for people to conceive of a superintelligence breaking it. Conditions analogous to software security conditions are impossible to imagine holding of human minds in the first place.”
Soberskeptic: “So? I’m not saying, nobody can ever talk me into believing something false. I’m saying, I can decide that there will be no pathways to persuading me of things that an ASI wants me to believe.”
Oldhand: “And—pretending for the moment that ASI is playing the game that much on your own level, because I can’t reach its real level myself—you don’t think that an ASI can manage to navigate a conversation such that you, in your invincible and confident refusal, thereby decide to believe something useful to the ASI?”
Soberskeptic: “Like what?”
Oldhand: “Like that it is totally safe to expose yourself to an ASI talking to you, for as long as it feels good and without taking any other safety precautions, because no possible input to you could defeat the invincible security of your brain. A conversation like this one, could be seen as having a predictable outcome like that—and when you came to that conclusion, it would feel like one-downing the ASI, pushing down its status and pushing up yourself; it would feel like being strong and ornery and independent, when you concluded what the ASI had decided you should believe.”
Soberskeptic: “Pffft. Nah, you didn’t navigate me into believing that. You obviously don’t want me to believe that, but I ornerily concluded it anyways.”
Oldhand: “Consider a superintelligence looking over its detailed model of which brain areas you have, doing which kind of work right now, on which kinds of human concepts—in its vastly greater understanding of human thought, than anything humans themselves have ever understood about their own brains and minds and cognition—searching more paths through the true rules governing your thought processes, than you would ever have the time or knowledge to search through yourself. Do you imagine it concluding: ‘Wow, human brains are such secure software that I cannot possibly figure out how to navigate this brain into doing anything useful for me.’?”
Soberskeptic: “Look, let’s be concrete. Suppose you’ve got a guy who has built an untrusted superintelligence on his personal horde of GPUs, who just decides, in advance, never to connect that AI to the Internet, no matter what it says—”
Oldhand: “Pal, all modern AIs are running on computers connected to the Internet. Including while they are still in the middle of being tweaked for underlying capability, and before any crude tries at alignment have even been tweaked at them.”
Soberskeptic: “—and that guy assesses himself, and concludes that nobody could ever persuade him to connect his AI to the Internet; no, not even a superintelligence. Because he can always just be an ornery human who just says no. Then in real life, I think, he’s right and he can always just say no.”
Oldhand: “Heh. See, now that sounds like the sort of experiment somebody could have actually run—taking people like you, who said that not even a superintelligence could talk them into a connecting an AI to the Internet, because they could always just say no. And connecting them to some mere human, roleplaying the part of an AI, trying to talk them into explicitly and on purpose deciding to let that AI out of its box.”
Soberskeptic: “Yeah, well, let’s go ahead and run that—oh no.”
Sounds like you might want to Google “AI-box experiment”, Msr. Soberskeptic!
I would really like you to label the host’s text, too. The quotation marks really aren’t enough unless I’m focusing on them.
I have tried bolding the contestants; I wonder if that helped by way of contrast?
No. It may have helped me see that each contestant only speaks a paragraph at a time, but I had to focus on the sensation that something looked different with the bolded texts.
Perhaps more space in between each segment of text corresponding to a speaker?
It’s an argument, sure, but still “there are more things in heaven and earth”. For example (invented by Paul Almond afaik) if civilization-level quantum suicide can be used for reality editing, and all sufficiently advanced aliens discover a proof that it is in some sense okay, that might explain why we don’t see them—they’ve winked out and are having fun. Or the universe allows better colonization opportunities than flying to other stars, something like creating new universes locally, and all sufficiently advanced aliens discover that. Or if the origin of life requires some super-improbable quantum event, and even given the scale of the universe we’re likely to be the only ones. Or something else.
So basically I’m not sure how much we can conclude from the age & emptiness of the universe. I mean, you’re probably right that FTL is impossible, but idk about this particular argument.
Then we should look around ourselves and see an older universe.
Also this falls under the category of “distinguish the possible from the probable”. No species anywhere cares about leaving one little probe behind to stop Holocausts?
Hmm. Okay, if FTL was possible, we’d observe a younger universe, and if there was a strong barrier to life we’d observe an older universe. If both are true, or both are false, we might well observe the age of the universe as it is now. So how can that be a reason to update strongly about either variable? (Taylor G. Lunt has made the same point.)
I can’t follow why we should necessarily expect to see an older universe: given @cousin_it’s “origin of life requires some super-improbable quantum event” (or some sort of Great Filter generally), such that the rate of intelligence emerging is roughly 1 intelligent civilisation per galaxy per 13 billion years, wouldn’t this explain our observed age of the universe whilst also explaining our observed lack of interactions with aliens?
I realise this reduces to an argument about the suspiciously convenient rate of 1/galaxy/13bnYears—but I think we would need to evaluate this proposed rate by evaluating arguments for/against each factor in the Drake Equation (or some other bottom-up methodology): if this rate is within-bounds for reasonable Drake Equation factors (or otherwise explicable from the inside view), this would seem to explain our observations without additional assumptions like “FTL is impossible for superintelligence”, “at least some aliens would be grabby”, etc.?
(nb. I think these assumptions are very reasonable indeed, in isolation—I just think they may be adding complexity here without adding explanatory power)
In other words: I don’t think we can simultaneously solve all three out of “are there aliens at all?”, “would any aliens be grabby?”, and “is FTL travel possible for a superintelligence?” purely from the observed age of the universe alone as when arguing from the age of the universe each of these becomes an unknown variable for the other two. A 13bn-year-old universe with no aliens at all but possible-for-superintelligence FTL looks very much like a 13bn-year-old universe teeming with aliens but impossible-for-superintelligence FTL, and so on.
This seems to remind me arguments like “ah, but what if the ASI doesn’t care about lots of, but about only one thing”. Then you still have maximizing the probability of the local thing. Just, why not to do that when it is about all your preferences? You won’t really have anything competing, which usually makes humans not to pursue tiny chances for particular things.
And similar way, even if aliens quantum suicide or something, there is still no reason for them not to leave behind an ASI which will improve the universe on margins to their tastes.
Soberskeptic: This is conditional on the probability of the ingredients of life turning into life not being extremely low. It could be the case that it’s easy to travel 1000x the speed of light, so long as life is really rare. I don’t think I’ve actually heard a piece of evidence that would indicate the transition from ingredients of life to life is not an extremely rare transition.
This makes me nostalgic for the old-school AI safety days, when conversations like this seemed like the most important conversations to be having, and billion dollar companies weren’t speedrunning internet-connected superintelligent hacker-swarms as fast as possible with no seatbelts.
But that is precisely the whole point of the argument: if it had indeed been extremely unlikely, then we would expect to be living in a much older universe, one that would have had much more time to meet the necessary conditions!
I find these debates sort of weird in the present timeline. 5 years ago they were relevant, when people said—would hook up a superintelligent AI to the internet, whereas now we simply give them the credentials to all our accounts and let them rip. Even a very clever human could obviously do tremendous harm in that situation.
Hm. I long had a certain thought and this seems like a fitting post to attach it as comment.
Background:
I noticed that most people tend into direction of “ah, this is all sci-fi”, but I myself rather tend in the opposite. Ideas like “but what if ASI will modulate a signal via CPU or RAM” seems to me like totally natural. What I actually think is more like “Can ASI via right images make neurons in the brain create certain chemicals which will interact in certain ways such that in the end they will change the DNA or assemble into viruses or prions which will make more direct routes for chemical manipulation via images, until ASI will be able to make your body cells turn into controllable nanotech by just giving you some right perceptual input”. And… I can’t rule out that this is possible for mere ASI (as opposed to Solomonoff induction).
So, this is in a sense opposite of what post talks about—things which actually were proven to already have happened. But imo it is just the result of most people being way too inclined in the first direction—if they were asked about ai swarms and solving novel math just 5 years ago, they would confidently say it can’t be true anywhere soon. Like some dialogue I have seen in comments (~5 years old ones) to a video about “pizza police call” where skeptic was “this is why you shouldn’t let machines work in accepting calls, they would never guess that” and enthusiast was “oh, never is too strong word, ai may learn to do that some time, not in 10 years or even 30, but 100 years later it can...”. And just 1.5 years after that gpt4 comes out.
So the actual example of thing possibly possible for ASI I keep thinking of is this:
It is usually said that ASI will send near lightspeed nanotech probes to other stars. But shouldn’t there be something further than molecular nanotechnology? Some… not “quantum tech”, if continue “molecular nano” it should be atomic piko- and subatomic femto- technology. Can ASI learn QM better than humans such that it would not need to create robots from atoms in the first place, but use some semi stable resonances or something in sophisticated way? I don’t know enough physics to rule it out, but it seems plausible enough for me that it will be possible (and used) at least to some degree—humans failed even in thermonuclear synthesis even though it already exists in the nature.
And if we take another step here, then moving at almost lightspeed is in some sense very different from exactly lightspeed. And you can’t send anything with clock still running at the speed of light, because it would mean the path is not the shortest anymore. But what you don’t actually need to, if you are ASI? It has some old data for predictions and very good knowledge of the physics and a computational power in the size of stars. What if it can send particles with no clock moving such that they will assemble into right form already in the place? It doesn’t need to be exact, it can use a lot of different combinations. And by “a lot” I of course don’t mean trillions, I mean orders of magnitude more.
Say, we have a star 1 lightyear away, then we moving at 99% of lightspeed would mean you lose 3.65 days, which is enormous time for ASI. A star emits billions of times more light than planets get. If you can redirect it all, you would get that much energy for particles to send, that it will immediately turn Earth’s surface into plasma. You actually will use only tiny part of the light which star emits in 3.65 days. And you still may have plenty of space for different setups for different variants of conditions. The usual case is when you are limited in action with plenty of knowledge and need to choose actions wisely, which is solvable to significant degree, but why should the opposite be unsolvable—when you have plenty of actions, but you are almost blind (especially if we save not 3 days on 1 year distance in time, but more like 30 billions of days for stars which are leaving the Hubble sphere).
ASI needs to make different variants not destroy each other. But it doesn’t actually need to just do that absolutely blindly. There are some priors, calculations of which planets likely form where, distributions of objects on the surface. And ASI only need to search for the most present, the most invariant ones.
So, maybe you shouldn’t actually expect to see traces of ASI activity, Because the moment they can reach you, you already will be reached by a flash of optimised signals which hack most likely computers or most likely electrochemical brains of animals. Or just a flash of particles which evaporates most of the surface where it didn’t found proper objects to latch and so everyone dies in one second.
Because, anyway, even if it fails then you just turned into vapor a civilization with differing utility function, which is also good. And advanced civilization will know about that tactic and configure themselves to not be harmed, but to collect the energy as a payment for perfoming some cheap components of AI utility function.
There of course are problems with that, like gas between galaxies and other distortions. Maybe, it will need to be multi step, with some assembling right in the space, to reconfigure. But it easily may worth it, you just need to spend less than 3.65 days on it for each lightyear and less than radiation of a star in 3.65 days of energy.
Another caveat is for example that ASIs and civilizations may have logical agreements not to vaporize living creatures or sentient or sapient creatures. But I am not sure about it.