If thats right, and its almost always low-res sims that are sufficient then that destroys the main ancestor sim argument for our conscious experience being simulated. Low res is not conscious in the same way we are, different reference class to base reality bio-consciousness
RussellThor
Karma: 264
If Windows95 was ever conscious (shock!) it would be very sure it was in a virtual machine (i.e like simulated) if it existed at the time when VM’s existed. It would reason about Moores law/resources going up exponentially. and be convinced it was in a VM. However I am pretty sure it would be wrong most of the time? Most Win95 instances in history were not run in VM and we have stopped bothering now? An analogy sort of but gives an interesting result.
Random ideas to expand on
https://newatlas.com/computers/human-brain-chip-ai/
https://newatlas.com/computers/cortical-labs-dishbrain-ethics/
Could this be cheaper than chips in an extreme silicon shortage? How did it learn, can we map connections forming and make better learning algorithms.
Birds vs ants/bees.
A flock of birds can be dumber than the dumbest individual bird, a colony of bees/ants can be smarter than than the individual, and smarter than a flock of birds! Bird avoiding predator in geometrical pattern—no intelligence as predictability like fluid has no processing. Vs bees swarming the scout hornet or ants building a bridge etc. Even though no planning in ants, no overall plan in individual neurons?
The more complex pieces the less well they fit together. Less intelligent units can form a better collective in this instance. Not like human orgs.
Progression from simple cell to mitochondria—mito have no say anymore but fit in perfectly. Multi organism like hive are next level up—simpler creatures can have more cohesion in upper level. Humans have more effective institutions in spite of complexity b/c of consciousness, language etc.
RISC vs CISC Intel vs NVIDIA, GPU for super computers. I though about this years ago, led to prediction that Intel or other CISC max business would lose to cheaper.
Time to communicate a positive singularity/utopia
Spheres of influence, like we already have, uncontacted tribes, Amish etc. Taking that further, Super AI must leave earth, perhaps solar system, enhanced ppl to of earth eco-system, space colonies, or Mars etc.
Take the best/happy nature to expand, don’t take suffering to >million stars.
Humans can’t do interstellar faster than AI anyway even if that was the goal, it would have to prepare it first, and can travel faster. So no question majority of humanity interstellar is AI. Need to keep earth for people. What is max CEV? Well keep earth ecosystem, humans can progress, discover on their own?
Is the progression to go outwards, human, posthuman/Neuralink, WBE? it is is some sci-fi Peter Hamilton/ Culture (human to WBE)
Long term all moral systems don’t know what to say on pleasure vs self determination/achievement. Eventually we run out of inventing things—should it go asymptotically slower.
Explores should be on the edge of civilization. For astronomers, shouldn’t celebrate JWST, but complain about Starlink—that is inconsistent. Edge of civilization has expanded past low earth orbit, that is why we get JWST. Obligation then to put telescopes further out.
Go to WBE instead of super AI—know for sure it is conscious.
Is industry, tech about making stuff less conscious with time? e.g. mechanical things have zero, vs a lot when done by people. Is that a principle for AI/robots? then there are no slaves etc.
Can ppl get behind this? - implied contract with future AI? acausal bargaining.
https://www.lesswrong.com/posts/qZJBighPrnv9bSqTZ/31-laws-of-fun
Turing test for WBE—how would you know?
Intelligence processing vs time
For search, exponential processing power gives linear increate in rating, Chess, Go. However this is a small search space. For life, does the search get bigger the further out you go.
e.g. 2 steps is 2^2 but 4 steps is 4^4. This makes sense if there are more things to consider the further ahead you look. e.g. house price for 1 month, general market, + economic trend. 10+ years then demographic trends, changing govt policy, unexpected changes in transport patterns, (new rail nearby or in competing suburb etc)
If applies to tech, then regular experiments shrink the search space, need physical experimentation to get ahead.
For AI, if its like intuition/search then need search to improve intuition. Can only learn from long term.
Long pause or not?
How long should we pause? 10 years? Even in stable society there is diminishing returns—seen this with pure maths, physics, philosophy, when we reach human limits, then more time simply doesn’t help. Reasonable to assume with CEV like concept also.
Pause carries danger? Is it like the clear pond before a rapid, are we already in the rapid, then trying to stop is dangerous having baby is fatal etc. “Emmett Shear” of go fast slow, stop, pause, Singularity seems ideal, though possible? WBE better than super AI—cultural as elder?
1984 quote “If you want a vision of the future, imagine a boot stamping on a human face—forever.”
“Heaven is high and the emperor is far away” is a Chinese proverb thought to have originated from Zhejiang during the Yuan dynasty.
Not possible earlier but is possible now. If democracies go to dictatorship but not back then pause is bad. Best way to keep democracies is to leave hence space colonies. Now in Xinjiang, the emperor is in your pocket, LLM can understand anything—how far back to go before this is not possible? 20 years, if not possible, then we are in the white water, and we need to paddle forwards, can’t stop.
Deep time breaks all common ethics?
Utility monster, experience machine, moral realism tiling the universe etc. Self determination and achievement will be in the extreme minority over many years. What to do, fake it forget it and keep achieving again? Just keep options open until we actually experience it.
All our training is about intrinsic motivation and valuing achievement rather than pleasure for its own sake. Great asymmetry in common thought “meaningless pleasure” makes sense and seems bad or not good, but “meaningless pain” doesn’t make it less bad. Why should that be the case. Evolution has biased us to not value pleasure or experience it as much as we “should”? Learn to take pleasure regard thinking “meaningless pleasure” is itself a defective attitude? If you could change yourself, should you dial down the need to achieve if you lived in a solved world?
What is “should” in is-ought. Moral realism in the limit? “Should” is us not trusting our reason, as we shouldn’t. If reason says one thing, then it could be flawed as it is in most cases. Especially as we evolved, then if we always trusted it, then mistakes are bigger than benefits, so the feeling “you don’t do what you should” is two systems competing, intuition/history vs new rational.
“most likely story you can think of that would make it be wrong”—that can be the hard part. For investments its sometimes easy—just they fail to execute, their competitors get better, or their disruption is itself disrupted.
Before the debate I put Lab leak at say 65-80%, now more like <10%. The most likely story/reason I had for natural origin being correct (before I saw the debate) was that the host was found, and the suspicious circumstances where a result of an incompetent coverup and general noise/official lies mostly by the CCP around this.Well I can’t say for sure that LL was wrong of course, but I changed my mind for a reason I didn’t anticipate—i.e. a high quality debate that was sufficiently to my understanding.
For some other things its hard to come up with a credible story at all, i.e. AGW being wrong I would really struggle to do.
Some advice I heard that was for investing was when committing to a purchase, write a story of what you think is most likely to make you lose your money. Perhaps you could identify your important beliefs that also perhaps are controversial and each year write down the most likely story you can think of that would make it be wrong? I also believe that you can only full learn from you own experience so building up a track record is necessary.
Good article. I listened to all the rootclaim debate and found it informative. After that debate, I have a lot less belief in the credibility of giving accurate bayes estimates for complicated events, e.g. both debaters attempted it but their estimates where different by >>> like >1e20 I think.
I think this applies even more for P(doom) for AI, after all its about something that hasn’t even happened yet—I agree with the criticism that P(doom) is more a feeling rather than the result of rationality.
I have thought the same with young kids. After a little thought I decided its best not to really change anything. As you said there is no benefit of taking kids out of school even if you believe the skills wont be useful. If they are happy at school and gain a sense of mastery and purpose with peers etc then that is good.
If you want specifics then sure asking them “what do you want to be when you grow up” is definitely not a good idea, if it ever was. Also our society is all geared towards feeling worthwhile if you make something rather than just being intrinsically worthwhile. You can wonder how a parent in the “Culture” universe would bring up their child.
Perhaps the more open a child is to a Brain Computer Interface such as Neuralink, the more they will contribute in the future? I keep the kids away from ChatGPT and image generators—if kids get a sense of achievement from drawing then I don’t see any good that can come of letting them make artwork in seconds.
Also I consider it plenty likely enough that we won’t see that future anytime soon. If you have read “Chip wars” you will realize how incredibly fragile the semi-conductor supply chain is. If Taiwan is invaded, then that guarantees say 5+ years of a setback and I could realistically see 2 decades if things really turn to strife and supply chains collapse. Additionally if we have a “slow takeoff” as I believe once again China will see that they will essentially have no chance of a Chinese century and will be motivated to destabilize things—likewise for any major power that thinks they are losing out. Where I live thats supply chain shortages but not a vastly changed world.
Thanks for the effort.
In the discussion about selfishness on this post it seems a bit implied that we know how to make a “self” or it will just appear like a humans. However that is not my experience with GPT-4. Often I have found its lack of self-awareness a significant handicap in its ability to be useful—I assume it has some self awareness, it doesn’t and wastes my time as a result. Consider a game engine that does “intuition” and “search” such as a GO engine. It is very good at examining possibilities and “intuits” what moves to consider and can model GO very well, but not itself at all.
If there is an algorithmic structure that self-awareness requires to be efficient and effective(why wouldn’t there be), then just throwing compute to get GPT-X won’t necessarily get there at all. If we do get a capable AI it won’t act in a way we would expect.
For humans it seems there is evolutionary pressure for us not only to have a “self” but to appear to have a consistent one to others so they can trust us etc. Additionally our brain structure prevents us from must being in a flow state the whole time where we do a task without questioning whether we should do something better, or whether it is the right thing to do. We accept this and furthermore consider this to be a sign of a complete human mind.
Our current AI seems more like creating “mind pieces” than a creature with a self/consciousness that would question its goals. Is there a difference between “what it wants and what we want” or just “what is wanted”?
I agree in general terms that “alignment has a basin of attraction” and GPT-4 is inside is somewhat justified.
Rootclaim covid origins debate:
This piece relates to this manifold market
and these videosI listened to most of the 17+ hours of the debate and found it mostly interesting, informative and important for someone either interested in COVID origins or practicing rationality.
I came into this debate about 65-80% lab leak, and left feeling <10% is most likely.
Key takeaways
The big picture of the lab leak is easy to understand and sounds convincing, however the details don’t check out when put under scrutiny.
Both sides attempted Bayesian estimates and probabilities and got absolutely absurd differences in estimates.
Rootclaim failed to impress me—the takeaway I got is that they are well suited to say murder cases where there is history to go off, but when it comes to such a large messy, one-off event as COVID origins they didn’t know what evidence to include, how to properly weight it etc. They didn’t present a coherent picture of why we should accept their worldview and estimates. An example is where they asserted that even if Zoonosis was the origin then the claimed market was not the origin because the details of infected animals and humans wasn’t what they expected. This seems an absurd claim to make with confidence judging on the data available. When forced to build models (rather than rely on multiplying probabilities) they were bad at it and overconfident in their conclusions from such models.
More generally this led me to distrust Bayesian inference type methods in complicated situations. Two smart reasonably well prepared positions could be off by say >1e12 in relative estimates. Getting all the details right, building consistent models that are peer reviewed by experts cannot be made up for by giving uncertainties to things.
Regarding AI, I have now more sympathy to the claim that P(Doom) is a measure of how the individual feels, rather than a defensible position on what the odds actually are.
RussellThor’s Shortform
I expect that if drones ever become a serious threat will see the proliferation of lots of Gatling guns mounted on tanks and other vehicles, linked to a decentralised radar system combining lots of different radars of different specs.
Yes—fair enough and I hope you are right—I would be happier if defense wins too. I hope soon Europe/USA develops such a system, including the ability to mass produce it in the quantities needed.
Firstly some context:
Missile vs gun
Radio comms and protection against jammingFor your points
Guided bullets—yes good, unsure whether they can be made cheap yet but if they can of course such a system would use them
Chaff etc—yes probably correct, however it seems this is not needed for missiles to destroy current guns.
Fly to 1000m—Yes it would, however for sniper drone we are comparing the cost to a actual soldier. I have in mind something like https://newatlas.com/drones/huntress-turbojet-drone/ for heavier drones. Other sniper drones could be electric with a very short flight time, carried by the huntress or logistics drone
Relay drones—the idea is most of them fly over territory that has been secured—think a drone with flapping wings like a bird circling at 1000m—if you shoot it down with a big gun you give away your position. Also such drones will be doing constant surveillance of territory.
Anti-armor only—yes however infantry holed up in a building can’t stop the invasion, it can route around them.
Flak guns—yes guns can take down drones economically, however it becomes missiles vs flak gun.
Aircraft—yes I overstated a bit—for the initial invasion conducted with stockpiled materiel, they can’t easily stop it. However taking out the aircraft is very important for the drone army. The drones can take out the airbases—so it could be a race between the fast aircraft trying to bomb the logistics before the drones reached the airfields. Most countries are ~1000 kilometers or less in length, which is in range of a cheap Cessna type logistics drone before they even do mesh network fuel drops to extend the range. Such low slow cheap aircraft would be protected by MANPAD carrying drones, or just equipped with them. fighter jets would be forced to shoot expensive missiles to destroy them, rather than get in close with the cannon etc. Even if the fighters can do 1,5000-2000 kilometers conventional forces could still enter at the edge of their range and help with logistics as the aircraft could not fly many sorties.
For a specific idea, consider a country on one side of a conflict or potential conflict. E.g. Armenia vs, Azerbaijan, Iran vs Saudi Arabia, or Russia vs Europe (through say Latvia) the side planning a drone army invasion stops active conflict for 1-3 years and quietly builds up a stockpile of the drones. They build enough of the cheap long distance logistics drones (and believable decoys) so they have more of them than the enemy has fighter jet missiles.
They then launch a somewhat surprise invasion—its easier to hide a deployment of drones than soldiers. They try to cut a long narrow path as quickly as possible to take out the key defenses. They route round defenses where possible and destroy the air power of the enemy first by destroying the airports and air bases. The missile drones fan out about 10K or so from the logistics drones destroying armor that attack them. Sniper and cheap recon drones continually launch from logistics drones or are carried by missile drones.
After the fast fighter jets are gone, the army then spreads out and attacks any armor that is not dug in. Well protected areas are isolated so they can’t be resupplied.
If the attacking side is prepared to commit war crimes the attacked side would surrender by now as the drone army can attack most towns/cities.
Finally even a weak country has conventional forces. These can then enter mostly unopposed, infantry in buildings are helpless against basic artillery, and tunnels etc cannot protect the civilian population.
So my point is that in a apparently even conflict (judged by conventional strength) one side could suddenly get a large advantage—Iran could take Saudi Arabia, and reach Riyadh quickly. Also Russia could suddenly threaten Europe in a way it can’t do with its existing forces.
Trophy sounds less effective than phalanx for missile defense in this situation:
https://en.wikipedia.org/wiki/Trophy_(countermeasure)
“The system is currently incapable of defeating kinetic energy anti-tank weapons.”
So a Javelin type missile that released a rail gun type slug when on the outside of the Trophy defense range would destroy the target.
“In the ATGM’s case, the EFP will affect the shaped plasma jet, dramatically decreasing its penetration ability.”—This sounds a normal missile will still cause damage, and since Phalanx is not as armored as a tank, probably destroy it.
I don’t see why https://newatlas.com/drones/huntress-turbojet-drone/ should cost >$100K when mass produced. In the context of this article, before we talk about strengths/weaknesses of Iron dome type defenses, the vast majority of countries don’t have them currently deployed and can’t afford to.
For the phalanx or similar—check this link this link. The consensus seems to be that the gun can’t take out multiple missiles.
“We’re gonna assume that by “missile” you mean a trans-sonic rocket propelled guided warhead, which is incapable of significant evasive or deceptive course changes within the last 1500 meters of its approach. We’re further going to assume that by CIWS you mean a guns only Phalanx installation or foreign equivalent, firing 4200–4500 rounds per minute of 20mm kinetic energy projectiles at 1200 meters per second. The unit would have the standard assemblage of radars and fire control.
A typical Phalanx installation has about a eighty percent chance of destroying two of these missiles if they are detected within ten seconds of each other in an unobstructed field of fire. If the missiles are “jinking” to avoid destruction, the chance of getting both with one Phalanx unit drops to thirty percent. If the missiles enter the kill zone more than ten seconds apart, the kill rate climbs to well over ninety percent.”
Wikipedia gives a Javelin (surface to air) with a max speed of “Mach 1.7+ approx.” EDIT the ground one is much slower, probably because its has more penetrating power. I don’t think we need that against the gun, it is not as armored as a tank.
I don’t doubt that a fixed gun can take out drones, but its missiles vs gun that matters here. At 115 degrees per sec, 2 Javelin 120% apart going Mach 1.7 is going to be a serious problem for it. I get a cost quoted of about US$5M for the land based gun vs $78K for the missile. https://www.thedefensepost.com/2023/03/02/us-uk-javelin-missile/
If it was made cheaper, would it be less effective? Also note that the land based system needs to withstand armor piercing rounds from the heaviest gun something like this drone could carry. Sure you could destroy the drone but probably not before it can get some rounds off. Especially if it engaged from >1500m out. That drone could also carry the Javelin and I doubt someone (soldier) could shoot it down easily.
OK don’t agree with that—I think there is great benefit to flying low/slow/cheap. For a start the logistics drones wont be in maximally hostile—the other drones will have taken care of anything that could easily destroy a logistics drone. I was thinking of this drone as a logistics drone, but I its load is too low. Instead imagine the logistics drones are modified fixed wing from micro-light to Cessna specs (single prop, chemical powered). This gives large range and reasonable carrying capacity. Add some elec powered quadcopter like rotors for short bursts of power. This could fly low over roads as i have said, and even perhaps have strengthened landing gear wheels so it can cruise along the road to save fuel. The elec rotors let it take off much faster than normal aircraft. It could be say $~100K rather than F35 cost.
I don’t see how this can be easily attacked by bombs—for a start the bomb needs to out-run it laterally. Secondly if the bomb is locking onto the rotor, then it will be giving off radar and be able to be picked up and targeted. How is it going to stop a cheaper suicide drone exploding next to its fins and disabling them so it can’t track?
If it can handle that, then how will it tell from prop signature which is a logistics drone vs cheap drone? Cheap drones could deliberately have a similar rotor signature or spoof one with a radar transmitter.
When the bomb is detected (say 10 seconds warning) how about if the logistics drone stops its propellor, and puts a brake on it to slow it down fast, at the same time as the jet powered drone mentioned before takes off from it? Surely the bomb will be tricked into chasing and not catching that instead?
Finally (and I am not expert on radar) can the prop signal be concealed by a metal plate mounted above it (think a spoiler but over the front and square) thus blocking the signal from say +20% from any angle above. Can the plate be filled with material that will either absorb or reflect radar so that the prop underneath cannot be detected?
Thanks for the info. What about RF weapons that is a focused short or EMP pulse against a drone. What range and countermeasures?
Ok sure without going into the details I don’t dispute an advanced gun system can take out the drones. However such a system counts as armor in my system. It would get the anti armor attack I have described elsewhere. Surely the gun costs more than a suitably sized missile so we get back to coordinated drone swarm with different units vs fixed armor
Upon further thought the logistics drones fly low over roads and disguise themselves e.g same color so that they cannot be seen from that altitude. Cheap decoy drones with the same cross sectional area from above would also seem to work very well
Good point. Am on holiday so can’t reply in detail. However drone deploys chaff turns sideways changing profile in a way heli can’t. Bomb loses lock. Also cheaper drones can attack bomb sensor and control surface. Logistics drone can also deploy decoys such as large parachute or similar looking decoy drones. Bomb can be optimized however to being cheaper faster glide and turn speed. Also swarm of bombs can communicate. Where do you think that would lead if both sides optimized
OK but why would you need high res for the minds? If its an ancestor sim and chatbots can already pass the Turing test etc, doesn’t that mean you can get away with compression or lower res? The major arc of history won’t be affected unless they are pivotal minds. If its possible to compress the sims so they experience lesser consciousness than us but still are very close to the real thing (and havn’t we almost already proven that can be done with our LLM’s), then an ancestor simulator would do that.