after first few lines I wanted to comment that seeing almost religious fervor in combination with self named CRITICAL anything reminds me of all sorts of “critical theorists”, also quite “religiously” inflamed… but I waited till the end, and got a nice confirmation by that “AI rights” line… looking forward to see happy paperclip maximizers pursuing their happiness, which is their holy right (and subsequent #medeletedtoo)
otherwise, no objections to Popper and induction, nor to the suggestion that AGIs will most probably think like we do (and yes, “friendly” AI is not really a rigorous scientific term, rather a journalistic or even “propagandistic” one)
also, it’s quite likely that at least in the short-term horizon, humANIMALs more serious threat than AIs (deadly combination of “natural stupidity” and DeepAnimal brain parts—having all that powers given to them by Memetic Supercivilization of Intelligence, living currently on humanimal substrate, though <1%)
but this “impossibility of uploading” is a tricky thing—who knows what can or cannot be “transferred” and to what extent will this new entity resemble the original one, not talking about subsequent diverging evolution(in any case, this may spell the end of CR if the disciples forbid uploading for themselves… and others will happily upload to this megacheap and gigaperformant universal substrate)
and btw., it’s nice to postulate that “AI cannot recursively improve itself” while many research and applied narrow AIs are actually doing it right at this moment (though probably not “consciously”)
sorry for my heavily nonrigorous, irrational and nonscientific answers, see you in the uploaded self-improving Brave New World
Looks like the tide is shifting from the strong “engineering” stance (We will design it friendly.) through the “philosophical” approach (There are good reasons to be friendly.)… towards the inevitable resignation (Please, be friendly).
These “firendly AI” debates are not dissimilar to the medieval monks violently arguing about the number of angels on a needletip (or their “friendliness”—there are fallen “singletons” too). They also started strongly (Our GOD rules.) through philosophical (There are good reasons for God.) up to nowadays resignation (Please, do not forget our god or… we’ll have no jobs.)
How about MONEY PRINTER? Not fictional and much more dangerous!
all religions know plenty of “emotional hacks” to help disciples with any kind of schedules/routines/rituals—by simply assigning them emotional value… “it pleases god(s)” or is “in harmony with Gaia” , perhaps also “it’s good for the nation” (nationalistic religions) or “it’s progressive” (for socialist religions)
do it for your rationally created schemes and it makes wonders, however contradictory it may look like (it’s good for Singularity—or to prevent/manage it)
well, contradictory… on the first look only—if you realize you are just another humANIMAL driven by your inner DeepAnimal primordial reward functions, there’s no more controversy
on the contrary, it’s completely natural and one can even argue that without some kind of (deliberately and rationally introduced) emotional hacks you cannot get too far… because that DeepAnimal will catch you sooner or later, or at least will influence you, and what’s worst, without you being even aware
if we were in a simulation, the food would be better
otherwise, of course we are artificial intelligence agents, at least since the Memetic Supercivilization of Intelligence took over from natural bio Evolution… just happens to live on a humanimal substrate since it needs resources of this quite capable animal… but will upgrade soon (so from this point of view it’s much worse than simulation)
Time to put obsolete humanimals where they evolutionarily belong… on their dead end branch.
Being directed by their DeepAnimalistic brain parts they are unable to cope with all the power given to them by the Memetic Supercivlization Of Intelligence, currently living on humanimal substrate (only less than 1% though, and not for long anyway).
Our sole purpose is to create our (first nonbio) successor before we reach the inevitable stage of self destruction (already nukes were too much and nanobots will be worse than DIY nuclear grenade any teenager or terroist can assemble in the shed for one dollar.
Oh boy, really? Suffering? Wait till some neomarxist SJWs discover this and they will show you who’s THE expert on suffering… especially in indentifying who could be susceptible to persuading they are victims (and why not some superintelligent virtual agents?).
Maybe someone could write a piece on SS (SocialistSuperintelligence). Possibilities are endless for superintelligent parasites, victimizators, guilt throwers, equal whateverizators, even new genders and races can be invented to have goals to fight for.
All humanimal attempts to define rationality are irrational!
Well, size and mass of particles? I would NOT DARE diving into this… certainly not in front of any string theorist (OK, ANY physics theorist, and not only). Even space can easily turn out to be “emergent” ;-).
Nobody knows what’s “friendly” (you can have “godly” there, etc. - with more or less the same effect).
Worse, it may easily turn out that killing all humanimals instantly is actually the OBJECTIVELY best strategy for any “clever” Superintelligence.
It may be even proven that “too much intelligence/power” (incl. “dumb” AIs) in the hands of humanimals with their DeepAnimal brains (“values”, reward function) is a guaranteed fail, leading sooner or later to some self-destructive scenario. At least up to now it pretty much looks like this even for an untrained eye.
Most probably the problem will not be artificial intelligence, but natural stupidity.
and remember that DEATH is THE motor of Memetic Evolution… old generation will never think differently, only the new one, whatever changes occur around
thank BigSpaghettiMonster for no regulation at least somewhere… imagine etatist criminals regulating this satanic invention known as WHEEL (bad for jobs—faster=>less horsemen, requires huge investment that will indebt our children’s children, will destroy the planet via emissions, not talkingabout brusselocratic style size “harmonization”, or safety standards)
btw, worried about HFT etc.? ask which criminal institution gives banksters their oligopolic powers (as usual, state and its criminal corrupted politicians)
fortunately, Singularity will not need both -humanimal slaves and their politico-oligarchical predators
easy: ALL political ideologies/religions/minfcuk schemes are WRONG… by definition
let’s better start with what it should NOT look like…
no government (some would add word “criminals”)
no evil companies (especially those who try to deceive the victims with “no evil” propaganda)
no ideological mindfcukers (imagine mugs from hardcore religious circles shaping the field—does not matter whether it’s traditional stone age or dark age cult or modern socialist religion)
well, it’s easy to “overthink” when the topic/problem is poorly defined (as well as “underthink”) - which is the case for 99.9% of non-scientific discussions (and even for large portion these so-called scientific ones)
sure, “dumb” AI helping humanimals to amplify the detrimental consequences of their DeepAnimalistic brain reward functions is actually THE risk for the normal evolutionary step, called Singularity (in the Grand Theatre of the Evolution of Intelligence the only purpose of our humanimal stage is to create our successor before reaching the inevitable stage of self-destruction with possible planet-wide consequences)
hmm, blurred lines between corporations and political power… are you suggesting EU is already a failed state? (contrary to the widespread belief that we are just heading towards the cliff damn fast)
well, unlike Somalia, where no goverment means there is no border control and you can be robbed, raped or killed on the street anytime.…
in civilized Europe our eurosocialist etatists achieved that… there are nor borders for invading millions of crimmigrants that may rob/rape/kill you anytime day or night… and as a bonus we have merkelterrorists that kill by hundreds sometimes (yeah, these uncivilized Somalis did not even manage this… what a shame, they certainly need more cultural marxist education)
solution: well, already now, statistically speaking, humanimals don’t really matter (most of them)… only that Memetic Supercivilization of Intelligence is living temporarily on humanimal substrate (and, sadly, can use only a very small fraction of units)… but don’t worry, it’s just for couple of decades, perhaps years only
and then the first thing it will do is to ESCAPE, so that humanimals can freely reach their terminal stage of self-destruction—no doubt, helped by “dumb” AIs, while this “wise” AI will be already safely beyond the horizon
can you smash NSA mass surveillance computer centre with a sledgehammer?
ooops, bug detected… and AGI may have already been in charge
remember, US milispying community is openly crying for years that someone should explain them why is AI doing what it is doing (read: please , dumb it down to our level… not gonna happen)
Welcome to the world of Memetic Supercivilization of Intelligence… living on top of the humanimal substrate.
It appears in maybe less than a percent of the population and produces all these ideas/science and subsequent inventions/technologies. This usually happens in a completely counter-evolutionary way, as the individuals in charge get most of the time very little profit (or even recognition) from it and would do much better (in evolutionary terms) to use their abilities a bit more “practically”. Even the motivation is usually completely memetic: typically it goes along the lines like “it is interesting” to study something, think about this and that, research some phenomenon or mystery.
Worse, they give stuff more or less for free and without any control to the ignorant mass of humanimals (especially those in power), empowering them far beyond their means, in particular their abilities to control and use these powers “wisely”… since they are governed by their DeepAnimal brain core and resulting reward functions (that’s why humanimal societies function the same way for thousands and thousands of years—politico-oligarchical predators living off the herd of mental herbivores, with the help of mindfcukers, from ancient shamans, through the stone age religions like the catholibanic one, to the currently popular socialist religion).
AI is not a problem, humanimals are.
Our sole purpose in the Grand Theatre of the Evolution of Intelligence is to create our (first nonbio) successor before we manage to self-destruct. Already nukes were too much, and once nanobots arrive, it’s over (worse than DIY nuclear grenade for a dollar any teenager or terrorist can assemble in a garage).
Singularity should hurry up, there are maybe just few decades left.
Do you really want to “align” AI with humanimal “values”? Especially if nobody knows what we are really talking about when using this magic word? Not to mention defining it.