Old man 1: Life is one trouble after another. I’d be better off dead, better yet, I wish I was never born
Old man 2: True, true, but who has such luck ?.. maybe one in a thousand.
My blog: https://cerebralab.com
I’m also building an open source generic ML library: https://github.com/mindsdb/mindsdb & https://github.com/mindsdb/lightwood …. which I guess might be of interest to some people here
I would point to AIXI as the most “intelligent” agent possible in the limit. This has a very formalized definition of “intelligent agent”. Intelligence doesn’t have to be human-like at all.
This seems to be a silly though-experiment to me:
To quote from the article
There is an agent, and an environment, which is a computable function unknown to the agent
which is equivalent to
There is an agent and a computable function unknown to the agent
If you want to reduce the universe to a computable function where randomized exploration is enough for us to determine the shape… and exploration is free… then yeah, I can’t argue against that reductionist model.
In the reality we live in, however, we don’t know:
a) Random exploration is not cheap, indeed, random exploration is quite expensive. Acquiring any given “interesting” insight about our shared perception (aka the world) is probably costly to the tune of a few hundreds of millions of dollars, or at least to the tune of a lot of wasted time and elecricity if we lower the bar for interesting.
b) “Simpler computations are more likely a priori to describe the environment than more complex ones” … there are people working on encoding that have wrestled with a version of this problem, it ends up not being a very simple heuristic to implement, even when your domain is as limited as “The sum of all 50s jazz songs”, I think a heuristic that even roughly approximates simplifying an equation that describes something as complex as the world is impossible to reach.… and if the simplification is done one a small part of the equation there’s no guarantee the equation you end up with won’t be more complex than if you were to not simplify anything.
c) Whether the universe contains enough resources to define itself, or even a small portion of itself within some given margin of errors, is unkown to use. It might be that all the resource in the universe are not enough to simulate the behavior of the fundamental particles in an apple (or of all humans on Earth). There’s actual observations about seeming “randomness” in particle physics that would back up this view. This argument I make, since I assume most proponents of AIXI will assume it might try to gain “real” insights via some sort of simulation-building.
But maybe I’m miss-understanding AIXI… in that case, please let me know
Recruiting and training humans takes time, whereas if you already have an ML model that performs well on a given task, you only need to acquire the relevant hardware to run the model. If hardware is cheap enough, I can see how using artificial intelligence could be much cheaper than spending money on {training + recruiting + wages} for a human. Automation in jobs such as audio transcription exemplify this trend -- although I think the curve for automation is smooth as the software services require continuously less supervision over time as they improve.
In theory I agree with this, in practice:
It seems that the scope of ML has remained stable over the last 40 years or so (various NLP tasks, image classification and element outlining/labeling, numerical equation construction to predict a category/number… with added generative models for images that seem to have only recently gained interest).
In spite of the reduced scope of tasks it seems that the amount of people working on maintaining ML infrastructure and working in ML research is increasing.
A specialized company seems to pop up in every field from cabbage maturation detection to dog breed validation with it’s dozens of employees our of which at least a few are actually responsible for the task of copy-pasting code from github, and often enough they seem to fail at it or perform unreasonably badly.
Ever had to figure out why the specific cudnn/pytroch/tensorflow setup on a given environment is not working ?
Granted, again, I do agree in theory with your point. I don’t think my argument relies on replication cost. But I can’t see a future where replication costs are not a huge issue the same way I can’t see a future where everyone agrees on {X}, it’s no theoretically impossible, far from it, but technical over-complexity and competing near-equivalent standards is an issue with social roots that humans can’t fix.
I agree with points 2 and 3. But I think it’s only really a counter for the equivalency between AI and human-brain-baring systems. I think the article in itself stands in spite of that. (See my other comments, especially the one in reply to Daniel Kokotajlo>
Is this list still being maintained and/or discussed over ?
I feel like the ML text-book being recommended *could* at least use an alternative in the form of: http://www.deeplearningbook.org/ , it takes a purely frequentist perspective (but consider that’s basically the “practical” perspective at the moment, with even the bayesianNN work being… not so Bayesian), but it’s much more concise, does a good job at explaining the math and skips over historical stuff that people either know of already (e.g DT) or that is essentially useless outside of niche applications and legacy systems (e.g. kernel trick).
Or possibly even the fast-ai course http://course18.fast.ai/ml , granted, it’s no a text-book per-say, but the combination of the notes is textbook-like.
At least it would be worth creating a “Modern automatic differentiation modeling” section or something for it, if people disagree that ML can be essentially reduced to “whatever the top papers on paperswithcode are doing in the last 4 or 5 years”.
I’ve taken the principle of never learning anything (or claiming to know it) unless I can deeply understand it to a very extreme place.
By “deeply understand” I mean, be able to apply it on a whole other set of inputs/problems than those that you saw it applied to. Or, if that is not doable, the more abstract version of “be able to explain it with your own words in such a way that it could reach vastly different people than those that would have understood it from the original author’s words” (which, to some extent, is based of Feyman idea).
By “never learning” I mean not starting to learn something unless I plan to reach this point and see it as feasible to reach.
Which essentially removes the “complicated” problem since using this method nothing stays complicated once you learn it almost by definition. (not, nothing stays complicated but most things still stay complex, by “complicated” I understand “hard to navigate or use”, by “complex” I understand “requiring a very large map or frequent checking of the manual”).
I think it helps create some very efficient primitives in one’s brain, at least for certain things, but I’m not sure it’s an approach I’d recommend, it puts you at odds with many things. I think wrote-memorization has it’s place and in a way it’s good to “learn” things that way, or at least necessary.
Retracted, as I can’t delete it and the content was *really* bad.
Hmh, I interpret standard nerual netwroks (which are the ones it focuses on) to be frequentist, since you are essentially maximising a likelihood without any priors and without an built-in uncertainty.
There’s the whole bayesian nn world where the focus is on being able to easily embed priors and treating every cell as a probability distribution and obtaining a probability distribution for every output cell (which is the important part).
In practice this doesn’t differ much, since you’re essentially just adding a few more terms to every weight and bias, but it seems to be a field that’s picking up speed… then again, I might just be stuck in my own reading bubble.
I guess upon further consideration I could scratch that whole thing, I’m honestly unsure if baesyan/frequentist is even a relevant distinction to be made anymore about modern ML/statistics/
Math is naught but thought experiments, and yet unreasonably effective in science.
A reduce sub-set of mathematics, yes, but that reduced sub-set is all that survives. Numerology, for example, has been and still is useless to science, however that hasn’t stopped hundreds of thousands of people from becoming numerologists.
Further more, math is often used as a way to formalize scientific finding, but the fact that a mathematical formalism for a thing exist doesn’t mean that thing exists.
Also, AIXI has been directly approximated using Monte Carlo methods, and the resulting agent systems do show “intelligent” behavior, so the formalism basically works. I am not suggesting this is a good path to AGI, that’s not the point.
This is the point where I start to think that, although we both seem to speak English, we certainly understand fully different things from some words and turns of phrase.
Why bring up AIXI if you yourself admit you are not going to defend the approach as a good path to AGI ?
Why is a system with “some” intelligent behavior proof that the paradigm is useful at all ?
I can use pheromones to train and ant to execute certain “intelligent” tasks, or food to train a parrot or a dog. Yet the fact that African parrots and indeed learns simple NLP tasks leads no credence to the idea that and African parrot based bleeding-edge NLP system is something worth pursuing.
And I’m pointing out that we have a definition already, and that definition is AIXI.
If we have a definition that is non-functional, i.e. one that can’t be used to actually get an intelligent agent, I would claim we don’t have a definition.
We have some people that imagine they have a definition but have no proof said definition works.
Would you consider string theory to be a definition of how the universe works in spite of essentially no experimental evidence backing it up ? If so, that might be the crux of our argument here (though one that I wouldn’t claim I’m able to resolve)
That’s called “physics”! We’re using computer programs to model the universe. The map is not the territory.
Yes, and physics is *very bad* at modeling complex systems, that’s why 2 centuries were spent pondering about how to model systems consisting of 3 interacting objects.
Physics is amazing at launching rockets into space, causing chain reactions and creating car engines.
But if you were to model even a weak representation of a simple gene (aka one that doesn’t stand up to reality, one that couldn’t be used to model an entire’s bacteria DAN plus afferent structural elements), it would take you a few days and some super-computers to get a few nano-seconds of that simulation: https://onlinelibrary.wiley.com/doi/abs/10.1002/jcc.25840
That is not to take anything away from how amazing the fact that we can model the above is, but if you are assuming that an efficient yet reasonably cheap (computationally) model of the universe exists, or if you are assuming it *can* exist you are ignoring all evidence thus far which is pointing towards the fact that:
a) It doesn’t exist
b) There’s no intuition or theory pointing us towards the fact that one could be built, and indeed there’s some evidence (again, see inherent randomness or co-dependences of various particles that can propagate into large differences in macro systems if even a single particle is modeled incorrectly).
Also known as Occam’s Razor. Solomonoff induction, which AIXI is based on, is a formalization of the principle. Since the hypothesis space is literally all possible computer programs, the set is infinite. We can’t very well assign them all equal probability of being the correct model, or our probabilities would add up to infinity instead of the expected 100%.
No idea what you’re on about here
We humans predict small portions of the universe all the time. It’s called “planning”. More formally, it’s called “physics”. To the extent that parts of the universe are truly random, it’s irrelevant to the question of artificial intelligence, which need only concern itself with predicting what it can, and accounting for uncertainty in the rest. Humans are no better. But even quantum physics is Turing computable. We have no good reason to think that the laws of physics are not computable, but there are unknown initial conditions.
Humans are better, since humans can have sensory experience and they can act with the environment.
A hypothetical system that *learns* the universe can’t interact with the environment, that’s the fundamental difference and I spend a whole chapter trying to explain it.
Assuming a human can’t model “X” that’s fine, because a human can design an experiment to see how “X” happens, that’s how we usually do science, indeed, our models are usually just guidelines for how to run experiments.
All but madmen would use theory alone to build or understand a complex system. On the other hand, we have managed to “understand” complex systems many times with not theoretical backing whatsoever (e.g. being able to cure bacterial infections via antibiotics before being able to even see single bacteria under a microscope, not to even mention having some sort of coherent model about what bacterias are and how one might model them, which is still an issue today).
If our hypothetical AGI is bound by the same modeling limitations we are, then it has to run experiments in the real world, and once again we come up to the cost problem. I.e. the experiments needed to better understand the universe might not be too hard to think of, they might just take too long, be physically impossible with our current technology or too expensive… and then even a SF-level AGI becomes as slightly better scientists, rather than a force for unlocking the mysteries of the universe.
I wonder if Social Justice missionaries will become a thing.
It seems to me that the SJ value-set/religion is growing more powerful and widely accepted, and the more this happens the more you get outliers. Either trying to pander to it in order to unscrupulously raise their social standing at the cost of others or “true believers” which take it’s dictums to the extreme.
To some extent, it must be that the heavily religious Europe of the 13th to 18th century suffered from these same issues. It also seems plausible that outliers might have been swayed to become missionaries. Since “spreading the values” can be very profitable for those with few scrupules and very appealing to the extreme moralists which want to value-maximize as much as possible.
It seems like the kind of thing that could happen without centralized intervention, it’s almost a natural conclusion that can be reached once you get enough outliers and enough people supporting the religion that to them it seems “obvious” it must be spread.
People want the outliers to leave (but can’t make this an open preference, since they couldn’t defend it within the dogma of the religion) and the incentives needed for it to happen are very easy to produce. So missionaries start being socially accepted and praised.
Granted, I can’t see any ‘good’ examples of this happening yet, so maybe I’m just speculating on an imaginary foundation.
Implementing true AIXI is not physically possible. It’s uncomputable. I did not say that “AIXI” is not a good path to AGI. I said that a Monte Carlo approximation of AIXI is not a good path.
So, you as saying “it’s no a good path”
*but* then you claim:
And it’s not that it can’t work, clearly it does (and indeed any intelligent agent, humans included, is going to be some kind of an approximation of AIXI) but that the Monte Carlo AIXI has certain problems that make the approach not good:
My argument here is that it doesn’t, it’s an empty formalism with no practical application.
You claim that this approach can work, but again, I’m saying you don’t understand the practical problems of the universe we live in: There are literal physical limitations to computational resources. This approach is likely too stupid to be able to simulate anything relevant even if all the atoms in the universe were to be arrange to create an optimal computer to run it. Or, at least, you have no leg to stand on claiming otherwise considering current performance of an optimize yet inferior implementation.
So essentially:
You are claiming that an in-practical model is proof that a practical model could exist because the in-practical model would work in a fictional reality thus a practical.
It makes no sense to me.
That seems like a potentially very unhealthy thing when applied to “basic” desires such as food and sex… Unless yoloing your way through a life of hookers, coke (the sugary kind) and jelo seems appealing.
Our first order desires usually conflict with our long terms desires, and those are usually much better to aim for.
But maybe I’m getting something wrong here. Where did you get this idea from ?
Now you are literally using Zeno to steal cattles.
The problem with your very wide perspective seems to be that you are basically taking Pascal’s wager.
But for now I honestly don’t have time to continue this argument, especially if you’re gonna take the read-this-book-peasant style approach to filling understanding/interpretation gaps.
Though this discussion has given me an idea about a “Why genetically engineered parrots will bring about the singularity” as a counter-argument to this kind of logic.
Other than that, congrats you “win”, I’m afraid however that I no further understand your position or why you hold it then when we began. Nor do I understand what would change your position or what it’s principal pillars are… :/
I don’t see why, it seems that the paper doesn’t challenge the assumption that the “enlightened” state or the “dmn reduced activation” sate or whatever you want to call it results in less happiness.
Even more so, a lot of the effects described could also fall under aging and might be confounded by the fact that most of these people have aged during the time they trained (e.g. not being able to remember appointments).
Finally, you don’t have to bring meditation/introspection practices to the point where you get that “enlightened” state a constant in your life. I think there’s a fair argument to be made for dzogchen-like practices (e.g. the kind of quackery people like Sam Harris and Loch Kelly expose), if done in moderation, can allow you to “silence the dmn” or “separate awareness/attention/consciousness”, in a selective way rather than all the time. I.e. get to a point where your brain is able to experience something close to a small-medium dose of psilocybin for a short amount of time.
But, again, this final point is based on empirical quackery,but then again, so is most of the “research” around meditation when you compare it to other types of research (Note: I’m not blaming the way the research is done, I’m saying that the thing that they are trying to observer in combination with the tools they have means they will get very little insight… you can only do so many diffusion tensor MRIs on healthy people until it becomes unethical and the data you get from is very broad compare to, say, that you would get about kidney function from a biopsy + blood/urine testing using a broad range of separation techniques and chemical assays within reasonable cost)
That’s actually a good point.
But what about places that are closed-off from the “global” virtual space (e.g. China, Arabia and possibly various African countries once their dictators get up to speed with technology) ?
All your needs are legitimate, but not all of your strategies to meet those needs are legitimate
Even in this form I don’t believe this sentence holds.
For example, I am a smoke (well, vaper, but you get the point, nicotine user). I can guarantee you I have a very real need for:
a) Nicotine’s effect on the brain
b) The throat hit nicotine gives
c) The physical “action” of smoking
Are those needs legitimate in the sense you seem to understand them ? Yes, they are pretty legitimate, or at least I can associate them to be on the same degree as other needs that most people would consider legitimate (e.g. need to take a piss, need to talk with a friend, w/e)
Must those needs stay legitimate ? No, actually, having taken breaks of up to half a year from the practice I can actually tell those needs get less relevant the longer you go without smoking
Should those needs stay legitimate ? Well, I’d currently argue “yes”, since otherwise I wouldn’t be vaping as I’m writing this. But, I’d equally argue that from a societal perspective the answer is “no”, indeed, for parts of my brain (the ones that don’t want to smoke), the answer is “no”.
1. Now, either smoking is a legitimate need
OR
2. Some needs that “seem” legitimate should actually be supressed
OR
3. Needs not only need to “feel/seem” legitimate, they also need to have some other stamp of approval, such as being natural
1 - is a bad perspective to hold all things considered, you wouldn’t teach your kid that you caught smoking that he should keep doing it because it’s a legitimate need now that he kinda likes it.
2 - seem to counter-act your point because we can now claim any legitimate need should actually be suppressed rather than indulged in some way.
3 - You get into a nurture vs nature debate… in which case, I’m on the “you can’t really tell” side for now and wouldn’t personally go any further in that direction.
I’m wondering if wealth-re-distribution in the “best case” scenario would have any positive effects.
So, assume a world in which the top-x wealthy have wealth that resides purely in the form of gold. Not stocks, or bond or whatever, so taking it away from them won’t de-stabilize the economy in the same sense that taking away stock from a CEO will break down the incentives chain that keep a company running properly.
Also, assume that everyone likes gold so much, that it will keep it’s inherent value once it’s re-distributed.
Would this have any effect besides a sudden~ish price inflation for stuff like food, housing, cars… etc, since everyone can now buy more/better, thus demand goes up at once with no modification in production.
As in, would this wealth redistribution be able to re-direct more resources into relevant industries to keep them more efficient ?
It seems to me that a lot of common-usage products are already heavily efficient in terms of production compared to luxury product, since the market is much broader.
Where, improvements can be made, the issue usually seems to be regulatory/ethical/consensus-related (e.g. zoning laws for housing, experimental ethics and drug-trail statistical power regulations for medical research).
So, for example, custom-cat production is very expensive not because the materials are expensive, but because a custom yacht requires loads of specialized artisan work. However, if all those artisans were to go out of business and be forced to get jobs at a mass-production facility… would that bring any benefits in terms of how the cars are designed to make production cheaper ? Even assuming 10 years pass and they are re-trained as highly-skilled mass-car producers, would that help ? Or are there physical limitations (e.g. cost of materials, time needed to ensamble an engine) or regulatory limitations (e.g. safety testing) that keep the prices of cars above a certain threshold (seems to be ~4,000euros) for the cheapest possible car one could make.
Same question goes for all mass-produced items.
Again, I think you are veering into religious thinking here, just because something has Eliezer Yudkowsky’s name on it it doesn’t mean that it’s true.
Personally I know the essay and I happen to fundamentally disagree with it. I’m a pragmatic bayesian at the best of times and a radical empiricist at my worst, so the kind of view Eliezer espouses has very little sway on me.
But despite the condescending voice you give this reply, if I am to make a very course assumption, I can probably summarize our difference here as me putting too much weight behind error accumulation in my model of the world or you not taking into account how error accumulation works (not saying one perspective or the other is correct, again, this is I think where we differ and I assume our difference is quite fundamental in nature).
Given the fact that your arguments seem to be mainly based on simple formal model working in what I see as an “ideal” universe, from which you then draw your chain of inferences leading to powerful AGI, I assume you might have a background in mathematics and/or philosophy.
I do think that my article is actually rather bad at addressing AGI from this angle.
I’m honestly unsure if the issue could even be addressed from this perspective, but I do think it might be worth a broader piece addressing why this perspective is flawed (i.e. an argument for why a perspective/model/world-view based on a long inferential distances is inherently flawed).
So, I honestly think this conversation might not have been pointless after all, at least not from my side, because it gives me an idea for an essay and a reason to write it.
Granted, I assume you have still gained nothing in terms of understanding my perspective, because quite frankly I did a bad job at addressing it in such a way that you would understand, I was not addressing the correct problem. So for that I am sorry.
Then again, I might be making too many assumptions about your perspective and background here, stacking imperfect inference upon imperfect inference and creating a caricature that does not match reality in any meaningful way.
That is actually a good point, I was focused too much on material goods and not thinking of service jobs.
Indeed, even if you take the real form of redistribution, which is closer to evening out social status than redistributing any form of real wealth. It would probably incentivize people to go into arguably useful service jobs more. (e.g. there are probably a lot of people which would be good medical researchers that become traders or “tech entrepreneurs” because in our current world it yields much more social status, even if the difference in actual material goods is not so great, wealth itself allows for signaling high status).
For some reason, despite reading socialist philosophers/activists which make these arguments… I’m just unable to stick them anywhere in my brain in such a way that I can remember them next time I even think about trying to argue a strong-man representation of redistribution.
Thanks for pointing this out.
a)
The first part of the article was not meant to me taken on it’s own, rather, it was meant to be a premise on which to say: “There is no guarantee AGI will happen or be useful and based on current evidence of how the things we want to model AGI after scale I’m inclined to think it won’t be useful”.
Compare it, if you wish, to a someone giving the argument “planes have flown through the clouds and satellites have taken photos of space and studies have been done on the effects of prayers and up until now no gods have been found nor their effects seen”. It’s a stupid argument, it doesn’t in itself prove that there is no God, but it’s necessary and I think it can help when everyone thinks there is a God.
Since everyone except for (funny enough) most ML researchers I’ve ever meet (as in, people that are actually building the AI, guys like Francois Chollet, not philosophers/category-theorists/old professors that haven’t publish a relevant paper in 30 years :p), seem to come from the (seemingly irrational) premise that AGI is a given considering how technology advances.
I don’t particularly think that this argument is back-up-able more so than the “there is no God” or “money has no inherent value” argument is back-up-able. Since, funnily enough, arguing against the purely imaginary things is basically impossible.
It’s impossible to prove the in-existance of AGI or the equivalency of humans with AGI or argue about power consumption and I/O overhead + algorithmic complexity required for synchronization.… on computer that don’t exist. At most I can say “computer today are much less efficient in terms of power consumption than the human brain at tasks like NLP and image-detection” and I can back it up with empirical evidence like how much power a GPU consumes vs how much claories your average human requires, and comparing those as W and as cost-of-production. At most I can come up with examples like Alexa and Google using m-turk like systems for hard NLP tasks rather than ML algorithms (despite both of them having invested literal billions in this are and academia having worked on it for 60+ years).
But at the end of the day I know that these argument don’t disprove AGI, they just prove that I don’t understand technology enough to realize that AGI is inevitable.
Still, I think these kind of arguments are useful to hopefully make fence-sitters realize how silly the AGI position is, the later two chapters are my arguments for *why* even the AGI God converts imagine will not be as all-powerful as we might think.
b)
I think there’s a lot of place where I’m unclear in the article because I oscillate between what kind of language to use. E.g.:
Here by “test” I mean something like “Given an intrinsic motivation agent augmented by bleeding-edge semi-supervised models to help it at complex & common tasks, it would still need to be given a small amount of physical resources for it to train on how to use them in a non-simulated environment and for us to evaluate it’s performance.… which would take a lot of time, since you can’t speed up real life and would be expensive” rather than “Allow skynet to take control of the nuclear arsenal for experimental purposes”
I think that it’s mainly my fault for not being more explicit with stuff like this, but the other side of that is articles turning into boring 10,000 page essays with a lot of ****, I will try to update that particular statement though.
c)
I actually think that, from your perspective, I could be seen as arguing this.
I’m pretty sure that from your perspective I would actually hold this view, the clarification I made was to specify that I don’t think this view is absolute (i.e. I think that AI will leads to
xhuman progress and most proponents of AGI seem to think it will lead tox * 100,000,000, but in spite of that difference I think evenxwill be significant).At least if you count human progress in something simple to measure like how much energy we capture and how little energy we have to spend on building nice human housing an delicious human food (e.g. a civilization with a Dyson sphere would be millions of times as advanced as one without one under this definition)