Not sure if this help but:
A program that generates “exactly” the sequence HTHTT
Alongside this program there is program that generates HHHHH and HHHHT and HHHTT etc − 2^5 of such programs, and before seeing evidence HTHTT is just one of those not standing out in any way. (but I don’t know how specifically Solomonoff induction accounts for it, if that is your question)
Computer viruses belong to the first category while biological weapons and gain of function research to the second.
I think it is not even goals but means. When you have big hammer every problem looks like a nail, if you good at talking you start to think you can talk your way out of any problem.
I’d add that correctness often is security: job poorly done is an opportunity for hacker to subvert your system, make your poor job into great job for himself.
Have you played something like Slay the spire? Or Mechabellum that is popular right now? Deck builders don’t require coordination at all but demands understanding of tradeoffs and managing risks. If anything those skills are neglected parts of intelligence. And how high is barrier of entry to something like Super Auto Pets?
Heard that boxfan works best if placed some distance from window, not in window.
I remember reading about zoologist couple that tried to rise their child together with baby gorilla. Gorilla development stopped at certain age and that stopped human development so they had to be separated.
Example1: Timmie gets cookies and I am glad he did even if I am dead and cant see him.
Example2: Timmie sucked into blackhole but I don’t care as I cant see him.
Today is raining and asking “why?” is a mistake because I am already experiencing rain right now, and counterfactual me isn’t me?
It seems to me explanation is confused in such a way as to obscure decision making process of which questions are useful to consider.
I would go so far as to say that the vast majority of potential production, and potential value, gets sacrificed on this alter, once one includes opportunities missed.
altar? same here:
A ton of its potential usefulness was sacrificed on the alter of its short-term outer alignment.
If some process in my brain is conscious despite not being part of my consciousness, it matters too! While I don’t expect it to be the case, I think there is bias against even considering such possibility.
Not long ago I often heard that AI is “beyond horizon”. I agreed to that while recognising how close horizon become. Technological horizon isn’t fixed time into future and not just property of (~unchanging) people but also available tech.
Now I hear “it takes a lot of time and effort” but again it does not have to mean “a lot of time and effort in my subjective view”. It can take a lot of time and effort and at the same time be done in a blink of an eye—but my eye, but not eye of whoever doing it. Lot of time and effort don’t have to subjectively feel like “lot of time and effort” to me.
Got Mega Shark Versus Giant Octopus vibes from this, but your prompts was much more sane, which made things feel inconsistent. Still somewhat enjoyed it.
Hyperloop? I am not sold on his talent being “find good things to do” as opposed to “successfully do things”. And second has a lot to do with energy/drive, not only intellect. Hence I expect his intelligence be overestimated. But I agree with your estimate, which is not what I expected?
It is important to remember that our ultimate goal is survival. If someone builds a system that may not meet the strict definition of AGI but still poses a significant threat to us, then the terminology itself becomes less relevant. In such cases, employing a ‘taboo-your-words’ approach can be beneficial.
Now lets think of intelligence as “pattern recognition”. It is not all that intelligence is, but it is big chunk of it and it is concrete thing we can point to and reason about while many other bits are not even known.[1]
In that case GI is general/meta/deep pattern recognition. Patterns about patterns and patterns that apply to many practical cases, something like that.
Obvious thing to note here: ability to solve problems can be based on a large number of shallow patterns or small number of deep patterns. We are pretty sure that significant part of LLM capabilities is shallow pattern case, but there are hints of at least some deep patterns appearing.
And I think that points to some answers: LLM appear intelligent by sheer amount of shallow patterns. But for system to be dangerous, number of required shallow patterns is so large that it is essentially impossible to achieve. So we can meaningfully say it is not dangerous, it is not AGI… Except, as mentioned earlier there seem to be some deep patterns emerging. And we don’t know how many. As for the pre-home-computer era researchers, I bet they could not imagine amount of shallow patterns that can be put into system.
I hope this provided at least some idea how to approach some of your questions, but of course in reality it is much more complicated, there is no sharp distinction between shallow and deep patterns and there are other aspects of intelligence. For me at least it is surprising that it is possible to get GPT-3.5 with seemingly relatively shallow patterns, so I myself “could not imagine amount of shallow patterns that can be put into system”
I tried Chat GPT on this paragraph, like the result but felt too long:
Intelligence indeed encompasses more than just pattern recognition, but it is a significant component that we can readily identify and discuss. Pattern recognition involves the ability to identify and understand recurring structures, relationships, and regularities within data or information. By recognizing patterns, we can make predictions, draw conclusions, and solve problems.
While there are other aspects of intelligence beyond pattern recognition, such as creativity, critical thinking, and adaptability, they might be more challenging to define precisely. Pattern recognition provides a tangible starting point for reasoning about intelligence.
If we consider problem-solving as a defining characteristic of intelligence, it aligns well with pattern recognition. Problem-solving often requires identifying patterns within the problem space, recognizing similarities to previously encountered problems, and applying appropriate strategies and solutions.
However, it’s important to acknowledge that intelligence is a complex and multifaceted concept, and there are still many unknowns about its nature and mechanisms. Exploring various dimensions of intelligence beyond pattern recognition can contribute to a more comprehensive understanding.
So I’m not super confident in this, but: I don’t have a model which suggests descaling a kettle (similar to mine) will have particularly noticeable effects. If it gets sufficiently scaled it’ll self-destruct, but I expect this to be very rare. (I don’t have a model for how much scaling it takes to reach that point, it’s just that I don’t remember ever hearing a friend, family member or reddit commenter say they destroyed their kettle by not descaling.)
Scale falls off by itself, so it is not really possible to self-destruct kettle under normal circumstances. For my 30l water heater, it took more than 5l of scale to self destruct, just no way to not notice spalled scale on such scale.
Read a bit about interaction between gas ethanol and water, fascinating!
Bullshit. I don’t believe it. Gas do not turn into water. I am sorry, but somebody “borrowing” your gas and returning water is more likely explanation. (No special knowledge here, tell me I am wrong!)
Here he touched on this (“Large language models” timestamp in video description), and maybe somewhere else in video, cant seem to find it. It is much better to get it directly from him but it is 4h long so...
My attempt of summary with a bit of inference so take with dose of salt:
There is some “core” of intelligence which he expected to be relatively hard to find by experimentation (but more components than he expected are already found by experimentation/gradient descent so this is partially wrong and he afraid maybe completely wrong).
He was thinking that without full “core” intelligence is non-functional—GPT4 falsified this. It is more functional than he expected, enough to produce mess that can be perceived as human level, but not really. Probably us thinking of GPT4 as being on human level is bias? So GPT4 have impressive pieces but they don’t work in unison with each other?
This is how my (mis)interpretation of his words looks like, last parts I am least certain about. (I wonder, can it be that GPT4 already have all “core” components but just stupid, barely intelligent enough to look impressive because of training?)
Important fact about rocket is that it provides some fixed amount of change of velocity (delta-v). I think your observation of “how long strong gravity slows me” combined with thinking in terms of delta-v budget and where it is best spent brought intuitive understanding of Oberth effect to me. Analysing linear trajectory instead of elliptic one also helps.