Well, yes, it will be shaped by what it learns, but it’s not what you train it on that matters, since you don’t get to limit the inputs and then hope it only learns “good behavior”. Human values are vague and complex, and not something to optimize for, but more of a rough guardrail. All of human output, informational and physical, is relevant here. “Safe” training data is asking for trouble once the AI learns that the real world is not at all like training data.
shminux
Karma: 27,098
That’s not how it works. If a transformative AI cannot tell dystopian fiction from actual human values (whatever they turn out to be), all is lost.
Well, if you are dealing with an adversarial situation against an equal or stronger opponent, the NFL implies that you should plan for the worst case, not a likely or average or median case. Unless I understand it wrong.
Black swans are unexpected events… AGI has been predicted for ages, just not the exact shape or timing. “An event that comes as a surprise” doesn’t seem like a good description.
Sorry, tried reading it a few times, the meaning escapes me...
Well, I don’t know who Ben Hoffman is, but the obvious answer is “good schools are good and bad schools are bad, and everything in between.”
Personally, I had a variety of experiences from quite bad to very good throughout my school years. It all depended on the mix of teachers, students, admins and my personal emotional place in the system. My own children were schooled, unschooled, private-schooled, public-schooled, depending on what was necessary and available at the moment.
The questions you are asking appear uncorrelated with what you want to learn though. Evaluate job candidates on merits, of which credentials are a part, but not a huge part. Ignore all considerations based on the conflict theory approach, like “class war.” Pick an educational framework that works best for a specific kid, unencumbered by ideological considerations. In general, keep your ideological identity small and such.
This black-and-white thinking doesn’t sound like you.
What in your mind qualifies this unfortunate event as a “catastrophe”?
I didn’t read the whole post, but wanted to chime in on the Constructor theory in physics. As a trained (but not practicing) physicist I make a categorical pronouncement that it is a load of bunk. (Were I a practicing physicist, I would make a much more careful and qualified statement.) David Deutsch is a genius with a lot of fantastic contributions to science, but that part is one of those where a genius goes off the deep end. Roger Penrose, Albert Einstein, Isaac Newton and many other top-notch physicists have or had their own pet ideas that are… not very well connected to reality.
Why I think it is bunk:
There have been zero interesting contributions from the constructor theory to our understanding of physics, let alone new testable ideas.
The main claim that physics is based on “dynamical laws and initial conditions” is patently false. Like, completely false. General relativity, Maxwell equations, Fermat principle, any least action principle are timeless and initial-condition-free.
As you say
Deutsch claims that there are certain problems in physics which are difficult or impossible to solve using the dynamical laws approach.
Indeed there are. And they are solved, but not that way, instead other approaches are used.
Some examples of egregious falsity of Deutsch’s claim that physics is based on time evolution of initial conditions with dynamical laws:
The first ever exact solution of the Einstein equation, the Schwarzschild metric, was found purely through symmetries, without any reference to initial conditions and subsequent time evolution. In fact, it is static and timeless, despite time being a variable in it. It took some half a century to even figure out an approximate approach to constructing it using initial conditions (stellar collapse).
The Godel metric, one that contains closed timelike curves everywhere and cannot be described as time evolution from initial conditions even in principle, no matter how much you try.
Simulated annealing-type approaches where each successive step is “unphysical”, but the result corresponds to a physically realizable configuration.
S-matrix approaches, where the calculation is non-local to begin with.
I have no idea about the constructor theory-like approaches in AI alignment, but my credence of it being a useful contributor to physics some day is at the noise level. That is, lost among a multitude of other unpromising ideas.
Good point, noticing is always how one starts.
Well written. Do you have a few examples of pivoting when it becomes apparent that the daily grind no longer optimizes for solving the problem?
Not sure I follow your point. Say, an alien civilization shows up and convincingly explains how humanity is a pox on the universe, and there is no way to “cure” it beyond getting rid of us. What should we do then?
[Question] If humanity one day discovers that it is a form of disease that threatens to destroy the universe, should it allow itself to be shut down?
I know very little about this area, but I suspect that a writeup like this classic explanation of Godel Incompleteness might be a step in the right direction: Godel incompleteness.
I briefly went through the google hits for “acquired savant syndrome”, and all the stories appear to be just that, nothing published/peer-reviewed. The only scientific-looking review is by Darold A. Treffert, which is published in the author’s own journal, from the looks of it.
Edit: oops, not an answer, was intended to be a comment.
well, yes, but why the dichotomy?
We may legitimately not want any civilisation that can trace its genealogy to Earth (any of our descendants) to engage in behaviours we consider especially heinous.
Hmm, trying to constrain your far descendants seems both a terrible and a futile idea. Making sure that the far descendants can replicate our reasoning exactly seems much more useful and doable.
Why is it a terrible idea? Imagine that our ancestors thought that regular human sacrifices to God of Rain are required for societal survival, and it would be “especially heinous” to doom the society by abandoning this practice, so they decided to “lock in” this value. We have a lot of these grandfathered values that no longer make sense already locked in, intentionally or accidentally.
On the other hand, it would be super useful to have a simulator that lets a future civilization trace our thinking and see the reasons for various Chesterton fences we have now.
I agree with everything, though this is just a very long way to say the https://en.wikipedia.org/wiki/Serenity_Prayer
30 is indeed roughly the inflection age, when the usual liberties one was able to take earlier with their body and mind are no longer consequence-free for an average person. If this transition was during the two plus years of covid, it makes sense that this gradual process feels like a discontinuous jump.
Thanks!