I’m Steve Byrnes, a professional physicist in the Boston area. I have a summary of my AGI safety research interests at: https://sjbyrnes.com/agi.html
steve2152
Karma: 2,883
Reminds me of a quote from this Paul Christiano post: “It’s a solution built to last (at most) until all contemporary thinking about AI has been thoroughly obsoleted...I don’t think there is a strong case for thinking much further ahead than that.”
Good insight! Haven’t seen that one before!
Think about a banging sound at 0dB, 1dB, …, 90dB. Everyone will be bothered at 90dB. Nobody will notice at 0dB. Somewhere in between, it transitions from effortless-to-ignore to impossible-to-ignore. Where is that point? It’s not determined by the laws of physics, it’s arbitrary, it’s a setting in your brain, and it’s set differently in different people. Similar for touch. Neurotypical people will not be able to ignore a woodpecker pecking them in the back, but may find it effortless to ignore a shirt tag. Other people find that the shirt tag keeps drawing their attention, but a gentle enough touch sensation would not rise to attention.
The “intense world theory of autism” (about which I’ll finish a blog post draft one of these years...) says that autism happens when empathetic social interactions are so overwhelming that the person learns early in life to just avoid thoughts and situations that solicit those feelings, including deliberately avoiding eye contact etc. Not coincidentally, people with autism are liable to have sensory sensitivity too, i.e. to feel overwhelmed by levels of sound and touch that are far lower than what it takes to overwhelm neurotypical people.
Anyway, long story short, we all have innate reactions to different stimuli, and the thresholds can be set differently. That’s just the way it is, I think.
Yeah I was really only thinking about “not yet trust the AGI” as the main concern. Like, I’m somewhat hopeful that we can get the AGI to have a snap negative reaction to the thought of deceiving its operator, but it’s bound to have a lot of other motivations too, and some of those might conflict with that. And it seems like a harder task to make sure that the latter motivations will never ever outbid the former, than to just give every snap negative reaction a veto, or something like that, if that’s possible.
I don’t think “if every option is bad, freeze in place paralyzed forever” is a good strategy for humans :-P and eventuality it would be a bad strategy for AGIs too, as you say.
Hmm, maybe I’m confused. Couple more questions, sorry if you’ve already answered them: (1) What are the differences / advantages / disadvantages between what you’re proposing vs “make an off switch but don’t tell the AGI about it”? (2) do you expect there to be another copy of the off-switch and its consequences (M) inside the St nodes? If so, is it one of “the arrows which traverse the walls of the node St”? Because I don’t see any arrows from M to St.
I imagine an AGI world-model being a bit like a giant souped-up version of a probabilistic graphical model that can be learned from scratch and updated on the fly. I agree that if there’s a node that corresponds to “I get turned off”, and you know where it is, then you can block any chain of inference that passes through that node, which amounts to the same thing as deleting the node, i.e. “making the agent not know that this is a thing that can happen”. Or a different approach would be, you could prevent that node from getting painted with a negative value (= reward prediction), or something like that, which vaguely corresponds to “I kinda like the idea that I can get turned off” if you do it right.
The big problem where I’m a bit stumped is how to reliably find the “I get turned off” node in the model. The world-model is going to be learned and changeable (I assume!). If you delete the node, the system could reinvent it. The system could piece together the existence of “I get turned off” as an abstract possibility having never seen it, or come up with four disconnected ways to think about the same thing, and then you need to find all four. I have thoughts but I’m interested in hearing yours. Or do you imagine that the programmer puts in the world-model by hand, or something?
Wow, neat!
Yeah Sowell’s books says that immediately speaking full sentences is a pretty common pattern, or at least not unheard of. I think Teller was in that category.
In fact this is one reason I’ve long been skeptical of the people who say you need “embodied cognition” to get AGI. Passive predictive (“self-supervised”) learning gets you pretty far by itself, such that you learned to speak complete sentences purely from predictive learning, not trial-and-error (or at least minimal trial-and-error).
I agree on all counts: “moo” is a word, <more realistic cow sound> is a non-word, and my kid like yours can only do the non-word version.
Interesting!
Human brain planning algorithms (and I expect future AGI systems too) don’t have a special status for “one timestep”; there are different entities in the model that span different lengths of time in a flexible and somewhat-illegible way. Like “I will go to the store” is one thought-chunk, but it encompasses a large and unpredictable number of elementary actions. Do you have any thoughts on getting myopia to work if that’s the kind of model you’re dealing with?
I was going to say Myth of Mirror Neurons by Hickok (excellent book by the way—maybe the only neuroscience book I’ve ever read where I feel like I can treat every word as gospel truth). But I just went back and double-checked, and he actually only said this about receptive language, not production.
Specifically, Hickok cites evidence that the task of distinguishing nonword sounds from each other (e.g. “ba” vs “da”) is “double dissociated” from the task of distinguishing different words (e.g. “bad” vs “dad”). In simpler terms, some people get brain damage that causes them to be able to distinguish ba-vs-da but not bad-vs-dad, and other people get brain damage that causes them to be able to distinguish bad-vs-dad but not ba-vs-da. This proves that they’re at least partly processed in different brain regions.
I still think what I wrote is probably correct (i.e. that production of animal sounds or other “sound effects” involves different brain regions than production of speech, although of course they’ll overlap by both passing through low-level motor control on the way out). But until I find direct evidence, I better fix the wording! Thanks for calling me out on that :-)
Thanks!
pretty much everyone learns to speak normally eventually
Well, Camarata says that 60% of late talkers catch up within a year or two.
In my kid’s particular case, the odds are much higher than 60% because I have a lot more information about him than just the one datapoint that he’s a late talker. I’m not worried.
It’s apparently more common for boys to have this sort of delay.
Yes, I forget the ratio, maybe 80⁄20 or 85/15? I don’t know why it’s so lopsided.
Yeah I haven’t thought about it much, but it does seem like there’s a fine line between “you get this early intervention because Society doesn’t like the way your brain is functioning and wants to change it” (=bad) vs “you get this early intervention because different kids have different needs and you’ll do best with an education tailored to your own needs” (=good). Like, other things equal, we want kids to grow into adults who can live independently, right? So if a kid is on a trajectory towards not being able to live independently as an adult, then in a sense, we do want to change that trajectory, and that might implicitly mean changing the kind of adult they grow up into, at least to some extent. Not that there would be anything wrong with the person if they followed the default trajectory. I dunno. I know nothing about ABA by the way.
As for “society isn’t on the ball”, you can say that again! Like for example, Camarata’s book says over and over, in chapter after chapter, “Do not let anyone strap your kid into a chair unless it’s for orthopedic support.” “Always ask explicitly if they strap kids into Rifton chairs.” “Write into the education plan that they will not strap your child into a Rifton chair under any circumstances”, Over and over. He says it so many times that I really get the idea that people are doing this everywhere! He says there isn’t a shred of evidence that strapping a kid into a chair is helpful for teaching anyone, with or without autism, in any circumstance (and lots of evidence that it isn’t helpful). It’s just awful.
Cool! Oh wow, yeah I’m not nearly as much of a late talker as some kids, just compared to typical kids.
To answer your question: I don’t think so. Obviously in horrific cases (like those Romanian orphanages) where the kid is never exposed to language, they won’t learn to talk. He says a bit about how a child in an early intervention program will do better if they feel safe around adults, but I don’t recall him saying anything about that kind of thing as an underlying cause.
Late-talking kids and “Einstein syndrome”
Funny story is “Unscented Kalman Filter”. The guy (Uhlmann) needed a technical term for the new Kalman Filter he had just invented, and it would be pretentious for he himself to call it an Uhlmann filter, so he looked around the room and saw an unscented deodorant on someone’s desk, and went with that. Source
I think all the things we identify as “intelligence” (including data-efficient learning) are things that the neocortex does, working in close conjunction with the thalamus (which might as well be a 7th layer of the neocortex), hippocampus (temporarily stores memories before gradually transferring them back to the neocortex because the neocortex needs a lot of repetition to learn), basal ganglia (certain calculations related to reinforcement learning including the value function calculation I think), and part of the cerebellum (you can have human-level intelligence without a cerebellum, but it does help speed things up dramatically, I think mainly by memoizing neocortex calculations).
Anyway, it’s not 100% proven, but my read of the evidence is that the neocortex in mammals is a close cousin of the pallium in lizards and birds and dinosaurs, and the neocortex & bird/lizard pallium do the same calculations using the same neuronal circuits descended from the same ancestor which also did those calculations. The neurons are arranged differently in space in the neocortex vs pallium, but that doesn’t matter, the network is what matters. Some early version of the pallium dates back at least as far as lampreys, if memory serves, and I would not be remotely surprised if the lamprey proto-pallium (whatever it’s called) did data-efficient learning, albeit learning relatively simple things like 1D time-series data or 3D environments. (That doesn’t sound like it has much in common with human intelligence and causal reasoning and rocket science but I think it really does...long story...)
Paul Cisek wrote this paper which I found pretty thought-provoking. He’s now diving much deeper into that and writing a book, but says he won’t be done for a few years.
I don’t know anything about octopuses by the way. That could be independent.
I don’t really know. My vague impression is that weird hardware could plausibly make many-orders-of-magnitude difference in energy consumption, but probably less overwhelming of a difference in other respects. Unless there’s an overwhelming quantum-computing speedup, but I consider that quite unlikely, like <5%. Again this is based on very little thought or research.
Maybe I’d be less surprised by a 100x speedup from GPU/TPU to custom ASIC than a 100x speedup from custom ASIC to photonic / neuromorphic / quantum / whatever. Just on the theory that GPUs are highly parallel, but orders of magnitude less parallel than the brain is, and a custom ASIC could maybe capture a lot of that difference. Maybe, I dunno, I could be wrong. A custom ASIC would not be much of a technological barrier the way weirder processors would be, although it could still be good for a year or two I guess, especially if you have cooperation from all the state-of-the-art fabs in the world...
Happy to have (maybe) helped! :-)
How dependent is the AGI on idiosyncratic hardware? While any algorithm can run on any hardware, in practice every algorithm will run faster and more energy-efficiently on hardware designed specifically for that algorithm. But there’s a continuum from “runs perfectly fine on widely-available hardware, with maybe 10% speedup on a custom ASIC” to “runs a trillion times faster on a very specific type of room-sized quantum computer that only one company on earth has figured out how to make”.
If your AGI algorithm requires a weird new chip / processor technology to run at a reasonable cost, it makes it less far-fetched (although still pretty far-fetched I think) to hope that governments or other groups could control who is running the AGI algorithm—at least for a couple years until that chip / processor technology is reinvented / stolen / reverse-engineered—even when everyone knows that this AGI algorithm exists and how the algorithm works.
My impression is that difficult for two people to raise kids while both are pursuing intense, ambitious careers. If it’s only one of the two people, no problem. See Anne-Marie Slaughter’s writings on this, for example. I’m interested if you know of counterexamples to that.