ML engineer, interested in phenomenology.
Cool post, thank you.
Where do you read that?
For discussions between friends about capabilities for, say estimating timelines, if I need to convince a friend that timelines are short, what is the policy you would recommend?
Great summary, I’ve read the full sequences 4 years ago, but this was a nice refreshing.
I also recommend from time to time to go to the concept lists and to focus randomly on 4-5 tags, and to try to generate/remember some thoughts about them.
Very Interesting, thank you.
So I want to propose the hackathon. Do you think we can simplify the rules?
e.g., instead of having to train multiple networks for each mix rate, can’t we just choose a single mix-rate, e.g. 10%, and see if they can beat your algorithm and your performance of 0.92?
How do we prevent people from cropping the lower part of the image to train on the text and cropping the upper part of the image to train on the face?
Interesting. This dataset could be a good idea of hackathon.
Is there an online list with this type of datasets of interest for alignment? I am trying to organize a hackathon, I am looking for ideas
no, superintelligent machines are not replacing humans, and they are not even competing with us.
I do not think the author has read Superintelligence.
In fact, these large language models are merely tools made so well that they manage to delude us
Eliminativist philosophers would say approximately the same thing of the neural net in the Brain.
I would be happy to hear an argument in favor of developing models of ‘conscious’ artificial intelligence. What would be its purpose, aside from proving that we can do it? But that is all it would be
I believe consciousness is a prerequisite for moral agency. Determining what is conscious or not therefore a very important moral problem; I think Robert Wiblin summarize it correctly:
Failing to recognise machine consciousness is one moral catastrophe scenario. But prematurely doing so just because we make machines that are extremely skilled at persuasive moral advocacy is another path to disaster
And also the number of views
Ok. But don’t you think “reverse engineering human instincts” is a necessary part of the solution?
My intuition is that value is fragile, so we need to specify it. If we want to specify it correctly, either we learn it or we reverse engineer it, no?
consider scaling up in ML to become an ML engineer or ML researcher. If it’s still possible, try to join the best engineering school in your region, and then join your local EA group, and start community building to nudge your smart friends towards AI safety. ML engineering does not necessitate a genius level of IQ.
I’m myself an ML engineer, you can dm me for further questions. I’m far from being a genius, I’ve never been the best in my class, but I’m currently able to contribute meaningfully.
The first plane didn’t emulate birds. The first AGI probably won’t be based on a retro engineering of the brain. The blue brain project is unlikely to finish reproducing the brain before DeepMind finds the right architecture.
But I agree that being able to retro engineer the brain is very valuable for alignment, this is one of the path described here, in the final post of intro-to-brain-like-agi-safety, section Reverse-engineer human social instincts.
Why won’t this alignment idea work?
Researchers have already succeeded in creating face detection systems from scratch, by coding the features one by one, by hand. The algorithm they coded was not perfect, but was sufficient to be used industrially in digital cameras of the last decade.
The brain’s face recognition algorithm is not perfect either. It has a tendency to create false positives, which explains a good part of the paranormal phenomena. The other hard-coded networks of the brain seem to rely on the same kind of heuristics, hard-coded by evolution, and imperfect.
However, it turns out that humans, despite these imperfect evolutionary heuristics, are generally cooperative and friendly.
This suggests that the seed of alignment can be roughly coded and yet work.
1. Can’t we replicate the kind of research effort of hand-crafting human detectors, and hand-crafting “friendly” behaviour?2. Nowadays, this quest would be facilitated by deep learning: no need to hand-craft a baby detector, just train a neural network that recognizes babies and triggers a reaction at a certain threshold that releases the hormones of tenderness. There is no need to code the detector, just train it. And then, only the reaction corresponding to the tenderness hormone must be coded.3. By this process, there will be gaping holes, which will have to be covered one by one. But this is certainly what happened during evolution.
The problems are:- We are not allowed to iterate with a strong AI- We are not sure that this would extrapolate well to higher levels of capability
But if we were to work on it today, it would only have a sub-human level, and we could iterate like on a child. And even if we had the complete code of the brain stem, and we had “Reverse-enginered human social instincts” as Steven Byrnes proposes here, it seems to me that we still would have to do all this.
What do you think?
I’ve not seen the idea of bit flip idea before, and anthropic are quasi-alone on that, they might have missed it
Then it forgets both the prompt it was given and the response it outputted.
If the memory of this AI is so limited, it seems to me that we are speaking about a narrow agent. An AGI wouldn’t be that limited. In order to execute complex tasks, you need to subdivide the task into sub-tasks. This requires a form of long term memory.
here we are, a concrete example of failure of alignment
I really want this to happen.
And why stop at Terry Tao? We could also email other top mathematicians and physicists.
If Facebook AI research is such a threat, wouldn’t it be possible to talk to Yann LeCun?