Someone who is interested in learning and doing good.
My Twitter: https://twitter.com/MatthewJBar
My Substack: https://matthewbarnett.substack.com/
Matthew Barnett
Karma: 9,135
It’s worth not taking the common-sense concept of consciousness seriously, as it more or less just looks like the idea of a soul dressed up in 21st century jargon. However, as you noted, this idea doesn’t rely on the existence of consciousness, but rather is a sort of panpsychist frame of looking at the world. I tend to agree with this view.
If one truly internalizes this view—not just intellectually, but on a deep emotional level—I think that it is probably the best argument for altruism that exists. If I told you that you were going to be in excruciating pain in five minutes unless you did something to avoid it, you would probably go to great lengths to avoid the suffering. The human mind is good at internalizing future person-slices as a continuation of its current self, and therefore acts altruistically towards those future selfs. If only we were able to do the same for all person slices on Earth.
Perhaps I did not make my point clear. If you are asking for me to justify the terminal value of altruism, I can’t. By definition, terminal values cannot be justified by appealing to other values. However, I was simply pointing out that our concept of identity can break very easily, as you noted as well. If one thinks that all they should care about are “continuations of their current self” and then they think that only this chunk of matter is a continuation of their current self, then no, this is insufficient to justify altruism. However, as your post reveals, one can imagine switching between person slices spatially, just as one switches between time slices temporally.
Asking “Why should I care about person-slices the conscion visits if they are not my own?” well, you’ve presupposed that they aren’t your own. I am making the opposite connection.
It’s unclear to me how this is different from other boxing designs which merely trade some usefulness for safety. Therefore, like the other boxing designs, I don’t think this is a long term solution. There isn’t an obvious question that, if we could just ask an Oracle AI, the world would be saved. For sure, we should focus on making the first AGIs safe, and boxing methods may be a good way to do this. But creating AI’s with epistemic design flaws seems like a risky solution. There are potentially many ways that, if the AI ever got out of the box, we would see malignant instantiations due to its flawed understanding of the world.
You won’t be horrified if you’re dead. More seriously though, if we got an Oracle AI that understood the intended meaning of our questions and did not lie or decieve us in any way, that would be an AI-alignment complete problem—in other words, just as hard as creating friendly AI in the first place.
The way I view it, the purpose of designing low-impact desiderata is that it might give us an idea of how to create a safety measure that doesn’t include any value-laden concepts.
The issue with saying that the AI should offset certain variables, such as nitrogen concentrations, is that it seems like an arbitrary variable that needs to be offset. If you say, “Well, the AI should offset nitrogen, but not offset our neurons that now know about the AI’s existence” then you are introducing values into the discussion of low impact, which kind of defeats the purpose.
Of course, the AI *should* offset the nitrogen, but whether it ought to be part of a low-impact measure is a separate question.
Could you clarify?
Surely the interpretations have different implications about the nature of reality, right?
Clearly, thought some idiot, consciousness causes the quantum wave function to collapse, the universe doesn’t like us knowing which slit the photon goes through.
In my honest opinion, if I held an opposing view I’d probably shut off mentally at this point and click on another article. Calling John Von Neumann an idiot is always a bit far, but furthermore, I never like it when a critique turns into name calling.
There have been no quantum mechanics experiments that show consciousness to have any relevance to particle physics. The laws of physics do not say what is or is not conscious, in much the same way that they don’t say what is or is not a work of art.
I’d say this is far from obvious and would require a bit more philosophical legwork than what was actually presented. Many believe that qualia (individual instances of conscious, or subjective experience) are indivisible atoms of the universe, the ontological fire in our equations. Debunking that is going to take some careful nuance and distinction, rather than dismissal.
I looked into the claim I made about John Von Neumann and found this interesting physicsforums post on the topic. It looks like from my cursory research that it might be overstating things to say that he claimed consciousness causes collapse.
Basically, December of 2018 seems like a bad time to “go abstract” in favor of transhumanism, when the implementation details of transhumanism are finally being seriously discussed
Wouldn’t this then be the best time to go abstract, since it would necessarily distinguish bad things done in the name of transhumanism from the actual values of the philosophy.
I understand that other people have different feelings about in-group identification, and so I sympathize with the idea. That being said, I have a pretty strong negative reaction to identity symbols. It would essentially be an explicit gesture that some people are part of the tribe and other people aren’t. It’s hard to explain exactly why, but I would probably be less likely to interact with the community by a non-trivial amount if this because widespread.
Maybe figuring out the rules of the game is an inference problem itself :).
The AI alignment problem seems to be a problem inherent to seeking out sufficiently complex goals.
I would add that the alignment problem can still occur for simple goals. In fact, I don’t think I can come up with a “goal” simple enough that I could specify it on an advanced artificial intelligence without mistake, even in principle. Of course, this might just be a limitation of my imagination.
The alignment problem really occurs whenever one agent can’t see all possible consequences of their actions. Given our extremely limited minds in this universe, the problem ends up popping up everywhere.
For example, if I think I see an oasis, but it is actually an illusion, then it is the oasis that is illusionary and not my experience of seeing. In other words, if an experience is an illusion, then we still have the experience of seeing that illusion.
I don’t find this convincing personally :).
I think an illusion can simply refer to a false belief which is caused by corrupted cognitive processes. In this case, the false belief really is sincerely held, and I do not deny this. But, every time we introspect, we’re not accessing some realm of direct metaphysical truths, we’re just running a cognitive algorithm. It’s similar to a computer that has an internal check, and runs a subroutine asking itself if it’s conscious. If the computer always returns “Yes, I am conscious, and my conscious states have these properties...” I can always point to an explanation which sheds light on this reply without referring to the conscious properties.
Walkthrough: The Transformer Architecture [Part 1/2]
Walkthrough: The Transformer Architecture [Part 2/2]
Understanding Batch Normalization
Some things I didn’t explain about batch normalization in this post:
Why batch normalization reduces the need for regularization (see section 3.4 in the paper).
New techniques which build on batch normalization (such as layer normalization), and the corresponding limitations of batch normalization.
Things I’m not sure about:
I may have messed up my explanation of why we use the learned parameters and . This was something I didn’t quite understand well. Also, there may be an error in the way I have set up batch normalization step; in particular, I’m unsure whether I am using “input distribution” accurately and consistently.
I might have been a bit unclear for the one dimensional neural network example. If that example doesn’t make sense, try reading the citation from the Deep Learning Book.
How do we check empirically or otherwise whether this explanation of what batch normalization does is correct?
Great question. I should be giving a partial answer in tomorrow’s post. The bare minimum we can do is check if there’s a way to define internal covariate shift (ICS) rigorously, and then measure how much the technique is reducing it. What Shibani Santurkar et. al. found was
Surprisingly, we observe that networks with BatchNorm often exhibit an increase in ICS (cf. Figure 3). This is particularly striking in the case of [deep linear networks]. In fact, in this case, the standard network experiences almost no ICS for the entirety of training, whereas for BatchNorm it appears that G and G0 are almost uncorrelated. We emphasize that this is the case even though BatchNorm networks continue to perform drastically better in terms of attained accuracy and loss.
Large internal covariate shift means that if we choose ε>0, perform some SGD steps, get some θ, and look at the function’s graph in ε-area of θ, it doesn’t really look like a plane, it’s more curvy like. And small internal covariate shift means that the function’s graph is more like a plane. Hence gradient descent works better. Is this intuition correct?
Interesting. If I understand your intuition correctly, you are essentially imagining internal covariate shift to be a measure of the smoothness of the gradient (and its loss) around the parameters . Is that correct?
In that case, you are in some sense already capturing the intuition (as I understand it) for why batch normalization really works rather than why I said it works above. The newer paper puts a more narrow spin on this, by saying roughly that the gradient around has an improvement in the Lipschitzness.
Personally, I don’t view internal covariate shift that way. Of course, until it’s rigorously defined (which it certainly can be) there’s no clear interpretation either way.
Why does the internal covariate shift become less, even though we have μ and β terms?
This was the part I understood the least, I think. But the way that I understand it is that by allowing the model to choose and , it can choose from a variety of distributions, while maintaining structure (specifically, it is still normalized). As long as and don’t change too rapidly, I think the idea is that it shouldn’t contribute too heavily towards shifting the distribution in a way that is bad.
Can we fix it somehow? What if we make an optimizer that allows only 1 weight to change sign at each iteration?
This is an interesting approach. I’d have to think about it more, and how it interacts with my example. I remember reading somewhere that researchers once tried to only change one layer at a time, but this ended up being too slow.
Does batch normalization really cause the distribution of activations of a neuron be more like a Gaussian? Is that like an empirical observation of what happens when a neural network with batch normalization is optimized by an SGD-like optimizer?
I will admit to being imprecise in the way I worded that part. I wanted a way of conveying that the transformation was intended to control the shape of the distribution, in order to make it similar across training steps. A Gaussian is a well behaved shape, which is easy for the layer to have as its distribution.
In fact the original paper responds to this point of yours fairly directly,
In reality, the transformation is not linear, and the normalized values are not guaranteed to be Gaussian nor independent, but we nevertheless expect Batch Normalization to help make gradient propagation better behaved.
As for posting about deep learning, I was just hoping that there would be enough people here who would be interested. Looks like there might be, given that you replied. :)
I agree with this. Recently I started blogging about ML and (in future posts) AI safety. I intended this to primarily be a learning experience. I found self-teaching myself material without the aid of a public forum to be pretty boring. This way I feel much more engaged. It also adds an adversarial aspect since I am forced to perform a mental check of “does what I wrote actually make sense” lest someone correct me. I hypothesize that this helps destroy a lot of beginner errors, and strengthens my ability to communicate in the process. Also writing is just a lot of fun too.
I actually found it pretty funny that you posted this the day I started blogging.
I’ve heard before that an argument against banning AI research (even if you could do such a thing) is that hardware will continue to improve. This is bad because it enables less technically abled parties to weild supercomputer-level AI developments. It’s better that a single company stays ahead in the race than the remote possibility that someone can create a seed-AI in their basement.