I argue that my brain right now contains a lossless copy of itself and itself two words ago!
I’d argue that your brain doesn’t even contain a lossless copy of itself. It is a lossless copy of itself, but your knowledge of yourself is limited. So I think that Nick Szabo’s point about the limits of being able to model other people applies just as strongly to modelling oneself. I don’t, and cannot, know all about myself—past, current, or future, and that must have substantial implications about something or other that this lunch hour is too small to contain.
How much knowledge of itself can an artificial system have? There is probably some interesting mathematics to be done—for example, it is possible to write a program that prints out an exact copy of itself (without having access to the file that contains it), the proof of Gödel’s theorem involves constructing a proposition that talks about itself, and TDT depends on agents being able to reason about their own and other agents’ source codes. Are there mathematical limits to this?
I’d argue that your brain doesn’t even contain a lossless copy of itself. It is a lossless copy of itself, but your knowledge of yourself is limited. So I think that Nick Szabo’s point about the limits of being able to model other people applies just as strongly to modelling oneself. I don’t, and cannot, know all about myself—past, current, or future, and that must have substantial implications about something or other that this lunch hour is too small to contain.
How much knowledge of itself can an artificial system have? There is probably some interesting mathematics to be done—for example, it is possible to write a program that prints out an exact copy of itself (without having access to the file that contains it), the proof of Gödel’s theorem involves constructing a proposition that talks about itself, and TDT depends on agents being able to reason about their own and other agents’ source codes. Are there mathematical limits to this?
I never meant to say that I could give you an exact description of my own brain and itself ε ago, just that you could deduce one from looking at mine.