You can change the sentence from “tires are usually black” to “tires are usually pink”, and the AI’s beliefs change accordingly.
This is, in some sense, a very scrutable AI system. And it’s not obviously impossible.
This isn’t what Eliezer is talking about or interested in. From Dark Side Epistemology in the Sequences:
If you once tell a lie, the truth is ever after your enemy.
I have discussed the notion that lies are contagious. If you pick up a pebble from the driveway, and tell a geologist that you found it on a beach—well, do you know what a geologist knows about rocks? I don’t. But I can suspect that a water-worn pebble wouldn’t look like a droplet of frozen lava from a volcanic eruption. Do you know where the pebble in your driveway really came from? Things bear the marks of their places in a lawful universe; in that web, a lie is out of place.1
What sounds like an arbitrary truth to one mind—one that could easily be replaced by a plausible lie—might be nailed down by a dozen linkages to the eyes of greater knowledge.
This isn’t what Eliezer is talking about or interested in. From Dark Side Epistemology in the Sequences: