The story is simple: Mickey is an apprentice to a powerful sorcerer whose magic comes from his hat. Mickey is tasked with carrying buckets of water up a long flight of stairs and dumping them into a basin–hard work for a mouse! When the sorcerer steps out, however, Mickey takes his hat and uses its magic to enchant a broomstick to do the water carrying for him. Aha! Mickey is able to rest now, and he falls asleep to dream about all the wonderful magic he can do. But he awakes to a terrible discovery: the enchanted broomstick won’t stop carrying and dumping water and the basin is now overflowing! Mickey attempts to alter the broomstick’s course of action, but he can’t; it’s too late! So he tries to hack it to pieces, thinking that will solve the problem. But no, the enchantment enables the broomstick to make copies of itself from the hacked pieces! Now there are even MORE of them doing the unwanted task, and Mickey, try as he might, can’t possibly hope to overcome them. It’s not until the sorcerer returns (the one who truly knows how to wield the magic hat) that all is put to rights.
I thought it fascinating how this popular story from when we were all little actually encapsulates the alignment problem pretty neatly. Just wanted to share!
Yeah, this example is excellent and is being brought up repeatedly on this site and elsewhere. Here is Gary Marcus in 2012:
https://www.newyorker.com/news/news-desk/moral-machines (https://archive.is/1Q7zp)
FWIW this story is much older than Mickey: https://en.wikipedia.org/wiki/The_Sorcerer’s_Apprentice
Totally! Just figured for many like myself this might have been the first/only exposure to the tale.
And it goes deeper. Because what if Mickey never actually woke up, and the brooms had been keeping him asleep the whole time? The Sleeping Beauty problem is actually present in quite a lot of Disney media where the MC goes to sleep. It’s also a theme in Mulan. Maybe she never went to war and made her parents proud. It may well have been a dream she just didn’t wake up from.