Realism : Direct or Indirect?

Stanford Encyclopedia : Perception
Wikipedia : Direct and Indirect Realism

On various philosophy forums I’ve participated on, there have been arguments between those who call themselves ‘direct realists’ and those who call themselves ‘indirect realists’. The question is apparently about perception. Do we experience reality directly, or do we experience it indirectly?

When I was first initiated to the conversation, I immediately took the indirect side—There is a ball, photons bounce off the ball, the frequency of those photons is changed by some properties of the ball, the photons hit my retina activating light-sensitive cells, those cells send signals to my brain communicating that they were activated, the signals make it to the visual cortex and...you know...some stuff happens, and I experience the sight of a ball.

So, my first thought in the conversation about Indirect vs Direct realism was that there was a lot of stuff in between the ball and my experience of it, so, it must be indirect.

But then I found that direct realists don’t actually disagree about any part of that sequence of events I described above. For them as well, at least the few that have bothered to respond, photons bounce off a ball, interact with our retinas, send signals to the brain, etc. The physical process is apparently the same for both sides of the debate.

And when two sides vehemently disagree on something, and then when the question is broken down into easy, answerable questions you find that they actually agree on every relevant question, that tends to be a pretty good hint that it’s a wrong question.

So, is this a wrong question? Is this just a debate about definitions? Is it a semantic argument, or is there a meaningful difference between Direct and Indirect Realism? In the paraphrased words of Eliezer, “Is there any way-the-world-could-be—any state of affairs—that corresponds to Direct Realism being true, or Indirect Realism being true?”