Thirdly, it regards the action as an one-time action. But just it isn’t. If you teach .people to push the fat guy to kill it. You just not only will have three people less dead. You’ll also have a bunch of emotionless people who think it is ok to kill people if it is for the greater good. Fourthly, people don’t always come immediately to the truth. You can’t say you should kill the fat guy if you really think that’s gonna save the other people.
These objections suggest that you are actually applying consequentialism already! You are worrying that other consequences of killing one person to save five might outweigh the benefit saving four lives, which is exactly the sort of thing a good consequentialist should worry about.
I think simply fully accepting materialism clears up all hard philosophical problems related to consciousness, including “qualia.” We can simply go and look at the how the brain works, physically. Once we understand all the physical facts (including e.g. the physical causes of people talking about qualia) there are no other facts to understand.
As such, I feel like someone treating “qualia” seriously is a big red (ha) flag. Either they have not embraced materialism, or they are worrying about whether a falling tree that no one hears makes a sound.