When I tried playing musical instruments as a teenager, I usually transcribed the music to my own notation. Something like “C D E F”. Then I could play much faster.
In hindsight, maybe this was a wrong approach. Something like when people learn foreign languages the wrong way (most of them do), and instead of “foreignWord → concept” associations they train “foreignWord → nativeWord” associations. Then they speak really slowly because they have to translate twice “foreignWord → nativeWord → concept”. The key to speak foreign languages fluently is to associate the foreign words directly with the concepts, skipping the translation to your own language. This usually also happens to people who learn languages the wrong way, but it can happen years later, and come as a surprise. Learning the right way, you can do it from day 1.
Analogically, if we tried learning music this way, we should skip the “C”, “D”, “E” sounds and try to create an immediate association between the shape on the screen and the position of the fingers. I have never tested this hypothesis, so I have no idea whether this is what skilled musicians do.
So I see two possible approaches here. They can also be combined: just because you may have learned to use some notation in a wrong way, it does not prove it is an optimal notation. The approaches are: 1) invent a better notation, and 2) invent exercises to associate the notation with movement on a level of reflex.
If I tried to invent a perfect piano notation today, it would probably be written top to bottom (instead of left to right). Because the piano keys are horizontal. A longer sound could correspond to a visually longer symbol; but the symbols would also slightly differ in shape (to make it easier to see how long exactly a long sound is). For example, there could be vertical lines corresponding to the black keys, small filled circles for short sounds, and something like filled letter “8” for twice as long sounds. That alone could be enough for a beginner.
My piano teacher usually hummed the first few bars of a piece to identify it when the title was not sufficient. When I try to emulate this with a new piece it’s much slower to go “note → D → D-sound”. If it’s a piece I’ve heard before, at some point my head fills in the rest without reading further, and then I only have to verify the score matches the sound in my head. This seems to agree with your hypothesis.
Top-down notation reminds me of vertical scrolling rhythm games like DDR and also Synthesia (though I have no experience with Synthesia specifically). My difficulties in learning to read piano sheet music were/are:
different clefs and clef changes
notes with many ledger lines
off-by-one-line errors (e.g. reading G as B)
following fingering recommendations
The last one might be entirely on me, I have a tendency to improvise fingerings and mess things up. But I think the other three are quite common problems to have. One of the things that trips me up with ledger lines is that the same note up an octave has different line/space parity.
My experience with rhythm games (and reading some sheet music rotated 90 degrees just now) suggests to me that fencepost errors are just as likely with top-down reading, and are greatly exacerbated by increasing the number of columns. I’m not sure it’s worth losing consistency with other music notation or western left-to-right reading systems. Maybe an anchor-note-position (say, a small mark on every bar line indicating C) would help with this, and possibly with clef changes too.
One other thing I used to not pay much attention to in piano were rests, and I would often hold notes until I had to play the next note regardless of whether there was a rest between. Playing rhythm games with hold notes has gone a long way in stopping me from doing this, as I have to pay attention to when to release the note also—extending the note-symbol to span its length could accomplish the same thing.