The problem we see more is that many later works inherit these imperfections without questioning them, and already try to build early conclusions about EM (as a broad phenomenon) despite these evaluation/​dataset limitations
OK, I agree here. If we predicted that EM will be popular and get many follow-ups, and that follow-up work will use our judges etc, then probably we would have spent more time there.
OK, I agree here. If we predicted that EM will be popular and get many follow-ups, and that follow-up work will use our judges etc, then probably we would have spent more time there.