I think it’s because with the words in -ly you know they’re supposed to refer to the noun? With a “stern looking man”, you might have doubts whether the guy is stern-looking or both “stern” and “looking” at something, because stern is an adjective. A sternly looking man can’t both be sternly and be looking.
What is the purpose of the -ly exception? What’s wrong with “hopefully-corrigible agent” other than that it breaks the rule?
I think it’s because with the words in -ly you know they’re supposed to refer to the noun? With a “stern looking man”, you might have doubts whether the guy is stern-looking or both “stern” and “looking” at something, because stern is an adjective. A sternly looking man can’t both be sternly and be looking.