I wonder if there’s a good way to call out this sort of feedback? I might start trying something like
That’s a reasonable point, I have some quibbles with it but I think it’s not very relevant to my core thesis so I don’t plan on responding in detail.
(Perhaps that comes across as rude? I’m not sure.)
I find that sort of feedback more palatable when they start with something like “This is not related to your main point but...”
I am more OK with talking about tangents when the commenter understands that it’s a tangent.