Although it’s not a strict enforcement mechanism, what about the ability to mark posts to be included in a robots.txt file to disallow them from being scraped (assuming the bots respect it)?
robots.txt
Although it’s not a strict enforcement mechanism, what about the ability to mark posts to be included in a
robots.txtfile to disallow them from being scraped (assuming the bots respect it)?