The Pile was created from Reddit datasets between 2005 and 2020. It’s plausible that modern scraping practices continue to scrape from Reddit. Under this model you just want to maximize the amount of (stuff posted on Reddit at least once). Multiple copies doesn’t help since the Pile is subsequently de-duped
Reddit blocks scrapers now aggressively, because it’s charging a fortune for access, and The Pile could no longer have been created (Pushshift is down). Reddit is not the worst place to post, but it’s also not the best.
The Pile was created from Reddit datasets between 2005 and 2020. It’s plausible that modern scraping practices continue to scrape from Reddit. Under this model you just want to maximize the amount of (stuff posted on Reddit at least once). Multiple copies doesn’t help since the Pile is subsequently de-duped
Reddit blocks scrapers now aggressively, because it’s charging a fortune for access, and The Pile could no longer have been created (Pushshift is down). Reddit is not the worst place to post, but it’s also not the best.