Google, Reddit, and the Robots.txt Rebellion
July 31, 2024
46 min
Full
51 of 194
July 31, 2024
Google, Reddit, and the Robots.txt Rebellion
0:00 / 46:05
Excerpt playback isn’t available for this episode — use the full-episode link.
Search the Record
Moment search isn’t open for this episode yet
When it opens, you’ll be able to type any phrase and jump straight to the second it was said.
404 Media Episodes Around July 31, 2024
Episode 51 of 194 — walk the feed in the order it was published.
Google, Reddit, and the Robots.txt Rebellion
Summary
We've got a bumper episode today. First off, a whole series of stories about robots.txt and its increased use against AI, as well as Reddit only being available in Google results now (and not other search engines like DuckDuckGo). Then after the break, Sam discusses her amazing scoop which showed that multibillion dollar company Runway scraped videos from individual YouTube creators. In the subscribers-section, everyone educates Joseph on what Skibidi is.
The Backlash Against AI Scraping Is Real and Measurable
Websites are Blocking the Wrong AI Scrapers (Because AI Companies Keep Making New Ones)
Google Is the Only Search Engine That Works on Reddit Now Thanks to AI Deal
AI Video Generator Runway Trained on Thousands of YouTube Videos Without Permission
Skibidi Toilet Copyright Takedown Request to Garry’s Mod Is Very Dumb
Subscribe at 404media.co for bonus content.
Learn more about your ad choices. Visit megaphone.fm/adchoices
Episode details and artwork are published by the show’s own feed. Sentinel plays short cited excerpts and links every episode back to its publisher.