Last week Digiday reported that Yahoo became the first licensee of a “Spotify-like” collective AI content licensing model. The same week, ExchangeWire’s digest led with the headline that Anthropic signed its first publisher licensing deal, and Digiday explained the AI training opt-out policy token.
A couple of years ago the publisher conversation about AI was mostly about blocking crawlers. Last week’s headlines were about pricing them. That’s a real change, and it’s happening while the traffic that pays for the content keeps shrinking.
Before 2025, scraping and crawlers barely registered in the stories we collected at Advertising Perspectives. So far in 2026, they come up in about one in every 200.
From “block” to “license”
Blocking was always a weak position. It works only if the companies doing the crawling respect it, and it gives the publisher nothing except the satisfaction of saying no. A sponsored post on Digiday last week still made that case, arguing why publishers must act now to protect content from AI scrapers. Protection still matters. But the news side of the week was about turning protection into a price.
The collective model is the interesting part. Most publishers are too small to negotiate a deal with an AI company on their own. A shared licensing pool, the way music rights are pooled, gives them a way in. Yahoo going first matters because it’s a buyer of content at scale, not just a seller.
Individual deals keep coming too. ExchangeWire’s headline put Anthropic on the list of AI companies signing publisher licenses. We’d treat each of these deals the same way, whoever signs them: the headline number matters less than whether the publisher keeps control of how its content is used and gets paid again when usage grows.
The opt-out problem
The opt-out token sounds like a clean answer: tell AI companies, in a standard machine-readable way, that your content can’t be used for training. The trouble is that training and search are increasingly the same company and sometimes the same crawler.
Digiday laid out the bind in its headline: as publishers await click data, Google’s AI visibility push complicates the opt-out decision. Opt out, and you may disappear from the AI answers people increasingly see first. Stay in, and you’re feeding the summaries that answer the reader’s question without a click. Publishers are being asked to choose without the data that would tell them which costs more.
The courts are starting to weigh in on scraping from the other direction. MediaPost reported last week that a judge sided against Google in its scraping battle with SerpApi, a case where Google was the one objecting to being scraped. And the pressure on AI developers is broadening beyond publishers: MediaPost also reported that advocacy groups sued AI companies, seeking an order halting development.
Reddit shows both sides
If you want one company that captures the whole tension, it’s Reddit. VideoWeek’s charts last week showed Reddit traffic hit by Google AI Overviews. At the same time, MediaPost wrote that Reddit finds a strategy to cash in on candor, and ExchangeWire’s digest noted that Reddit is testing AI-generated product reviews.
Reddit’s content is valuable to AI precisely because it’s human and unfiltered. AI answers built on it reduce the need to visit Reddit. So Reddit tries to sell the candor directly and use AI on its own pages. Every publisher with a distinctive archive is looking at a version of the same math.
We’ve watched this movie before
Publishers have rebuilt their businesses around someone else’s traffic more than once. In 2017, the pivot to video chased Facebook’s priorities, and Digiday was soon reporting the side effect of the pivot to video: audience shrinkage. A few months later, as promised, Facebook traffic to news publishers declined again after a news feed change.
Then came the licensing era for search and social. In 2020, TechCrunch reported that Google committed $1B to pay publishers, on Google’s terms, through its News Showcase program. By 2025, AdExchanger was asking whether AI companies would pay publishers at all.
The lesson from each round is the same. When a platform pays for content, it pays to keep the relationship, and the terms follow its priorities. The publishers who came out of the Facebook years in better shape were the ones that owned a direct relationship with readers.
What we’d do now
Know who’s crawling you. Log which bots hit your site, how often and what they take. You can’t price what you can’t see.
Separate training from search in your policy. Use the opt-out controls deliberately, and revisit the decision when click data arrives. Don’t make a permanent call on a guess.
Look at collective deals, not just direct ones. If you’re not big enough to negotiate alone, a pool may be the only seat at the table.
Measure AI referral traffic as its own channel. Track it separately from classic search so you can see what AI answers actually send you, and what they take.
Invest in the direct relationship. Newsletters, registration and first-party data held in a CDP or cloud data platform are the parts of the business no crawler can summarize away.
Every headline in this piece links to the original story. Browse the archive’s coverage of AI scraping, the pivot to video and platforms paying publishers, or read our look at the walled gardens argument.
Kevin Reynolds is the founder of Advertising Perspectives, MadTech Consulting and data4orce.ai. He is a marketing technology architect and C-level advisor, with a focus on Salesforce and data-driven advertising.