https://www.politico.com/newsletters/digital-future-daily/2025/12/02/how-to-... (extracted) SAN FRANCISCO — In a startup hub near Chinatown, a few dozen programmers
grabbed light beers and flipped open their laptops for a series of demonstrations on how to get around data scraping blockers.
The programmers in the room had their own agenda, trying to find ways to
keep public data accessible in the spirit of the original Web.
They were also there to figure out a pretty important question: What
exactly are the rules around data scraping?
It turns out that there aren’t many — in part because government regulators
haven’t really focused on this increasingly important issue. Internet data is now becoming a crucial resource in the AI boom, and there are ongoing clashes between major tech platforms that are trying to both hoard and capitalize on that data.
Caught in the crossfire are advocates for an open internet, and small-time
content creators who want to get compensation for their work. It’s a thorny dilemma: The internet was built on a communal ethos of free-flowing information, but that has also allowed AI companies to hoover up content for free, and make immense profits from it.
Legally speaking, they’re largely doing it in a void. Federal law only
loosely touches on data scraping, and courts have been slow to develop a body of law around the field. Instead, the industry mostly follows a set of customs and voluntary standards.
The meetup was partly a tech demo, and partly a discussion on how to update
those standards and influence the process. “If we’re not setting our own best practices, someone else will be doing it for us,” said Jo Levy, chair of the Alliance for Responsible Data Collection <https://responsibledatacollection.org/>, which advocates for preserving open access to internet data.
Levy has been trying to convince more people who want to uphold the
foundational openness of the internet — but who might not have the resources or the cachet of a big company — to get involved in the important drudgery of standards setting. During the San Francisco event, she encouraged attendees to sign up for the email lists where the boards often discuss the development of new protocols.
“A lot of what we’re trying to do with ARDC is to make sure that interested
parties are aware of these discussions and can participate,” Levy told DFD. “If the standards are going to be more about helping one business [...] those standards are not going to be as widely adopted.”
-- -------------------------------------- Joly MacFie +12185659365 -------------------------------------- -