Meta Removes Controversial Web Content Usage Language from Terms

Image: Illustrative image · Nokia621 · CC BY-SA 4.0 · Source

Meta has revised its commercial terms to remove wording that implied it could scrape content from websites, including news sites, to train AI—clarifying that robots.txt preferences remain respected.

Meta recently updated its commercial terms to eliminate language that had suggested the company could scrape content from various websites, including news sites, for training its artificial intelligence products. The removed text previously stated that Meta could use any content from certain domains and URLs to develop and improve its products and services, a phrase that raised concerns among media and legal experts.

Critics, including Neiman Lab and Cornell law professor James Grimmelmann, interpreted this language as Meta attempting to assert a broad right to harvest web content for AI training without compensation or consent, potentially overriding site owners’ blocking instructions set via robots.txt files. This implication caused significant backlash over the company’s data usage policies.

Meta’s vice president of communications, Andy Stone, and spokesperson Daniel Roberts responded by denying any changes to how Meta employs web crawlers and emphasized that content owners’ preferences expressed through robots.txt are respected. To address the controversy, Meta took a definitive step by removing the contentious language entirely from its commercial terms.

In a follow-up statement to The Verge, Roberts clarified that the crawling terms only pertain to Meta’s advertising and business products, not affecting its broader web crawling practices. He reiterated that the updates do not grant Meta new rights to override site owners’ directives concerning content usage, nor are they related to scraping news sites specifically.

This revision underscores Meta’s effort to clarify its stance on content usage amid growing scrutiny over AI training data sources and highlights the ongoing tension between content creators’ rights and AI developers’ data needs. The change does not signal any alteration in Meta’s technical web crawling methods but aims to reassure that website operators can control indexing preferences through standard protocols.

As AI development continues to rely heavily on large datasets, including potentially web-sourced information, companies like Meta face increased pressure to maintain transparency and respect intellectual property boundaries. Meta’s removal of the problematic language reflects responsiveness to public concern and a commitment to align terms with established internet content management norms.