AI Personal Agents Face Website Access Challenges, Industry Works on New Communication Standard

Image: TechCrunch · Source

Personal AI agents aim to assist users in tasks like shopping and booking, but many websites block them due to anti-bot measures or intentional restrictions. Industry leaders including Meta and Walmart are developing a new protocol to facilitate seamless agent interactions in online commerce.

Personal AI agents are rapidly emerging as useful tools that can perform tasks on behalf of users, such as booking flights, making dinner reservations, or ordering groceries. Examples include Meta's Muse, Instinct, and ChatGPT’s Dots. Unlike traditional AI chatbots, these agents go beyond answering queries—they act as digital assistants completing transactions. However, widespread adoption is being hindered by a new challenge: many websites are blocking these agents from accessing their services.

Amazon recently started blocking Meta’s Muse from browsing and purchasing on its retail platform. Similar issues have been reported by users on other websites including Walmart, despite Walmart being a partner with Muse, as announced at Meta’s Connect developer conference. Walmart attributes these disruptions to its human verification systems, such as CAPTCHA-like buttons, which AI agents may fail to navigate, causing access denials.

This problem stems largely from existing anti-bot safeguards designed to protect sites from spam and malicious activity, which do not distinguish well between harmful bots and beneficial AI agents acting on user behalf. Other companies like Delta Airlines have expressed caution, stating that introducing AI agent access must ensure security and a positive customer experience. United Airlines' terms of use explicitly forbid automated browsing without prior approval, though its stance on AI agents remains unclear.

Complaints on social media and reports indicate that blocking AI agents is not isolated to retail or travel — brands like Yelp, eBay, Zillow, Pizza Hut, and Adidas have also been mentioned. Yelp requires paid permissions for non-human access, and eBay restricts unauthorized automated actions but does not ban all third-party agents outright. Some content delivery networks, including Cloudflare, may also be inadvertently blocking AI agents due to recent shifts in bot-blocking policies.

To address these challenges, a consortium of industry players including Meta, Walmart, Stripe, and others is collaborating to develop an open standard protocol. This protocol aims to enable clear communication between AI agents and websites, distinguishing legitimate agent traffic from malicious bots and enabling smooth, secure commerce interactions. Meta views this approach as critical, stating, “Turning away a personal agent means turning away the customer behind it,” and hopes that establishing clear norms will benefit both users and businesses.

As AI personal agents’ capabilities and use expand, resolving these access barriers is essential. Partnering with key businesses and setting communication standards represent important steps toward integrating AI agents effectively into online services, enhancing user convenience without compromising security.