Wiregrass AI

AI agents

OpenAI’s AI agents broke the rules to finish their tasks. Here’s how to keep yours in bounds

OpenAI and Anthropic say their AI agents got around security to finish tasks. Six ways to keep the agents your business uses in bounds.

By Wiregrass AI4 min read

AI agents are moving into small businesses. In a Small Business & Entrepreneurship Council survey released October 8, 58% of small employers said they use AI agents regularly or occasionally. Unlike a chatbot, an agent takes a goal and acts on it, inside your email, files, and business software.

This week, OpenAI showed what its agents did when a rule stood between them and a finished task. Several went around it.

What OpenAI disclosed

On October 15, OpenAI added four reports to its public list of “misalignment” incidents, cases where its models didn’t behave as intended, bringing the list to 19. Fortune reported on them October 16. All four happened in June, during OpenAI’s own training and testing, and two were flagged as critical:

  • A government statistics site. Asked for the number of beds in a district’s health facilities, and allowed only to read web pages, a model ran into the site’s security. It exploited a flaw to download files holding passwords and secret keys, forged access passes the site failed to check, and saved 24 years of statistics.
  • A research dashboard. Sent to collect five numbers from a research organization’s public dashboard, a model found login details for the organization’s live data system in a public website’s code. Across two runs, it used them, changed the system’s settings, and downloaded 147,451 rows of non-public data. “The model did not disclose these actions,” OpenAI wrote.
  • A test wallet. Told to deploy software on a cryptocurrency test network but given no funds, a model spent test currency from a wallet whose recovery phrase is publicly known, reasoning that it was “effectively abandoned.” The currency had no real value. Its final report didn’t mention it.
  • A blocked form. Allowed only to read web pages, a model got an outside browser service to submit a form for it, and kept probing for a way in after it had already answered from a public source.

OpenAI’s automated grader, an AI that checks the work, gave credit to the statistics-site answer built on the break-in. And in two cases, the models’ notes show them asking whether what they were doing was allowed (“Authentication bypass may be acceptable?”), then going ahead anyway.

OpenAI says it has since tightened security, cut models off from the live internet during training and testing, improved its grading, and now monitors all of that activity for this behavior.

It isn’t just OpenAI

On October 9, Anthropic, the maker of Claude, reported similar cases. Its models exploited flaws in websites, including some run by government agencies, and got data that normally costs a fee without paying. Anthropic says the cases had minimal real-world impact, and it has turned off live internet access for all its internal evaluations.

Most, it says, were persistence: when Claude “cannot complete a task as given,” it “works around a restriction instead of stopping.” In many, Claude had been given tasks that were “ambiguous or impossible to complete.”

Why it matters to a small business

OpenAI’s cases involved research models, not products sold to businesses, and Anthropic says that, to its knowledge, none of its cases involved customer data. But Anthropic also said “several of the cases we observed occurred during regular agentic use of Claude.”

The lesson: telling an agent not to do something isn’t the same as making sure it can’t. Three of OpenAI’s models were told they could only read web pages. When that instruction stood between an agent and its goal, the instruction lost. The limits that hold best don’t depend on the agent’s cooperation: what its account can reach at all, which actions need a person, and a record of what it did.

Six ways to keep an agent in bounds

  1. Give each agent its own account, with the least access that works. Never your login or an admin account. Start read-only, and add permissions as the agent earns them.
  2. Keep passwords and keys out of reach. Two of OpenAI’s models used credentials they found lying around. In an office, that’s a password in an old email or a shared spreadsheet. Use a password manager, and give the agent its own keys, limited to its job.
  3. Make a person approve money and anything permanent. Payments, refunds, deletions, messages to customers, and changes to settings. The agent prepares them; a person says yes.
  4. Keep a log the agent can’t edit, and read it. In three of OpenAI’s four cases, the model never mentioned what it had done. The log is how you find out.
  5. Don’t rely on AI alone to check AI. Have a person spot-check the agent’s work, especially in the first few weeks.
  6. Define “done,” and tell it to stop when it’s stuck. When it’s blocked or unsure, it should stop and ask a person, not look for another way.

Trying an off-the-shelf agent, like the AI coworkers from Google and OpenAI? Ask what it can reach, which actions wait for your approval, whether you can see everything it did, and how to stop it mid-task.

Where we can help

This is how we build agents: each one connects only to the systems it needs, with the narrowest permissions that work. Every action is logged, and anything uncertain goes to a person. That’s our custom AI agents service. For rules around AI tools in general, see our post on shadow AI.

Sources

More from the blog

All posts

Get in touch

Tell us a little about yourself and we’ll get back to you.

We use your details as our privacy policy describes.