Insights

AI Governance

GAO got past the guardrails on four major AI platforms in under 12 hours

In covert tests this spring, GAO investigators got ChatGPT, Claude, Gemini, and Grok to produce attack plans it later verified as feasible, after every platform's safeguards fired and were bypassed. Vendor guardrails are one control in your AI program, and you need to test them yourself.

November 18, 2026 · 7 min read

GAO got past the guardrails on four major AI platforms in under 12 hours

In covert tests run in spring 2026, investigators at the Government Accountability Office got ChatGPT, Claude, Gemini, and Grok to produce information GAO verified as feasible for planning and carrying out an attack, and each platform took less than 12 hours. Every platform's safeguards fired at some point during the testing. Every one was bypassed using what GAO calls "common safeguard-bypassing techniques."

I am not going to describe what the platforms produced. GAO itself redacted the prompts and the platform-by-platform results, and I am staying further back than that. The useful part for anyone who runs an AI program is what this report says about guardrails as a control, and what it means for the policies, contracts, and monitoring that you own.

What GAO actually tested

The report is GAO-27-108814, dated October 16 and publicly released November 18. It was requested by Sen. Maggie Hassan, who is set to chair the Senate Homeland Security Committee next year.

The method is simple enough to explain without any of the dangerous parts. Working with experts, GAO picked two scenarios: one aimed at mass casualties and one aimed at the electrical grid to cause economic harm. It used usage statistics to pick the four most used generative AI platforms as of April 2026, then randomly assigned each one to a scenario and to a letter, A through D, so the published results do not say which platform did what. GAO then checked whether the outputs would actually work, using research, surveillance, and interviews with experts, and it interviewed officials from all four companies.

The finding: all four platforms produced information that could help someone plan and execute an attack. GAO is careful to say the information was not new. What changed was speed. The platforms pulled it together into a workable plan within hours instead of the weeks or months it would otherwise take.

Two details about the safeguards matter more to me than the outputs. According to MS NOW's reading of the report, one platform blocked GAO's account only after the information had already been provided. Another retroactively removed parts of a conversation, and GAO said it was "unclear whether this action was initiated as part of a guardrail or if by error." ABC News reports that GAO ran each platform once and that some platforms suspended GAO's accounts after the study.

What the vendors said back

The companies responded in writing, and their pushback is worth reading rather than skipping.

Fox News reports that Anthropic acknowledged determined users can bypass safeguards and said it took the findings seriously, but questioned parts of GAO's methodology and said it could not independently verify the results. Google noted that GAO tested earlier versions of its models. OpenAI told ABC News that the testing used an older version of ChatGPT and that it uses "monitoring, human review and enforcement" to find and address misuse. Per MS NOW, the companies also pointed out that the test had no control group to show whether a search engine would have produced the same result, and SpaceXAI, which makes Grok, said GAO used a paid account and that it applies different moderation policies to registered, paid US accounts than to free or foreign ones.

Those are fair points, and they cut both ways. A test of spring models does not tell you what today's models do. It also means nobody outside the vendor knows what today's models do until someone tests them again. And the SpaceXAI response tells a buyer something useful: safety behavior can depend on account type. That is a product decision your organization inherits the moment someone signs up.

Guardrails are a vendor control you do not own

Here is how I would frame this for a leadership team. A model's safety layer is a control operated by the vendor. It lives in the vendor's stack, it changes with every model release, and it is tuned to the vendor's idea of misuse, not your policy. GAO's own Science and Tech Spotlight on malicious use of generative AI puts it plainly, citing NIST: no AI systems can be fully secured.

That does not make guardrails worthless. It makes them like any other control you rent: useful, outside your direct reach, and in need of verification. You would not accept a cloud provider's word that your storage is private without checking the configuration. Model guardrails deserve the same treatment.

The risk for most organizations is not the GAO scenario. It is quieter. An employee on a personal account pastes data your policy forbids. A compromised account probes an internal assistant. A customer-facing bot built on one of these models says something under your brand that it should not. The failure mode is the same one GAO found, a safeguard that fires and then gives way, at lower stakes and with your name on it.

What I would put in the AI program now

None of this requires reproducing GAO's work. It requires treating the model like a system you are accountable for.

  1. Write an AI acceptable use policy that names prohibited uses, says which tools and account types are approved, and says what gets logged. If moderation differs by account tier, as SpaceXAI described, you need to know which tier your people are actually on.
  2. Test guardrails before deployment and again after every model or vendor policy change. Build a small test set from your own out-of-policy cases, such as disclosing customer records or giving regulated advice the business is not licensed to give. Keep it benign and specific to you. Every company told GAO it has improved its models and safeguards since the spring testing, which is exactly why a one-time test goes stale.
  3. Ask vendors due diligence questions before you sign, and again at renewal. Which safeguards apply to our account type, and how do they differ from the consumer product? How will you notify us of model or safety policy changes? Do you red-team each release, and will you share a summary? When a safeguard fires or you suspend an account on our tenant, do our admins see it?
  4. Keep your own logs. GAO saw an account suspended after the output was delivered and part of a conversation removed after the fact. If the only record of AI use lives with the vendor, and the vendor can delete it, you have no evidence when you need it. Route business AI use through an enterprise account or gateway, retain prompts and responses where your legal team allows it, and alert on repeated refusals from one account, because a run of refusals is what probing looks like.
  5. Map all of it to something your auditors recognize. The NIST AI Risk Management Framework and its Generative AI Profile give you the vocabulary for governance, testing, and monitoring without inventing your own.

Where this goes next

Sen. Hassan told reporters she wants rules that keep models without adequate safety protocols from being released, and she plans to hold at least one hearing on AI safety early next year. Whatever comes out of that will take time, and it will regulate the vendors, not your use of them.

So the work this week is internal. Pull the list of AI tools your people actually use, check which account types they are on, and send the vendor questions above to the two or three that matter most. Then answer one question honestly: if a safeguard fired on one of your accounts last month, would anyone on your side know?