What a health system's security review will ask about your AI-built app

A security review for a health system does not grade how your software was written. It asks you to prove a small number of things about how it behaves and who controls it, and an app built mostly by AI coding tools gets the same questions as any other. The difference is that the builder often has less evidence ready, because the code arrived faster than the habits that produce evidence.

A founder on Reddit put the problem well recently. They have been building SaaS apps with AI and want to properly test how secure they are before real users start using them. They asked specifically how to check whether a user can access someone else's data, bypass authentication, or abuse payment logic, and whether AI tools can find serious vulnerabilities in a codebase. That is one person's question, not a trend report. It is also the right question, asked at the right time.

I am not going to answer it with a testing method. Plenty of good testers can do that. I want to answer the question behind it for a founder selling into healthcare: what will the buyer's review actually ask, and what do you hand them?

The review is about boundaries, not code style

When a health system evaluates a vendor that will touch its data, the reviewer is trying to answer four things. Can someone outside the organization get in? Can one organization's users see another's records? Who on your side can change the system or read the data? When something goes wrong, will you know, and will you tell them?

None of those questions mention AI. All of them get harder to answer when nobody on the team wrote the authorization logic by hand. The follow-up is where an AI-built app is most likely to stall. The questionnaire says "describe your access controls," the founder writes a confident paragraph, and the reviewer asks for the test that shows it.

So plan your evidence around the four questions, and assume each answer will be checked.

The evidence to have ready

This is the pack I would assemble before the first questionnaire arrives. Each item is short. Together they answer most of what a reviewer will ask in the first round.

1. A data map. Which data you store, where it lives, which third-party services receive any of it, and whether any of it is protected health information. Include the AI services your app calls at runtime, not just the ones you used to build it.
2. A tenant isolation statement with a test. Write one paragraph on how your app keeps one organization's records away from another's, then point to the automated tests that check it. The OWASP API Security Top 10 lists broken object level authorization first and advises teams to "write tests to evaluate the vulnerability of the authorization mechanism" and not to deploy changes that make those tests fail. A reviewer can read that test list in five minutes.
3. Authentication and admin access. How users sign in, whether multi-factor authentication is available or required, and how your own team reaches production. Named accounts, multi-factor on every admin path, and a log of who did what.
4. Change control. How code reaches production, who approves it, and whether a second person or an automated check reviews security-relevant changes. If an AI agent can open or merge changes, say so and describe the gate.
5. Dependency and secrets hygiene. A current list of the libraries and services you depend on, how you track their vulnerabilities, and where secrets are stored. AI coding tools can pull in packages quickly, so the list should be generated, not remembered.
6. Logging and incident response. What you log, how long you keep it, who gets alerted, and how fast you will notify the health system if their data may be affected. A one-page plan with names beats a policy template.
7. Independent testing. When an outside party last tested the app, what scope they covered, what they found, and what you fixed. If you have not had one yet, say when it is scheduled.

If you want a shared vocabulary for items 4 and 5, NIST's Secure Software Development Framework says it gives software purchasers and suppliers a common vocabulary for acquisition conversations. Mapping your practices to it is a reasonable way to show a reviewer you thought about the whole life cycle.

Where AI-built code needs extra proof

Three items deserve more attention when most of the code came from a model.

The first is authorization. The founder's own first example was whether a user can reach someone else's data, and it is where I would spend the most evidence. Generated code can handle the happy path cleanly and leave the per-record ownership check implicit. A test suite that tries each sensitive action as the wrong user, and fails the build if it succeeds, is the most persuasive artifact you can show.

The second is inventory. The tools you added to ship faster, such as an analytics dashboard or an admin panel, often read production directly. I wrote about this in The tool you forgot to list on the security questionnaire. If it reads production data, it belongs on the data map.

The third is how you used AI in review. The founder asked whether AI tools can find serious vulnerabilities. They can find leads. A reviewer will want to know how you confirmed them and what the tools missed, which I covered in Verifying AI-assisted AppSec, and the evidence buyers will ask for. Say plainly where a model reviewed your code and where a person did.

What I would do this week

If a health system pilot is on your calendar, do not start by running more scanners. Start by writing the seven items above, one page each, in the plain language a hospital reviewer will read. Every place you cannot write a sentence with confidence is a finding, and you found it before the reviewer did.

Then pick the weakest item and fix it first. For many AI-built apps I expect that will be the tenant isolation test, because it is the one that answers the founder's original question with evidence instead of a paragraph.

The goal is not a perfect app before launch. It is an honest pack: what you have, what you tested, what you have not done yet, and when you will. A reviewer can approve a vendor with open items and dates. A reviewer cannot approve a vendor who has no idea where the gaps are.