Verifying AI-assisted AppSec, and the evidence buyers will ask for

A language model reviewing code for vulnerabilities produces leads, and a lead becomes a finding only after someone reproduces it. The same goes in reverse: a clean AI review means the model did not report anything, which is a weaker statement than "there is nothing there."

That distinction is showing up in practitioner forums. In a recent r/AskNetsec thread, a team describes running static analysis, threat modeling, and code review through several commercial models with custom prompts, alongside traditional tools, and asks how others independently validate the results and catch what AI-assisted reviews miss. It is one team's question. But it is the right question, and HealthTech founders will be asked a version of it by buyers soon: was your security testing done by a model, and how do you know it was right?

Why AI review needs its own verification

Three properties of model-driven review make verification non-optional.

Results vary between runs. Ask the same model to review the same code twice and you may get different findings. That is fine for brainstorming and a problem for evidence. If a review cannot be reproduced, it cannot be audited.

False positives cost time, and false negatives cost more. A model can describe a vulnerability fluently that does not exist. It can also miss one that does, especially where the flaw depends on context the model was not given, such as another service, a configuration file, or how a deployment is actually wired.

The same blind spot can sit on both sides. When AI helps write code and AI helps review it, the reviewer may share the writer's assumptions. The thread names this concern directly: how do you catch vulnerabilities that escape both?

None of this argues against using models. They are fast, they read a lot of code, and they can surface things a tired human skips. It argues for treating their output the way you would treat a junior analyst's first pass: useful, and checked.

A verification approach that holds up

Here is the structure I would put around AI-assisted security testing for any product team.

1. Treat every AI finding as a hypothesis. A finding is confirmed only when a person reproduces it in a test environment, or a deterministic tool flags the same issue. Record which.
2. Keep a deterministic baseline. Run conventional static analysis and dependency scanning on every change, independent of any model. The model adds coverage on top of that baseline. It does not replace it.
3. Write a regression test for every confirmed issue. Once a finding is fixed, a test that fails if it comes back is far better evidence than a note saying a model once flagged it.
4. Pin and record the setup. For each AI-assisted review, record the model and version, the prompt or prompt template, what code and context were provided, and the date. Without that, nobody can rerun it.
5. Test the reviewer. Periodically seed known issues into a test branch and see whether the AI review catches them. That is the only honest way to estimate what it misses.
6. Keep humans on the high-risk paths. Authentication, authorization, data access, and anything touching patient data get human review and manual testing, whatever the model says.
7. Scope the model's access. Give AI review tools only the repositories and context they need, and know where your code goes when you send it to a model provider.

The evidence buyers will ask for

Security questionnaires already ask about penetration testing and secure development. The next version will ask whether testing was AI-assisted and how it was validated. A health system's reviewer is not going to accept "we ran it through a model" as assurance, and they should not.

A credible evidence pack is short and concrete:

1. A one-page description of your testing approach, stating plainly where models are used and where they are not.
2. The deterministic tools you run and how often.
3. Your rule for confirming AI findings, and a sample of confirmed findings with the reproduction and the fix.
4. Your regression tests for past security issues.
5. Results from your seeded-issue checks, including what the model missed.
6. Where independent human testing fits, and the date of the last one.

The fifth item will make some founders nervous. It should be the most persuasive. A team that can show what its AI review misses has clearly thought about the problem. A team that claims it misses nothing has not.

The leadership question

For a founder or board, AI-assisted security testing is an AI readiness question like any other model use. Who owns it, what data does it see, how much does it act on its own, and what do you not know about its accuracy? The last one is the one to press on.

Ask your engineering lead a single question: of the security issues we fixed this year, which were first found by a model, which by a tool, and which by a person, and how was each confirmed? If nobody can answer, the evidence a buyer will ask for does not exist yet.

Start with one seeded test

Pick a recent, already-fixed vulnerability in your own code, reintroduce it on a test branch, and run your AI-assisted review against it. Whether it gets caught tells you more about your process than any vendor claim, and the result belongs in your evidence pack either way.