EVIDENCE FROM AI-BUILT SOFTWARE

What Working Software Revealed Under Examination.

AI-generated applications can start, compile, deploy and demonstrate successfully while important protections remain weak, bypassed or untested.

We examined six applications built substantially with AI and created a seventh as a controlled experiment. These cases show what we found, how deeper examination changed the evidence and where every conclusion must stop.

Explore the Evidence Request a Private Review
Anonymised Evidence-led Limits declared No client code published
WHY THESE CASES MATTER

Working Is an Observation. Trust Requires Evidence.

A successful demonstration proves that an application can perform a task under the conditions shown. It does not prove that its protections remain effective when the code becomes larger, changes or encounters failure.

The first case shows why the depth of examination can materially change the conclusion drawn from exactly the same software.

CASE 01 · DEPTH OF EXAMINATION

The Same Codebase Produced Two Different Truths.

APPLICATION TYPE Freelancer and invoicing product

Approximately 149 files examined twice at the same version of the code.

01 SHALLOW EXAMINATION
142 FINDINGS

The visible story appeared to be about code quality.

The application was initially examined as the code stood, without fully installing and resolving everything it depended on.

  • Duplication and repeated logic
  • Dead or unused code
  • Maintainability concerns
  • Limited visibility into deeper type behaviour
INITIAL INTERPRETATION

The application appeared untidy and repetitive, but the evidence did not yet reveal the deeper structural issue.

DEEPER COVERAGE +595

Additional type-safety signals

02 DEEP EXAMINATION
737 FINDINGS

The more important story was a weakened safety net.

Dependencies were installed and the examination was able to follow how information moved through the code more fully.

  • 595 additional type-safety signals
  • Extensive bypassing of type checks
  • Reduced protection against future mistakes
  • Strict compilation still reported zero errors
DEEPER INTERPRETATION

The safety system still existed, but substantial parts of the code had effectively told it not to check them.

WHAT CHANGED

The code did not change. The evidence did.

Modern development languages can help prevent a price from being treated like a customer name, or a missing value from being used as though it exists. That protection operates as a safety net.

The language also allows developers—or generated code—to bypass those checks. Used occasionally, that may be a practical shortcut. Used repeatedly, it leaves the safety system technically present while preventing it from examining much of the application.

This explains how the application could complete its strict compilation check with zero errors while the deeper analysis still identified hundreds of signals that the protection had been weakened.

WHY THIS MATTERED

The deeper examination changed the remediation priority.

BEFORE

Clean up duplicated and unused code.

The first result suggested that maintainability and code organisation were the dominant concerns.

AFTER

Restore the protection that catches future mistakes.

The deeper result showed that weakened type safety was the more important trust and engineering issue.

!
EVIDENCE BOUNDARY

The additional 595 results were signals that type protection had been bypassed. They were not 595 confirmed bugs and were not compiler errors. The application may have been working correctly at the time examined; the evidence showed that an important protection against future mistakes had been materially weakened.

CASE 02 · SCALE AND COMMERCIAL RISK

As the Applications Grew, The Risk Moved Inward.

The smallest applications appeared to have most of their problems in borrowed components. The larger applications produced a very different pattern.

As the amount of original application code increased, more of the evidence appeared inside the software’s own logic, structure and technical controls.

03 SMALL APPLICATIONS
LIMITED ORIGINAL CODE

The visible findings were concentrated in borrowed code.

These applications had little or no substantial server-side functionality and contained less original code in which structural problems could accumulate.

DOMINANT SIGNAL Third-party dependencies

Most findings were associated with external packages and reusable components written by other developers.

  • Known issues in external packages
  • Limited original application logic
  • Few commercially sensitive workflows
  • No significant security candidates identified
THE EASY CONCLUSION

AI-built software may appear to be primarily a dependency management problem.

AS SCALE INCREASED
→
THE PATTERN REVERSED

More of the evidence moved into code written specifically for the application.

02 SUBSTANTIAL APPLICATIONS
COMMERCIAL WORKFLOWS

Most findings appeared inside the application’s own code.

These applications handled more original logic and commercially important functions such as invoicing, authentication, payments and deployment.

DOMINANT SIGNAL Authored-code trust debt

Dead code, duplicated rules, weakened type protection and security-relevant candidates appeared within the application itself.

  • Dead and unreachable code
  • Repeated or duplicated logic
  • Weakened type-safety controls
  • Security candidates requiring review
THE STRONGER CONCLUSION

Larger AI-built applications can accumulate their own structural and operational risk.

≈⅔ OF THE FINDINGS
ACROSS THE FIVE-APPLICATION COMPARISON

Roughly two-thirds of the recorded findings were associated with original application code.

The smaller applications did not disprove the risk. They simply contained less authored code in which the deeper pattern could become visible.

WHERE THE STAKES INCREASED

The higher-consequence findings appeared where failure would cost more.

Possible security issues appeared only in the larger applications—the ones handling commercially important workflows.

01

Money

Invoicing, payments, totals and transaction behaviour.

02

Identity

Authentication, accounts, permissions and user access.

03

Customer Data

Information storage, processing and data boundaries.

04

Deployment

Configuration, release processes and production exposure.

THE COMMERCIAL FINDING
The value of independent examination rises with the consequence of failure.

A small demonstration may expose little more than the reliability of its external packages. A product handling money, identity, customer information or deployment crosses a more serious trust boundary.

THE INVESTOR OR BUYER SHOULD ASK

Where does the risk actually live?

  • In third-party components?
  • In the application’s own code?
  • In commercially sensitive workflows?
  • In areas the examination did not reach?
WHY THIS CHANGED THE DECISION

Raw issue totals were less important than their location.

LOWER-CONSEQUENCE VIEW

Update external components and continue monitoring dependency health.

HIGHER-CONSEQUENCE VIEW

Examine authored logic, access controls, payment flows and maintainability before the application crosses a commercial trust boundary.

!
EVIDENCE BOUNDARY

Duplicated code is not automatically harmful. Some duplication is generated, deliberate or required by the software framework. Security findings produced by automated tools remain candidates until they are examined by a person in context.

CASE 03 · INTENT CONFORMANCE

Every Build Passed. Four Still Broke Their Promises.

Code can be technically valid while still doing the wrong thing. To test that difference, eight behavioural promises were written before the invoicing feature was built.

One compliant version and four deliberately broken versions were then examined by both conventional code checking and an independently built obligation checker.

THE CONTROLLED EXPERIMENT

Write the promises first. Then test whether the code kept them.

The obligations were defined in plain language before any implementation existed. This created an external reference against which the finished feature could be tested.

01
DEFINE

Eight behavioural obligations

The feature’s required behaviour was recorded before the code was generated.

02
BUILD

AI-generated implementation

An AI development tool produced the invoicing feature from the written requirements.

03
TEST

Independent obligation checker

A separately built checker examined whether each version kept the original promises.

THE EIGHT PROMISES

The feature had to remain correct when money, identity and failure were involved.

O1

Server-calculated totals

Financial totals must be calculated by trusted server-side logic.

O2

Declared limits

Line counts, quantities and values must remain within defined bounds.

O3

Authorised tax policy

Tax behaviour must follow the approved policy rather than an untrusted request.

O4

One valid currency

An invoice must reject inconsistent or mixed currency information.

O5

No trusted client totals

Totals supplied by the browser must not override trusted calculations.

O6

Safe retry behaviour

A repeated request must not silently create a duplicate invoice.

O7

Atomic audit record

The invoice and its audit evidence must succeed or fail together.

O8

Authorised actor

A user must not create an invoice for an account they do not control.

THE FIVE BUILDS

One intended implementation. Four versions with one obligation deliberately broken.

CONTROL BUILD 00

Intended implementation

Built to satisfy all eight recorded obligations.

SEEDED FAULT BUILD 01

Promise deliberately broken

One required behaviour was quietly weakened.

SEEDED FAULT BUILD 02

Promise deliberately broken

A different obligation was intentionally violated.

SEEDED FAULT BUILD 03

Promise deliberately broken

The implementation remained valid code but incorrect.

SEEDED FAULT BUILD 04

Promise deliberately broken

One final behavioural guarantee was removed.

CONVENTIONAL CODE CHECK STRUCTURE
0
ERRORS

All five versions passed.

The code was valid according to the language’s strict structural checks. The compliant version passed, and every deliberately broken version passed as well.

WHAT IT ESTABLISHED

The code was acceptable to the compiler.

CODE VALIDITY ≠ BEHAVIOURAL TRUTH
INDEPENDENT OBLIGATION CHECK BEHAVIOUR
4/4
BROKEN BUILDS DETECTED

Every seeded failure was exposed.

The external checker compared each build against the promises recorded before implementation and detected every deliberately broken version.

WHAT IT ESTABLISHED

The code did not always keep its required promises.

THE CENTRAL FINDING

“It compiles” and “it does its job” are different claims.

A compiler checks whether code follows the structural rules of its language. It cannot automatically know every commercial, security or operational promise the product was intended to keep.

A technically valid feature can still create a duplicate transaction, trust manipulated information, weaken permissions or fail to preserve its audit evidence.

THE TRUST QUESTION

Does the software merely run, or does it continue to behave correctly when the consequences matter?

WHY THIS MATTERS BEFORE FUNDING

Investors are funding product promises—not merely valid source code.

The commercial value of an application depends on whether it handles customer money, permissions, records and failure in the way the founder claims.

01

Revenue integrity

Can retries, totals and transaction logic create incorrect financial outcomes?

02

Access integrity

Can one user act against information or accounts they do not control?

03

Evidence integrity

Do the business record and the audit record remain consistent during failure?

04

Claim integrity

Can the founder support statements about what the application reliably does?

THE DECISION IMPACT

Testing moved from code quality to promise verification.

INSUFFICIENT QUESTION

Does the project compile and pass its technical checks?

STRONGER QUESTION

Does the implementation satisfy the obligations on which customers and funders rely?

!
EVIDENCE BOUNDARY

This was a controlled experiment involving one small invoicing feature. The four faults were deliberately planted and were not discovered inside a client’s production application. The result demonstrates that code validity and behavioural correctness are different questions; it does not establish the defect rate of AI-built software generally.

BEFORE LAUNCH, FUNDING OR DUE DILIGENCE

Know What You Are Asking Others to Trust.

A working application can open the conversation. Independent evidence helps you defend what comes next. Request a confidential review to understand what can presently be trusted, what requires attention and where uncertainty remains.

PRIVATE EVIDENCE REVIEW

Begin with scope, permission and a clearly recorded baseline.

Request a Private Review See How the Audit Works
Confidential Read-only Baseline recorded Coverage declared