Review AI-generated code as untrusted proposed code before production: inspect behavior, secrets, dependencies, authorization, licensing information and test evidence. A plausible explanation or a successful build does not prove that a change is secure or suitable for the application.

Key takeaways

• Assign an engineer who understands and owns the accepted change.
• Inspect the entire diff, including build scripts, dependencies and CI configuration.
• Keep secrets outside agent-readable workspaces and routine logs.
• Use independent tests and security tooling with reviewed findings.
• Record dependency provenance and route license exceptions through the organization’s process.

What can go wrong in a generated change?

OWASP’s Secure Coding with AI guidance highlights risks including invented dependencies, insecure generated patterns, secret exposure and overly privileged coding agents. Use it as a reference for a risk-based review. The scenarios below are original review exercises, not reported incidents.

Start with the effect of the change, not whether it was generated. The same authorization defect matters in human-written code. AI assistance adds workflow concerns such as context exposure, misleading explanations and automated edits beyond the requested scope.

The production review checklist

1. Scope and behavior

Compare the diff with the approved task. Identify unrelated refactoring, new outbound calls and changed defaults. Review error handling and partial failure. A small feature should not quietly replace a framework or change public contracts without a design decision.

2. Secrets and sensitive data

Inspect configuration, fixtures, logs and generated documentation for credentials or unnecessary personal data. Use approved secret storage and access boundaries. A file ignored by version control may still be readable by a coding tool; repository exclusion and tool access are different controls.

GitHub’s secret-scanning documentation describes finding supported exposed credentials. Use the appropriate scanning capability in the workflow, but do not assume a clean result detects every secret or every form of data leakage.

3. Authorization and tenant boundaries

Verify the caller, resource and permitted action at the trusted server boundary. Test another user’s record and another tenant’s identifier. A hidden button or a client-side filter is not sufficient authorization.

4. Input and output handling

Review how untrusted values reach database queries, templates, file paths, commands and external requests. Use the platform’s safe APIs and explicit validation. Check generated output before passing it into another interpreter or business action.

5. Runtime and infrastructure controls

Inspect timeouts, retries, certificate validation, filesystem access and process execution. Confirm that a local debugging workaround has not become the production default. Review any change to CI permissions and deployment configuration separately.

The NIST SSDF provides broader secure-development practices for protecting software, verification and vulnerability response. This checklist supports engineering review; it does not establish compliance or certify the application.

Dependencies, provenance and license review

For every new dependency, record why it is needed, its actual registry identity, version, upstream source and relevant review evidence. Check transitive dependencies and lockfile changes. A generated package name should never be treated as proof that the package exists or is the intended one.

Use the organization’s vulnerability and dependency-review tooling. Inspect findings and documented exceptions rather than disabling a failing check to complete the task. Keep update ownership after release.

GitHub’s license-compliance overview describes tooling for tracking dependency licenses and enforcing configured policy. Tool availability depends on the repository’s product configuration. A tool’s classification is evidence for review, not a universal legal conclusion.

Route unclear provenance, attribution requirements or license-policy exceptions to the responsible reviewer. Do not ask a coding assistant to invent a legal determination from a package name. Preserve notices and source information according to the approved process.

A small static-analysis review signal

We ran a local Python AST fixture with six assertions. It flags direct eval/exec calls and calls containing the literal keyword verify=False, while checking several simple counterexamples. Fixture strings were parsed, not executed.

def review(source):
    findings=[]
    for node in ast.walk(ast.parse(source)):
        if isinstance(node,ast.Call) and isinstance(node.func,ast.Name) and node.func.id in ('eval','exec'):
            findings.append((node.lineno,'dynamic-execution-review'))
        if isinstance(node,ast.Call):
            for kw in node.keywords:
                if kw.arg=='verify' and isinstance(kw.value,ast.Constant) and kw.value.value is False:
                    findings.append((node.lineno,'disabled-verification-review'))
    return sorted(findings)

This is a deliberately limited teaching example, not a security scanner. It does not understand aliases, interprocedural data flow, framework semantics or whether a keyword named verify controls TLS. It can miss dangerous code and flag harmless code. A finding means “review this,” not “a vulnerability has been proven.”

For production, use appropriate language/framework analysis and manual review. Keep a tested rule set and documented suppressions. Do not claim a clean tiny fixture proves an application’s security.

Verify tests as well as implementation

Require negative cases for authorization, malformed input and dependency failures where relevant. Keep independent acceptance cases outside the assistant’s ability to redefine the intended behavior. Review whether tests assert a meaningful outcome instead of simply mirroring implementation details.

For an API change, useful evidence includes allowed access, denied cross-tenant access, invalid input and a downstream failure. For an integration, include duplicate delivery and uncertain outcomes. The idempotency guide provides a focused example of replay behavior.

Inspect test removal, skipped checks, broad mocks and weakened assertions in the diff. Passing CI is less meaningful if the change has made CI stop checking the important behavior.

Keep a truthful check record: command or test suite, result, environment and limitations. Distinguish compiled, unit-tested, integration-tested and not tested. Do not promote an assistant’s summary into evidence without checking it.

A practical pull-request policy template

AI-assisted change review
Owner: named engineer accountable for the accepted change
Scope: requested behavior and files/components affected
Sensitive data: no unauthorized prompts, fixtures, tokens or logs
Dependencies: identity, necessity, version and review evidence
Authorization: caller/resource/action and tenant-boundary tests
Security checks: findings reviewed; exceptions authorized
Provenance/licensing: approved review path followed
Verification: independent cases and checks actually run
Infrastructure: CI, permissions and deployment changes reviewed
Release: normal approvals, recovery and operational ownership
Unresolved items: explicit blockers or accepted limitations

Adapt the template to the application’s risk. A formatting-only edit does not need the same evidence as a payment or identity change, but it still should not introduce unrelated code or weaken the workflow.

Use the AI-assisted SDLC playbook to connect PR review with requirements and release. For tool selection, the coding-workflow comparison provides a proposed governance evaluation.

After release, keep vulnerability response and dependency maintenance active. Generated code does not become safer merely because it was reviewed once.

Common questions

Make generated changes reviewable before release

Bring the repository, dependency workflow and current review gaps. Discuss AI-assisted engineering adoption with practical security gates and ownership.