Tech, ExplainedNetworking, infrastructure and security
Product

AI code governance

Checking the code the AI assistant wrote.

BEFORE ITAFTER ITAI code governanceEndor LabsBlack DuckCheckmarxSonatypeCOVERSEditor checksPackage checksAI code recordsendssuggestedcodeStatic application securitytesting (SAST)sendssuggestedlibrariesSoftware compositionanalysis (SCA) and SBOMtagsAI-writtencodeCode repository↗ DEVOPSA developer receives asuggestion from an AIcoding assistant in thecode editorCONCEPTS IT USESAI in cybersecurityOpen-source dependenciesBEFORE ITAFTER ITAI code governanceEndor LabsBlack DuckCheckmarxSonatypeCOVERSEditor checksPackage checksAI code recordsendssuggestedcodeStaticapplicationsecuritytesting (SAST)sendssuggestedlibrariesSoftwarecompositionanalysis (SCA)and SBOMtagsAI-writtencodeCode repository↗ DEVOPSA developerreceives asuggestion froman AI codingassistant inthe code editorCONCEPTS IT USESAI in cybersecurityOpen-source dependenciesBEFORE ITA developer receives a suggestion from anAI coding assistant in the code editor↓AI code governanceEndor LabsBlack DuckCheckmarxSonatypeEditor checksPackage checksAI code recordAFTER IT↓ sends suggested codeStatic applicationsecurity testing (SAST)↓ sends suggested librariesSoftware compositionanalysis (SCA) and SBOM↓ tags AI-written codeCode repository↗ DEVOPSCONCEPTS IT USESAI in cybersecurityOpen-source dependencies
AI code governance checks an AI assistant's suggestion in the editor, flags a six-day-old package and an unchecked login token, passes the code and libraries to SAST and SCA, and tags AI-written lines in the repository.
opens a page inside this one a page in the same scene a page in another scene a concept page a label only, with no page

AI code governance is software that checks the code and the libraries suggested by AI coding assistants, at the moment they are suggested, before they become part of a company's software. It also records which code was written by AI, so that the company can apply its own rules to that code.

An AI coding assistant is an AI model inside the developer's code editor. It suggests whole lines and functions as the developer types, and a large share of new code is now written this way. The assistant learned from millions of public programs, and some of those programs contained security flaws. So it can repeat the same flaws. It can also suggest importing a library that nobody in the company chose, or even a library that does not exist.

Say a developer at an edtech company in Indore asks the assistant for code that checks student logins. The assistant suggests 30 lines of code, and an import of a small package for reading login tokens. AI code governance checks the suggestion as it appears in the editor, and finds two problems.

First, the package was published only six days earlier, by an unknown account, and it has been downloaded 40 times. Attackers publish packages under names that AI assistants tend to invent, and wait for someone to install them. So the tool warns the developer before the package is installed, and suggests a well-known library instead.

Second, the suggested code accepts a login token without checking its signature. So anyone could make a token by hand and log in as any student. The tool marks the line and shows the corrected version.

The checks themselves are not new. Checking libraries is the job of software composition analysis (SCA), and checking code is the job of static application security testing (SAST). What is new is the timing and the record. The checks run inside the editor, seconds after the AI writes the code, instead of later in the pipeline. And the tool keeps a record of which lines came from the assistant. A company can then set rules. For instance, every AI-written change to the payments code must be approved by a senior developer before it joins the main code in the code repository.

AI code governance is different from AI code review. AI code review uses an AI model to read each finished pull request and comment on it. AI code governance checks what an AI model writes, against the company's security rules. Both are part of a wider change described in AI in cybersecurity: attackers and defenders now use the same AI models.

Tech, Explained is a part of ingrid.fyi. Every page explains one piece of technology in plain words.