

Detecting & Mitigating Unowned AI‑Generated Code (Claude, Codex, Hermes) in Corporate Networks
Unowned AI‑generated code – code produced by large language models without clear provenance, licensing, or ownership – can slip into repositories, build pipelines, or runtime environments and introduce hidden security, compliance, and IP risks. This guide shows how to safely invoke the Claude, Codex, and Hermes APIs, capture their output, and apply lightweight provenance checks so you can spot unowned AI code before it becomes a liability.
<a name="step-1-prerequisites"></a>
| Item | Why you need it | Recommended version |
|---|---|---|
| Anthropic API access (Claude) | To call the Claude model via its REST API. | Account at https://www.anthropic.com; API key enabled. |
| OpenAI API access (Codex) | To call the Codex model (code‑completion variant of GPT‑4). | OpenAI account with access to the code-davinci-002 or gpt-4 code model. |
| HuggingFace Hub token (Hermes) | Hermes is hosted on HuggingFace (e.g., NousResearch/Hermes-2-Pro-Llama-3-8B). | HF account with read access to the model repo. |
| Python 3.9+ | Core language for the Python examples. | python --version |
| Node.js 18+ (or latest LTS) | For the JavaScript/TypeScript examples. | node --version |
| Git | To clone sample repos & manage SBOMs. | git --version |
| IDE / Editor (VS Code, PyCharm, etc.) | For development & debugging. | Any modern editor. |
Basic networking – outbound HTTPS allowed to api.anthropic.com, api.openai.com, huggingface.co. | Required for API calls. | – |
Security note: Store all API keys in a secret manager (AWS Secrets Manager, HashiCorp Vault, Azure Key Vault, or GitHub Secrets). Never hard‑code them in source control.
<a name="step-2-installation-and-setup"></a>
# 1️⃣ Create a clean virtual environment
python -m venv .venv
source .venv/bin/activate # on Windows: .venv\Scripts\activate
# 2️⃣ Upgrade pip & install core libraries
pip install --upgrade pip
pip install anthropic openai transformers[sentencepiece] python-dotenv tenacity
# 3️⃣ (Optional) Install pre‑commit hooks for linting
pip install pre-commit
pre-commit install
# 1️⃣ Initialize a new npm project (if you don’t have one)
npm init -y
npm install typescript ts-node @types/node --save-dev
# 2️⃣ Install API clients & utilities
npm install anthropic-sdk openai @huggingface/inference dotenv axios
npm install --save-dev typescript-eslint eslint prettier
Create a .env file (add it to .gitignore) with the following keys:
# .env
ANTHROPIC_API_KEY=your_anthropic_key_here
OPENAI_API_KEY=your_openai_key_here
HF_TOKEN=your_huggingface_token_here
Load them in code via python-dotenv (Python) or dotenv (Node).
<a name="step-3-basic-implementation"></a>
The following snippets demonstrate a secure, reproducible workflow:
The code is deliberately minimal – in production you would wrap each call in retry logic, rate‑limit handling, and SBOM generation.
# file: ai_code_provenance.py
"""
Securely invoke Claude, Codex, and Hermes to generate code,
attach provenance metadata, and detect unowned output.
Requirements:
pip install anthropic openai transformers[sentencepiece] python-dotenv tenacity
"""
import os
import hashlib
import json
import datetime as dt
from typing import Tuple
import anthropic
import openai
from transformers import pipeline
from dotenv import load_dotenv
from tenacity import retry, stop_after_attempt, wait_exponential
# ----------------------------------------------------------------------
# Load environment variables from .env (ignore if not present)
load_dotenv()
ANTHROPIC_KEY = os.getenv("ANTHROPIC_API_KEY")
OPENAI_KEY = os.getenv("OPENAI_API_KEY")
HF_TOKEN = os.getenv("HF_TOKEN")
if not all([ANTHROPIC_KEY, OPENAI_KEY, HF_TOKEN]):
raise RuntimeError("Missing one or more required API keys in environment.")
# ----------------------------------------------------------------------
# Initialize clients
anthropic_client = anthropic.Anthropic(api_key=ANTHROPIC_KEY)
openai_client = openai.OpenAI(api_key=OPENAI_KEY)
# Hermes – using HuggingFace Inference API (text generation)
hermes_generator = pipeline(
"text-generation",
model="NousResearch/Hermes-2-Pro-Llama-3-8B",
token=HF_TOKEN,
device=-1, # CPU; set to 0 for first GPU if available
)
# ----------------------------------------------------------------------
def _sha256(text: str) -> str:
"""Return hex SHA‑256 digest of the supplied string."""
return hashlib.sha256(text.encode("utf-8")).hexdigest()
def _add_provenance_header(code: str, model: str) -> str:
"""
Prepend a JSON‑style comment block containing provenance info.
The comment style is language‑agnostic; adjust for target language if needed.
"""
metadata = {
"generated_by": model,
"generated_at": dt.datetime.utcnow().isoformat() + "Z",
"code_sha256": _sha256(code),
}
header = f"// PROVENANCE: {json.dumps(metadata, separators=(',', ':'))}\n"
return header + code
@retry(stop=stop_after_attempt(3), wait=wait_exponential(multiplier=1, min=2, max=10))
def generate_with_claude(prompt: str) -> str:
"""
Call Claude (Anthropic) to complete the prompt.
Returns raw model output (no provenance added yet).
"""
response = anthropic_client.messages.create(
model="claude-3-opus-20240229", # latest as of writing
max_tokens=500,
temperature=0.2,
system="You are a helpful coding assistant. Return only code, no extra explanation.",
messages=[{"role": "user", "content": prompt}],
)
# The API returns a list of content blocks; we concatenate text blocks.
return "".join(block.text for block in response.content if block.type == "text")
@retry(stop=stop_after_attempt(3), wait=wait_exponential(multiplier=1, min=2, max=10))
def generate_with_codex(prompt: str) -> str:
"""
Call OpenAI Codex (code‑davinci-002) to complete the prompt.
"""
response = openai_client.completions.create(
model="code-davinci-002",
prompt=prompt,
max_tokens=500,
temperature=0.2,
stop=["\n\n"], # stop at blank line to avoid extra commentary
)
return response.choices[0].text.strip()
@retry(stop=stop_after_attempt(3), wait=wait_exponential(multiplier=1, min=2, max=10))
def generate_with_hermes(prompt: str) -> str:
"""
Call Hermes via HuggingFace Inference API.
"""
# The pipeline returns a list of dicts; we take the first generated text.
outputs = hermes_generator(
prompt,
max_new_tokens=500,
temperature=0.2,
do_sample=True,
return_full_text=False, # only the newly generated part
)
return outputs[0]["generated_text"].strip()
def generate_and_provenance(model_name: str, prompt: str) -> Tuple[str, str, bool]:
"""
Dispatch to the correct generator, add provenance header,
and flag whether the output is considered *owned* (i.e., we added the header).
Returns:
(final_code, model_used, is_owned)
"""
if model_name.lower() == "claude":
raw = generate_with_claude(prompt)
elif model_name.lower() == "codex":
raw = generate_with_codex(prompt)
elif model_name.lower() == "hermes":
raw = generate_with_hermes(prompt)
else:
raise ValueError(f"Unsupported model: {model_name}")
# Heuristic: if the model output already contains a provenance‑like comment,
# we treat it as potentially owned (could be a previous run). Otherwise,
# we add our own header and mark as owned.
if "PROVENANCE:" in raw:
# Existing provenance – keep as‑is and consider it owned.
final_code = raw
is_owned = True
else:
final_code = _add_provenance_header(raw, model_name)
is_owned = True # we just added provenance, so we own it now
return final_code, model_name, is_owned
# ----------------------------------------------------------------------
if __name__ == "__main__":
# Example prompt: generate a factorial function in Python
prompt = """Write a Python function named `factorial` that computes the factorial of a non‑negative integer.
Include a docstring and raise ValueError for negative inputs."""
for model in ["Claude", "Codex", "Hermes"]:
code, used_model, owned = generate_and_provenance(model, prompt)
print(f"\n=== {used_model} output (owned={owned}) ===")
print(code)
# Optionally write to a file for later review / SBOM generation
out_path = f"generated_{used_model.lower()}_factorial.py"
with open(out_path, "w", encoding="utf-8") as f:
f.write(code)
print(f"💾 Saved to {out_path}")
.env.tenacity).// file: ai-code-provenance.ts
/**
* TypeScript implementation that mirrors the Python workflow:
* - Calls Claude (Anthropic), Codex (OpenAI), and Hermes (HF Inference)
* - Adds a provenance header if missing
* - Returns the final code plus ownership flag
*
* Prerequisites:
* npm install anthropic-sdk openai @huggingface/inference dotenv axios
* npm install --save-dev typescript ts-node @types/node
*/
import "dotenv/config";
import { Anthropic } from "@anthropic-ai/sdk";
import { OpenAI } from "openai";
import { InferenceApi } from "@huggingface/inference";
import axios from "axios";
import crypto from "crypto";
// ----------------------------------------------------------------------
// Load environment variables (already done by dotenv/config)
const ANTHROPIC_KEY = process.env.ANTHROPIC_API_KEY ?? "";
const OPENAI_KEY = process.env.OPENAI_API_KEY ?? "";
const HF_TOKEN = process.env.HF_TOKEN ?? "";
if (!ANTHROPIC_KEY || !OPENAI_KEY || !HF_TOKEN) {
throw new Error("Missing required API keys in environment.");
}
// ----------------------------------------------------------------------
// Initialise clients
const anthropic = new Anthropic({ apiKey: ANTHROPIC_KEY });
const openai = new OpenAI({ apiKey: OPENAI_KEY });
const hf = new InferenceApi(HF_TOKEN);
// ----------------------------------------------------------------------
/**
* Compute SHA‑256 hex digest of a string.
*/
function sha256(str: string): string {
return crypto.createHash("sha256").update(str, "utf8").digest("hex");
}
/**
* Add a provenance header (JSON comment) to the supplied code.
* Header format: // PROVENANCE: {"generated_by":"<model>","generated_at":"<ISO>","code_sha256":"<hash>"}
*/
function addProvenanceHeader(code: string, model: string): string {
const metadata = {
generated_by: model,
generated_at: new Date().toISOString(),
code_sha256: sha256(code),
};
const header = `// PROVENANCE: ${JSON.stringify(metadata)}\n`;
return header + code;
}
// ----------------------------------------------------------------------
/**
* Retry helper – exponential backoff, max 3 attempts.
*/
async function retry<T>(fn: () => Promise<T>, retries = 3): Promise<T> {
let attempt = 0;
while (true) {
try {
return await fn();
} catch (err) {
attempt++;
if (attempt >= retries) throw err;
// wait 2^attempt * 100ms
await new Promise((r) => setTimeout(r, 2 ** attempt * 100));
}
}
}
// ----------------------------------------------------------------------
/**
* Call Claude (Anthropic) to complete a prompt.
*/
async function generateClaude(prompt: string): Promise<string> {
return retry(async () => {
const msg = await anthropic.messages.create({
model: "claude-3-opus-20240229",
max_tokens: 500,
temperature: 0.2,
system: "You are a helpful coding assistant. Return only code, no extra explanation.",
messages: [{ role: "user", content: prompt }],
});
// Concatenate all text blocks
return msg.content
.filter((block): block is { type: "text"; text: string } => block.type === "text")
.map((b) => b.text)
.join("");
});
}
/**
* Call OpenAI Codex (code-davinci-002) to complete a prompt.
*/
async function generateCodex(prompt: string): Promise<string> {
return retry(async () => {
const resp = await openai.completions.create({
model: "code-davinci-002",
prompt: prompt,
max_tokens: 500,
temperature: 0.2,
stop: ["\n\n"],
});
return resp.choices[0]?.text?.trim() ?? "";
});
}
/**
* Call Hermes via HuggingFace Inference API.
*/
async function generateHermes(prompt: string): Promise<string> {
return retry(async () => {
const result = await hf.textGeneration({
model: "NousResearch/Hermes-2-Pro-Llama-3-8B",
inputs: prompt,
parameters: {
max_new_tokens: 500,
temperature: 0.2,
return_full_text: false,
},
});
// HF returns an array; we take the first generated text.
return Array.isArray(result) && result[0]?.generated_text
? result[0].generated_text.trim()
: "";
});
}
/**
* Dispatch to the correct generator, add provenance header if missing,
* and return (finalCode, modelUsed, isOwned).
*/
async function generateAndProvenance(
modelName: string,
prompt: string
): Promise<{ code: string; model: string; owned: boolean }> {
let raw: string;
switch (modelName.toLowerCase()) {
case "claude":
raw = await generateClaude(prompt);
break;
case "codex":
raw = await generateCodex(prompt);
break;
case "hermes":
raw = await generateHermes(prompt);
break;
default:
throw new Error(`Unsupported model: ${modelName}`);
}
// If the model already emitted a provenance‑like comment, treat as owned.
const owned = raw.includes("PROVENANCE:");
const final = owned ? raw : addProvenanceHeader(raw, modelName);
return { code: final, model: modelName, owned };
}
// ----------------------------------------------------------------------
// Example usage
(async () => {
const prompt = `Write a TypeScript function named \`factorial\` that computes the factorial of a non‑negative integer.
Include JSDoc documentation and throw an Error for negative inputs.`;
for (const model of ["Claude", "Codex", "Hermes"]) {
const { code, model: used, owned } = await generateAndProvenance(model, prompt);
console.log(`\n=== ${used} output (owned=${owned}) ===`);
console.log(code);
// Optional: write to file for later review / SBOM
const fs = await import("fs");
const path = await import("path");
const outPath = path.join(__dirname, `generated_${used.toLowerCase()}_factorial.ts`);
fs.writeFileSync(outPath, code, "utf8");
console.log(`💾 Saved to ${outPath}`);
}
})();
@anthropic-ai/sdk, openai, @huggingface/inference).<a name="step-4-configuration"></a>
| Variable | Description | Example | Where to set |
|---|---|---|---|
ANTHROPIC_API_KEY | API key for Claude (Anthropic) | sk-ant-api03-... | .env, secret manager, CI/CD variable |
OPENAI_API_KEY | API key for Codex (OpenAI) | sk-proj-... | Same as above |
HF_TOKEN | Personal access token for HuggingFace (read access to Hermes repo) | hf_... | Same as above |
LOG_LEVEL (optional) | Verbosity of internal logging (debug, info, warn, error) | info | .env or process env |
OUTPUT_DIR (optional) | Directory where generated code files are written | ./generated | .env or process env |
PROVENANCE_STYLE (optional) | Comment style for provenance header (js, py, txt) | py | .env – adjust header function accordingly |
Loading in code
Python: from dotenv import load_dotenv; load_dotenv()
TS/JS: import "dotenv/config"; (already done at top of file)
Secret management tip – In production, replace direct .env reads with calls to your secret store (e.g., AWS Secrets Manager get_secret_value, HashiCorp Vault vault read, Azure Key Vault getSecret). The code above only shows the simplest approach for local dev.
<a name="step-5-common-patterns"></a>
Both language examples use a small retry wrapper (tenacity in Python, custom retry in TS) to handle transient HTTP 429/502 errors.
The JSON block inside the comment can be extracted by a simple regex and fed into an SBOM generator (e.g., cyclonedx-bom, syft).
import re, json
pattern = r"// PROVENANCE: (\{.*\})"
match = re.search(pattern, code)
if match:
metadata = json.loads(match.group(1))
# metadata now contains generated_by, generated_at, code_sha256
Before committing AI‑generated code to a repo, run it in an isolated container or sandbox (e.g., Docker with --read-only filesystem, limited capabilities) to verify it behaves as expected and does not perform unwanted network/file system calls.
Add a CI step that scans new or changed files for the PROVENANCE: marker. If a file lacks it and the file matches known AI‑model output patterns (e.g., typical LLM phrasing, lack of author tags), fail the build and notify security.
# .github/workflows/ai-provenance.yml
name: AI Provenance Check
on: [push, pull_request]
jobs:
check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install Node (for JS check)
uses: actions/setup-node@v3
with:
node-version: '20'
- name: Run provenance scanner
run: |
npm i -g @oxsecurity/provenance-scanner # hypothetical scanner
provenance-scan --src . --fail-on-missing
Track usage per model (e.g., via a simple Redis counter) and pause calls when you approach your plan’s limits. This prevents unexpected bills and helps stay within corporate usage policies.
<a name="step-6-troubleshooting"></a>
| Symptom | Likely Cause | Fix |
|---|---|---|
401 Unauthorized from Anthropic/OpenAI/HF | Missing or incorrect API key | Verify .env values; ensure the key has the right permissions (e.g., models:read for HF). |
429 Too Many Requests | Exceeded rate limit | Implement/rely on retry backoff; consider upgrading plan or adding a usage throttler. |
Empty model output ("") | Prompt too long, temperature too low, or model refused | Shorten prompt, increase max_tokens, or adjust temperature. Check model’s content policy. |
ModuleNotFoundError: No module named 'anthropic' | Package not installed in active venv | Run pip install anthropic inside the same virtual environment you’re using to run the script. |
TS2304: Cannot find name 'process' | Running TS without dotenv/config import | Ensure import "dotenv/config"; is at the very top of the file. |
| Generated code fails linting (e.g., missing semicolons) | Model output style differs from project standards | Add a post‑generation formatting step (e.g., prettier --write or autopep8). |
| Provenance header duplicated on successive runs | Header detection logic only looks for exact string | Normalize whitespace/comments before checking, or strip existing header before re‑adding. |
Docker sandbox denies network calls but code tries to fetch | Generated code attempts outbound calls | Review generated code for unintended side‑effects; tighten sandbox (--network none) or add egress allowlist. |
Debug tip: Set environment variable LOG_LEVEL=debug and add simple print/console.log statements around each API call to see raw responses and timestamps.
<a name="step-7-production-checklist"></a>
| ✅ Item | Why it matters | How to verify |
|---|---|---|
| Secrets never in repo | Prevents key leakage | Run git grep -i "api_key|secret"; use pre‑commit hook detect-secrets. |
| Least‑privilege API keys | Limits blast radius if compromised | Create scoped keys (e.g., Claude key with only messages:write permission). |
| TLS enforcement | Protects credentials in transit | Ensure outbound calls use https://; disable HTTP fallbacks. |
| Input validation & sanitization | Stops prompt injection that could exfiltrate data | Validate prompt length, reject strings containing <<SYS>> or other model‑specific escape sequences. |
| Output sandboxing | Prevents malicious code from affecting host | Run generated code in a restricted container (docker run --read-only --network none --cap-drop ALL). |
| Provenance metadata extraction | Enables SBOM creation & audit trails | Implement a CI step that scans for // PROVENANCE: and feeds data to an SBOM tool. |
| Rate‑limit monitoring | Avoids unexpected costs & service denial | Export API response headers (x-ratelimit-remaining) to a monitoring system (Prometheus, Datadog). |
| Dependency vulnerability scanning | Ensures libraries used to call AI models are safe | Run npm audit / pip check or use Snyk/Dependabot. |
| License compliance check | AI‑generated code may inadvertently copy snippets | Use a tool like Scancode or FOSSology to detect copied snippets and verify licensing. |
| Incident response plan | Fast containment if unowned code is detected | Document steps: quarantine artifact, revoke keys, notify legal/security, forensic analysis. |
| Regular model version pinning | Guarantees reproducibility | Lock to specific model versions (e.g., claude-3-opus-20240229) and update only after review. |
| Training & awareness | Developers understand risks of unowned AI code | Conduct short workshops; include this guide in onboarding. |
By following the steps above you can:
Treat AI‑generated code like any third‑party dependency: track its origin, verify its integrity, and enforce policy before it reaches production.
Happy—and secure—coding! 🚀
Source: Ars Technica AI
Follow ICARAX for more AI insights and tutorials.
