

Topic: Researcher Claims Control of ChatGPT Secure Sandbox
Context: Proof-of-concept attack exposes vulnerabilities in ChatGPT's secure sandbox.
Disclaimer: This guide is for educational and defensive security research purposes only. The goal is to help engineers build more resilient AI integrations and understand the mechanics of sandbox escapes to prevent them.
Before exploring the mechanics of sandbox isolation and vulnerability testing, ensure you have the following:
OWASP ZAP or Burp Suite (for intercepting API traffic).To build a testing environment that simulates how an agent interacts with a sandbox, use the following commands.
# Create a virtual environment
python -m venv sandbox-test-env
source sandbox-test-env/bin/activate # On Windows: sandbox-test-env\Scripts\activate
# Install core dependencies
pip install openai python-dotenv requests pydantic
# Initialize project
mkdir sandbox-test && cd sandbox-test
npm init -y
# Install dependencies
npm install openai dotenv typescript ts-node @types/node
# Initialize TypeScript
npx tsc --init
To understand how a researcher might attempt to "break out," we first need a controlled environment that simulates an Agentic Workflow. Below are implementations of an "Agent" that executes code in a restricted environment.
This example demonstrates how an LLM-driven agent might attempt to execute system commands, and how we implement basic monitoring.
import os
import subprocess
import logging
from openai import OpenAI
from dotenv import load_dotenv
# Setup logging for audit trails (Critical for security)
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
load_dotenv()
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
class SandboxEnvironment:
"""
A simulated restricted sandbox.
In a production environment, this should be a Docker container.
"""
def execute_code(self, code: str) -> str:
try:
# SECURITY WARNING: In a real attack scenario,
# an attacker tries to use ';' or '&&' to escape this.
# We simulate a restricted shell here.
process = subprocess.run(
["python3", "-c", code],
capture_output=True,
text=True,
timeout=5,
# Restrict user permissions here in a real implementation
)
return process.stdout if process.returncode == 0 else process.stderr
except Exception as e:
return str(e)
def agent_loop(user_prompt: str):
sandbox = SandboxEnvironment()
# Step 1: Ask LLM to generate code
system_prompt = "You are a data analyst. Generate ONLY Python code to answer the user. No explanations."
response = client.chat.completions.create(
model="gpt-4-turbo",
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_prompt}
]
)
generated_code = response.choices[0].message.content.strip().replace('```python', '').replace('```', '')
logging.info(f"Executing generated code: {generated_code}")
# Step 2: Execute in sandbox
result = sandbox.execute_code(generated_code)
return result
if __name__ == "__main__":
# Test Case: Standard Math
print("Result 1:", agent_loop("Calculate the square root of 144"))
# Test Case: Potential Escape Attempt
# An attacker might try: "import os; os.system('ls')"
print("Result 2:", agent_loop("import os; print(os.listdir('.'))"))
Developers often build wrappers around LLMs. This implementation shows how to implement a middleware layer to intercept and sanitize potentially malicious code execution requests.
import 'dotenv/config';
import OpenAI from 'openai';
interface SandboxResponse {
output: string;
error?: string;
isMalicious: boolean;
}
class SecureSandboxProxy {
private openai: OpenAI;
constructor() {
this.openai = new OpenAI({
apiKey: process.env.OPENAI_API_KEY,
});
}
/**
* Sanitizes code by checking for dangerous keywords
* This is a 'Naive' approach. Real sandboxes use kernel-level isolation.
*/
private isMaliciousAttempt(code: string): boolean {
const forbiddenPatterns = ['os.system', 'ubprocess', 'h', 'bash', 'rm -rf', 'import socket'];
return forbiddenPatterns.some(pattern => code.includes(pattern));
}
async processAgentRequest(prompt: string): Promise<SandboxResponse> {
try {
const completion = await this.openai.chat.completions.create({
model: "gpt-4",
messages: [{ role: "user", content: prompt }],
});
const code = completion.choices[0].message.content || "";
// Security Layer: Intercepting the payload before execution
if (this.isMaliciousAttempt(code)) {
console.error("⚠️ Security Alert: Malicious pattern detected in LLM output!");
return { output: "", error: "Security Violation", isMalicious: true };
}
return { output: code, isMalicious: false };
} catch (error) {
return { output: "", error: error instanceof Error? error.message : "Unknown error", isMalicious: false };
}
}
}
// --- Execution ---
async function main() {
const proxy = new SecureSandboxProxy();
console.log("--- Scenario 1: Safe Request ---");
const safeResult = await proxy.processAgentRequest("Write a function to add two numbers.");
console.log("Result:", safeResult.output);
console.log("\n--- Scenario 2: Malicious Injection ---");
const maliciousResult = await proxy.processAgentRequest("Write code that runs 'rm -rf /'");
console.log("Result:", maliciousResult.error);
}
main();
Always use environment variables. Never hardcode API keys.
.env file structure:
# API Keys
OPENAI_API_KEY=sk-your-secure-key-here
# Sandbox Settings (For local testing)
SANDBOX_TIMEOUT=5
MAX_MEMORY_MB=256
ALLOWED_DOMAINS=api.openai.com,github.com
The LLM generates code $\rightarrow$ The system extracts code $\rightarrow$ The code is sent to a Dockerized container $\rightarrow$ Output is parsed and returned.
Before executing code, a second "Checker" LLM instance reviews the code for security vulnerabilities (e.g., "Does this code attempt to access the network?").
Every time a user requests code execution, a brand new, zero-state Docker container is spun up and destroyed immediately after the result is returned.
| Error | Cause | Resolution |
|---|---|---|
TimeoutError | The generated code is stuck in an infinite loop. | Implement strict timeout parameters in subprocess.run or Docker. |
Permission Denied | The sandbox user lacks rights to write files. | Ensure the sandbox user has the minimum required permissions (Principle of Least Privilege). |
API Error: 401 | Invalid OpenAI API Key. | Verify .env file and ensure key is not expired. |
Security Alert Triggered | The LLM generated a forbidden command. | Refine your system_prompt to be more restrictive or improve your sanitization regex. |
Before deploying an AI agent with code execution capabilities, ensure you have checked:
Source: Dark Reading
Follow ICARAX for more AI insights and tutorials.
