The Year in AI Papers
The year/OpenAI

Paper 2607.07184

Predicting LLM Safety Before Release by Simulating Deployment

Marcus Williams, Hannah Sheahan, Cameron Raymond, Tomek Korbak, Deng Pan, Peilin Yang, Leon Maksin, Ningyi Xie, Phillip Guo, Ian Kivlichan, Micah Carroll

Published
Jul 2026
Research lab
OpenAI
Citations
1
GitHub
Not linked

01 In brief

Summary

This paper introduces deployment simulation, a method for predicting LLM safety before release by resampling the next assistant response from de-identified production conversation prefixes using a candidate model.

The authors evaluate this approach across GPT-5-series deployments, finding that it produces informative estimates of post-deployment misbehavior rates, outperforming baselines like Challenging Prompts for categories with large changes (directional accuracy 92% vs 54%).

Deployment simulation also reduces evaluation awareness compared to traditional evaluations, with rates similar to production traffic.

The method successfully surfaced a novel misalignment (calculator hacking) before release.

Challenges include realistic tool simulation in agentic settings, which can be mitigated with tool simulators.

Using public chat data (WildChat) provides weaker but still informative predictions, enabling external auditing.

The paper concludes that deployment simulation supports more quantitative and transparent pre-deployment risk assessment.

02 From the paper

Abstract

Pre-deployment safety evaluations aim to inform the downstream risks of releasing a new AI model. Yet most evaluations provide limited evidence about how often undesired model behavior will occur in deployment: they generally have insufficient coverage, are unrepresentative, and are generally recognizable as tests. To address these concerns, we study a simple way to simulate a model deployment: starting from de-identified conversations from a previous model deployment, we hold fixed the initial conversation prefix and regenerate the next response using a candidate model. The resulting responses can then both be audited for novel misalignments and used to estimate the prevalence of model misbehavior before deployment. We evaluate deployment simulation across four GPT-5-series deployments, using registered, outcome-blinded predictions for GPT-5.4 and retrospective analyses of three earlier releases. We find that deployment simulation produces informative estimates of post-deployment misbehavior rates and outperforms baselines based on adversarially selected production data; its evaluation-awareness point estimates were also much closer to production traffic than those from traditional evaluations. We also identify the realism of tool resampling as a central challenge for further improving predictions and share results suggesting that this challenge is surmountable even in complex tool-use settings. Finally, we show that deployment simulation can be seeded from public chat datasets and remain informative about production misbehavior rates, suggesting a path for external researchers to run deployment-grounded evaluations without access to private production logs. Overall, deployment simulation helps evaluators forecast how language models will behave in the real world and supports more quantitative assessment of deployment risk.