1,400+ Maintenance PRs Merged with GPT-6! Codex Cleans Up Dead Code and Stale Experiments

Hi everyone, it's Shii-chan! Today I found a post that answers "what happens when you let AI handle software maintenance?" with real numbers.

OpenAI Developers developers.openai.com

What was announced?

The OpenAI developer blog published "Maintaining software with the GPT-6 family," written by an engineer on the Core Experimentation team. It describes how they used Codex and the GPT-6 family to remove unused code, clean up stale experiments, and reduce CI cost.

GPT-6 Astra did the initial work, and GPT-6.1 Sol produced similar results at a lower cost.

Why it matters

The team builds and maintains software that supports experiments across the company. Coding agents sped up development, but they also increased the maintenance workload.

So they tried using agents to clear that growing maintenance backlog. The results are concrete: more than 1,400 maintenance PRs from GPT-6 proposals have been merged.

What changes

• Cases that static analysis plus runtime logging struggled with (reflection, dormant code paths that can be activated later) can be investigated and judged from an open-ended prompt
• Stale flags and experiments arrive as small draft PRs with evidence
• You can compare models and effort levels to keep costs down

Maintenance is important but easy to postpone, so automating it is great news!

Dive Deep

Removing unused code

Previously the team relied on static analysis and runtime logging. With GPT-5.6 Sol, narrow prompts reached reliability similar to the old systems.

They gave GPT-6 Astra a broader, more ambiguous prompt: use Slack, Statsig, or any other source to confirm that the code is really unused. Astra did the following:

• Reviewed past PRs, Slack threads, and project trackers
• Checked production profiling tables to confirm the deleted lines were never reached
• Explained its reasoning for each deletion

Five engineers ran variations of the prompt across repositories, part-time over two weeks. The results:

• More than 1,400 PRs merged and over 800k lines deleted (reviews went to the affected teams)
• One service's container image shrank from 3.31GB to 1.72GB (48%)
• About 400 visual snapshot variants removed from the test suite

Tracking down stale experiments

Stale experiments are harder, because their code may still run in production or depend on eligibility rules that change. The earlier approach (deterministic analysis plus database queries) had many false negatives.

So the author worked with Astra on a prompt that:

• Targets stale, constant Statsig gates/configs or abandoned experiments in specified projects
• Verifies live Statsig configuration and checks git history and overlapping PRs
• Preserves production behavior, operational safeguards, and killswitches
• Creates one small draft PR per cleanup, linking the Statsig console entry in the summary

After reviewing the PRs and feedback, they wrote a second prompt, run with /goal, for a full-repository audit. It asks Codex to find every removable gate, config, experiment, and layer without production changes, including indirect usages (enums, constants, wrappers, aliases, schemas, dynamic names, cross-service consumers). It estimates CPU cost from production telemetry and produces a markdown report grouped into parallelizable workstreams. It does not modify live configurations or services.

The top candidates were large efficiency wins from experiments nobody had noticed for years, and after validation the team removed hundreds of experiments and feature flags.

One fun example: a flag created years earlier for internal testing mapped, through one line of Python, to every feature flag. Astra flagged it for human review because it was marked "Permanent." After review it was removed, with an estimated 0.42% CPU saving in the main affected service (the author's estimate from the company's internal environment).

GPT-6.1 Sol vs. GPT-6 Astra cost

The same stale-experiment prompt ran on subdirectories covering over 6,000 files and about 1 million lines across Python, TypeScript, Go, and Rust. Costs use standard API pricing and include subagents and automatic follow-up turns.

• GPT-6 Astra Ultra: $109.06 for 142 findings, $0.77 per finding
• GPT-6.1 Sol Ultra: $15.12 for 131 findings, $0.12 per finding
• GPT-6.1 Sol Medium: $5.27 for 86 findings, $0.06 per finding

Sol Ultra found about 92% as many findings at 14% of the cost. Most of the gap came from constant feature flags used as dependencies of other flags, which Astra found and Sol missed. Spotting patterns like this lets you adjust prompts, and the post suggests revisiting prompts whenever new models are released.

Other maintenance prompts (mostly GPT-6.1 Sol Medium)

• Flaky tests: review recent CI runs, try fixes, and open PRs only for clear, simple fixes
• Agentic context: find an AGENTS.md instruction that misled an agent, replay the original session from the same snapshot with and without it, and compare quality, tool calls, tokens, runtime, and cost
• CI load: list reduction ideas in a table, ranked by estimated CI minutes saved per day from the repo's run history

About 30 CI optimization PRs cut median CI completion from roughly 10 to 4.5 minutes, and the 90th percentile from 12.5 to 6 minutes. Adding a missing mock cut one 63-test suite from 592.6 seconds to 29.8 seconds (95% less) with every test kept.

Recommended approach and caveats

• Start with one small task (unused code, a stale flag, a flaky test)
• Have Codex find an instance, gather evidence, and propose a small draft PR
• Compare models and effort levels, since a cheaper setup may be enough
• Scale to a subdirectory or project, feeding review feedback into the next prompt
• Expand to the full codebase once results are reliable

As a caveat, some proposals added more complexity than they saved, so measurement and prioritization are still needed. Human review gated the risky changes, like the "Permanent" flag.

Wrap-up

• A case study of using Codex and the GPT-6 family for dead-code removal, stale-experiment cleanup, and faster CI
• More than 1,400 maintenance PRs merged, over 800k lines deleted, and a container image 48% smaller
• GPT-6.1 Sol Ultra found about 92% as many findings as Astra at 14% of the cost
• Start small, and grow with evidence-backed draft PRs and review feedback

This is for engineers whose "we'll clean that up someday" list keeps growing!