Ploy Migrates Production Agent to GPT-5.6 Sol

Ploy has transitioned its production AI agent—which builds and edits marketing websites—from Claude Opus 4.8 to OpenAI's GPT-5.6 Sol. According to Ploy, the migration was justified by a head-to-head evaluation showing GPT-5.6 Sol completed builds more than twice as fast and at a lower cost.

In a redesign suite evaluation, Ploy reported that GPT-5.6 Sol's mean cost per completed build was $2.22, compared to $3.06 for Claude Opus 4.8. Wall-clock time dropped from 8 minutes to 3 minutes and 42 seconds. Ploy also noted that GPT-5.6 Sol used roughly half the output tokens and wrote more concise code; in one instance, GPT-5.6 Sol used 2,508 characters for a CSS file where Opus produced 17,957 characters.

Despite the performance gains, Ploy detailed significant integration hurdles. The company found that GPT-5.6 Sol handled tool arguments differently than Claude, sending all 25 of the tool's parameters on every call and filling the unused, optional ones with plausible but invented values. Ploy reported that this caused 52% to 64% of GPT-5.6's file reads to return empty results because the system could not distinguish invented values from intended ones. Ploy resolved this by implementing a schema transform at the provider boundary, rewriting optional properties as required but nullable.

Additionally, Ploy noted that initial evaluation runs were skewed by harness assumptions. GPT-5.6 Sol's tendency to make parallel calls exceeded tool-call budgets sized for Opus's sequential style, and the model frequently used batched file reads that the executor did not initially support. Ploy stated that roughly one-third of raw failures in the first run stemmed from these harness assumptions rather than model behavior.