The Migration Trap: Why LLM-Driven Refactoring is Just Technical Debt in a New Language

The current rush to automate legacy code migration—specifically the transition from COBOL to Java—is often framed as a liberation from the mainframe. But from a practitioner's perspective, there is a critical distinction between *migration* and *modernization*. If you are simply using LLMs to translate old logic into a new syntax, you aren't solving technical debt; you are just migrating it to a different language.

This tension is evident in a recent paper published via arXiv, as first reported by the platform, titled "Agentic Method for Deterministic Validation of Legacy Code Migration." Authors Andras Ferenczi, Jordan Docherty, Mariya Bessonov, Matthew Findlay, and Krishna Lingamneni describe a process designed to ensure that generated Java code behaves exactly like its COBOL predecessor. They call this the "Locksmith Loop."

***

**Opinion: The Parity Paradox**

As a developer, the goal of refactoring is typically to improve the architecture, remove redundancies, and optimize performance. However, the "Locksmith Loop" focuses on "deterministic parity checks." The objective is to ensure the Java target matches the COBOL reference. While this is a necessary step for safety, it reveals the limitation of agentic coding: the AI is being used to mirror the legacy system's behavior, not to rethink it. When we prioritize parity over architectural intuition, we risk creating "Java-COBOL"—code that lives in a modern environment but retains the rigid, outdated logic of the 1970s.

***

According to the arXiv paper, the Locksmith Loop addresses the common hurdles of legacy migration, such as a lack of test data and the difficulty of validating edge cases. The method involves instrumenting both the COBOL source and the generated Java target with mocks and executing them on commodity hardware. From there, an iterative agentic loop performs a "Witness Search" over input mocks to penetrate program branches, utilizing parity-preserving mutations.

When the system hits a boundary that prevents deeper exploration, the authors identify this as a "Locked Paragraph." By identifying these blocks, the researchers claim they can push coverage beyond typical input-search plateaus.

In three case studies—which included two open-source programs and one internal production-like COBOL program ranging from 430 to 4,114 source lines—the Locksmith method achieved nearly complete coverage on the open-source examples and 91.90% branch coverage on the internal production-like program. The researchers found that for every accepted test case, the generated Java was consistent with the COBOL reference under deterministic parity checks.

While the technical achievement of achieving 91.90% branch coverage is impressive, it underscores the reality of automated migration. The agentic loop is acting as a high-speed validator for a translation task. It ensures the machine did not hallucinate, but it does not ensure the resulting code is actually *better* from a software engineering standpoint. Until AI can provide the architectural intuition to tell a developer *why* a legacy structure should be discarded rather than mirrored, these tools remain sophisticated translators, not architects.