Introduction: Amidst the relentless drumbeat of AI revolutionizing every facet of industry, a sobering reality is beginning to surface in the trenches of software development. As one seasoned engineer’s candid account reveals, the much-touted LLM “co-pilot” might be less a helpful navigator and more a back-seat driver steering us towards unforeseen technical debt and profound disillusionment.

Key Points

• The “LLM as an assistant, human as the architect” paradigm is not merely a preference but a critical necessity, highlighting AI’s current inability to grasp architectural complexity or maintain code quality autonomously.
• For non-trivial, feature-complete development or infrastructure rebuilds, LLMs can actively decrease productivity and introduce significant technical debt, turning perceived “shortcuts” into costly detours.
• The initial “wow” factor of LLMs often masks deeper, persistent issues around accuracy, coherence, and the introduction of subtle, hard-to-debug errors, leading to a cycle of fixing AI-generated fixes.

In-Depth Analysis

Alberto Fortin’s experience, while deeply personal, resonates with a growing chorus of senior developers confronting the chasm between AI hype and its practical application in complex engineering environments. His initial embrace of LLMs, fueled by the industry’s pervasive narrative of transformative productivity gains, quickly gave way to a stark realization: for serious infrastructure work involving Go and ClickHouse, these tools were more hindrance than help. The core problem, as Fortin articulates, wasn’t just about functional bugs, but a deeper malaise of poor code quality, architectural incoherence, and a persistent “two weeks away from fixing” loop, where LLM-generated solutions introduced new, often insidious, problems.

This isn’t merely a critique of specific models; it’s a fundamental challenge to the notion of generative AI as a reliable partner in anything beyond highly constrained, trivial tasks. For an engineer tasked with maintaining a robust, scalable codebase for years, the “neatness” and logical consistency of the code are paramount. LLMs, despite their linguistic fluency, demonstrably struggle with this, often generating solutions that are syntactically plausible but architecturally naive or fundamentally flawed, akin to a skilled orator who lacks subject matter expertise.

The critical insight from Fortin — “I am the software engineer, the senior software engineer, I am the architect. The LLM is the assistant. The assistant responds to me; I make the plan” — encapsulates the essential mental shift required. The fantasy of offloading significant cognitive load or creative problem-solving to AI is, for now, just that: a fantasy. Instead, engineers find themselves performing an exhausting new role: that of a hyper-vigilant AI quality assurance specialist, constantly debugging not just their own code, but the AI’s suggestions, and often, the cascading effects of those suggestions. This overhead negates any perceived efficiency gains, leading to a loss of trust that forces developers back to the tried-and-true method: understanding the codebase themselves. The “hundred percent understanding” derived from manual debugging, Fortin found, was not a slower alternative but the quickest path to a stable solution.

Contrasting Viewpoint

Proponents of LLM-driven development might argue that Fortin’s experience is an outlier, perhaps stemming from insufficient prompt engineering, overly ambitious use cases, or a failure to adapt to a new paradigm. They would point to countless examples where LLMs successfully generate boilerplate, refactor minor code snippets, or assist in debugging error messages. The argument often shifts to “It’s about how you use it” or “You’re asking it to do too much, too soon.” Furthermore, the pace of AI advancement is undeniable; models are improving rapidly, and perhaps the next iteration will overcome these very hurdles. They might contend that even if LLMs only handle 10-20% of routine coding tasks, that still represents a significant aggregate productivity boost across a large engineering team, freeing up senior developers for more strategic work.

Future Outlook

The realistic 1-2 year outlook for LLMs in complex software engineering suggests continued evolution as powerful assistants, but not as autonomous or even semi-autonomous engineers. The industry will likely settle into a more nuanced understanding, where LLMs excel at specific, narrow tasks: generating unit tests, boilerplate code, documentation, or offering targeted refactoring suggestions. True architectural reasoning, maintaining code quality over time, and understanding the intricate dependencies of a large-scale system remain firmly within the human domain. The biggest hurdles to overcome are not just about model accuracy, but about building genuine trust, ensuring deterministic and explainable outputs, and proving a net positive return on investment when factoring in the cognitive overhead of verification and the cost of potential technical debt. The “revolution” will be incremental, focusing on augmenting human capabilities rather than replacing them.

For more context, see our deep dive on [[The True Cost of Software Tooling]].

Further Reading

---

Original Source: I’m dialing back my LLM usage (Hacker News (AI Search))

阅读中文版 (Read Chinese Version)