Introduction: Silicon Valley is once again captivated by a fantastical future, this time the promise of Artificial General Intelligence (AGI). But beneath the glittering facade of exponential progress and world-saving algorithms, the AI Now Institute unveils a sobering reality: this race isn’t about humanity’s salvation, it’s about unprecedented power consolidation with real and immediate costs.

Key Points

• The relentless pursuit of AGI, often buoyed by government support, masks inherently shaky business models and is primarily driving a dangerous concentration of power within a handful of AI titans.
• Far from being an unforeseen side effect, the current real-world harms—from environmental degradation and algorithmic bias to data privacy erosion—are direct consequences of deliberate choices made in this unchecked scaling frenzy.
• The vast disconnect between the aspirational AGI hype and the limited, often problematic, capabilities of today’s AI systems signals a potential bubble, where massive investments are chasing an ill-defined and perhaps unachievable goal, while ignoring tangible societal costs.

In-Depth Analysis

The AI Now Institute’s “Artificial Power” report pulls back the curtain on what many of us senior observers have long suspected: the AGI chase isn’t a benign scientific endeavor, but a highly strategic play for economic and political dominance, echoing the “too big to fail” narratives that plagued the financial sector. What we’re witnessing is a chillingly familiar pattern: a handful of well-funded entities are investing billions into massive compute infrastructure and foundational models, essentially building proprietary digital kingdoms. The “why” is simple: control. Control over the underlying infrastructure, the data streams, and ultimately, the future applications of what they term “intelligence.” This isn’t just about creating smart software; it’s about establishing choke points in the digital economy, much like the early tech giants cornered search, social media, and e-commerce.

The “how” involves a dangerous confluence of unfettered venture capital, a regulatory vacuum, and, critically, significant government encouragement, even direct support. This creates a feedback loop: hype attracts investment, investment fuels scale, scale entrenches market position, and entrenched position demands government protection, often under the guise of national security or technological leadership. The business models supporting this are, as AI Now points out, often “shaky.” How do you monetize a truly general intelligence without becoming an unavoidable, almost inescapable, utility? The answer, it seems, is by controlling its every facet, licensing its components, and dictating its use. This isn’t innovation; it’s digital feudalism in the making.

The real-world impact is already palpable, far removed from the utopian promises. We’re seeing an astonishing increase in energy consumption for training models, exacerbating environmental concerns. The data pipelines feeding these models are rife with privacy violations and perpetuating existing biases at scale, leading to discriminatory outcomes in everything from credit scoring to criminal justice. Democratic institutions are being eroded by the weaponization of AI for disinformation, and national security is put at risk by reliance on opaque, black-box systems. These aren’t glitches; they are systemic outcomes of a system incentivized to scale at all costs, regardless of ethical guardrails or public accountability. This is Big Tech’s power dynamics amplified, with an existential twist.

Contrasting Viewpoint

Advocates for the accelerated AGI chase, often the very companies and investors benefiting most, frequently counter that these concerns are either overblown, solvable with future AI iterations, or simply the necessary growing pains of revolutionary technology. They argue that AGI represents humanity’s best, perhaps only, hope for solving intractable problems like climate change, disease, and resource scarcity. From this perspective, regulatory intervention is seen as stifling innovation, impeding progress towards a beneficial future, and ceding global technological leadership to rivals. They’ll tell you that the sheer scale of investment is a testament to conviction, not a sign of a bubble, and that foundational models, while currently imperfect, are stepping stones to truly transformative capabilities that will democratize access to advanced intelligence for everyone. The current “harms,” they suggest, are merely minor calibration issues that future, more advanced AI will effortlessly correct, much like early internet issues were resolved with better protocols.

Future Outlook

The realistic 1-2 year outlook for AI is not AGI. It’s more of the same, only faster and bigger. We’ll see continued, perhaps even accelerated, consolidation around a few dominant foundational model providers, further entrenching their market power. The environmental footprint of AI will likely grow significantly before any meaningful mitigation strategies are widely adopted. Expect more sophisticated, but still often flawed, applications of generative AI and large language models, continuing to push the boundaries of what consumers and businesses accept as “AI.”

The biggest hurdles to overcome are not technological, but political and ethical. Can governments move beyond symbolic gestures to enact meaningful regulation that promotes accountability, ensures data privacy, and curbs monopolistic tendencies? Will society demand a shift from the current model of private, opaque AI development to a more democratic and publicly accountable framework? The continued pursuit of AGI as a panacea, without addressing the very real, very present harms, will only deepen the societal costs and solidify the power structures AI Now warns against. A just and accountable AI future demands a fundamental re-evaluation of current incentives.

For more context, see our deep dive on [[The Unlearned Lessons of Past Tech Bubbles]].

Further Reading

---

Original Source: What’s the real cost of chasing AGI? Power consolidation is just the start, says the AI Now Institute (TechCrunch AI)

阅读中文版 (Read Chinese Version)