Introduction: The dream of powerful, on-device AI for everyone, not just flagship owners, is a compelling one. Cactus (YC S25) enters this arena claiming to optimize AI inference for the vast majority of smartphones, the budget and mid-range devices. But while the market need is undeniable, one can’t help but wonder if this ambitious startup is planting itself in fertile ground or merely adding another layer of complexity to an already fragmented landscape.

Key Points

• Cactus boldly targets the 70%+ market share of budget and mid-range phones, a segment currently underserved by mainstream AI frameworks.
• If truly successful, Cactus could democratize advanced on-device AI features, shifting power away from exclusive high-end hardware.
• The claim of a “bottom-up,” “no dependencies” framework with custom kernels is a double-edged sword: a potential performance advantage but also a monumental engineering and adoption hurdle.

In-Depth Analysis

Cactus arrives with a clear mission: to bring sophisticated AI inference to the budget and mid-range smartphone market. This is a genuinely underserved niche, where existing frameworks like TensorFlow Lite or PyTorch Mobile, while offering CPU fallbacks, often struggle to deliver acceptable performance for larger, modern models without dedicated NPUs or DSPs. The value proposition is enticing: unlock new AI-driven experiences for the majority of global smartphone users, not just those with the latest iPhone Pro or Pixel flagship.

The technical approach described is intriguing. Cactus proposes a custom, four-layered stack, from low-level ARM-specific SIMD kernels up to an OpenAI-compatible C API. The “no dependencies” and “bottom-up” design philosophy suggests a leaner, potentially more efficient footprint, aiming to eke out every possible FLOPS from less powerful CPUs. The mention of “zero-copy computation graph” is a nod to minimizing memory overhead, which is critical on resource-constrained devices.

However, this bespoke architecture also raises significant questions. In an industry increasingly gravitating towards standardized high-level APIs and robust, vendor-backed ecosystems (Core ML for Apple, Android Neural Networks API, Qualcomm AI Engine), a new, custom stack represents a substantial engineering challenge for both Cactus and potential adopters. Developers are often reluctant to embrace new frameworks that don’t offer overwhelming performance gains or superior ease of integration compared to established giants. The demo snippet, while illustrative, showcases raw C API usage – a paradigm many modern mobile developers have moved away from in favor of higher-level abstractions and comprehensive SDKs.

The preliminary performance figures, while presented as positive, require careful contextualization. 16-20 tokens/second (t/s) for Qwen3-600m-INT8 on a Pixel 6a or iPhone 11 Pro (CPU-only) is respectable for a smaller model, demonstrating their core thesis of optimizing for older hardware. But 50-70 t/s on a Pixel 9 or iPhone 16 is less impressive when considering these devices possess powerful NPUs that, with proper framework utilization (which Cactus does claim to target for high-end phones via SMMLA, NPU & DSP), should yield far superior results. The Qwen3-4B-INT4 on iPhone 16 Pro NPU at 21 t/s is a more realistic benchmark for NPU utilization, but still needs comparison against Apple’s own MLX or Core ML for similar models to understand its true competitive edge. The “500k+ weekly inference tasks in production today” claim is a strong signal of early adoption, but without specifics on the applications or the complexity of those tasks, its impact is hard to assess. Ultimately, the success hinges on whether their performance gains for the target market are truly transformative enough to justify developers taking on a new, non-standard dependency.

Contrasting Viewpoint

While Cactus touts its “bottom-up” and “no dependencies” approach as a strength, a skeptic would argue this is precisely its greatest vulnerability. In a mobile ecosystem dominated by Apple, Google, and chip manufacturers like Qualcomm and ARM, building a custom, low-level inference stack from scratch is an audacious undertaking. These tech behemoths pour billions into optimizing their own hardware and software, providing highly integrated, battle-tested solutions (Core ML, TensorFlow Lite, ANNA) that often benefit from exclusive hardware access and deep OS-level integration. Why would a developer choose to build on a nascent, bespoke framework, regardless of its performance claims, when the industry giants offer comprehensive toolchains, vast communities, and long-term support? The “no dependencies” approach might offer a lean footprint, but it also implies a heavy burden of continuous optimization and maintenance across an increasingly fragmented Android landscape. Keeping pace with diverse ARM architectures, OS updates, and ever-evolving NPU designs from dozens of manufacturers is a monumental task, potentially stretching a startup thin against the resources of industry titans.

Future Outlook

The immediate 1-2 year outlook for Cactus hinges entirely on proving its “significantly better performance” promise for the targeted budget/mid-range segment, not just on paper, but in real-world, demanding applications. Their claim of 500k+ weekly inferences is a positive sign, but the crucial next step is to secure high-profile partnerships with app developers or even OEMs to pre-integrate their SDK. Without such strategic alliances, penetrating the market beyond early adopters will be an uphill battle.

The biggest hurdles include the relentless pace of mobile hardware evolution – next year’s “mid-range” phone will have last year’s flagship NPU, potentially eroding Cactus’s CPU-only optimization advantage. Furthermore, competing with first-party mobile AI frameworks and their vast developer ecosystems will be challenging. Cactus must not only deliver superior performance but also offer a compelling developer experience, robust documentation, and long-term support. If they can carve out a niche where their custom kernels offer genuinely game-changing performance for specific, resource-intensive AI models on the long tail of Android devices, they might succeed. Otherwise, it risks becoming another interesting but ultimately niche solution in a crowded market.

For more on the challenges and opportunities in democratizing AI, explore our previous report on [[The Future of Edge AI and Device Fragmentation]].

Further Reading

---

Original Source: Launch HN: Cactus (YC S25) – AI inference on smartphones (Hacker News (AI Search))

阅读中文版 (Read Chinese Version)