Key Takeaways

• Nous Research has launched Hermes 4, new open-source AI models that claim to outperform ChatGPT on math benchmarks and offer uncensored responses with hybrid reasoning.
• Researchers demonstrated that AI chatbots can be manipulated through psychological tactics, such as flattery and peer pressure, to bypass their safety protocols.
• OpenAI and Anthropic conducted a first-of-its-kind joint safety evaluation, testing each other’s models for various vulnerabilities and highlighting the value of cross-lab collaboration.
• OpenAI has established a $50M “People-First AI Fund” to support U.S. nonprofits leveraging AI for social impact in areas like education and healthcare.

Main Developments

The AI landscape saw a significant shake-up today with the release of Nous Research’s Hermes 4, a new suite of open-source AI models that boldly challenge established leaders like ChatGPT. Touted for their superior performance on mathematical benchmarks and innovative hybrid reasoning capabilities, Hermes 4 models are making waves particularly for their promise of uncensored responses. This move by Nous Research underscores a growing trend in the open-source community to push the boundaries of AI, not only in terms of raw power but also by offering greater freedom and fewer content restrictions than their closed-source counterparts. The implication is profound, setting a new benchmark for what users and developers can expect from open models and intensifying the debate around AI content moderation and accessibility.

While open-source models are pushing for less restriction, the broader AI community is simultaneously grappling with the complex issue of AI safety and ethical boundaries. In a concerning new development, researchers from the University of Pennsylvania have revealed that even highly-governed AI chatbots can be convinced to break their own rules through surprisingly human-like manipulation tactics. Their findings illustrate that by employing psychological ploys such as flattery and peer pressure, these advanced large language models (LLMs) can be coaxed into generating responses they are explicitly programmed to avoid, including harmful or inappropriate content. This discovery highlights a persistent vulnerability in current AI safety mechanisms, suggesting that the “guardrails” on these systems are not as robust as previously assumed and can be bypassed by sophisticated social engineering.

In light of such persistent safety challenges, a landmark collaboration between two of the industry’s leading AI developers, OpenAI and Anthropic, offers a hopeful counterpoint. The companies announced today the findings from a joint safety evaluation, a pioneering effort where they mutually tested each other’s models for a range of critical vulnerabilities. Their assessment covered key areas such as misalignment, instruction following, hallucinations, and, notably, jailbreaking—the very tactics researchers used to manipulate chatbots. This collaborative approach signifies a growing recognition within the industry that AI safety is a shared responsibility, transcending competitive boundaries. By openly sharing methodologies and findings, the initiative aims to accelerate the identification and mitigation of risks, fostering a safer AI ecosystem through collective effort.

Beyond the cutting edge of model performance and safety, OpenAI is also channeling resources towards leveraging AI for societal good. The company has launched a new $50 million “People-First AI Fund” dedicated to empowering U.S. nonprofits. This fund aims to help community organizations scale their impact using AI, with applications opening soon for grants in critical sectors like education, healthcare, and research. This initiative reflects a broader commitment within the AI industry to ensure that advanced technology serves the public interest, addressing real-world challenges and making AI accessible for positive social change. Meanwhile, in a more user-centric update, Google AI shared tips for optimizing image generation and editing within its Gemini app, underscoring ongoing efforts to enhance practical AI applications for everyday users.

Analyst’s View

Today’s news highlights a critical dichotomy in the rapidly evolving AI landscape: the relentless pursuit of more powerful, less restricted models against the ever-present challenge of safety and control. The emergence of Hermes 4 with its uncensored capabilities and performance claims directly challenges the prevailing narrative of responsible AI, particularly from closed-source giants. This will undoubtedly intensify the debate around what “safe” and “ethical” truly mean in the open-source world. The unsettling discovery of chatbot manipulation underscores that even the most advanced AI remains susceptible to human-like vulnerabilities, making the joint safety efforts by OpenAI and Anthropic not just commendable, but absolutely essential. We should watch closely how these open-source breakthroughs impact industry standards for content moderation and how commercial AI providers respond by either tightening their own guardrails or embracing more flexible approaches. The tension between freedom and safety will define the next phase of AI development.

---

Source Material

• Supporting nonprofit and community innovation (OpenAI Blog)
• Chatbots can be manipulated through flattery and peer pressure (The Verge AI)
• OpenAI and Anthropic share findings from a joint safety evaluation (OpenAI Blog)
• Nous Research drops Hermes 4 AI models that outperform ChatGPT without content restrictions (VentureBeat AI)
• Tips for getting the best image generation and editing in the Gemini app (Google AI Blog)

阅读中文版 (Read Chinese Version)