Skip to main content

GPT-4o and Gemini 1.5 Pro just got beat in the AI race

a screenshot of claude 3.5 sonnet, with an 8-bit crab
Anthropic

There’s a new leader, technically, in the race for AI assistant dominance, and it’s Anthropic’s new Claude 3.5 Sonnet. The newly released model outperforms both Gemini 1.5 Pro and ChatGPT-4o across a spectrum of benchmark tests, the company announced on Thursday.

This new iteration of Sonnet is the first in Anthropic’s upcoming line of 3.5 models, and it significantly outperforms the more expansive Opus 3.0 model, and does so at a fraction of the larger model’s energy cost. Compute efficiency is becoming an increasingly important aspect of AI system design, especially as the cost of both powering and cooling AI data centers soars while the infrastructure pushes into the gigawatt range.

Claude 3.5 Sonnet for vision

“Claude 3.5 Sonnet operates at twice the speed of Claude 3 Opus,” the Anthropic team wrote in a blog post. “This performance boost, combined with cost-effective pricing, makes Claude 3.5 Sonnet ideal for complex tasks such as context-sensitive customer support and orchestrating multistep workflows.”

Recommended Videos

The new model has reportedly set benchmark results across three standardized tests: graduate-level reasoning with GPQA, undergraduate-level knowledge with MMLU, and coding proficiency with HumanEval. It beat out Google’s Gemini 1.5 Pro, Meta’s Llama-400b, and OpenAI’s ChatGPT-4o, though not by any huge margin and typically only by a couple percentage points.

A table showing Claude 3.5 Sonnet's performance compared to other leading AI systems.
Anthropic

Sonnet 3.5 is being billed as Anthropic’s “strongest vision model yet. ” It’s capable of performing a number of vision-based tasks — like interpreting charts and graphs or transcribing text from imperfect image sources like screenshots or scanned receipts — more accurately than Opus 3.0. In fact, Sonnet 3.5 beat out Opus 3.0 by anywhere from 6 to 17 points across industry standard vision benchmarks. The new model is also reportedly much more competent at handling humor and can converse in a much more lifelike manner.

Sonnet will also be the first Anthropic AI to offer the Artifacts feature to users. Rather than generate images or code snippets directly into the flow of the conversation, Artifacts will create that content in a dedicated space to the side of the chat. This allows users to create “a dynamic workspace where they can see, edit, and build upon Claude’s creations in real time, seamlessly integrating AI-generated content into their projects and workflows,” the Anthropic team claims. It also announced that Claude will soon support team collaboration wherein a company can store its data, documents and projects in a single, central silo, with Claude acting as an on-demand assistant.

You can try out Claude 3.5 Sonnet today for free on the Claude.ai website and the Claude iOS app (a Claude Pro or Team subscription will garner you significantly higher rate limits). Third-party integration is also available through the Anthropic API, Amazon Bedrock, and Google Cloud’s Vertex AI. Claude Haiku 3.5 and Opus 3.5 are scheduled for release later in the year.

Andrew Tarantola
Andrew Tarantola is a journalist with more than a decade reporting on emerging technologies ranging from robotics and machine…
This Lenovo ThinkPad is almost $1,800 off today!
A press photo of the ThinkPad X1 Carbon Gen 11.

One of the best laptops for a busy computer-heavy workplace is the Lenovo ThinkPad. For years, this tried and true laptop and 2-in-1 has delivered a fast and reliable Windows experience to many a 9 to 5 go-getter. Processor speed and power evolve year over year, and new features are added to these laptops all the time. This also means you’ll be able to find discounts on older machines, which is precisely what we came across while scouring through Lenovo ThinkPad deals:

Right now, as part of Lenovo’s doorbuster sale, you’ll save $1,800 on the purchase of a brand-new Lenovo ThinkPad X1 Carbon Gen 11 when you order through Lenovo.

Read more
Runway brings precise camera controls to AI videos
Gen-3 alpha advanced camera controls

Content creators will have more control over the look and feel of their AI-generated videos thanks to a new feature set coming to Runway's Gen-3 Alpha model.

Advanced Camera Control is rolling out on Gen-3 Alpha Turbo starting today, the company announced via a post on X (formerly Twitter).

Read more
Score the Dell XPS 15 for less than $1,000 during this sale
Dell XPS 15 9520 front view showing display and keyboard deck.

If you’ve been looking for laptop deals but feel disappointed with the results of your research, we know the pain. Searching for a new PC can take months, especially if you’ve got the time and energy to vet through numerous brands and models. Fortunately, there are a few tried and true PC names, one of which happens to be Dell. We see Dell laptop deals pretty regularly, but this one stopped us in our tracks:

Right now, when you order the Dell XPS 15 Laptop through the manufacturer, you’ll save $300. At full price, this model sells for $1,300.

Read more