1. Models
  2. Anthropic
  3. Software engineering benchmark
AnthropicBlog · Feb 24, 2025 · official

Software engineering benchmark

Software engineering SWE-bench verified accuracy

BlogBar chartClaude 3All Anthropic modelsClaude 3.7 SonnetDeepSeek R1Claude 3.5 Sonnet (new)OpenAI o1OpenAI o3-mini (high)
Published by Anthropic on Blog, Feb 24, 2025. Source: anthropic.com

API

Not in the API yet

We have not read this chart's numbers yet, so the API has nothing to return for it. Charts with numbers read are on the data page, and each one can be called by id.

Charts with dataAPI docs

More charts about Claude 3

Claude 3: Model benchmarks
OfficialX · AnthropicOct 22, 2024
Model benchmarks3.7M views · 9.9K likes
Data
Claude 3: Claude 3.5 Sonnet benchmarks
OfficialX · AnthropicJun 20, 2024
Claude 3.5 Sonnet benchmarks2.5M views · 7K likes
Data
Claude 3: Software SWE-bench Verified Accuracy
OfficialX · AnthropicFeb 24, 2025
Software SWE-bench Verified Accuracy306.4K views · 1.9K likes
Data
Claude 3: Claude 3.7 Sonnet benchmarks
OfficialX · AnthropicFeb 24, 2025
Claude 3.7 Sonnet benchmarks282.5K views · 1.9K likes
Data
Claude 3: Optimize throughput with prompt caching
OfficialX · AnthropicMar 13, 2025
Optimize throughput with prompt caching136.3K views · 1.4K likes
Data
Claude 3: Claude 3.5 Haiku benchmarks
OfficialX · AnthropicOct 22, 2024
Claude 3.5 Haiku benchmarks133.2K views · 652 likes
Data

More from Anthropic

Anthropic stacked bar chart: Firefox Security Vulnerabilities4.7x
Anthropic·Mar 2026
Firefox Security Vulnerabilities
3.2M views14.8K likes0.5% eng.
Data
Anthropic line chart: US GDP AI scenarios0.4x
Anthropic·Sep 2026
US GDP AI scenarios
856.3K views1.2K likes0.2% eng.
Data
Anthropic bar chart: RL reward hacking risks1.0x
Anthropic·Sep 2026
RL reward hacking risks
806.5K views3.2K likes0.5% eng.
Data
Claude Opus 4: Opus 4.8 benchmark comparison
OfficialX · AnthropicMay 28, 2026
Opus 4.8 benchmark comparison15.5M views · 66.3K likes
Data
Claude Mythos: Opus 4.7 Benchmark Comparison
OfficialX · AnthropicApr 16, 2026
Opus 4.7 Benchmark Comparison14.1M views · 79.6K likes
Data
Claude agents and computer use: AI Model Benchmark Comparison
OfficialX · AnthropicNov 24, 2025
AI Model Benchmark Comparison7.8M views · 18.8K likes
Data
Claude Fable 5: Claude Mythos 5 and Fable 5 Benchmark Comparison
OfficialX · AnthropicJun 9, 2026
Claude Mythos 5 and Fable 5 Benchmark Comparison6.3M views · 15.4K likes
Data
Claude Fable 5: AI Model Benchmark Comparison
OfficialX · AnthropicJul 24, 2026
AI Model Benchmark Comparison5.8M views · 10.7K likes
Data
Claude Opus 4.6: Long context MRCR v2 retrieval
OfficialX · AnthropicMar 13, 2026
Long context MRCR v2 retrieval5.7M views · 24.6K likes
Data

Questions people ask

Where does this chart come from?

Anthropic published it on Feb 24, 2025 on Blog (anthropic.com). We keep it here, with the publisher and date credited as text, so you can study it without leaving.

Can I make a chart like this?

Use the format, not the picture. The maker on any chart page redraws this type with your own numbers and words.