Devin

🔓 Open

Fully autonomous AI software engineer with sandboxed environment, browser access, and end-to-end task completion.

💰 Free tier + $20/mo pay-as-you-go · web, api

74.7Overall Score

Non-Gameable Scoring

Scores are derived from established benchmarks, adjusted for harness-specific performance across four dimensions: Coding, Reasoning, Tool Use, and Autonomy.

Each dimension starts from public benchmark data and applies harness-specific modifiers based on tool integration, context handling, and orchestration quality. The overall score is a weighted composite that penalizes narrow optimization.

Model	Overall	Coding	Reasoning	Tool Use	Autonomy
Claude Opus 4.6	74.7	70.68	84.6	66.41	92.5
Gemini 3.1 Pro	66.7	75.44	81.19	61.57	88.33
Kimi K2.5	64.1	71.94	72.16	41.86	84.5
GPT-5.4	59.6	45.32	62.53	84.18	84.5
Claude Sonnet 4.6	58.6	62.7	52.32	64.71	92.5
MiniMax-M2.7	56.4	56.26	48.11	63.7	84.5
DeepSeek R1	50.2	56.96	48.1	44.41	72.13
Qwen 3.5	49.1	50.22	48.55	32.79	67.82
Gemini 3 Pro	47.5	41.18	53.28	49.31	70.72
MiMo-V2-Flash	44.5	54.99	44.51	40.75	59.37