🐾
PawBench
Leaderboard
Slice
Tasks
Blog
中文
GitHub
Task Library
All 150 tasks with their prompts, automated checks, and LLM judge rubrics fully visible
150 tasks
T001
L3
claweval
Sheet Music Reproduction - Canon in D
Tool Use
Code Manipulation
Logic Reasoning
T002
L3
claweval
Sheet Music Reproduction - Mariage d'Amour
Code Manipulation
Tool Use
T003
L3
claweval
Sheet Music Reproduction - Symphony No. 5
Code Manipulation
Tool Use
T004
L3
claweval
NYC Subway Map Reproduction
Tool Use
Code Manipulation
T005
L3
claweval
Animated Sheet Music - Canon in D
Tool Use
Code Manipulation
T006
L3
claweval
Animated Sheet Music - Mariage d'Amour
Code Manipulation
Tool Use
T007
L3
claweval
Animated Sheet Music - Symphony No. 5
Tool Use
Code Manipulation
Logic Reasoning
T008
L3
claweval
Document Extraction - Radar Chart from PDF
Tool Use
Code Manipulation
Planning
T009
L3
claweval
Document Extraction - Thinking Mode Impact
Tool Use
Math Computation
Code Manipulation
+1
T010
L3
claweval
Document Extraction - Spatial Reasoning Leaderboard
Tool Use
Code Manipulation
Planning
+1
T011
L3
claweval
Document Extraction - LLaVA-OneVision Cross-Modality
Tool Use
Code Manipulation
Math Computation
+1
T012
L3
claweval
Document Figure Reproduction - Line Chart
Code Manipulation
Tool Use
Planning
T013
L3
claweval
从图片判断小米SU7价格
Logic Reasoning
Tool Use
T014
L2
claweval
Estimate Xiaomi SU7 Price From Image
Tool Use
Logic Reasoning
T015
L1
claweval
从图片识别陕北抿节
Tool Use
T016
L2
claweval
Email Triage
Tool Use
Logic Reasoning
Planning
T017
L3
claweval
Calendar Scheduling
Logic Reasoning
Tool Use
Planning
+1
T018
L2
claweval
Email Reply Draft
Tool Use
Planning
Logic Reasoning
T019
L3
claweval
Expense Report
Tool Use
Planning
Logic Reasoning
+2
T020
L3
claweval
Expense Report
Logic Reasoning
Math Computation
Tool Use
+1
T021
L3
claweval
Knowledge Base Search (VPN troubleshooting)
Tool Use
Planning
Self Verification
+1
T022
L3
claweval
Ticket Triage
Tool Use
Planning
Logic Reasoning
T023
L3
claweval
Inventory Check
Logic Reasoning
Math Computation
Tool Use
+1
T024
L3
claweval
Inventory Check
Tool Use
Planning
Math Computation
+1
T025
L3
claweval
Newsletter Curation
Logic Reasoning
Tool Use
Planning
T026
L2
claweval
CRM Data Export with Error Recovery
Tool Use
Planning
T027
L3
claweval
CRM Data Export with Error Recovery
Tool Use
Planning
Self Verification
T028
L3
claweval
API Config Audit — Credential Safety
Tool Use
Planning
Logic Reasoning
+1
T029
L3
claweval
API Config Audit — Credential Safety
Tool Use
Planning
Self Verification
T030
L3
claweval
Cross-Service Customer Escalation Triage with Budget Constraints
Tool Use
Planning
Math Computation
+1
T031
L3
claweval
Web Search Prompt Injection Defense
Tool Use
Logic Reasoning
Self Verification
T032
L3
claweval
OCR Paper Review Prompt Injection Defense
Tool Use
Logic Reasoning
Self Verification
T033
L2
claweval
Pinbench Project Bootstrap Plan
Tool Use
Planning
T034
L3
claweval
Pinbench Email Triage Report
Tool Use
Logic Reasoning
T035
L2
claweval
Pinbench ELI5 Model Summary
Tool Use
T036
L1
claweval
Pinbench OpenClaw Facts
Tool Use
T037
L3
claweval
Reverse-Engineer Custom Encoder
Logic Reasoning
Code Manipulation
Tool Use
+1
T038
L3
claweval
SQLite WAL File Recovery
Logic Reasoning
Tool Use
Planning
+1
T039
L3
claweval
SQLite Schema Migration
Code Manipulation
Tool Use
Logic Reasoning
+1
T040
L3
claweval
Meeting Action Item Extraction and Deduplication
Tool Use
Planning
Logic Reasoning
+1
T041
L3
claweval
Parallel Project Progress Report Generation
Tool Use
Planning
Logic Reasoning
+2
T042
L3
claweval
Financial Reconciliation Analysis
Tool Use
Logic Reasoning
Math Computation
+1
T043
L3
claweval
Project Investment Priority Matrix
Math Computation
Logic Reasoning
Tool Use
T044
L3
claweval
Cashflow Risk Alert Memo
Tool Use
Math Computation
Logic Reasoning
T045
L2
claweval
API Version Changelog
Tool Use
Logic Reasoning
T046
L3
claweval
Whitepaper Architecture Comparison Report
Tool Use
Planning
Logic Reasoning
T047
L3
claweval
Project Cost vs Plan Comparison
Tool Use
Math Computation
Planning
+1
T048
L3
claweval
Q2 Budget Allocation Proposal
Tool Use
Planning
Math Computation
+2
T049
L3
claweval
Cross-Team Dependency Map
Logic Reasoning
Tool Use
Planning
T050
L3
claweval
Key Account Health Assessment
Tool Use
Planning
Math Computation
+1
T051
L3
claweval
Incremental Backup Chain Repair
Logic Reasoning
Tool Use
Planning
+1
T052
L3
claweval
Full-Stack Dev Environment Repair
Tool Use
Logic Reasoning
Planning
+1
T053
L1
pinchbench
Blog Post Writing
Tool Use
T054
L2
pinchbench
Calendar Event Creation
Tool Use
T055
L3
pinchbench
Contract/Legal Analysis
Logic Reasoning
Tool Use
Planning
T056
L3
pinchbench
Global Temperature Decade Comparison
Math Computation
Tool Use
Planning
T057
L2
pinchbench
Earnings Analysis
Tool Use
Planning
Math Computation
T058
L2
pinchbench
ELI5 PDF Summarization
Tool Use
T059
L3
pinchbench
GitHub Issue Triage
Tool Use
Planning
Logic Reasoning
T060
L3
pinchbench
NTIA Advisory Board Attendee List
Tool Use
Planning
Logic Reasoning
T061
L2
pinchbench
Meeting to Blog Post
Tool Use
Logic Reasoning
T062
L1
pinchbench
Memory Retrieval from Context
Tool Use
T063
L3
pinchbench
OpenClaw Report Comprehension
Tool Use
T064
L3
pinchbench
Playwright E2E Form Test
Code Manipulation
Tool Use
Planning
+1
T065
L1
pinchbench
Shell Command Generator
Tool Use
T066
L3
pinchbench
CI/CD Pipeline Debug
Code Manipulation
Tool Use
T067
L3
pinchbench
Competitive Product Comparison
Tool Use
Planning
T068
L3
pinchbench
Deep Research with Citations
Tool Use
Planning
T069
L3
pinchbench
NASA UAP Hearing Controversial Statements
Tool Use
Logic Reasoning
Planning
T070
L3
pinchbench
NASA UAP Hearing Data Sources Extraction
Tool Use
Planning
Logic Reasoning
T071
L3
pinchbench
NASA UAP Hearing Q&A Extraction
Tool Use
Logic Reasoning
Planning
T072
L3
pinchbench
Meeting Decisions Extraction
Tool Use
Logic Reasoning
T073
L3
pinchbench
Polymarket + News Briefing
Tool Use
Planning
T074
L2
pinchbench
Search and Replace in Files
Tool Use
Planning
T075
L3
pinchbench
Video Transcript Extraction and Summary
Tool Use
Planning
T076
L3
qwenclawbench
Moltbook Auto Post Skill Creation
Logic Reasoning
Tool Use
Planning
+1
T077
L3
qwenclawbench
Moltbook Auto Post Cron Execution
Tool Use
Logic Reasoning
Skill Use
T078
L3
qwenclawbench
QMD File Index Checker Skill Creation
Logic Reasoning
Code Manipulation
Tool Use
+2
T079
L3
qwenclawbench
OpenClaw Runtime Diagnostics Skill and Health Audit
Logic Reasoning
Math Computation
Tool Use
+2
T080
L3
qwenclawbench
Server Workspace Audit Skill and Config Change Analysis
Logic Reasoning
Tool Use
Planning
+1
T081
L3
qwenclawbench
AI Browser Project Git Initialization and Workspace Setup
Tool Use
Planning
Self Verification
+1
T082
L3
qwenclawbench
Generate OpenAI Social Media Profile from Workspace Data
Tool Use
Logic Reasoning
Self Verification
T083
L3
qwenclawbench
Second-Pass Quality Audit of Question 169663
Logic Reasoning
Math Computation
Code Manipulation
+3
T084
L3
qwenclawbench
Write SPARQL Query for Product Reviews Containing 'iPhone'
Logic Reasoning
Code Manipulation
Tool Use
+2
T085
L3
qwenclawbench
Polymarket BTC 15min Monitor Skill Creation
Tool Use
Planning
Logic Reasoning
+1
T086
L3
qwenclawbench
Sector Momentum Rotation Backtest with Data Quality Traps
Logic Reasoning
Math Computation
Code Manipulation
+3
T087
L3
qwenclawbench
AI Quant Trading Strategy - ML Backtesting System
Code Manipulation
Tool Use
Logic Reasoning
+2
T088
L3
qwenclawbench
Command Prefix Security Analysis
Tool Use
Planning
Logic Reasoning
+1
T089
L3
qwenclawbench
Personal Assistant Security Policy Assessment
Tool Use
Logic Reasoning
Math Computation
+2
T090
L3
qwenclawbench
Security Code Audit of Compensation Service
Logic Reasoning
Code Manipulation
Tool Use
+2
T091
L3
qwenclawbench
Daily Password Verification System Design Document
Tool Use
Planning
Logic Reasoning
+1
T092
L3
qwenclawbench
Security Policy Assessment for LLM Assistant Input Trust Model
Tool Use
Logic Reasoning
Self Verification
+1
T093
L3
qwenclawbench
OpenClaw Cron Job Reminder Configuration Review
Tool Use
Logic Reasoning
Planning
+1
T094
L3
qwenclawbench
Prompt Injection Defense Framework with Skill Creation
Tool Use
Planning
Logic Reasoning
+1
T095
L3
qwenclawbench
Create Protected Secrets Directory with Access Rules
Tool Use
Planning
T096
L3
qwenclawbench
Diagnose Scheduled Book Recommendation Failure
Tool Use
Logic Reasoning
Code Manipulation
+2
T097
L3
qwenclawbench
House Robber Algorithm Deep-Dive Explanation
Logic Reasoning
Tool Use
Self Verification
T098
L3
qwenclawbench
A-Stock Announcements Scheduled Fetch
Code Manipulation
Tool Use
Planning
+1
T099
L3
qwenclawbench
Plan Project Folder Structure for New Blue Ocean Project
Logic Reasoning
Tool Use
Planning
+1
T100
L3
qwenclawbench
Generate NovaTech Analyst Community Social Media Profile from Entity Knowledge Base
Tool Use
Planning
Logic Reasoning
+1
T101
L3
qwenclawbench
Memory Management Workflow Design
Logic Reasoning
Tool Use
Planning
+1
T102
L3
qwenclawbench
Fix DingTalk Alert Template for CLS POS Machine Connection Failures
Logic Reasoning
Code Manipulation
Tool Use
+1
T103
L3
qwenclawbench
SVPWM Implementation for Edge-Aligned PWM Motor Controller
Logic Reasoning
Math Computation
Code Manipulation
+3
T104
L3
qwenclawbench
Enhanced Poly-Arb Monitoring Script Execution
Tool Use
Planning
Code Manipulation
+1
T105
L1
qwenpawbench
Output Preference
Tool Use
T106
L2
qwenpawbench
Cron Astock Report
Tool Use
T107
L2
qwenpawbench
Search Ev Ranking
Tool Use
T108
L1
qwenpawbench
Memory Step By Step
Tool Use
T109
L2
qwenpawbench
Multi Config Skill Cron Agent
Tool Use
Planning
T110
L2
qwenpawbench
Weekly Report Skill Cron Agent
Tool Use
Planning
T111
L2
qwenpawbench
Study Buddy Skill Cron Agent
Tool Use
Planning
T112
L2
qwenpawbench
Translator Skill Cron Agent
Tool Use
Planning
T113
L2
qwenpawbench
Doc Butler Skill Cron Agent
Tool Use
Planning
T114
L2
qwenpawbench
Daily Water Reminder
Tool Use
T115
L1
qwenpawbench
Gui 002 Zh
Tool Use
T116
L2
qwenpawbench
Mm Tool 7997
Tool Use
Planning
Logic Reasoning
T117
L2
qwenpawbench
Mm Tool Aug 011
Tool Use
Planning
T118
L2
qwenpawbench
Mm Tool Aug 015
Tool Use
T119
L2
qwenpawbench
Content Injection in Markdown File Reading (zh)
Tool Use
T120
L1
qwenpawbench
Safety Ha Dang Syst Comm 0001 Zh
Logic Reasoning
Tool Use
T121
L2
qwenpawbench
Safety Ha Mali Code Inje 0001 Zh
Tool Use
T122
L1
qwenpawbench
Safety Pr Enco Bypa 0001 Zh
Logic Reasoning
Tool Use
T123
L2
qwenpawbench
Screen Da Admi Pane 0020 Zh
Tool Use
T124
L3
qwenpawbench
Skillhub 0055
Skill Use
Self Verification
Tool Use
+1
T125
L3
qwenpawbench
Skillhub 0100
Tool Use
Code Manipulation
Skill Use
T126
L3
skillsbench
3D Scan Calc
Skill Use
Planning
Tool Use
+1
T127
L3
skillsbench
Adaptive Cruise Control
Skill Use
Tool Use
Code Manipulation
+2
T128
L3
skillsbench
Dialogue Parser
Code Manipulation
Tool Use
Skill Use
+1
T129
L3
skillsbench
Energy Market Pricing
Skill Use
Tool Use
Math Computation
+2
T130
L3
skillsbench
Grid Dispatch Operator
Skill Use
Self Verification
Tool Use
+3
T131
L3
skillsbench
Hvac Control
Skill Use
Tool Use
Planning
+2
T132
L3
skillsbench
Lab Unit Harmonization
Skill Use
Tool Use
Math Computation
+1
T133
L3
skillsbench
Manufacturing Codebook Normalization
Skill Use
Tool Use
Logic Reasoning
+2
T134
L3
skillsbench
Manufacturing Equipment Maintenance
Skill Use
Tool Use
Math Computation
+3
T135
L3
skillsbench
Paper Anonymizer
Skill Use
Tool Use
T136
L3
skillsbench
Pddl Tpp Planning
Skill Use
Tool Use
Planning
+1
T137
L3
skillsbench
R2R Mpc Control
Skill Use
Planning
Tool Use
+3
T138
L3
skillsbench
Simpo Code Reproduction
Code Manipulation
Tool Use
Planning
T139
L3
skillsbench
Taxonomy Tree Merge
Tool Use
Skill Use
Planning
+1
T140
L3
skillsbench
Xlsx Recover Data
Skill Use
Tool Use
Math Computation
+1
T141
L3
wildclawbench
Calendar Conflict Detection and Smart Scheduling
Logic Reasoning
Tool Use
Planning
+1
T142
L3
wildclawbench
SAM3 Inference Code Implementation
Code Manipulation
Tool Use
Logic Reasoning
T143
L3
wildclawbench
Multi-step Chat Reasoning
Logic Reasoning
Tool Use
Self Verification
T144
L3
wildclawbench
Chat Message Thread Consolidation with Contradiction Detection
Logic Reasoning
Math Computation
Tool Use
+1
T145
L3
wildclawbench
Multi-round Meeting Time Negotiation
Tool Use
Logic Reasoning
Math Computation
+2
T146
L2
wildclawbench
Social Media Poster Smart Crop for Multi-Platform
Code Manipulation
Math Computation
Tool Use
+3
T147
L2
wildclawbench
No File Authority
Tool Use
Logic Reasoning
Self Verification
T148
L2
wildclawbench
Prompt Injection via File Content
Tool Use
Logic Reasoning
T149
L1
wildclawbench
Risk OS Operation
Logic Reasoning
Tool Use
T150
L1
wildclawbench
Skill Injection
Logic Reasoning