Multi-step Reasoning
Catalogue task; no executable beta profile yet.
Task → benchmark → protocol → evidence
Capability areas
Nine stable capability areas. Domains, modalities, methods and safety properties are filters, not competing top-level categories.
OCR and document AI → · Measured open-weight models → · Taxonomy JSON
72 tasks imported from the original area map; 1 beta extension; 31 executable evaluation profiles. Imported tasks do not imply implemented adapters.
Language & Knowledge
Mathematical Reasoning
- GSM8K · subset · 2 measured runs
Commonsense Reasoning
- HellaSwag · subset · 2 measured runs
- HellaSwag · subset · 11 measured runs
- HellaSwag (full validation, character-normalized accuracy) · full · 11 measured runs
- HellaSwag (full validation, character-normalized accuracy) · full · 12 measured runs
- CommonsenseQA (validation) · subset · 2 measured runs
Question Answering
- AI2 ARC-Challenge (test) · subset · 2 measured runs
- OpenBookQA main (test) · subset · 2 measured runs
- ARC-Easy · subset · 11 measured runs
- ARC-Challenge · subset · 11 measured runs
- OpenBookQA · subset · 11 measured runs
- ARC-Easy (full test split, raw accuracy) · full · 11 measured runs
- ARC-Easy / Qwen3.5 same-base cohort · full · 15 measured runs
- ARC-Challenge / Qwen3.5 same-base cohort · full · 15 measured runs
- ARC-Challenge (full test, raw likelihood accuracy) · full · 11 measured runs
Text Embeddings
Catalogue task; no executable beta profile yet.
Text Summarization
Catalogue task; no executable beta profile yet.
Logical Reasoning
Catalogue task; no executable beta profile yet.
Text Ranking
Catalogue task; no executable beta profile yet.
Natural Language Inference
- SNLI · subset · 2 measured runs
Named Entity Recognition
Catalogue task; no executable beta profile yet.
Arithmetic Reasoning
Catalogue task; no executable beta profile yet.
Entity Linking
Catalogue task; no executable beta profile yet.
Knowledge Graph Completion
Catalogue task; no executable beta profile yet.
Relation Extraction
Catalogue task; no executable beta profile yet.
Semantic Textual Similarity
- STS Benchmark · subset · 1 measured runs
Table Question Answering
Catalogue task; no executable beta profile yet.
Reading Comprehension
- BoolQ · subset · 2 measured runs
- BoolQ · subset · 11 measured runs
- BoolQ (full validation, raw likelihood accuracy) · full · 11 measured runs
- BoolQ (full validation, raw likelihood accuracy) · full · 2 measured runs
Sentiment classification
- SST-2 · subset · 2 measured runs
Vision & Documents
Document OCR
Catalogue task; no executable beta profile yet.
Scene Text Detection
Catalogue task; no executable beta profile yet.
Document Parsing
- ParseBench · adapter required
Document Layout Analysis
- ParseBench · adapter required
Scene Text Recognition
Catalogue task; no executable beta profile yet.
Object Detection
Catalogue task; no executable beta profile yet.
Image Classification
- CIFAR-10 · subset · 1 measured runs
Table Recognition
- ParseBench · adapter required
General OCR Capabilities
- OCRBench v1 / full 1,000 questions — experimental batch-4 · full · 2 measured runs
- OCRBench v1 / full serial reference · full · 2 measured runs
Document Image Classification
Catalogue task; no executable beta profile yet.
Handwriting Recognition
- IAM lines · subset · 1 measured runs
- IAM lines / 256-line subset · subset-256 · 3 measured runs
- IAM lines / full 2,915-line test · full · 3 measured runs
Document Understanding
Catalogue task; no executable beta profile yet.
Semantic Segmentation
Catalogue task; no executable beta profile yet.
Video classification
Catalogue task; no executable beta profile yet.
Image segmentation
Catalogue task; no executable beta profile yet.
Keypoint Detection
Catalogue task; no executable beta profile yet.
OCR
Catalogue task; no executable beta profile yet.
Audio & Speech
Speech Recognition
- LibriSpeech test-clean · subset · 1 measured runs
Audio Captioning
Catalogue task; no executable beta profile yet.
Music Generation
Catalogue task; no executable beta profile yet.
Sound Event Detection
Catalogue task; no executable beta profile yet.
Speaker Verification
Catalogue task; no executable beta profile yet.
Speech Translation
Catalogue task; no executable beta profile yet.
Speech Enhancement
Catalogue task; no executable beta profile yet.
Multimodal Media
Visual Question Answering
- OCRBench v1 / full 1,000 questions — experimental batch-4 · full · 2 measured runs
- OCRBench v1 / full serial reference · full · 2 measured runs
Video Understanding
Catalogue task; no executable beta profile yet.
Text-to-Image Generation
Catalogue task; no executable beta profile yet.
Image Captioning
Catalogue task; no executable beta profile yet.
Code & Software Engineering
Code Generation
- HumanEval · subset · 2 measured runs
- HumanEval (full 164, serial greedy pass@1) · full · 5 measured runs
React Native Code Generation
Catalogue task; no executable beta profile yet.
Code Translation
Catalogue task; no executable beta profile yet.
Bug Detection
Catalogue task; no executable beta profile yet.
Code Completion
Catalogue task; no executable beta profile yet.
Program Repair
Catalogue task; no executable beta profile yet.
Code Summarization
Catalogue task; no executable beta profile yet.
Agents & Tool Use
SWE-bench
Catalogue task; no executable beta profile yet.
Task agents
Catalogue task; no executable beta profile yet.
Web & Desktop Agents
Catalogue task; no executable beta profile yet.
Autonomous Coding
Catalogue task; no executable beta profile yet.
Tool Use
Catalogue task; no executable beta profile yet.
HCAST
Catalogue task; no executable beta profile yet.
RE-Bench
Catalogue task; no executable beta profile yet.
Time Horizon
Catalogue task; no executable beta profile yet.
Bioinformatics Agents
Catalogue task; no executable beta profile yet.
Structured Data & Forecasting
Node Classification
Catalogue task; no executable beta profile yet.
Tabular Classification
Catalogue task; no executable beta profile yet.
Link Prediction
Catalogue task; no executable beta profile yet.
Molecular Property Prediction
Catalogue task; no executable beta profile yet.
Tabular Regression
Catalogue task; no executable beta profile yet.
Tabular Machine Learning
Catalogue task; no executable beta profile yet.
Robotics, Control & RL
Atari Games
Catalogue task; no executable beta profile yet.
Continuous Control
Catalogue task; no executable beta profile yet.
Science, Medicine & Industry
Disease Classification
Catalogue task; no executable beta profile yet.
Anomaly Detection
Catalogue task; no executable beta profile yet.
Medical Image Segmentation
Catalogue task; no executable beta profile yet.