Language / BoolQ (full validation, raw likelihood accuracy)

BoolQ / full sub-1B common core (qwen35)

Provided context/question + candidate answers → selected answer

Measurements Evaluation protocol Planned runs Cost & information

Model size vs score

Static, watermarked figures. SlayerLab models are red. The close-up expands the crowded small-model cluster without changing any coordinates.

BoolQ (full validation, raw likelihood accuracy): all measured models, with a dotted Pareto frontier and peach favourable region — Overview · Download PNG · Download SVG

BoolQ (full validation, raw likelihood accuracy): small-model close-up, with a dotted Pareto frontier and peach favourable region — Small-model close-up · Download PNG · Download SVG

Measured results

Full pinned test split · 3,270 examples · accuracy higher is better. No global SOTA claim.

System	Score	Samples	Elapsed¹	Evidence
Qwen/Qwen3.5-0.8B-Base dc7cdfe2ee41	74.3%	3270	126s	Inspect run → 17 Oct 2026, 15:02 UTC

¹ Evaluation elapsed time includes model loading after dataset selection. It is not serving latency or a fair speed comparison.

What this measures

Benchmark origin

BoolQ (full validation, raw likelihood accuracy) ↗

Dataset / split

aps/super_glue / validation

Immutable dataset revision

3de24cf8022e94f4ee4b9d55a6f539891524d646

Upstream license metadata

other · consult upstream terms

Protocol

codesota-common-core-qwen35-boolq-lm-eval-v2 / choice

Selection

All 3270 examples in the pinned test split. SNLI rows without a valid label are excluded.

Configuration

Seed 42. Greedy generation for math and code. Summed conditional answer-token log likelihood; raw accuracy. Character-normalized accuracy is recorded separately. No model judge.

Read scoring and comparison rules →

Planned runs

0 qualified visits in 30 days · priority 1000.0. Interest changes execution order, within the approved baseline list.

System	Status
Qwen/Qwen3.5-0.8B-Base	completed